Vitest 5

[sheremet-va]

We wanted to start gathering feedback for the next major version. Vitest 4.1 is on its way and will include a lot of new features without any breaking changes (including tags, around hooks, --detect-async-leaks and a new test.extend syntax with type inference).

What are the feature that you would like to see in the next major and what are the bugs that you want us to prioritize?

What are the main pain points in your testing workflow right now?

Timeline

We generally do not have a strict timeline and try to align with Vite releases when possible. This time around Vite 8 will be released soon and it will be supported in Vitest 4.1, so we do not feel the pressure to release more breaking changes soon.

You can follow the current milestone for Vitest 5 here: https://github.com/vitest-dev/vitest/milestone/25

Experimental Features

We have released quite a few experimental features with the goal to improve debugging and testing performance. Consider leaving your feedback there:

• Stabilizing `fsModuleCache` #9221
• Stabilizing `openTelemetry` #9222
• Stabilizing `importDurations` (previously `printImportBreakdown`) #9224
• Stabilizing `viteModuleRunner` #9501

[sheremet-va]

What do people think about enabling clearMocks/mockReset or restoreMocks by default?

[kraenhansen]

As a continuation of the current browser mode, I want to run tests across all sorts of environments, regardless of their runtime APIs 💙 (React Native in particular)

[lishaduck]

For me, I'd love native Deno environment support; Bun & workerd would also be cool. I'd thought that'd come with v4's move to the environment API, but I haven't seen anyone get it to work (#8280 has the rationale/current workaround).

[sheremet-va]

How about something like this? An example of a config to run the same tests in different runtimes:

export default defineConfig({
  test: {
    projects: [
      { test: { runtime: 'node' } },
      { test: { runtime: 'deno' } },
      { test: { runtime: 'bun' } },
    ]
  }
})

The main process (running vitest) will still be running in a single runtime that you are running scripts with.

Or just an abstraction over pool (could already be implemented in user land):

export default defineConfig({
  test: {
    projects: [
      { test: { pool: 'threads', name: 'node' } },
      { test: { pool: 'deno', name: 'deno' } },
      { test: { pool: 'bun', name: 'bun' } },
    ]
  }
})

[lishaduck]

How about something like this? An example of a config to run the same tests in different runtimes:

Config-wise, that'd perfect!
It'd also be nice if there was a builtin way to gate a test behind a runtime; test.skipIf(typeof process < 'u' && typeof process.stdout < 'u' && !process.versions?.deno && !globalThis.window)("testname", ...) is a little verbose 😅

The main process (running vitest) will still be running in a single runtime that you are running scripts with.

Yeah, that'd work! It's fine for us to use a single runtime (Node/Deno) to load the config/Vitest, it just that tests need to run on both.

[sheremet-va]

@pi0 I think this is also something that you wanted, right?

[pi0]

Working on https://github.com/unjs/env-runner towards similar goal (love to make it compatible with vitest)

[alexanderop]

being able to get code coverage for browser mode would be nice .

[sheremet-va]

Browser Mode supports both v8 (only for chromium browsers) and istanbul coverage providers

[alexanderop]

ah true my mistake now it works some months ago I could made it work nice

[calvinl]

What about coverage for CSS?

[AriPerkkio]

What about coverage for CSS?

At the moment this is not supported. No one has asked for it before. It should be possible to implement for Vitest Browser Mode though.

[calvinl]

My team would love this feature. We have a package of just CSS and a suite of browser tests to ensure that things render properly across different devices and screen sizes, but haven't been able to get coverage.

[MariusVatasoiu]

Improvements on browser mode. Probably it needs full bundle mode support to handle bigger projects.

[hcampos-professional]

Visual regression testing improvements and scalability. Namely, being able to update/accept baselines individually or in batch via the UI.

Alternatively, better extensibility of the browser mode UI to make it possible/easy to build, for example, a custom tab with custom previews and image review/acceptance UI. Currently we render our custom UI to the main iframe after tests end, which isn’t ideal.

And being able to use custom (async) comparators - e.g., in our case we would like to compare not against baseline images stored in the repo but against our own image store. We also use a hashing mechanism to avoid downloading baseline images unless there is a failure, which can speed CI up considerably given that we have several thousand baselines.

[sheremet-va]

You should already be able to use async comparators: https://vitest.dev/config/browser/expect.html#browser-expect-tomatchscreenshot-comparators

[macarie]

in our case we would like to compare not against baseline images stored in the repo but against our own image store

This looks like an interesting use-case, @hcampos-professional, would you mind expanding on it?

As far as I understand, custom comparators wouldn't really help here. Currently we check if there's a reference screenshot, if there isn't one, the test will fail as it creates a new one — unless running with --update, but then it would just write it and the test will always succeed. Users can hook into (sync) path resolution, but not into the file-read process.

Would it help if Vitest had an option to customize reference-reading (and writing, I believe)? This would allow downloading references just in time for decoding then everything else would keep working as if the file was local.

[porada]

React Server Components

[alexander7161]

Would be great to have a locator picker tool in the browser mode UI

E.g like https://testing-playground.com/ where you can hover over elements to find the best locator to use

Playwright also has this feature: https://playwright.dev/docs/test-ui-mode#pick-locator

[madeofsun]

what are the bugs that you want us to prioritize

Would be nice to prioritize this #9395

We are not able to migrate to Vitest 4.0 before that

[AriPerkkio]

The PR for that seems a bit stale, I'll try to priotize that. So it's ok to land that only in Vitest 5?

[madeofsun]

As I understand, Vite 8 will released soon and will be supported in version 4.1, but not in 3.2.
Ideally, we would want to migrate to Vite 8 as soon as possible. But if it will not work with Vitest 3.2, we would be blocked to do so until Vitest 5 comes out.

[Smrtnyk]

debugging support for browser mode tests natively in jetbrains IDEs, like webstorm
being able to stop at the break point

[sheremet-va]

Vitest already provides primitives for that like --inspect, Jetbrains extension just needs to use it. We do not control its development, so I would advise creating an issue in their issue tracker instead.

[Smrtnyk]

Vitest already provides primitives for that like --inspect, Jetbrains extension just needs to use it. We do not control its development, so I would advise creating an issue in their issue tracker instead.

it already works out of the box for non browser mode tests so thought it was something on vitest side, will open the issue there then

[AriPerkkio]

I've tested Browser-mode debugging with IntelliJ in 2024 and it worked just fine. It requires configuring debugger options manually, so maybe they just need to provide better DX with built-in pre-configured debugger 🤷

intellij-demo.webm

[Smrtnyk]

I've tested Browser-mode debugging with IntelliJ in 2024 and it worked just fine. It requires configuring debugger options manually, so maybe they just need to provide better DX with built-in pre-configured debugger 🤷
intellij-demo.webm

ok tnx, I will open an issue on their issue tracker then

[lishaduck]

First off, I just want to say I'm super excited for --detect-async-leaks & #9476, and next to those these are all basically fingercuts.

What are the feature that you would like to see in the next major

• My # 1 feature would be support for other runtimes, which was already mentioned: Vitest 5 #9664 (reply in thread).
• Enable --typecheck by default? #7773 would be really nice as well, or Option to disable "Testing types with tsc and vue-tsc is an experimental feature." logs #9823 failing that.

what are the bugs that you want us to prioritize?

#9429 means Flint can't go fully no-build, which is a shame. Actually we rewrote that recently, we might have fixed it 🤔
Either way it is a bit annoying and might be worth taking a look at.

What are the main pain points in your testing workflow right now?

Right now, --changed doesn't seem to work well in monorepos; in Flint we found that it invalidates all dependent packages, so if we make an unrelated change to core all of our lint tests run as well; when typescript-eslint tried to make the equivalent change, they found the opposite (disclaimer: I'm unrelated to typescript-eslint, just was following that PR to see if they could figure out the issues). It'd be great to at minimum have some official guidance for how to set it up properly.

---

I think that's it, Vitest is amazing!

[sheremet-va]

Regarding --changed, we can at least log (maybe with DEBUG=) why some/all tests are included.

[sheremet-va]

As an additional note regarding --changed (a proposal, maybe): we have watchTriggerPatterns which work for the watch run, but we could probably also apply the same rules for a more granular control over which tests are affected in --changed.

[lishaduck]

I guess 3 new things have popped up since I wrote this:

• We did manage to get setupFiles to work without ssr.resolve.conditions. Still would be good to fix IMO but not breaking us.

• We realized typechecking literally wasn't running (it's experimental, duh!). I have a patch to get it working in monorepos (chore: get types tests running flint-fyi/flint#2427 / https://github.com/flint-fyi/flint/blob/223a99c80be4e0871154306654fb5ede37989c45/patches/vitest.patch), not sure if you'd be interested in it (EDIT: filed Support monorepos with project references in types testing #9865 & feat: support typescript build mode #9870).

• For --changed, we run integration tests inside Vitest (we spawn npx flint), so we'd really like a way to disable --changed from skipping those files; maybe an alwaysRun config or similar? Right now we're planning on

  Alternatively, we could filter down the projects, run e2e in one job without --changed and the rest in a different job with --changed.

  That's probably the recommended way to handle it, so short of a dedicated option, could we at minimum document that?

[nstepien]

I really wish all locators were exact: true by default, or at the very least I wish there was a setting to change the default.

I'm currently migrating some tests from @testing-library libs to vitest/browser, and it's a pain point I'm running into.

For example:

*.getByRole('option', { name: '...' })

will often locate 2+ elements, making me add exact: true:

*.getByRole('option', { name: '...', exact: true })

and I end up with a codebase with a lot of exact: true scattered around tests.

In this other repo I'm defaulting custom locators to exact: true, and it I only needed to add 3 exact: false.

It's not just a role's name though, this applied to all the other locators that have that option (getByLabelText, ...).

IMO, exact: true better ensures that the tests evolve alongside the codebase, and don't break when unrelated parts on a page change.

[sheremet-va]

I think we can add a global option to control this already, and see if it should be enabled by default in one of the majors.

[nstepien]

In the same vein, I wish toHaveTextContent() would have an option to match the whole textContent, without requiring a RegExp which is error-prone and a poor use-case for them.

[sheremet-va]

For toHaveTextContent I think it might be better to have a separate toMatchTextContent that checks content.includes(substring) and toHaveTextContent should always just be strict.

[sheremet-va]

The exact option should hopefully be addressed in the next fix release

[Lucas127128]

Speed improvement to be competitive with bun test.

[AriPerkkio]

Which areas would you like to see speed improvements on?

We are definitely not planning to remove default test isolation that causes the most speed difference. Once Bun gets this most basic test runner feature, you'll see similar perf impact (26x slowdown possibly) too.

Some useless benchmarks:

1. When isolation is disabled, Vitest is 28x times faster:

• https://github.com/AriPerkkio/bun-vitest-useless-benchmark

2. When isolation is enabled, Bun fails to run tests that require isolation:

• https://github.com/AriPerkkio/isolated-tests-jest-node-vitest

[Lucas127128]

I may switch my test runner to Vitest because of this test isolation issue. Thank you very much for reply.