vitest/docs/guide/improving-performance.md

188 lines
6.0 KiB
Markdown

# Improving Performance
## Test isolation
By default Vitest runs every test file in an isolated environment based on the [pool](/config/#pool):
- `threads` pool runs every test file in a separate [`Worker`](https://nodejs.org/api/worker_threads.html#class-worker)
- `forks` pool runs every test file in a separate [forked child process](https://nodejs.org/api/child_process.html#child_processforkmodulepath-args-options)
- `vmThreads` pool runs every test file in a separate [VM context](https://nodejs.org/api/vm.html#vmcreatecontextcontextobject-options), but it uses workers for parallelism
This greatly increases test times, which might not be desirable for projects that don't rely on side effects and properly cleanup their state (which is usually true for projects with `node` environment). In this case disabling isolation will improve the speed of your tests. To do that, you can provide `--no-isolate` flag to the CLI or set [`test.isolate`](/config/#isolate) property in the config to `false`. If you are using several pools at once with `poolMatchGlobs`, you can also disable isolation for a specific pool you are using.
::: code-group
```bash [CLI]
vitest --no-isolate
```
```ts [vitest.config.js]
import { defineConfig } from 'vitest/config'
export default defineConfig({
test: {
isolate: false,
// you can also disable isolation only for specific pools
poolOptions: {
forks: {
isolate: false,
},
},
},
})
```
:::
:::tip
If you are using `vmThreads` pool, you cannot disable isolation. Use `threads` pool instead to improve your tests performance.
:::
For some projects, it might also be desirable to disable parallelism to improve startup time. To do that, provide `--no-file-parallelism` flag to the CLI or set [`test.fileParallelism`](/config/#fileparallelism) property in the config to `false`.
::: code-group
```bash [CLI]
vitest --no-file-parallelism
```
```ts [vitest.config.js]
import { defineConfig } from 'vitest/config'
export default defineConfig({
test: {
fileParallelism: false,
},
})
```
:::
## Pool
By default Vitest runs tests in `pool: 'forks'`. While `'forks'` pool is better for compatibility issues ([hanging process](/guide/common-errors.html#failed-to-terminate-worker) and [segfaults](/guide/common-errors.html#segfaults-and-native-code-errors)), it may be slightly slower than `pool: 'threads'` in larger projects.
You can try to improve test run time by switching `pool` option in configuration:
::: code-group
```bash [CLI]
vitest --pool=threads
```
```ts [vitest.config.js]
import { defineConfig } from 'vitest/config'
export default defineConfig({
test: {
pool: 'threads',
},
})
```
:::
## Sharding
Test sharding means running a small subset of test cases at a time. It can be useful when you have multiple machines that could be used to run tests simultaneously.
To split Vitest tests on multiple different runs, use [`--shard`](/guide/cli#shard) option with [`--reporter=blob`](/guide/reporters#blob-reporter) option:
```sh
vitest run --reporter=blob --shard=1/3 # 1st machine
vitest run --reporter=blob --shard=2/3 # 2nd machine
vitest run --reporter=blob --shard=3/3 # 3rd machine
```
Collect the results stored in `.vitest-reports` directory from each machine and merge them with [`--merge-reports`](/guide/cli#merge-reports) option:
```sh
vitest --merge-reports
```
<details>
<summary>Github action example</summary>
This setup is also used at https://github.com/vitest-tests/test-sharding.
```yaml
# Inspired from https://playwright.dev/docs/test-sharding
name: Tests
on:
push:
branches:
- main
jobs:
tests:
runs-on: ubuntu-latest
strategy:
matrix:
shardIndex: [1, 2, 3, 4]
shardTotal: [4]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Install pnpm
uses: pnpm/action-setup@v4
- name: Install dependencies
run: pnpm i
- name: Run tests
run: pnpm run test --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
- name: Upload blob report to GitHub Actions Artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: blob-report-${{ matrix.shardIndex }}
path: .vitest-reports/*
retention-days: 1
merge-reports:
if: ${{ !cancelled() }}
needs: [tests]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Install pnpm
uses: pnpm/action-setup@v4
- name: Install dependencies
run: pnpm i
- name: Download blob reports from GitHub Actions Artifacts
uses: actions/download-artifact@v4
with:
path: .vitest-reports
pattern: blob-report-*
merge-multiple: true
- name: Merge reports
run: npx vitest --merge-reports
```
</details>
:::tip
Test sharding can also become useful on high CPU-count machines.
Vitest will run only a single Vite server in its main thread. Rest of the threads are used to run test files.
In a high CPU-count machine the main thread can become a bottleneck as it cannot handle all the requests coming from the threads. For example in 32 CPU machine the main thread is responsible to handle load coming from 31 test threads.
To reduce the load from main thread's Vite server you can use test sharding. The load can be balanced on multiple Vite server.
```sh
# Example for splitting tests on 32 CPU to 4 shards.
# As each process needs 1 main thread, there's 7 threads for test runners (1+7)*4 = 32
# Use VITEST_MAX_THREADS or VITEST_MAX_FORKS depending on the pool:
VITEST_MAX_THREADS=7 vitest run --reporter=blob --shard=1/4 & \
VITEST_MAX_THREADS=7 vitest run --reporter=blob --shard=2/4 & \
VITEST_MAX_THREADS=7 vitest run --reporter=blob --shard=3/4 & \
VITEST_MAX_THREADS=7 vitest run --reporter=blob --shard=4/4 & \
wait # https://man7.org/linux/man-pages/man2/waitpid.2.html
vitest --merge-reports
```
:::