Skip to content

fix!(coverage): remove .all, reworked include/exclude#7837

Merged
AriPerkkio merged 1 commit intovitest-dev:mainfrom
AriPerkkio:fix/coverage-include-exclude
Jun 18, 2025
Merged

fix!(coverage): remove .all, reworked include/exclude#7837
AriPerkkio merged 1 commit intovitest-dev:mainfrom
AriPerkkio:fix/coverage-include-exclude

Conversation

@AriPerkkio
Copy link
Member

@AriPerkkio AriPerkkio commented Apr 15, 2025

Description

Breaking changes:

  • coverage.all option is removed
  • coverage.extensions option is removed
  • By default Vitest will show coverage only for the files that were imported during test run
  • To see coverage for untested files, users need to configure coverage.include with a pattern.
  • As coverage.extensions is removed, it is recommended to define extensions in include pattern, e.g. include: ["src/**.{js,ts,tsx}"].

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@AriPerkkio AriPerkkio added breaking change feat: coverage Issues and PRs related to the coverage feature labels Apr 15, 2025
@AriPerkkio AriPerkkio added this to the 4.0.0 milestone Apr 15, 2025
@netlify
Copy link

netlify bot commented Apr 15, 2025

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit b243a72
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/685266447c8d340008f59c94
😎 Deploy Preview https://deploy-preview-7837--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@AriPerkkio AriPerkkio force-pushed the fix/coverage-include-exclude branch 7 times, most recently from 3c1d9e7 to 0ddd0a1 Compare April 18, 2025 07:55
@AriPerkkio AriPerkkio force-pushed the fix/coverage-include-exclude branch from 0ddd0a1 to 1d7055c Compare June 16, 2025 14:20
@AriPerkkio
Copy link
Member Author

/ecosystem-ci run

@vitest-ecosystem-ci
Copy link

vitest-ecosystem-ci bot commented Jun 16, 2025

📝 Ran ecosystem CI: Open

suite result
elk ✅ success
aria-live-capture ✅ success
zustand ✅ success
nuxt ✅ success
nuxt-test-utils ✅ success
sveltejs-cli ✅ success
vite ✅ success
vitest-sonar-reporter ✅ success
vitest-coverage-large ✅ success
effect ✅ success
webcontainer-test ✅ success
vue ✅ success
vitest-benchmark-large ✅ success
lerna-lite ✅ success
vitest-browser-examples ✅ success
vitest-in-webcontainer ✅ success
vitest-reporters-large ✅ success
vitest-vscode ❌ failure

@pkg-pr-new
Copy link

pkg-pr-new bot commented Jun 16, 2025

@vitest/browser

npm i https://pkg.pr.new/@vitest/browser@7837

@vitest/coverage-istanbul

npm i https://pkg.pr.new/@vitest/coverage-istanbul@7837

@vitest/coverage-v8

npm i https://pkg.pr.new/@vitest/coverage-v8@7837

@vitest/expect

npm i https://pkg.pr.new/@vitest/expect@7837

@vitest/mocker

npm i https://pkg.pr.new/@vitest/mocker@7837

@vitest/pretty-format

npm i https://pkg.pr.new/@vitest/pretty-format@7837

@vitest/runner

npm i https://pkg.pr.new/@vitest/runner@7837

@vitest/snapshot

npm i https://pkg.pr.new/@vitest/snapshot@7837

@vitest/spy

npm i https://pkg.pr.new/@vitest/spy@7837

@vitest/ui

npm i https://pkg.pr.new/@vitest/ui@7837

@vitest/utils

npm i https://pkg.pr.new/@vitest/utils@7837

vite-node

npm i https://pkg.pr.new/vite-node@7837

vitest

npm i https://pkg.pr.new/vitest@7837

@vitest/web-worker

npm i https://pkg.pr.new/@vitest/web-worker@7837

@vitest/ws-client

npm i https://pkg.pr.new/@vitest/ws-client@7837

commit: b243a72

@AriPerkkio AriPerkkio force-pushed the fix/coverage-include-exclude branch from 1d7055c to e1ac12d Compare June 17, 2025 06:31
@AriPerkkio AriPerkkio marked this pull request as ready for review June 17, 2025 06:49
Comment on lines +183 to +184
// Run again through picomatch as tinyglobby's exclude pattern is different ({ "exclude": ["math"] } should ignore "src/math.ts")
includedFiles = includedFiles.filter(file => this.isIncluded(file))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use picomatch when checking covered files, and tinyglobby when checking uncovered files from file system. To make coverage.include and coverage.exclude behave similarly, we need to run tinyglobby's results through picomatch again here. This feels like duplicate work.

Minimal repro below. @SuperchupuDev any suggestions how to make tinyglobby behave identical to picomatch here?

import { writeFileSync, rmSync, readdirSync, mkdirSync } from "node:fs";
import { resolve } from "node:path";
import { glob } from "tinyglobby";
import pm from "picomatch";

{
  // Setup
  rmSync("./src", { force: true, recursive: true });
  mkdirSync("./src");

  writeFileSync("./src/first.ts", "export const a = 1;", "utf8");
  writeFileSync("./src/second.ts", "export const a = 2;", "utf8");
}

// Include everything inside src, but exclude anything that contains word "second"
const options = {
  include: ["src"],
  exclude: ["second"],
};

// ['/<root>/repros/tinyglobby/src/first.ts', '/<root>/repros/tinyglobby/src/second.ts']
const files = readdirSync(options.include[0]).map((file) => resolve(options.include[0], file));

const picomatchResult = files.filter((file) =>
  pm.isMatch(file, options.include, {
    ignore: options.exclude,
    contains: true,
  })
);

const tinyglobbyResult = await glob(options.include, {
  ignore: options.exclude,
  absolute: true,
});

console.log({ picomatchResult, tinyglobbyResult });
// {
//   picomatchResult: [
//     '/<root>/repros/tinyglobby/src/first.ts'
//   ],
//
//   tinyglobbyResult: [
//     '/<root>/repros/tinyglobby/src/first.ts',
//     '/<root>/repros/tinyglobby/src/second.ts'
//   ]
// }

Copy link
Contributor

@SuperchupuDev SuperchupuDev Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi! i just woke up. will investigate in some hours. off the top of my head you might want to use expandDirectories: false as i believe vitest uses that everywhere else? if it doesn't match after that then it's likely a picomatch issue, as iirc behavior between picomatch.isMatch and picomatch(pattern)(file) can differ (or at least i've seen behavior differences at least once in the past). still have to test and confirm though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll merge this for Vitest v4 as is, but in the future it would be nice to have part working just by using tinyglobby, without needing picomatch.

Copy link
Contributor

@SuperchupuDev SuperchupuDev Jun 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AriPerkkio sorry for the wait. after testing locally it looks like the reason why your example gives different results between tinyglobby and picomatch is because you are using the contains picomatch option, which tinyglobby does not expose as an option and neither does fast-glob. i suppose you can achieve the same functionality by wrapping the patterns with **/*<pattern>*/**. or you could move off contains and its behavior altogether

sheremet-va
sheremet-va previously approved these changes Jun 18, 2025
@AriPerkkio AriPerkkio merged commit 41a111c into vitest-dev:main Jun 18, 2025
55 of 60 checks passed
@AriPerkkio AriPerkkio deleted the fix/coverage-include-exclude branch June 18, 2025 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change cr-tracked feat: coverage Issues and PRs related to the coverage feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simplify coverage.exclude patterns, remove coverage.all

3 participants