fix: extraneous coverage for --project (fix #6331) by gtbuchanan · Pull Request #7885 · vitest-dev/vitest

gtbuchanan · 2025-04-24T14:33:17Z

Description

Coverage was reported for projects outside of the specified project filter. For example, given a monorepo with two projects, p1 and p2, vitest run --coverage --project=p1 reported coverage for code from p2 at 0% because no tests from p2 were run.

Resolves: #6331

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
Ideally, include a test that fails without this PR but passes with it.
Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

Run the tests with pnpm test:ci.

Documentation

If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

Coverage was reported for projects outside of the specified project filter. For example, given a monorepo with two projects, p1 and p2, `vitest run --coverage --project=p1` reported coverage for code from p2 at 0% because no tests from p2 were run.

gtbuchanan · 2025-04-24T14:38:48Z

@AriPerkkio Does this align with what you were thinking? I still need to familiarize myself with how the tests are written, but I wanted to ensure I was on the right track first.

Also, should I move the duplicate code to a common location (like BaseCoverageProvider)?

AriPerkkio

This change would only filter out uncovered files outside the workspace project's root. We should filter out covered files too.

To do this, we'll need to make this.testExclude an array, and create one for each project with their roots.

vitest/packages/coverage-istanbul/src/provider.ts

Lines 34 to 35 in 9be32fb

    
           this.testExclude = new TestExclude({ 
        
             cwd: ctx.config.root,

netlify · 2025-04-25T22:18:51Z

✅ Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`11107dc`
🔍 Latest deploy log	https://app.netlify.com/projects/vitest-dev/deploys/685c29ca9f575b0008d296bc
😎 Deploy Preview	https://deploy-preview-7885--vitest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

gtbuchanan · 2025-04-25T22:21:33Z

That should do it. It seems to work for my project. I will add tests when I figure that out 🙂

As an aside, I wasn't able to move the duplicate code to BaseCoverageProvider because vitest/node doesn't reference test-exclude. I figured it wasn't worth adding the extra reference for this fix.

AriPerkkio

Looks good so far.

As an aside, I wasn't able to move the duplicate code to BaseCoverageProvider because vitest/node doesn't reference test-exclude. I figured it wasn't worth adding the extra reference for this fix.

That's okey for now. In #7837 we'll remove test-exclude completely and utilize packages that are available in vitest package. That allows re-using logic from BaseProvider.

For test cases we'll need to verify following:

When --project filter is passed, files outside that project are excluded. This means covered and uncovered files.
When --project filter is not passed, files in projects and in the root should be visible.

You can add fixtures in test/coverage-test/fixtures/workspaces and create test cases in new file test/coverage-test/test/workspace.test.ts for example.

packages/coverage-istanbul/src/provider.ts

AriPerkkio · 2025-06-22T08:23:53Z

Hey @gtbuchanan, do you have any time to finalize this PR? I'm also happy to take over and finish it if you prefer that. Thanks for all the work so far! 💯

gtbuchanan · 2025-06-23T14:28:19Z

@AriPerkkio I would have liked to finish it myself, but I'm running into some issues:

I figured out how to write the tests, but the one testing my fix is actually failing. I'm trying to debug the coverage test in VS Code using these steps, but clicking "Run and Debug" immediately stops debugging. Do you have any tips? Is there a specific file I'm supposed to have open?
Obviously, my branch is now outdated. Out of curiosity, I added my unit tests on top of main to make sure one would fail, but they actually passed. I'm assuming my test is just wrong since main doesn't appear to have a fix for this. Will you glance at my tests to see if something stands out?

EDIT:
Disregard 2. There must have been some caching involved. I picked the PR back up on my work computer and the test fails there as expected. Now to reapply the fix...

This reverts commit 5716d0f.

gtbuchanan · 2025-06-23T16:29:42Z

I didn't figure out how to debug the coverage tests in VS Code, but I was able to get everything working! It seems to work without updating isIncluded, but let me know if I'm missing something.

packages/vitest/src/node/coverage.ts

AriPerkkio

There must have been some caching involved. I picked the PR back up on my work computer and the test fails there as expected. Now to reapply the fix...

The test cases are running against built assets so you'll need to run pnpm build or pnpm dev before running the test cases. That might explain the caching issue.

I didn't figure out how to debug the coverage tests in VS Code

These test cases are running "Vitest inside Vitest" so it's likely that VS Code extension is unable to inspect those.

test/coverage-test/test/workspace.test.ts

test/coverage-test/fixtures/configs/vitest.config.workspace.ts

AriPerkkio

Small improvements, otherwise this is looking good!

Btw as this PR is from a Github organization account (energyworldnet), Vitest maintainers cannot push any changes to your PR branch. Usually I would fix small errors like this myself and push changes to your branch directly, but now it's not possible due to permission requirements.

packages/vitest/src/node/coverage.ts

AriPerkkio

Looks good, thanks for the PR @gtbuchanan! 💯

AriPerkkio reviewed Apr 25, 2025

View reviewed changes

Filter out covered files

a02bd30

AriPerkkio requested changes Apr 29, 2025

View reviewed changes

packages/coverage-istanbul/src/provider.ts Outdated Show resolved Hide resolved

Inline createTestExcludes function

f79d78f

Add workspace tests

b6c42ff

gtbuchanan added 5 commits June 23, 2025 10:04

Revert "fix: extraneous coverage for --project (fix #6331)"

a1fc457

This reverts commit 5716d0f.

Merge branch 'main' into fix/project-filter-coverage

f032d1f

Migrate workspace test config to new style

a12ca30

Fix workspace coverage include

38b255d

Reapply fix

098a9a4

gtbuchanan marked this pull request as ready for review June 23, 2025 16:25

gtbuchanan commented Jun 23, 2025

View reviewed changes

packages/vitest/src/node/coverage.ts Show resolved Hide resolved

Localize project roots definition

377a643

AriPerkkio requested changes Jun 24, 2025

View reviewed changes

test/coverage-test/test/workspace.test.ts Show resolved Hide resolved

test/coverage-test/fixtures/configs/vitest.config.workspace.ts Show resolved Hide resolved

gtbuchanan added 3 commits June 24, 2025 07:42

Rename workspace test file

0073c2d

Add test case for shared ref

13cb63a

Fix included external covered files

ecb7978

gtbuchanan requested a review from AriPerkkio June 24, 2025 17:58

AriPerkkio requested changes Jun 25, 2025

View reviewed changes

packages/vitest/src/node/coverage.ts Show resolved Hide resolved

packages/vitest/src/node/coverage.ts Outdated Show resolved Hide resolved

packages/vitest/src/node/coverage.ts Outdated Show resolved Hide resolved

gtbuchanan added 4 commits June 25, 2025 11:30

Add clarifying comment about roots

bb4f9fb

Improve readability of included condition

b6458af

Dedup project roots

aad1474

Merge branch 'main' into fix/project-filter-coverage

11107dc

AriPerkkio approved these changes Jun 26, 2025

View reviewed changes

AriPerkkio merged commit 761beee into vitest-dev:main Jun 26, 2025
25 of 26 checks passed

AriPerkkio mentioned this pull request Jun 29, 2025

chore: add reminder about "Allow edits" in PR template #8233

Merged

7 tasks

This was referenced Nov 3, 2025

build(deps-dev): Bump @vitest/ui from 3.2.4 to 4.0.6 blinklabs-io/vpn-frontend#143

Closed

build(deps-dev): Bump @vitest/coverage-v8 from 3.2.4 to 4.0.6 blinklabs-io/vpn-frontend#145

Closed

AriPerkkio mentioned this pull request Feb 17, 2026

fix(coverage): check path boundaries in prefix matching #9682

Closed

7 tasks

Janpot mentioned this pull request Feb 27, 2026

[code-infra] Remove --coverage mui/mui-x#21410

Merged

Uh oh!

Conversation

gtbuchanan commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

Tests

Documentation

Changesets

Uh oh!

gtbuchanan commented Apr 24, 2025

Uh oh!

AriPerkkio left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

netlify bot commented Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for vitest-dev ready!

Uh oh!

gtbuchanan commented Apr 25, 2025

Uh oh!

AriPerkkio left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AriPerkkio commented Jun 22, 2025

Uh oh!

gtbuchanan commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gtbuchanan commented Jun 23, 2025

Uh oh!

Uh oh!

AriPerkkio left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

AriPerkkio left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AriPerkkio left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gtbuchanan commented Apr 24, 2025 •

edited

Loading

AriPerkkio left a comment •

edited

Loading

netlify bot commented Apr 25, 2025 •

edited

Loading

AriPerkkio left a comment •

edited

Loading

gtbuchanan commented Jun 23, 2025 •

edited

Loading

AriPerkkio left a comment •

edited

Loading

AriPerkkio left a comment •

edited

Loading