Provider v8 does not show test coverage #3555
Description
Describe the bug
The @vitest/coverage-v8 and @vitest/coverage-c8 packages are expected to provide identical coverage and test results. But that doesn't happen.
Package @vitest/coverage-v8 does not work correctly in terms of coverage for sure. Package @vitest/coverage-c8 is more accurate - sees all uncovered and skipped areas in Vue templates, takes into account missing test files and overall coverage.
Other things being equal settings, package @vitest/coverage-v8 simply always passes the tests and never outputs missing lines or coverage problems.
Reproduction
A simplified example is in the repository.
Install dependencies: yarn
Running tests: yarn unit
We see a report:
RUN v0.32.0 /home/user/projects/vitest-reproduction/tests
Coverage enabled with v8
✓ unit/UI/Icon.spec.js (11)
✓ unit/UI/Img.spec.js (5)
✓ unit/layouts/BottomNavigation.spec.js (3)
✓ unit/UI/Badge.spec.js (7)
✓ unit/views/AnyPage.spec.js (4)
✓ unit/layouts/TopBar.spec.js (1)
✓ unit/router/routes.spec.js (9)
✓ unit/App.spec.js (1)
✓ unit/layouts/MainLayout.spec.js (1)
✓ unit/views/InitError.spec.js (1)
✓ unit/stores.spec.js (2)
✓ unit/UI/Loader.spec.js (1)
✓ unit/services/helpers.spec.js (1)
↓ unit/main.spec.js (2) [skipped]
Test Files 13 passed | 1 skipped (14)
Tests 47 passed | 2 skipped (49)
Start at 16:01:38
Duration 4.23s (transform 170ms, setup 380ms, collect 2.21s, tests 695ms, environment 2.54s, prepare 1.46s)
% Coverage report from v8
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 0 | 0 | 0 | 0 |
----------|---------|----------|---------|---------|-------------------
Done in 5.24s.We go into the config and change the coverage provider from v8 to c8. Run the tests again and see the following report:
RUN v0.32.0 /home/user/projects/vitest-reproduction/tests
Coverage enabled with c8
✓ unit/UI/Badge.spec.js (7)
✓ unit/UI/Icon.spec.js (11)
✓ unit/UI/Img.spec.js (5)
✓ unit/router/routes.spec.js (9)
✓ unit/App.spec.js (1)
✓ unit/layouts/BottomNavigation.spec.js (3)
✓ unit/layouts/TopBar.spec.js (1)
✓ unit/views/AnyPage.spec.js (4)
✓ unit/layouts/MainLayout.spec.js (1)
✓ unit/UI/Loader.spec.js (1)
✓ unit/stores.spec.js (2)
✓ unit/views/InitError.spec.js (1)
✓ unit/services/helpers.spec.js (1)
↓ unit/main.spec.js (2) [skipped]
Test Files 13 passed | 1 skipped (14)
Tests 47 passed | 2 skipped (49)
Start at 16:02:56
Duration 4.62s (transform 163ms, setup 325ms, collect 2.32s, tests 975ms, environment 2.45s, prepare 1.69s)
% Coverage report from c8
-----------------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------------|---------|----------|---------|---------|-------------------
All files | 96.37 | 96.42 | 97.67 | 96.37 |
src | 75.23 | 84.61 | 87.5 | 75.23 |
App.vue | 100 | 83.33 | 100 | 100 | 2
main.ts | 0 | 0 | 0 | 0 | 1-26
stores.ts | 100 | 100 | 100 | 100 |
src/UI | 100 | 100 | 100 | 100 |
Badge.vue | 100 | 100 | 100 | 100 |
Icon.vue | 100 | 100 | 100 | 100 |
Img.vue | 100 | 100 | 100 | 100 |
Loader.vue | 100 | 100 | 100 | 100 |
src/layouts | 100 | 92.85 | 100 | 100 |
BottomNavigation.vue | 100 | 100 | 100 | 100 |
MainLayout.vue | 100 | 100 | 100 | 100 |
TopBar.vue | 100 | 75 | 100 | 100 | 46
src/router | 100 | 100 | 100 | 100 |
index.ts | 100 | 100 | 100 | 100 |
routes.ts | 100 | 100 | 100 | 100 |
src/services | 100 | 100 | 100 | 100 |
helpers.js | 100 | 100 | 100 | 100 |
src/views | 100 | 100 | 100 | 100 |
AnyPage.vue | 100 | 100 | 100 | 100 |
InitError.vue | 100 | 100 | 100 | 100 |
-----------------------|---------|----------|---------|---------|-------------------
ERROR: Coverage for lines (96.37%) does not meet global threshold (100%)
ERROR: Coverage for functions (97.67%) does not meet global threshold (100%)
ERROR: Coverage for branches (96.42%) does not meet global threshold (100%)
ERROR: Coverage for statements (96.37%) does not meet global threshold (100%)
error Command failed with exit code 1.Conclusion: at the moment v8 does not show coverage in principle
System Info
System:
OS: Linux 6.3 Manjaro Linux
CPU: (4) x64 Intel(R) Core(TM) i3-8100T CPU @ 3.10GHz
Memory: 3.41 GB / 11.04 GB
Container: Yes
Shell: 5.1.16 - /bin/bash
Binaries:
Node: 18.15.0 - /usr/bin/node
Yarn: 1.22.19 - ~/.node_modules_global/bin/yarn
npm: 9.6.6 - ~/.node_modules_global/bin/npm
npmPackages:
@vitejs/plugin-vue: ^4.2.3 => 4.2.3
@vitest/coverage-c8: ^0.32.0 => 0.32.0
@vitest/coverage-v8: ^0.32.0 => 0.32.0
vite: ^4.3.9 => 4.3.9
vitest: ^0.32.0 => 0.32.0Used Package Manager
yarn
Validations
- Follow our Code of Conduct
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
- The provided reproduction is a minimal reproducible example of the bug.