Skip to content

coverage not collected from sub-propcesses result from child_process APIs #7064

@osher

Description

@osher

Describe the bug

Consider a package which is a CLI tool.
Consider a test suite that does the testing-trophy: it runs e2e with few happy paths, generates coverage, and then write unit-tests to cover only the parts that are not covered by the e2e and component tests.
Now consider that the e2e test cases run the CLI using any of the APIs of child_process: exec, spawn, fork, or their sync equivalents.

The tool fails to collect coverage from the e2e suites.
The reproduction does only exec, because it is minimal. however, tried all of them.

I'd be delighted if it's just some setting that I'm failing to pass...

reproduction

scenario

  1. given a project (full structure below) that uses child_process to run the bin of the package in e2e test of CLI tools, and unit tests to cover code-paths that do not appear in the e2e coverage

  2. Run vitest run --coverage

expected

100% coverage. for all files.

Found

All tests run successfully, but only coverage collected using unit-tests is observed.

 RUN  v2.1.8 /home/user/ws/opensource/reproduction/vitest/child_process-not-covered
      Coverage enabled with v8

 ✓ test/calc.unit.mjs (1)
   ✓ calc (1)
     ✓ sub (1)
       ✓ should substract positive integers
 ✓ test/cli.e2e.mjs (2)
   ✓ fxgen command (2)
     ✓ when called with valid parameters (2)
       ✓ should not fail
       ✓ should return the right answer

 Test Files  2 passed (2)
      Tests  3 passed (3)
   Start at  17:13:40
   Duration  291ms (transform 18ms, setup 0ms, collect 16ms, tests 32ms, environment 1ms, prepare 128ms)

 % Coverage report from v8
-----------|---------|----------|---------|---------|-------------------
File       | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------|---------|----------|---------|---------|-------------------
All files  |   14.28 |    33.33 |      25 |   14.28 |
 bin       |       0 |        0 |       0 |       0 |
  cli.mjs  |       0 |        0 |       0 |       0 | 1-3
 lib       |   18.18 |       50 |   33.33 |   18.18 |
  calc.mjs |     100 |      100 |      50 |     100 |
  cli.mjs  |       0 |        0 |       0 |       0 | 1-9
-----------|---------|----------|---------|---------|-------------------

Project structure:

image

vitest.config.js

import { defineConfig } from 'vitest/config';
export default defineConfig({
  test: {
    globals: true,
    reporters: ['verbose'],
    include: [
      'test/**/*.unit.mjs',
      'test/**/*.e2e.mjs',
    ],
  },
});

package.json

{
  "name": "child_process-not-covered",
  "version": "1.0.0",
  "bin": {
    "calc": "bin/calc.mjs"
  },
  "scripts": {
    "test": "vitest run --coverage"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": "",
  "devDependencies": {
    "@vitest/coverage-v8": "^2.1.8",
    "vitest": "^2.1.8"
  }
}

bin/cli

#!/usr/bin/env node
import cli from '../lib/cli.mjs';
cli({ process, console });

lib/cli

import * as calc from './calc.mjs';
export default ({ process, console }) => {
  const { argv: [ ,, op, a, b ] } = process;
  const result = calc[op](Number(a), Number(b));
  console.log(result);
}

lib/calc

export const add = (a, b) => a + b;
export const sub = (a, b) => a - b;

test/calc.unit.mjs

import * as calc from '../lib/calc.mjs';

describe('calc', () => {
  describe('sub', () => {
    it('should substract positive integers', () => {
      expect(calc.sub(44,2)).toEqual(42);
    });
  });
});

test/cli.e2e.mjs

import { execSync } from 'node:child_process';

describe('fxgen command', () => {
  describe('when called with two integers', () => {
    const ctx = {};
    beforeAll(() => {
      try { 
        ctx.result = execSync('node bin/cli.mjs add 20 22').toString().trim();
      } catch(err) {
        ctx.err = err;
      }
    });

    it('should not fail', () => expect(ctx.err).toBeFalsy());
    it('should return their sum', () => expect(ctx.result).toEqual('42'));
  });
});

System Info

$ vitest --version
vitest/2.1.8 linux-x64 node-v20.18.0

Used Package Manager

npm

Validations

disclaimer

Feels like de j'avoux.
I encountered this before while trying to migrating a moch+nyc project, and that effort turned away from vitest.
If I opened such a ticket before - excuse me, for some reason I cannot find it :(
I'm now working with a new customer on a new codebase - new chance :)

Anyway, now I really put the effort to reproduce the but in isolation :)

Metadata

Metadata

Assignees

Labels

feat: coverageIssues and PRs related to the coverage featurep2-nice-to-haveNot breaking anything but nice to have (priority)

Projects

Status

Approved

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions