Background

we've been working on getting it to the point where it's elegant and simple to instrument scripts executed with Node.js with v8's native test-coverage feature.

It's my hope that this could represent a future direction for the Istanbul project (the tool that's currently most commonly used for facilitating test coverage for Node.js). Istanbul currently uses babel for instrumenting source-files; I believe it will make it easier to keep up with the evolving JavaScript language if we move to using the v8 engine itself.

I've created this issue to track the progress on this project, as it relates to Node.js.

Work So Far

• we need a way to collect coverage regardless of how a process exits:
  • we are now able to do this using the event Runtime.executionContextDestroyed (see Proposal: provide a straight-forward approach for collecting coverage via Chrome's Profiler #16531).
• we need to be able to collect coverage for binary and conditional operations (users have come to expect this):
  • we still have some work on the go, but we're now instrumenting most of what folks would expect (https://chromium-review.googlesource.com/c/v8/v8/+/754564/1, https://chromium-review.googlesource.com/c/v8/v8/+/568307).

Current Blockers

• the approach outlined in Proposal: provide a straight-forward approach for collecting coverage via Chrome's Profiler #16531 does not appear to work for .mjs files (see bug: Runtime.executionContextDestroyed event does not fire for .mjs files #17336 --inspect-brk does not work with ESM #17340).
• we need to take into account origin line and origin column when calculating test coverage offsets (see proposal: standardize header size prepended before executing scripts in v8 #17256, JS source should not be wrapped. #17396).
  • there's currently work in progress on this (https://bugs.chromium.org/p/v8/issues/detail?id=7119, https://chromium-review.googlesource.com/c/v8/v8/+/789373).
• we need a way to run the inspector in silent mode.
  • the inspector currently spits out quite a bit of text to the terminal when running, this is bad for tests that rely on terminal output, e.g., tests outputting tap format.
  • I'd be happy to take on this work, @addaleax, @bmeck is there a precedent for how I might toggle this feature? perhaps an environment variable or flag?
• when booting a subprocess with the inspector enabled, how do we know when it's actually bound to its port?
  • I've been using a library that polls on the port, but this feels like a hack; any other suggestions, thoughts? tldr; I would like to be able to spawn a subprocess in such a way that I know when it's safe to connect on the inspector ws://.
• spawn-wrap will need modifications to support the new inspector dance (spawn a subprocess with --inspect-brk, connect to the port, etc.).
  • @isaacs I know you have some thoughts around this.

I've been updating and maintaining the module c8 as we work on this feature (if you feel like playing, it's already partially functional).

👋 thanks a ton for everyone's help so far (as you may have noticed, I'm really excited about this project).

CC: @schuay, @bmeck, @hashseed, @ak239, @mikeal, @isaacs