Subject of the feature

Automate canary testing changes with downstream packages

Prior art: https://github.com/nodejs/citgm

Problem

Unified is a large and highly connected community of packages.
Changes to any package could have impact downstream packages.
Currently anticipating impact is a purely manual process.

It could be helpful to use downstream packages, which themselves have comprehensive test suites, to help automatically detect what changes may be breaking or have unintended impacts.

Expected behaviour

The flow could work something like:

1. add a dependency on the new downstream testing tool
2. a package adds a json config file with:
   • a description of the build process for the package
   • a listing of trusted downstream packages with unit tests
3. add an npm test script that calls the downstream testing package

From then on, for each run of the continuous integration process the downstream testing tool would

1. build the package using the instructions from the configuration
2. download each downstream package listed in the configuration
3. for each package downloaded
   1. setup that package (npm install)
   2. link the build of the package being updated, into the downstream package's dependencies
   3. run the test suite of the downstream package
   4. report results

Alternatives

• manual testing (which is needed anyway)
• Find an existing tool like https://github.com/nodejs/citgm but for node modules