feat(jest-worker): add gnumake jobclient to limit number of workers#12968
feat(jest-worker): add gnumake jobclient to limit number of workers#12968milahu wants to merge 1 commit intojestjs:mainfrom
Conversation
|
Hi @milahu! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks! |
dc45dc8 to
7eaa4be
Compare
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
859f57c to
3f1400c
Compare
3f1400c to
51a3857
Compare
|
This PR is stale because it has been open 1 year with no activity. Remove stale label or comment or this will be closed in 30 days. |
|
This PR was closed because it has been stalled for 30 days with no activity. Please open a new PR if the issue is still relevant, linking to this one. |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Summary
limit jobs when jest-worker is running under make / cmake / ninja
Details
use case: chromium or qtwebengine
call stack: devtools-frontend -> rollup -> terser -> jest-worker
by default, jest-worker spawns $nproc workers (100 workers on a 100 core machine)
even if jobs are limited with
make -j10 -l10this overload causes trouble on build servers -> downstream issue NixOS/nixpkgs#178171
based on my gnumake-tokenpool
needs some polish -> draft PR
Versions
Test plan
manual test in
packages/jest-worker/tests/jobserver/test.shtodo: integrate with automatic tests
the test requires gnumake to run
synthetic test is passing
i have integrated this change into chromium, now the job count is limited by ninja
API change
i had to change some internal APIs between Farm and WorkerPool (bump major version?)
i hope noone depends on the old APIs ...
if needed, this can be made backward compatible
debug prints
for production builds, the debug prints could be removed by typescript