-
-
Notifications
You must be signed in to change notification settings - Fork 835
bug: Copy task runs in parallel to build tasks creating a race condition #5592
Description
Prerequisites
- I have read the Contributing Guidelines.
- I agree to follow the Code of Conduct.
- I have searched for existing issues that already report this problem, without success.
Stencil Version
4.13.0
Current Behavior
When building with stencil, when the copy task is issued here:
https://github.com/ionic-team/stencil/blob/main/src/compiler/output-targets/index.ts#L30
there is Promise.all calling all tasks, including copy tasks, and regular building tasks. When the build (compilation, regular targets) takes longer, copy task is executed first, trying to copy non-existing directories. It's a race condition between compilation and copying.
I have problems replicating the issue with a smaller repo, and obviously, I can't share my work project. I'm guessing the issue happens only in big projects, or with poor hardware, as it's a race it depends on the performance.
For me - the workaround is to either create empty directories first (risking incomplete build), or not use copy-task at all, and instead copy files in standalone CI/CD script.
On developer machine, with watch mode, the problem is nearly nonexistent, as there are some files in /dist all the time. If something doesn't work, devs just restart the build script and then it goes smoothly. Such behavior in CI/CD is difficult to achieve.
Expected Behavior
Please consider moving await outputCopy(config, compilerCtx, buildCtx), after the Promise.all and before the www target. This should solve the race problem.
System Info
System: node 18.16.1
Platform: windows (10.0.19045)
CPU Model: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz (16 cpus)
Compiler: C:\foobar\node_modules\@stencil\core\compiler\stencil.js
Build: 1710170895
Stencil: 4.12.6
TypeScript: 5.3.3
Rollup: 2.56.3
Parse5: 7.1.2
jQuery: 4.0.0-pre
Terser: 5.29.1Steps to Reproduce
Set up a big repo with complex components,
Add lots of targets to be compiled
Set www target to be built
Have all targets be copied into www
run npx stencil build
Stencil runs all tasks as in Promise.all call, copy ends after build tasks are over, compilation breaks with error that some dirs don't exist in /dist.
Code Reproduction URL
no url :(
Additional Information
The log is as follows, it's that the clear copy task is ending before the build tasks are finished. So I'm concluding that moving the copy task out of Promise.all will solve the problem - because the copy task will be executed only after all build tasks are done.
[58:20.3] create workers, maxWorkers: 7 MEM: 142.1MB
[58:20.3] Starting compilation in watch mode... MEM: 142.1MB
[58:24.3] build, search, dev mode, started ...
[58:24.3] start build, 2024-03-26T17:58:24 MEM: 453.9MB
[58:24.3] cleaning 6 dirs ... MEM: 453.9MB
[58:24.3] cleaning dirs finished in 2 ms MEM: 454.0MB
[58:24.4] transpile started ...
[58:32.5] Transpiled modules: [ ... ]
[58:32.5] generated app types started ... MEM: 744.0MB
[58:32.5] generated app types finished: src/components.d.ts in 22 ms MEM: 747.4MB
[58:32.5] transpile finished in 8.18 s
[58:32.5] generate outputs started ... MEM: 747.6MB
[58:32.5] getComponentAssetsCopyTasks: 8 MEM: 747.6MB
[58:32.5] getComponentAssetsCopyTasks: 8 MEM: 747.6MB
[58:32.5] copy started ...
[58:32.5] generate hydrate app started ...
[58:32.5] generate lazy + source maps started ...
[58:32.5] generateEntryModules, 144 entryModules MEM: 747.7MB
[58:36.4] copy finished (862 files) in 3.84 s
[59:02.7] generate hydrate app finished in 30.14 s
[59:06.9] generate lazy + source maps finished in 34.35 s
[59:06.9] generate www started ... MEM: 2443.5MB
[59:06.9] generateIndexHtml, write: www/index.html MEM: 2443.5MB
[59:06.9] generate www finished in 6 ms MEM: 2443.5MB
[59:07.1] updated readme docs: (...)
[59:07.1] updated readme docs: (...)
[59:07.1] updated readme docs: (...)
[59:07.1] generate types started ... MEM: 2451.7MB
[59:07.1] generate types finished in 1 ms MEM: 2451.7MB
[59:07.1] generate outputs finished in 34.55 s MEM: 2451.7MB
[59:07.1] aborted build, 42750ms MEM: 2451.8MB
[ ERROR ] ENOENT: no such file or directory, stat 'C:\foobar\dist\lazy-loader'
[ ERROR ] ENOENT: no such file or directory, stat 'C:\foobar\dist\custom-elements'
[ ERROR ] ENOENT: no such file or directory, stat 'C:\foobar\dist\types'
[ ERROR ] ENOENT: no such file or directory, stat 'C:\foobar\dist\search'