Skip to content

bug: Copy task runs in parallel to build tasks creating a race condition #5592

@mt3o

Description

@mt3o

Prerequisites

Stencil Version

4.13.0

Current Behavior

When building with stencil, when the copy task is issued here:
https://github.com/ionic-team/stencil/blob/main/src/compiler/output-targets/index.ts#L30

there is Promise.all calling all tasks, including copy tasks, and regular building tasks. When the build (compilation, regular targets) takes longer, copy task is executed first, trying to copy non-existing directories. It's a race condition between compilation and copying.

I have problems replicating the issue with a smaller repo, and obviously, I can't share my work project. I'm guessing the issue happens only in big projects, or with poor hardware, as it's a race it depends on the performance.

For me - the workaround is to either create empty directories first (risking incomplete build), or not use copy-task at all, and instead copy files in standalone CI/CD script.
On developer machine, with watch mode, the problem is nearly nonexistent, as there are some files in /dist all the time. If something doesn't work, devs just restart the build script and then it goes smoothly. Such behavior in CI/CD is difficult to achieve.

Expected Behavior

Please consider moving await outputCopy(config, compilerCtx, buildCtx), after the Promise.all and before the www target. This should solve the race problem.

System Info

System: node 18.16.1
    Platform: windows (10.0.19045)
   CPU Model: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz (16 cpus)
    Compiler: C:\foobar\node_modules\@stencil\core\compiler\stencil.js
       Build: 1710170895
     Stencil: 4.12.6
  TypeScript: 5.3.3
      Rollup: 2.56.3
      Parse5: 7.1.2
      jQuery: 4.0.0-pre
      Terser: 5.29.1

Steps to Reproduce

Set up a big repo with complex components,
Add lots of targets to be compiled
Set www target to be built
Have all targets be copied into www
run npx stencil build

Stencil runs all tasks as in Promise.all call, copy ends after build tasks are over, compilation breaks with error that some dirs don't exist in /dist.

Code Reproduction URL

no url :(

Additional Information

The log is as follows, it's that the clear copy task is ending before the build tasks are finished. So I'm concluding that moving the copy task out of Promise.all will solve the problem - because the copy task will be executed only after all build tasks are done.

[58:20.3]  create workers, maxWorkers: 7  MEM: 142.1MB
[58:20.3]  Starting compilation in watch mode...  MEM: 142.1MB
[58:24.3]  build, search, dev mode, started ...
[58:24.3]  start build, 2024-03-26T17:58:24  MEM: 453.9MB
[58:24.3]  cleaning 6 dirs ...  MEM: 453.9MB
[58:24.3]  cleaning dirs finished in 2 ms  MEM: 454.0MB
[58:24.4]  transpile started ...
[58:32.5]  Transpiled modules: [ ... ]

[58:32.5]  generated app types started ...  MEM: 744.0MB
[58:32.5]  generated app types finished: src/components.d.ts in 22 ms  MEM: 747.4MB
[58:32.5]  transpile finished in 8.18 s
[58:32.5]  generate outputs started ...  MEM: 747.6MB
[58:32.5]  getComponentAssetsCopyTasks: 8  MEM: 747.6MB
[58:32.5]  getComponentAssetsCopyTasks: 8  MEM: 747.6MB
[58:32.5]  copy started ...
[58:32.5]  generate hydrate app started ...
[58:32.5]  generate lazy + source maps started ...
[58:32.5]  generateEntryModules, 144 entryModules  MEM: 747.7MB
[58:36.4]  copy finished (862 files) in 3.84 s
[59:02.7]  generate hydrate app finished in 30.14 s
[59:06.9]  generate lazy + source maps finished in 34.35 s
[59:06.9]  generate www started ...  MEM: 2443.5MB
[59:06.9]  generateIndexHtml, write: www/index.html  MEM: 2443.5MB
[59:06.9]  generate www finished in 6 ms  MEM: 2443.5MB
[59:07.1]  updated readme docs: (...)
[59:07.1]  updated readme docs: (...)
[59:07.1]  updated readme docs: (...)
[59:07.1]  generate types started ...  MEM: 2451.7MB
[59:07.1]  generate types finished in 1 ms  MEM: 2451.7MB
[59:07.1]  generate outputs finished in 34.55 s  MEM: 2451.7MB
[59:07.1]  aborted build, 42750ms  MEM: 2451.8MB

[ ERROR ]  ENOENT: no such file or directory, stat 'C:\foobar\dist\lazy-loader'

[ ERROR ]  ENOENT: no such file or directory, stat 'C:\foobar\dist\custom-elements'

[ ERROR ]  ENOENT: no such file or directory, stat 'C:\foobar\dist\types'

[ ERROR ]  ENOENT: no such file or directory, stat 'C:\foobar\dist\search'

Metadata

Metadata

Assignees

Labels

Bug: ValidatedThis PR or Issue is verified to be a bug within Stencil

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions