Skip to content

perf: parallelize independent async work#1071

Merged
fi3ework merged 2 commits intomainfrom
perf/parallelize-independent-async-work
Mar 19, 2026
Merged

perf: parallelize independent async work#1071
fi3ework merged 2 commits intomainfrom
perf/parallelize-independent-async-work

Conversation

@fi3ework
Copy link
Copy Markdown
Member

@fi3ework fi3ework commented Mar 18, 2026

Benchmark

Micro-benchmark comparing sequential await vs Promise.all for the two parallelization patterns in this PR.

This is a synthetic benchmark to demonstrate the theoretical benefit of the patterns, not an end-to-end measurement. In current rstest workloads, the parallelized operations use Rspack's in-memory filesystem (delay ≈ 0), so the improvement is negligible. However, as project count, test file count, or I/O latency increases (e.g., real disk I/O via writeToDisk, network FS, or complex config resolution), the benefit scales accordingly.

Pattern A: two independent awaits → Promise.all

Applies to getAssetFiles + getSourceMaps parallelization (pool/index.ts, runTests.ts, listTests.ts).

  • count = number of test files (each triggers one pair of calls)
  • delay = per-call I/O latency (0 ≈ in-memory FS, >0 ≈ real disk)
+-------+----------+----------------+----------------+-----------+
| count | delay_ms | sequential_ms  | parallel_ms    | speedup   |
+-------+----------+----------------+----------------+-----------+
|    10 |     0.00 |           0.00 |           0.00 |    0.52x  |
|    10 |     0.05 |          23.13 |          11.48 |    2.01x  |
|    10 |     0.50 |          23.16 |          11.53 |    2.01x  |
|   100 |     0.00 |           0.02 |           0.03 |    0.54x  |
|   100 |     0.05 |         228.08 |         114.39 |    1.99x  |
|   100 |     0.50 |         227.79 |         114.55 |    1.99x  |
|   500 |     0.00 |           0.12 |           0.20 |    0.60x  |
|   500 |     0.05 |        1135.12 |         570.36 |    1.99x  |
|   500 |     0.50 |        1137.39 |         567.81 |    2.00x  |
+-------+----------+----------------+----------------+-----------+

Theoretical ceiling: 2x (two independent operations fully overlapped).

Summary

Pattern B: for loop → Promise.all map

Applies to collectTestFiles, collectProjectEntries, resolveProjects (listTests.ts, hostController.ts, cli/init.ts).

  • count = number of projects
  • delay = per-project async work (glob, config resolution, etc.)
+-------+----------+----------------+----------------+-----------+
| count | delay_ms | sequential_ms  | parallel_ms    | speedup   |
+-------+----------+----------------+----------------+-----------+
|     3 |     0.00 |           0.00 |           0.00 |    0.31x  |
|     3 |     0.50 |           3.44 |           1.15 |    3.00x  |
|     3 |     2.00 |           6.84 |           2.30 |    2.98x  |
|     3 |    10.00 |          33.14 |          11.04 |    3.00x  |
|    10 |     0.00 |           0.00 |           0.00 |    0.54x  |
|    10 |     0.50 |          11.58 |           1.17 |    9.90x  |
|    10 |     2.00 |          22.81 |           2.30 |    9.91x  |
|    10 |    10.00 |         110.39 |          11.08 |    9.96x  |
|    50 |     0.00 |           0.01 |           0.01 |    0.61x  |
|    50 |     0.50 |          56.56 |           1.18 |   47.93x  |
|    50 |     2.00 |         113.04 |           2.28 |   49.53x  |
|    50 |    10.00 |         548.19 |          11.05 |   49.61x  |
+-------+----------+----------------+----------------+-----------+

Theoretical ceiling: Nx where N = project count (all projects processed concurrently).

Background

Startup and test discovery still serialized several independent async tasks, which added avoidable latency to CLI setup, project discovery, and asset preparation.

Implementation

  • Parallelized browser and core project discovery while preserving result order.
  • Overlapped asset and source map loading for global setup and worker asset filtering.
  • Started independent CLI and coverage module imports earlier so config I/O and module loading can run concurrently.

User Impact

Large multi-project runs, list commands, and startup paths spend less time waiting on serialized async setup.

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

Validation:

  • pnpm format attempted, but Biome hit an existing internal panic in packages/core/src/types/coverage.ts before completing.
  • pnpm run lint
  • pnpm run typecheck
  • pnpm run build
  • pnpm run test
  • cd e2e && pnpm test

@fi3ework fi3ework marked this pull request as draft March 18, 2026 17:22
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07d89340ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@fi3ework fi3ework marked this pull request as ready for review March 19, 2026 05:29
@fi3ework fi3ework requested a review from 9aoy March 19, 2026 05:55
@fi3ework fi3ework merged commit 8b9417f into main Mar 19, 2026
16 checks passed
@fi3ework fi3ework deleted the perf/parallelize-independent-async-work branch March 19, 2026 06:10
@9aoy 9aoy mentioned this pull request Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants