Skip to content

perf: drop some big data in rayon#11611

Merged
LingyuCoder merged 2 commits intomainfrom
perf/drop-in-rayon-spawn
Sep 8, 2025
Merged

perf: drop some big data in rayon#11611
LingyuCoder merged 2 commits intomainfrom
perf/drop-in-rayon-spawn

Conversation

@LingyuCoder
Copy link
Contributor

@LingyuCoder LingyuCoder commented Sep 8, 2025

Summary

We should drop some big data structs in rayon::spawn if the task only run on main thread.

Before:
image

After:
image

Related links

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

Copilot AI review requested due to automatic review settings September 8, 2025 07:28
@netlify
Copy link

netlify bot commented Sep 8, 2025

Deploy Preview for rspack canceled.

Name Link
🔨 Latest commit 07748bc
🔍 Latest deploy log https://app.netlify.com/projects/rspack/deploys/68be861c7bf02d0008df6000

@github-actions github-actions bot added release: performance release: performance related release(mr only) team The issue/pr is created by the member of Rspack. labels Sep 8, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes memory usage by moving the deallocation of large data structures to background threads using rayon::spawn. This prevents blocking the main thread during expensive drop operations for memory-intensive structs.

  • Use rayon::spawn to drop large data structures asynchronously
  • Capture and drop removed values from collections in background threads
  • Refactor iterator chains to collect intermediate results before parallel processing

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
module_group.rs Spawn background task to drop removed module group entries
mod.rs Spawn background task to drop combinator after processing
module_concatenation_plugin.rs Refactor parallel iterator and spawn background drop for cache data
exports_info.rs Spawn background task to drop replaced exports info entries

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

📦 Binary Size-limit

Comparing 07748bc to perf: improve create hash (#11598) by harpsealjs

🎉 Size decreased by 2.13KB from 47.24MB to 47.24MB (⬇️0.00%)

@codspeed-hq
Copy link

codspeed-hq bot commented Sep 8, 2025

CodSpeed Performance Report

Merging #11611 will not alter performance

Comparing perf/drop-in-rayon-spawn (07748bc) with main (6e6aedf)

🎉 Hooray! codspeed-node just leveled up to 4.0.1!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

✅ 17 untouched benchmarks

@LingyuCoder LingyuCoder merged commit 451a9a6 into main Sep 8, 2025
70 of 72 checks passed
@LingyuCoder LingyuCoder deleted the perf/drop-in-rayon-spawn branch September 8, 2025 09:22
@LingyuCoder LingyuCoder mentioned this pull request Sep 12, 2025
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release: performance release: performance related release(mr only) team The issue/pr is created by the member of Rspack.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants