ci: remove redundant critcmp benchmark job#8823
Merged
Conversation
✅ Deploy Preview for rolldown-rs canceled.
|
There was a problem hiding this comment.
💡 Codex Review
https://github.com/rolldown/rolldown/blob/c104b7391437e056e56ca408d988d7629bac819b/.github/workflows/benchmark-rust.yml#L82-L83
Preserve CI coverage for the
threejs10x Rust benchmark
Removing the cargo bench -p bench step drops the only CI path that still executes threejs10x. The remaining job runs cargo codspeed ... --features codspeed, but crates/bench/benches/bench.rs:19-20 explicitly excludes threejs10x when the codspeed feature is enabled. In practice, any regression that only shows up on that largest fixture will now merge without benchmark feedback, which is a real coverage loss rather than a duplicate signal.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The critcmp job compiles the project twice with fat LTO (~15 min per PR) and uses a dead cache key that is never saved. CodSpeed already provides the same regression comparison with better statistical analysis. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
c104b73 to
818d1c3
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 818d1c32bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -57,82 +57,3 @@ jobs: | |||
| run: cargo codspeed build -p bench --features codspeed && cargo codspeed run -p bench | |||
There was a problem hiding this comment.
Keep the non-CodSpeed large benchmark in CI
With this deletion, the only Rust benchmark invocation left is cargo codspeed ... --features codspeed here. That is not equivalent to the removed cargo bench -p bench path: crates/bench/benches/bench.rs:19-20 gates the threejs10x case behind #[cfg(not(feature = "codspeed"))], so enabling the codspeed feature skips the largest benchmark input entirely. As a result, PRs no longer exercise that workload, and performance regressions that only show up on larger module graphs will stop being caught by CI.
Useful? React with 👍 / 👎.
Merging this PR will not alter performance
Comparing 
Footnotes
|
Merged
shulaoda
added a commit
that referenced
this pull request
Mar 23, 2026
## [1.0.0-rc.11] - 2026-03-23 ### 🚀 Features - magicString replace with regex (#8802) by @IWANABETHATGUY - support `output.sourcemapExcludeSources` option (#8828) by @sapphi-red - support `getIndentString` in MagicString (#8775) by @IWANABETHATGUY - MagicString ignoreList support (#8773) by @IWANABETHATGUY ### 🐛 Bug Fixes - types: remove `pluginName` from `MinimalPluginContext` (#8864) by @sapphi-red - do not report eval?.() as direct eval (#8860) by @IWANABETHATGUY - handle negative indices, overlapping ranges, and moved content in MagicString remove (#8829) by @IWANABETHATGUY - enable arbitrary_precision for serde_json to fix JSON float parsing (#8848) by @elderapo - resolve TypeScript lint errors (#8841) by @Boshen - avoid panic on multi-byte UTF-8 chars in hash placeholder iterator (#8790) by @shulaoda - ci: skip failing vite build watch raw query test (#8840) by @Boshen - ci: use step-level env override to unset VITE_PLUS_CLI_BIN in vite tests (#8838) by @Boshen - ci: move vite tests into CI workflow by @Boshen - ci: unset all VITE_PLUS_* env vars in vite-tests workflow (#8837) by @Boshen - test: skip watch CLI tests on Windows (#8830) by @Boshen - ci: unset VITE_PLUS_CLI_BIN in vite-tests workflow (#8832) by @Boshen - remove redundant bare side-effect imports in entry/facade chunks (#8804) by @h-a-n-a - magicString prepend issues (#8797) by @IWANABETHATGUY - ci: use `vpx` instead of `vp exec` for `pkg-pr-new` (#8827) by @Boshen - set `order` for callable plugins (#8815) by @sapphi-red - handle reversed slice ranges with moved content (#8750) by @IWANABETHATGUY - update emnapi to latest to avoid version mismatch (#8781) by @sapphi-red - external.md on Windows OS (#8780) by @bddjr - align MagicString length/isEmpty with reference magic-string (#8776) by @IWANABETHATGUY ### 🚜 Refactor - extract canonical_ref_resolving_namespace helper (#8836) by @Boshen ### 📚 Documentation - improve external examples for cross-platform correctness (#8786) by @hyf0-agent - update reference to transform function in plugin API documentation (#8778) by @zOadT ### ⚡ Performance - reduce timing of `dervie_entries_aware_chunk_name` (#8847) by @AliceLanniste - bench: remove redundant sourcemap benchmark cases (#8825) by @Boshen - reduce intermediate allocations in `collapse_sourcemaps` (#8821) by @Boshen - enable parallel AST cloning on macOS (#8814) by @Boshen ### 🧪 Testing - watch: use polling watcher and retry for watch error test (#8772) by @sapphi-red ### ⚙️ Miscellaneous Tasks - justfile: skip setup-vite-plus if vp is already installed (#8862) by @Boshen - add expectWarning option to test config (#8861) by @IWANABETHATGUY - justfile: support windows for `just setup` (#8846) by @AliceLanniste - deps: update rust crates (#8852) by @renovate[bot] - deps: update endbug/version-check action to v3 (#8855) by @renovate[bot] - deps: update github-actions (#8853) by @renovate[bot] - deps: update dependency vitepress to v2.0.0-alpha.17 (#8854) by @renovate[bot] - deps: update npm packages (#8851) by @renovate[bot] - bench: use mimalloc as global allocator in bench crate (#8844) by @IWANABETHATGUY - reuse native build artifact in node-validation job (#8826) by @Boshen - speed up CodSpeed benchmark build by disabling LTO (#8824) by @Boshen - remove redundant critcmp benchmark job (#8823) by @Boshen - deps: update rust crate oxc_sourcemap to v6.1.0 (#8785) by @renovate[bot] - node: migrate oxlint and oxfmt to Vite+ (#8813) by @Boshen - revert namespace runners for release build jobs (#8820) by @Boshen - migrate runners to namespace (#8819) by @Boshen - test: relax test utils path assertion to support git worktrees (#8816) by @younggglcy - rename `examples/lazy` to `examples/lazy-compilation` (#8789) by @shulaoda - improve "needs reproduction" wording by @Boshen - deps: update dependency oxlint-tsgolint to v0.17.1 (#8807) by @renovate[bot] - enable 7 previously-skipped MagicString tests (#8771) by @IWANABETHATGUY - upgrade oxc to 0.121.0 (#8784) by @shulaoda - increase Windows dev drive size from 12GB to 20GB (#8779) by @Copilot ### ❤️ New Contributors * @elderapo made their first contribution in [#8848](#8848) * @younggglcy made their first contribution in [#8816](#8816) * @bddjr made their first contribution in [#8780](#8780) * @zOadT made their first contribution in [#8778](#8778) Co-authored-by: shulaoda <165626830+shulaoda@users.noreply.github.com>
This was referenced Mar 23, 2026
Closed
Merged
shulaoda
added a commit
that referenced
this pull request
Mar 23, 2026
## [1.0.0-rc.11] - 2026-03-23 ### 🚀 Features - magicString replace with regex (#8802) by @IWANABETHATGUY - support `output.sourcemapExcludeSources` option (#8828) by @sapphi-red - support `getIndentString` in MagicString (#8775) by @IWANABETHATGUY - MagicString ignoreList support (#8773) by @IWANABETHATGUY ### 🐛 Bug Fixes - forward test filters through vp run (#8870) by @younggglcy - types: remove `pluginName` from `MinimalPluginContext` (#8864) by @sapphi-red - do not report eval?.() as direct eval (#8860) by @IWANABETHATGUY - handle negative indices, overlapping ranges, and moved content in MagicString remove (#8829) by @IWANABETHATGUY - enable arbitrary_precision for serde_json to fix JSON float parsing (#8848) by @elderapo - resolve TypeScript lint errors (#8841) by @Boshen - avoid panic on multi-byte UTF-8 chars in hash placeholder iterator (#8790) by @shulaoda - ci: skip failing vite build watch raw query test (#8840) by @Boshen - ci: use step-level env override to unset VITE_PLUS_CLI_BIN in vite tests (#8838) by @Boshen - ci: move vite tests into CI workflow by @Boshen - ci: unset all VITE_PLUS_* env vars in vite-tests workflow (#8837) by @Boshen - test: skip watch CLI tests on Windows (#8830) by @Boshen - ci: unset VITE_PLUS_CLI_BIN in vite-tests workflow (#8832) by @Boshen - remove redundant bare side-effect imports in entry/facade chunks (#8804) by @h-a-n-a - magicString prepend issues (#8797) by @IWANABETHATGUY - ci: use `vpx` instead of `vp exec` for `pkg-pr-new` (#8827) by @Boshen - set `order` for callable plugins (#8815) by @sapphi-red - handle reversed slice ranges with moved content (#8750) by @IWANABETHATGUY - update emnapi to latest to avoid version mismatch (#8781) by @sapphi-red - external.md on Windows OS (#8780) by @bddjr - align MagicString length/isEmpty with reference magic-string (#8776) by @IWANABETHATGUY ### 🚜 Refactor - extract canonical_ref_resolving_namespace helper (#8836) by @Boshen ### 📚 Documentation - improve external examples for cross-platform correctness (#8786) by @hyf0-agent - update reference to transform function in plugin API documentation (#8778) by @zOadT ### ⚡ Performance - reduce timing of `dervie_entries_aware_chunk_name` (#8847) by @AliceLanniste - bench: remove redundant sourcemap benchmark cases (#8825) by @Boshen - reduce intermediate allocations in `collapse_sourcemaps` (#8821) by @Boshen - enable parallel AST cloning on macOS (#8814) by @Boshen ### 🧪 Testing - watch: use polling watcher and retry for watch error test (#8772) by @sapphi-red ### ⚙️ Miscellaneous Tasks - deps: update dependency @oxc-project/types to v0.122.0 (#8873) by @renovate[bot] - publish-to-npm: use correct vp pm publish (#8871) by @shulaoda - justfile: skip setup-vite-plus if vp is already installed (#8862) by @Boshen - add expectWarning option to test config (#8861) by @IWANABETHATGUY - justfile: support windows for `just setup` (#8846) by @AliceLanniste - deps: update rust crates (#8852) by @renovate[bot] - deps: update endbug/version-check action to v3 (#8855) by @renovate[bot] - deps: update github-actions (#8853) by @renovate[bot] - deps: update dependency vitepress to v2.0.0-alpha.17 (#8854) by @renovate[bot] - deps: update npm packages (#8851) by @renovate[bot] - bench: use mimalloc as global allocator in bench crate (#8844) by @IWANABETHATGUY - reuse native build artifact in node-validation job (#8826) by @Boshen - speed up CodSpeed benchmark build by disabling LTO (#8824) by @Boshen - remove redundant critcmp benchmark job (#8823) by @Boshen - deps: update rust crate oxc_sourcemap to v6.1.0 (#8785) by @renovate[bot] - node: migrate oxlint and oxfmt to Vite+ (#8813) by @Boshen - revert namespace runners for release build jobs (#8820) by @Boshen - migrate runners to namespace (#8819) by @Boshen - test: relax test utils path assertion to support git worktrees (#8816) by @younggglcy - rename `examples/lazy` to `examples/lazy-compilation` (#8789) by @shulaoda - improve "needs reproduction" wording by @Boshen - deps: update dependency oxlint-tsgolint to v0.17.1 (#8807) by @renovate[bot] - enable 7 previously-skipped MagicString tests (#8771) by @IWANABETHATGUY - upgrade oxc to 0.121.0 (#8784) by @shulaoda - increase Windows dev drive size from 12GB to 20GB (#8779) by @Copilot ### ❤️ New Contributors * @younggglcy made their first contribution in [#8870](#8870) * @elderapo made their first contribution in [#8848](#8848) * @bddjr made their first contribution in [#8780](#8780) * @zOadT made their first contribution in [#8778](#8778) --------- Co-authored-by: shulaoda <165626830+shulaoda@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
benchmark-rust(critcmp) job which compiles the project twice with fat LTO (~15 min per PR) and uses a dead cache key that is never savedcodspeed-benchmarkjob🤖 Generated with Claude Code
The wall clock benchmarks were flaky and provided unreliable results, so we always had to double-check them. They also made our CI too slow, so we remove them. We should get our codspeed benchmark to produce meaningful results.