Skip to content

fix: ensure ordering of assign chunk name#1981

Merged
underfin merged 4 commits intomainfrom
fix-chunk-name-order
Aug 14, 2024
Merged

fix: ensure ordering of assign chunk name#1981
underfin merged 4 commits intomainfrom
fix-chunk-name-order

Conversation

@underfin
Copy link
Contributor

@underfin underfin commented Aug 14, 2024

Description

The esbuild using Chunk#bits to sorted chunks, but the order of Chunk#bits is not stable, eg BitSet(0) 00000001_00000000 > BitSet(8) 00000000_00000001. It couldn't ensure the order of dynamic chunks and common chunks.
Consider the compare Chunk#exec_order should be faster than Chunk#bits, we use Chunk#exec_order to sort chunks.
Note Here could be make sure the order of chunks.

  • entry chunks are always before other chunks
  • static chunks are always before dynamic chunks
  • other chunks has stable order at per entry chunk level

@netlify
Copy link

netlify bot commented Aug 14, 2024

Deploy Preview for rolldown-rs canceled.

Name Link
🔨 Latest commit 3bd9ca1
🔍 Latest deploy log https://app.netlify.com/sites/rolldown-rs/deploys/66bc839b9baec300081a955e

// - entry chunks are always before other chunks
// - static chunks are always before dynamic chunks
// - other chunks has stable order at per entry chunk level
let sorted_chunk_idx_vec = chunks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chunks
.iter_mut()
.sorted_by(|a, b| {
let a_should_be_first = Ordering::Less;
let b_should_be_first = Ordering::Greater;
match (&a.kind, &b.kind) {
(
ChunkKind::EntryPoint { module: a_module_id, .. },
ChunkKind::EntryPoint { module: b_module_id, .. },
) => self.link_output.module_table.modules[*a_module_id]
.exec_order()
.cmp(&self.link_output.module_table.modules[*b_module_id].exec_order()),
(ChunkKind::EntryPoint { module: a_module_id, .. }, ChunkKind::Common) => {
let a_module_exec_order =
self.link_output.module_table.modules[*a_module_id].exec_order();
let b_chunk_first_module_exec_order =
self.link_output.module_table.modules[b.modules[0]].exec_order();
if a_module_exec_order == b_chunk_first_module_exec_order {
a_should_be_first
} else {
a_module_exec_order.cmp(&b_chunk_first_module_exec_order)
}
}
(ChunkKind::Common, ChunkKind::EntryPoint { module: b_module_id, .. }) => {
let b_module_exec_order =
self.link_output.module_table.modules[*b_module_id].exec_order();
let a_chunk_first_module_exec_order =
self.link_output.module_table.modules[a.modules[0]].exec_order();
if a_chunk_first_module_exec_order == b_module_exec_order {
b_should_be_first
} else {
a_chunk_first_module_exec_order.cmp(&b_module_exec_order)
}
}
(ChunkKind::Common, ChunkKind::Common) => {
let a_chunk_first_module_exec_order =
self.link_output.module_table.modules[a.modules[0]].exec_order();
let b_chunk_first_module_exec_order =
self.link_output.module_table.modules[b.modules[0]].exec_order();
a_chunk_first_module_exec_order.cmp(&b_chunk_first_module_exec_order)
}
}
})
.enumerate()
.for_each(|(i, chunk)| {
chunk.exec_order = i.try_into().expect("Too many chunks, u32 overflowed.");
});

chunk is already sorted here. Could we just use chunks.indices().collect<Vec<_>>?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The share chunks ordering is not stable because we using unstale ordering at https://github.com/rolldown/rolldown/blob/main/crates/rolldown/src/stages/generate_stage/code_splitting.rs#L141

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we fix it look like it could be get expected stable ordering for assgin name

Copy link
Member

@hyf0 hyf0 Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The share chunks ordering is not stable because we using unstale ordering at https://github.com/rolldown/rolldown/blob/main/crates/rolldown/src/stages/generate_stage/code_splitting.rs#L141

Shared chunks are sorted by their first module's execution order. It seems stable.

(ChunkKind::Common, ChunkKind::Common) => {
let a_chunk_first_module_exec_order =
self.link_output.module_table.modules[a.modules[0]].exec_order();
let b_chunk_first_module_exec_order =
self.link_output.module_table.modules[b.modules[0]].exec_order();
a_chunk_first_module_exec_order.cmp(&b_chunk_first_module_exec_order)

chunks.iter_mut().for_each(|chunk| {
chunk.modules.sort_unstable_by_key(|module_id| {
self.link_output.module_table.modules[*module_id].exec_order()
});
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chunks
.iter_mut()
.sorted_by(|a, b| {
let a_should_be_first = Ordering::Less;
let b_should_be_first = Ordering::Greater;
match (&a.kind, &b.kind) {
(
ChunkKind::EntryPoint { module: a_module_id, .. },
ChunkKind::EntryPoint { module: b_module_id, .. },
) => self.link_output.module_table.modules[*a_module_id]
.exec_order()
.cmp(&self.link_output.module_table.modules[*b_module_id].exec_order()),
(ChunkKind::EntryPoint { module: a_module_id, .. }, ChunkKind::Common) => {
let a_module_exec_order =
self.link_output.module_table.modules[*a_module_id].exec_order();
let b_chunk_first_module_exec_order =
self.link_output.module_table.modules[b.modules[0]].exec_order();
if a_module_exec_order == b_chunk_first_module_exec_order {
a_should_be_first
} else {
a_module_exec_order.cmp(&b_chunk_first_module_exec_order)
}
}
(ChunkKind::Common, ChunkKind::EntryPoint { module: b_module_id, .. }) => {
let b_module_exec_order =
self.link_output.module_table.modules[*b_module_id].exec_order();
let a_chunk_first_module_exec_order =
self.link_output.module_table.modules[a.modules[0]].exec_order();
if a_chunk_first_module_exec_order == b_module_exec_order {
b_should_be_first
} else {
a_chunk_first_module_exec_order.cmp(&b_module_exec_order)
}
}
(ChunkKind::Common, ChunkKind::Common) => {
let a_chunk_first_module_exec_order =
self.link_output.module_table.modules[a.modules[0]].exec_order();
let b_chunk_first_module_exec_order =
self.link_output.module_table.modules[b.modules[0]].exec_order();
a_chunk_first_module_exec_order.cmp(&b_chunk_first_module_exec_order)
}
}
})
.enumerate()
.for_each(|(i, chunk)| {
chunk.exec_order = i.try_into().expect("Too many chunks, u32 overflowed.");
});

chunk is already sorted here. Could we just use chunks.indices().collect<Vec<_>>?

Oh. I missing the your meanings. Here the order is not correct, because the share chunk execute order could be at first. But the entry chunks should be at first for assign name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ChunkKind::EntryPoint { module: a_module_id, .. }, ChunkKind::Common) => {
let a_module_exec_order =
self.link_output.module_table.modules[*a_module_id].exec_order();
let b_chunk_first_module_exec_order =
self.link_output.module_table.modules[b.modules[0]].exec_order();
if a_module_exec_order == b_chunk_first_module_exec_order {
a_should_be_first
} else {
a_module_exec_order.cmp(&b_chunk_first_module_exec_order)
}

Probably should this logic.

Copy link
Contributor Author

@underfin underfin Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we fix it look like it could be get expected stable ordering for assgin name

Here I have a try to changed it, but looke like the sorted_modules hasn't dynamic import ids.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 14, 2024

Benchmarks Rust

group                                                               pr                                     target
-----                                                               --                                     ------
bundle/bundle@multi-duplicated-top-level-symbol                     1.00     52.8±1.81ms        ? ?/sec    1.05     55.6±1.25ms        ? ?/sec
bundle/bundle@multi-duplicated-top-level-symbol-minify              1.03     81.5±2.70ms        ? ?/sec    1.00     78.9±1.42ms        ? ?/sec
bundle/bundle@multi-duplicated-top-level-symbol-minify-sourcemap    1.00     96.3±2.02ms        ? ?/sec    1.03     98.8±1.95ms        ? ?/sec
bundle/bundle@multi-duplicated-top-level-symbol-sourcemap           1.00     61.1±1.65ms        ? ?/sec    1.02     62.1±1.96ms        ? ?/sec
bundle/bundle@rome-ts                                               1.01    105.5±0.75ms        ? ?/sec    1.00    104.2±1.67ms        ? ?/sec
bundle/bundle@rome-ts-minify                                        1.00    195.1±7.13ms        ? ?/sec    1.09   212.1±10.17ms        ? ?/sec
bundle/bundle@rome-ts-minify-sourcemap                              1.00    227.5±6.72ms        ? ?/sec    1.04    236.0±8.34ms        ? ?/sec
bundle/bundle@rome-ts-sourcemap                                     1.03    123.1±1.37ms        ? ?/sec    1.00    119.7±1.13ms        ? ?/sec
bundle/bundle@threejs                                               1.00     37.9±0.74ms        ? ?/sec    1.00     37.9±0.65ms        ? ?/sec
bundle/bundle@threejs-minify                                        1.00     86.1±3.94ms        ? ?/sec    1.02     88.0±2.68ms        ? ?/sec
bundle/bundle@threejs-minify-sourcemap                              1.00    105.5±3.82ms        ? ?/sec    1.02    107.6±3.58ms        ? ?/sec
bundle/bundle@threejs-sourcemap                                     1.01     47.9±0.57ms        ? ?/sec    1.00     47.6±0.69ms        ? ?/sec
bundle/bundle@threejs10x                                            1.01    402.5±7.68ms        ? ?/sec    1.00    397.2±3.15ms        ? ?/sec
bundle/bundle@threejs10x-minify                                     1.01  1038.8±15.54ms        ? ?/sec    1.00   1029.8±9.16ms        ? ?/sec
bundle/bundle@threejs10x-minify-sourcemap                           1.00  1265.1±10.89ms        ? ?/sec    1.00  1264.1±17.10ms        ? ?/sec
bundle/bundle@threejs10x-sourcemap                                  1.00    476.3±5.11ms        ? ?/sec    1.00    477.5±4.86ms        ? ?/sec
remapping/remapping                                                 1.02     33.5±0.92ms        ? ?/sec    1.00     32.8±0.22ms        ? ?/sec
remapping/render-chunk-remapping                                    1.00     81.1±1.48ms        ? ?/sec    1.02     82.3±0.28ms        ? ?/sec
scan/scan@rome-ts                                                   1.00     83.3±1.12ms        ? ?/sec    1.02     84.6±0.72ms        ? ?/sec
scan/scan@threejs                                                   1.00     28.7±0.23ms        ? ?/sec    1.02     29.3±0.82ms        ? ?/sec
scan/scan@threejs10x                                                1.00    290.6±2.45ms        ? ?/sec    1.03    297.9±3.50ms        ? ?/sec

@underfin underfin added this pull request to the merge queue Aug 14, 2024
Merged via the queue into main with commit 7ac1381 Aug 14, 2024
@underfin underfin deleted the fix-chunk-name-order branch August 14, 2024 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants