Skip to content

[Panic]: byte index 21 is out of bounds of AQZUku11Kfs-GgIXiQXV #8270

@blixt

Description

@blixt

Panic message

Rolldown panicked. This is a bug in Rolldown, not your code.

thread 'tokio-runtime-worker' (4853) panicked at /home/runner/work/rolldown/rolldown/crates/rolldown/src/utils/chunk/finalize_chunks.rs:117:74:
byte index 21 is out of bounds of `AQZUku11Kfs-GgIXiQXV`
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Reproduction

I've previously reported very similar bugs (#4016, #4017) and this crash is haunting me once more. I believe it's due to base64 encoding not always giving us 21 characters. I think the math checks out to: 75% of the time we get 22 characters, 24.6% of the time we get 21 characters, and 0.4% of the time we get 20 characters (hence a crash if we request 21).

The fix could be something like this:

diff --git a/crates/rolldown_utils/src/xxhash.rs b/crates/rolldown_utils/src/xxhash.rs
--- a/crates/rolldown_utils/src/xxhash.rs
+++ b/crates/rolldown_utils/src/xxhash.rs
@@ -18,12 +18,25 @@ pub fn xxhash_with_base(input: &[u8], base: u8) -> String {
   let hash = if input.len() == 16 { input } else { &xxh3_128(input).to_le_bytes() };

   let chars = match base {
     64 => CHARACTERS_BASE64,
     36 => &CHARACTERS_BASE64[26..(26 + 36)],
     16 => CHARACTERS_BASE16,
     _ => {
       unreachable!()
     }
   };

-  to_string(hash, base, chars).unwrap()
+  let mut result = to_string(hash, base, chars).unwrap();
+  // `base_encode::to_string` produces variable-length output like decimal
+  // (no leading-zero padding). Left-pad with the zero-digit to the maximum
+  // length for a 128-bit value so that `&hash[..placeholder.len()]` in
+  // finalize_chunks.rs never panics on a shorter-than-expected string.
+  let max_len = match base {
+    64 => 22, // ceil(128 / log2(64))
+    36 => 25, // ceil(128 / log2(36))
+    16 => 32, // 128 / 4
+    _ => unreachable!(),
+  };
+  while result.len() < max_len {
+    result.insert(0, chars[0] as char);
+  }
+  result
 }

 #[test]
 fn test_xxhash_with_base() {
-  assert_eq!(&xxhash_with_base(b"hello", 64), "YOFJeqs95x38-Gwetwem1");
+  assert_eq!(&xxhash_with_base(b"hello", 64), "AYOFJeqs95x38-Gwetwem1");
   assert_eq!(&xxhash_with_base(b"hello", 36), "bpwli5k6mqm0gij09mxrh9npj");
   assert_eq!(&xxhash_with_base(b"hello", 16), "1838525eaacf79c77f3e1b07adc1e9b5");
 }

But I'm sure there are more performant ways to do it.

System Info

System:
    OS: macOS 26.3
    CPU: (16) arm64 Apple M3 Max
    Memory: 52.42 GB / 128.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 22.22.0 - /Users/blixt/.local/share/mise/installs/node/22.22.0/bin/node
    Yarn: 1.22.22 - /opt/homebrew/bin/yarn
    npm: 10.9.4 - /Users/blixt/.local/share/mise/installs/node/22.22.0/bin/npm
    pnpm: 10.28.1 - /Users/blixt/.local/share/mise/installs/node/22.22.0/bin/pnpm
    bun: 1.2.23 - /Users/blixt/.bun/bin/bun
    Deno: 2.6.8 - /opt/homebrew/bin/deno
  Browsers:
    Safari: 26.3

Additional context

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions