Skip to content

Conversation

@lemire
Copy link
Member

@lemire lemire commented Jul 17, 2025

This should fix #824 once completed.

Thanks for the report by @syg

This PR changes the following things...

  1. Previously, we intentionally removed (ignored) trailing ignorable characters. They are now included. This when decoding AAAA , we would declare having read 4 characters.
  2. When the input was empty (except for ignorable characters), we would report reading zero characters.
  3. With partial chunk handling, we would stop right before the partial chunk, we now end right after the last complete chunk (thus dropping the ignorable characters).

I have added some tests.

@lemire lemire requested a review from Copilot July 17, 2025 23:10

This comment was marked as resolved.

@lemire lemire marked this pull request as draft July 18, 2025 03:59
@lemire lemire changed the title fixing Issue 824 fixing Issue 824 (work in progress) Jul 18, 2025
@lemire
Copy link
Member Author

lemire commented Jul 18, 2025

@syg

I am currently busy but I will come back to this issue and complete it before the end of July.

Daniel Lemire and others added 2 commits July 31, 2025 18:09
@lemire lemire marked this pull request as ready for review July 31, 2025 22:47
@lemire lemire merged commit a3a268e into master Aug 1, 2025
69 of 70 checks passed
@sno2
Copy link
Contributor

sno2 commented Aug 10, 2025

@lemire
Copy link
Member Author

lemire commented Aug 10, 2025

@sno2 Yes. They resulted from a faulty test that I wrote hastily. The test is itself unsafe (but not the code).

syg added a commit to syg/WebKit that referenced this pull request Oct 21, 2025
https://bugs.webkit.org/show_bug.cgi?id=301152
Include a Radar link (OOPS!).

Reviewed by NOBODY (OOPS!).

The simdutf library had a bug that returned the incorrect read length when the
last chunk handling option is "stop-before-partial". This bug has since been
fixed upstream in simdutf/simdutf#825.

This PR update simdutf to 7.5.0 and removes the hack.

Testing is already covered by test262, specifically
built-ins/Uint8Array/prototype/setFromBase64.
syg added a commit to syg/WebKit that referenced this pull request Oct 21, 2025
https://bugs.webkit.org/show_bug.cgi?id=301158
rdar://163082878

Reviewed by NOBODY (OOPS!).

The simdutf library had a bug that returned the incorrect read length when the
last chunk handling option is "stop-before-partial". This bug has since been
fixed upstream in simdutf/simdutf#825.

This PR update simdutf to 7.5.0 and removes the hack.

Testing is already covered by test262, specifically
built-ins/Uint8Array/prototype/setFromBase64.
syg added a commit to syg/WebKit that referenced this pull request Oct 21, 2025
https://bugs.webkit.org/show_bug.cgi?id=301158
rdar://163082878

Reviewed by NOBODY (OOPS!).

The simdutf library had a bug that returned the incorrect read length when the
last chunk handling option is "stop-before-partial". This bug has since been
fixed upstream in simdutf/simdutf#825.

This PR updates simdutf to 7.5.0 and removes the hack.

Testing is already covered by test262, specifically
built-ins/Uint8Array/prototype/setFromBase64.
webkit-commit-queue pushed a commit to syg/WebKit that referenced this pull request Oct 21, 2025
https://bugs.webkit.org/show_bug.cgi?id=301158
rdar://163082878

Reviewed by Yusuke Suzuki.

The simdutf library had a bug that returned the incorrect read length when the
last chunk handling option is "stop-before-partial". This bug has since been
fixed upstream in simdutf/simdutf#825.

This PR updates simdutf to 7.5.0 and removes the hack.

Testing is already covered by test262, specifically
built-ins/Uint8Array/prototype/setFromBase64.

Canonical link: https://commits.webkit.org/301889@main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read count is incorrect when using last_chunk_handling::stop_before_partial

3 participants