Skip to content

add test for deterministic deflate output#461

Merged
folkertdev merged 1 commit intomainfrom
reset-deterministic
Feb 25, 2026
Merged

add test for deterministic deflate output#461
folkertdev merged 1 commit intomainfrom
reset-deterministic

Conversation

@folkertdev
Copy link
Copy Markdown
Member

fixes #459

Compressing an input after a reset should give the same output as if it was done with a fresh compression state. There was a bug where some of the state from before the reset could influence what match was picked for later input. The output was still valid, but it is inconsistent.

Thank you @QrczakMK for the detective work here!

@codecov
Copy link
Copy Markdown

codecov bot commented Jan 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
fuzz-compress 40.05% <100.00%> (ø)
fuzz-decompress 27.10% <0.00%> (-0.45%) ⬇️
test-aarch64-apple-darwin 85.57% <100.00%> (ø)
test-aarch64-unknown-linux-gnu 85.50% <100.00%> (ø)
test-i686-unknown-linux-gnu 85.20% <100.00%> (ø)
test-x86_64-apple-darwin 84.84% <100.00%> (-0.03%) ⬇️
test-x86_64-unknown-linux-gnu 90.92% <100.00%> (-1.56%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
zlib-rs/src/deflate/longest_match.rs 95.69% <100.00%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

const DATA_A: &[u8] = b"\0AAAA\0AAAAAAAA";
const DATA_B: &[u8] = &[1u8; DATA_A.len() + 1];

pub unsafe fn compress_data(src: &[u8], zs: &mut z_stream) -> Vec<u8> {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function can be simplified, e.g. as in https://github.com/zlib-ng/zlib-ng/pull/2102/files

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll hold off on that until they merge the PR.

Even then we're running into some failing tests here, so we'll have to wait for a new libz-ng release before we can merge this PR.

@QrczakMK
Copy link
Copy Markdown

The zlib-ng fix has just been merged to the develop branch: zlib-ng/zlib-ng#2102

@folkertdev
Copy link
Copy Markdown
Member Author

Cool, I've updated the implementation here but we'd need a release before we can merge here (plus I'd need to update libz-sys to use that release, but we do that at every zlib-ng release).

@folkertdev folkertdev force-pushed the reset-deterministic branch 3 times, most recently from d65d0d8 to 1c2609e Compare February 24, 2026 19:26
@folkertdev folkertdev requested review from bjorn3 February 24, 2026 19:42
@folkertdev
Copy link
Copy Markdown
Member Author

The libz-sys dependency has been updated, so now we can upgrade too.

Cargo.toml Outdated
libz-sys = { version = "1.1.23", default-features = false, features = ["zlib-ng"] } # use libz-ng in libz compat mode
arbitrary = { version = "1.0" }
quickcheck = { version = "1.0.3", default-features = false, features = [] }
getrandom = "0.2.17" # work around https://github.com/rust-random/getrandom/issues/775
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did say this is the proper fix. Something is broken with the older version, so you effectively have a dependency on the newer version already, this is just declaring said dependency.

Suggested change
getrandom = "0.2.17" # work around https://github.com/rust-random/getrandom/issues/775
getrandom = "0.2.17" # Earlier getrandom versions contain UB: https://github.com/rust-random/getrandom/issues/775

Does specifying this in workspace.dependencies have any effect though? Aren't workspace.dependencies and workspace.package just inert templates that an actual package can use with foo.workspace = true? Maybe move it to the dev-dependencies section of test-libz-rs-sys/Cargo.toml ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't, it's already bumped in the Cargo.lock file in d1aad5e, so this is not really needed.

compressing an input after a `reset` should give the same output as if it was done with a fresh compression state. There was a bug where some of the state from before the reset could 'leak' into influencing what match was picked. The output was still valid, but it is inconsistent.
@folkertdev folkertdev merged commit d45e960 into main Feb 25, 2026
29 checks passed
@folkertdev folkertdev deleted the reset-deterministic branch February 25, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deflate output not deterministic when the compressor is reused with deflateReset

3 participants