Skip to content

Fixed minideflate write buffers being overwritten.#1060

Merged
Dead2 merged 1 commit intozlib-ng:developfrom
nmoinvaz:fixes/minideflate-buffers
Dec 2, 2021
Merged

Fixed minideflate write buffers being overwritten.#1060
Dead2 merged 1 commit intozlib-ng:developfrom
nmoinvaz:fixes/minideflate-buffers

Conversation

@nmoinvaz
Copy link
Copy Markdown
Member

@nmoinvaz nmoinvaz commented Nov 12, 2021 •
edited
Loading

See #1059 and #1053. Certain stream info were not being set at the correct times on the stream, causing some data to be lost.

@nmoinvaz nmoinvaz added the bug label Nov 12, 2021
@nmoinvaz nmoinvaz mentioned this pull request Nov 12, 2021
Merged
@codecov
Copy link
Copy Markdown

codecov bot commented Nov 12, 2021 •
edited
Loading

Codecov Report

Merging #1060 (0350dc9) into develop (e12fc9d) will decrease coverage by 0.04%.
The diff coverage is 50.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1060      +/-   ##
===========================================
- Coverage    79.99%   79.95%   -0.05%     
===========================================
  Files           85       85              
  Lines         8678     8680       +2     
  Branches      1405     1397       -8     
===========================================
- Hits          6942     6940       -2     
- Misses        1193     1196       +3     
- Partials       543      544       +1
Flag Coverage Δ
macos_clang 70.15% <50.00%> (-0.04%) ⬇️
macos_gcc 68.43% <50.00%> (-0.05%) ⬇️
ubuntu_clang 69.74% <50.00%> (-0.04%) ⬇️
ubuntu_clang_debug 68.92% <50.00%> (-0.04%) ⬇️
ubuntu_clang_inflate_allow_invalid_dist 69.50% <50.00%> (-0.04%) ⬇️
ubuntu_clang_inflate_strict 69.69% <50.00%> (-0.04%) ⬇️
ubuntu_clang_mmap 69.74% <50.00%> (-0.04%) ⬇️
ubuntu_clang_msan 69.74% <50.00%> (-0.04%) ⬇️
ubuntu_clang_pigz 36.15% <ø> (ø)
ubuntu_clang_pigz_no_optim 40.90% <ø> (ø)
ubuntu_clang_pigz_no_threads 35.66% <ø> (ø)
ubuntu_clang_reduced_mem 70.00% <50.00%> (-0.04%) ⬇️
ubuntu_gcc 68.86% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_aarch64 70.65% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_aarch64_compat_no_opt 69.03% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_aarch64_no_acle 69.73% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_aarch64_no_neon 70.03% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_armhf 70.63% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_armhf_compat_no_opt 69.00% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_armhf_no_acle 70.82% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_armhf_no_neon 71.08% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_armsf 70.64% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_armsf_compat_no_opt 69.00% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_compat_no_opt 70.30% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_compat_sprefix 68.59% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_mingw_i686 0.00% <0.00%> (ø)
ubuntu_gcc_mingw_x86_64 0.00% <0.00%> (ø)
ubuntu_gcc_no_avx2 70.75% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_no_ctz 71.26% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_no_ctzll 70.95% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_no_pclmulqdq 67.19% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_no_sse2 69.85% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_no_sse4 67.43% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_o3 67.93% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_osb 70.48% <50.00%> (-0.04%) ⬇️
ubuntu_gcc_pigz 36.70% <ø> (+0.02%) ⬆️
ubuntu_gcc_pigz_aarch64 38.74% <ø> (-0.04%) ⬇️
ubuntu_gcc_ppc 68.33% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_ppc64 72.03% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_ppc64le 70.35% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_ppc_no_power8 71.10% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_s390x 73.56% <50.00%> (-0.05%) ⬇️
ubuntu_gcc_sparc64 72.31% <50.00%> (-0.06%) ⬇️
ubuntu_gcc_sprefix 68.43% <50.00%> (-0.05%) ⬇️
win64_gcc ∅ <ø> (∅)
win64_gcc_compat_no_opt ∅ <ø> (∅)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
test/minideflate.c 59.34% <50.00%> (-1.22%) ⬇️
trees.c 73.77% <0.00%> (-0.28%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e12fc9d...0350dc9. Read the comment docs.

@KungFuJesus
Copy link
Copy Markdown
Collaborator

KungFuJesus commented Nov 12, 2021 •
edited
Loading

So it doesn't bail out anymore with those changes, but something definitely seems wrong:

[adam@thinkpad build]$ ./minideflate -c /tmp/silesia.tar > ~/silesia.tar.df
[adam@thinkpad build]$ ./minideflate -d ~/silesia.tar.df >  ~/silesia.tar
[adam@thinkpad build]$ md5sum ~/silesia.tar
d41d8cd98f00b204e9800998ecf8427e  /home/adam/silesia.tar
[adam@thinkpad build]$ md5sum /tmp/silesia.tar 
42e120d96a85a495910b628718770599  /tmp/silesia.tar

Did I somehow invoke this wrong? The deflatebench test also fails, claiming something isn't in the gzip format:

Tool: minideflate
Activated single file mode
Level 0-9: /tmp/silesia.tar  202.1 MiB   211,957,760 B
Starting run 1 of 15
Testing level 0: cdv
gzip: /tmp/zlib-testfil.gz: not in gzip format
Failed, retval(1): gunzip -c /tmp/zlib-testfil.gz

@KungFuJesus
Copy link
Copy Markdown
Collaborator

KungFuJesus commented Nov 12, 2021

ahh, the gzip thing appears to be an oversight by the benchmarking tool. That's gunzip trying to verify checksums, which don't apply of course, with deflate compression.

@nmoinvaz
Copy link
Copy Markdown
Member Author

nmoinvaz commented Nov 12, 2021

I think your second command should be ./minideflate -c -d ~/silesia.tar.df > ~/silesia.tar

@KungFuJesus
Copy link
Copy Markdown
Collaborator

KungFuJesus commented Nov 12, 2021

Ah yes, you're correct. Reading the --help text would have served me there. The md5 checksums match after that.

@nmoinvaz
Copy link
Copy Markdown
Member Author

nmoinvaz commented Dec 1, 2021

@Dead2 this is also ready.

@Dead2 Dead2 merged commit 81154c5 into zlib-ng:develop Dec 2, 2021
Dead2 added a commit that referenced this pull request Dec 13, 2021
@Dead2
Version 2.0.6
2cf68ab 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050
@Dead2 Dead2 mentioned this pull request Dec 13, 2021
Merged
Dead2 added a commit that referenced this pull request Dec 20, 2021
@Dead2
Version 2.0.6
4c43f09 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1071
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
@Dead2
Version 2.0.6
0436e5c 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1071
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
@Dead2
Version 2.0.6
a7d7aea 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1071
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
@Dead2
Version 2.0.6
5618c06 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1049 #1071
 - Fix incorrect function declaration warning #1080
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1049 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
@Dead2
Version 2.0.6
9f99d5b 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1049 #1071
 - Fix incorrect function declaration warning #1080
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1049 #1051 #1056 #1063 #1067 #1079
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 24, 2021
@Dead2
Version 2.0.6
b56a2fd 
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1049 #1071
 - Fix incorrect function declaration warning #1080
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1049 #1051 #1056 #1063 #1067 #1079
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Dead2 Dead2 Dead2 approved these changes

Assignees

No one assigned

Labels

bug

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nmoinvaz @KungFuJesus @Dead2