Skip to content

Add support of decoding legacy frames#66

Merged
PSeitz merged 2 commits intoPSeitz:mainfrom
yestyle:main
Jan 30, 2023
Merged

Add support of decoding legacy frames#66
PSeitz merged 2 commits intoPSeitz:mainfrom
yestyle:main

Conversation

@yestyle
Copy link
Copy Markdown
Contributor

@yestyle yestyle commented Jan 27, 2023

Hi @PSeitz ,

While working on an utility ikconfig extracting Linux kernel images, I chose lz4_flex as a dependency to do lz4 decompression. However, Linux kernel is using legacy frame and lz4_flex doesn't support it yet. I added the support in my fork and thinking it'd be good to have it upstream so here's the pull request.

Cheers,
Philip

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jan 27, 2023

Codecov Report

Merging #66 (7d2185a) into main (e112ef3) will increase coverage by 0.02%.
The diff coverage is 88.57%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main      #66      +/-   ##
==========================================
+ Coverage   87.65%   87.67%   +0.02%     
==========================================
  Files          11       11              
  Lines        2292     2321      +29     
==========================================
+ Hits         2009     2035      +26     
- Misses        283      286       +3     
Impacted Files Coverage Δ
src/frame/decompress.rs 75.17% <80.00%> (+0.36%) ⬆️
src/frame/header.rs 88.31% <100.00%> (+0.75%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 27, 2023

Awesome thank you!
Can you add a decompression test for the legacy format? to make sure there's no regression later

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 27, 2023

I fixed the broken gh action

@yestyle
Copy link
Copy Markdown
Contributor Author

yestyle commented Jan 27, 2023

Awesome thank you! Can you add a decompression test for the legacy format? to make sure there's no regression later

Sure, let me try to add one.

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 28, 2023

I noticed you force pushed against the branch, but there's no test added

@yestyle
Copy link
Copy Markdown
Contributor Author

yestyle commented Jan 28, 2023 via email

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 29, 2023

Since the legacy format is used in the linux kernel, that may be a big enough use case.
I would put it behind a feature flag, if existing code is affected performance wise.

You can also add a special test only for legacy decompression out site the test suite.

`benches/dickens.lz4` was generated by cli utility lz4:

lz4 -l benches/dickens.txt benches/dickens.lz4
@yestyle
Copy link
Copy Markdown
Contributor Author

yestyle commented Jan 29, 2023

Thanks for the suggestion. As described in the commit message, I created benches/dickens.lz4 using CLI utility lz4:

lz4 -l benches/dickens.txt benches/dickens.lz4

and decompress and compare with benches/dickens.txt in the test case. Hope this is acceptable.

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 29, 2023

A 6MB binary is a little big for git, can you pick something smaller?

@yestyle
Copy link
Copy Markdown
Contributor Author

yestyle commented Jan 29, 2023

Thanks for the review. Since legacy format uses fixed 8MB block size, I think 10MB dickens.txt is a good source to test multiple blocks and more similar to the size of a Linux kernel image. Does it make sense?

I'm open to choose a smaller one (like compression_66k_JSON.txt) if you insist. Cheers.

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 30, 2023

Yes, good point. Thanks for the PR!

@PSeitz PSeitz merged commit 9f4f79d into PSeitz:main Jan 30, 2023
@yestyle
Copy link
Copy Markdown
Contributor Author

yestyle commented Jan 30, 2023

Sweet! By the way, any clue when will be the next release publish to crates.io?

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Jan 30, 2023

Published a new version on crates.io https://crates.io/crates/lz4_flex

@yestyle
Copy link
Copy Markdown
Contributor Author

yestyle commented Jan 30, 2023

Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants