Skip to content

lz4_block exposes option to reuse compression dict#207

Merged
PSeitz merged 3 commits intoPSeitz:mainfrom
matthewfollegot:main
Mar 6, 2026
Merged

lz4_block exposes option to reuse compression dict#207
PSeitz merged 3 commits intoPSeitz:mainfrom
matthewfollegot:main

Conversation

@matthewfollegot
Copy link
Copy Markdown
Contributor

Adding compress_into_with_table to avoid the HashTable allocation every time compress_into is called. In my service, I'm compressing small collection very often and when looking at CPU profiles, CPUs spend as much time allocating as they do in compress_internal.

@PSeitz
Copy link
Copy Markdown
Owner

PSeitz commented Mar 6, 2026

Thanks! This may be also useful for the frame compressor if the blocksize is small

@PSeitz PSeitz merged commit 7191df8 into PSeitz:main Mar 6, 2026
2 checks passed
@matthewfollegot
Copy link
Copy Markdown
Contributor Author

Thanks! This may be also useful for the frame compressor if the blocksize is small

Thanks for the prompt review! Good point, I see it's reused there but it's defaulting to HashTable4K. I'll whip something up to leverage this, but I think we need CompressTable to implement HashTable at least to support this, hopefully it isn't too ugly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants