Skip to content

[C++][Parquet] Expose the API to fine tune the window_bits parameter of ZLIB compression  #35287

@yyang52

Description

@yyang52

Describe the enhancement requested

ZLIB library supports history buffers of different sizes by setting windowBits parameter, while currently in Arrow it is set as a fixed value. It makes sense that setting the window size to the maximum number will give a better performance, while it does not provide too much flexibility. As we know there would be some scenarios where the memory efficiency is very limited, or the software stack is relatively old and do not have a large memory capacity. For those cases, the user may want to set the window bits to a small number to save some memory. At least it would be much more flexible to give users such an option to choose the window size when de/compressing. If that makes sense, we would like to add such a property for the user to set the ZLIB(GZIP) window_bits, and will ensure not affecting the existing function calls. Thanks!

Component(s)

C++, Parquet

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions