Skip to content

[C++][Parquet] DeltaBitPackDecoder expects all miniblock bitwidths to be present for the last block #14923

@rok

Description

@rok

Describe the bug, including details regarding any error messages, version, and platform.

This is a follow-up to this discussion.
Arrow's reader is not compliant with the spec, and requires all miniblock bitwidth values of the last block even if not necessary. This forces arrow writer to write all miniblock bitwidths to be able to roundtrip which is not a big issue. However it does mean Arrow reader would not be able to read bitpacked buffer written by another parquet writer that does not write all miniblock bitwidths.

Component(s)

C++, Parquet

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions