Skip to content

Fix: Handle null values in PlainFixedLenByteArrayEncoder gracefully#320

Merged
zeroshade merged 3 commits intoapache:mainfrom
singh1203:ByteArrayEncoder
Mar 24, 2025
Merged

Fix: Handle null values in PlainFixedLenByteArrayEncoder gracefully#320
zeroshade merged 3 commits intoapache:mainfrom
singh1203:ByteArrayEncoder

Conversation

@singh1203
Copy link
Copy Markdown
Contributor

Fixes: #71

Rationale for this change

Ensures PlainFixedLenByteArrayEncoder handles null values gracefully in non-nullable fields by writing zero-filled bytes instead of panicking, maintaining consistency with DictFixedLenByteArrayEncoder.

What changes are included in this PR?

  • Replaces panics on null values with writing zero-filled bytes to maintain byte alignment.
  • Ensures encoding consistency between dictionary and plain encoding.
  • Prevents unexpected failures when handling schema inconsistencies.

Are these changes tested?

Yes:

  • Passed existing test suite.
  • Verified with local Arrow-Go modifications.

Are there any user-facing changes?

Not Sure (Need possible guidance if required)

Signed-off-by: Saurabh Kumar Singh <singh1203.ss@gmail.com>
@singh1203 singh1203 changed the title fix: (GH-#71) Handle null values in PlainFixedLenByteArrayEncoder Fix: Handle null values in PlainFixedLenByteArrayEncoder gracefully Mar 20, 2025
@singh1203 singh1203 marked this pull request as ready for review March 20, 2025 12:57
@singh1203 singh1203 requested a review from zeroshade as a code owner March 20, 2025 12:57
Copy link
Copy Markdown
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a new test that would fail without this change?

@singh1203
Copy link
Copy Markdown
Contributor Author

Can we add a new test that would fail without this change?

Sure @zeroshade I'm on it.

Signed-off-by: Saurabh Kumar Singh <singh1203.ss@gmail.com>
@singh1203 singh1203 requested a review from zeroshade March 24, 2025 06:18
@zeroshade zeroshade merged commit 601311b into apache:main Mar 24, 2025
22 of 23 checks passed
@singh1203
Copy link
Copy Markdown
Contributor Author

Thank you. @zeroshade

@singh1203 singh1203 deleted the ByteArrayEncoder branch March 24, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Go][Parquet] PlainFixedLenByteArrayEncoder behaves differently from DictFixedLenByteArrayEncoder with null values where schema has Nullable: false

2 participants