Skip to content

Conversation

@Blizzara
Copy link
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

The bump to Arrow 55.1 brings with it apache/arrow-rs#7247. That causes runtime errors using ScalarStructBuilder if the struct is empty.

What changes are included in this PR?

Use try_new_with_length instead of try_new to allow empty struct. Our length is always 1 as we're building a scalar.

The try_new_with_length also does validation on the provided length against the arrays, so we can remove our validation.

Are these changes tested?

Existing tests for non-empty structs + a new unit test for empty struct.

Are there any user-facing changes?

The error thrown for a wrong-sized array is now different, as it comes from Arrow instead of being a DF InternalError. The Arrow error has some benefits, e.g. it mentions the field name, and also some downsides (I think Arrow doesn't include backtraces, tho I might be wrong):

Err(ArrowError::InvalidArgumentError(format!(
                    "Incorrect array length for StructArray field {:?}, expected {} got {}",
                    f.name(),
                    len,
                    a.len()
                )))

@github-actions github-actions bot added the common Related to common crate label May 28, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. Thank you @Blizzara

@xudong963 xudong963 merged commit 72f4eab into apache:main May 29, 2025
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants