Skip to content

Add efficient concatenation of StructArrays #7516

@asubiotto

Description

@asubiotto

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

We have observed excessive memory usage because StructArray concatenation falls back to concat_fallback which results in not merging nested dictionaries.

Describe the solution you'd like

Adding a concat_structs function to `concat.rs

Describe alternatives you've considered

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrowChanges to the arrow crateenhancementAny new improvement worthy of a entry in the changelog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions