Skip to content

Add DictionaryArray::gc method #7683

@davidhewitt

Description

@davidhewitt

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Related to #7185. It would be nice to also be able to compact dictionary arrays on IPC write. A first step would be to add a DictionaryArray::gc method.

Describe the solution you'd like

Calling this method would rebuild the dictionary's values array such that all unreferenced values are discarded. The keys array will need rebuilding too given the indices of the values array will be changed by the gc.

A relatively trivial implementation will just use dictionary builders to create new arrays, feeding the builder all current values and letting it create a new keys array. I don't know if that's considered prohibitive for code bloat reasons due to the combination of K and V generic parameters.

Describe alternatives you've considered

None so far.

Additional context

None so far.

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrowChanges to the arrow crateenhancementAny new improvement worthy of a entry in the changelog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions