Skip to content

Add Field::size and DataType::size #3147

@crepererum

Description

@crepererum

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In DataFusion, it would be nice to know how much data is allocated so we can bail out early instead of OOMing (a slight over-allocation is OK, so this can be measured after the fact). For that purpose, it would be nice to know how much memory a specific instance of Field/DataType requires.

Describe the solution you'd like
Add:

impl Field {
    /// Return size of this instance in bytes.
    ///
    /// Includes the size of `Self`.
    pub fn size(&self) -> usize {...}
}

impl DataType {
    /// Return size of this instance in bytes.
    ///
    /// Includes the size of `Self`.
    pub fn size(&self) -> usize {...}
}

Note that fields and data types have a cyclic dependency, so we probably need to implement both in a single PR.

Describe alternatives you've considered
Not adding this feature to arrow but let downstream users (e.g. DataFusion) do this.

Additional context
-

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrowChanges to the arrow crateenhancementAny new improvement worthy of a entry in the changelogparquetChanges to the parquet crate

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions