Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In DataFusion, it would be nice to know how much data is allocated so we can bail out early instead of OOMing (a slight over-allocation is OK, so this can be measured after the fact). For that purpose, it would be nice to know how much memory a specific instance of Field/DataType requires.
Describe the solution you'd like
Add:
impl Field {
/// Return size of this instance in bytes.
///
/// Includes the size of `Self`.
pub fn size(&self) -> usize {...}
}
impl DataType {
/// Return size of this instance in bytes.
///
/// Includes the size of `Self`.
pub fn size(&self) -> usize {...}
}
Note that fields and data types have a cyclic dependency, so we probably need to implement both in a single PR.
Describe alternatives you've considered
Not adding this feature to arrow but let downstream users (e.g. DataFusion) do this.
Additional context
-
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In DataFusion, it would be nice to know how much data is allocated so we can bail out early instead of OOMing (a slight over-allocation is OK, so this can be measured after the fact). For that purpose, it would be nice to know how much memory a specific instance of
Field/DataTyperequires.Describe the solution you'd like
Add:
Note that fields and data types have a cyclic dependency, so we probably need to implement both in a single PR.
Describe alternatives you've considered
Not adding this feature to arrow but let downstream users (e.g. DataFusion) do this.
Additional context
-