Currently we need to carry around components' metadata (e.g. ComponentName) everywhere, because the DataType in erased arrow arrays doesn't give us any information about the component type.
This is a real issue as it prevents using DataCell in more places (too much overhead if every cell carries a ComponentName).
The solution for this is arrow2's Extension datatype; but we need to improve on the ArrowField derive macro first, so that we can specify extension metadata easily... otherwise we'd have to manually implement ArrowField for everything.
Currently we need to carry around components' metadata (e.g.
ComponentName) everywhere, because theDataTypein erased arrow arrays doesn't give us any information about the component type.This is a real issue as it prevents using
DataCellin more places (too much overhead if every cell carries aComponentName).The solution for this is
arrow2'sExtensiondatatype; but we need to improve on theArrowFieldderive macro first, so that we can specify extension metadata easily... otherwise we'd have to manually implementArrowFieldfor everything.