We are now not too far from being able to support generic bit-sizes (e.g. a 64bit Point3D or a 16bit Scalar) for all things backed by primitive types or arrays of primitive types, for two reasons:
- The first reason is that
Chunks allow it, and therefore the ChunkStore also (mostly) allows it.
- The second reason is that space views now mostly work with primitive-casted data, e.g.:
let data = re_query2::range_zip_1x5(
all_positions_indexed,
all_colors.primitive::<u32>(),
all_radii.primitive::<f32>(),
all_labels.string(),
all_class_ids.primitive::<u16>(),
all_keypoint_ids.primitive::<u16>(),
)
I.e. one can easily imagine casting to different native types based on the current arrow datatype (likely tedious, but also probably straightforward in most cases).
Note that I'm only talking about bit-size transformations here, nothing beyond that, and definitely nothing that involves changes in semantics (so e.g. Color is likely out).
Still, even that kind of limited change will likely have ripple effects across the entire stack, and those need to be planned for appropriately:
- What's the impact on our data model (interactions with all things tensor-like in particular, probably also all things transforms).
- What's the impact on logging APIs: C++, Python & Rust.
- What's the impact on our query APIs.
- What's the impact on the different space views implementation (e.g. I imagine a 64bit vertex would not make the renderer happy, so some transformation is required, maybe?).
- What other impact? E.g. wrt blueprint defaults/overrides?
Looking further into the future, what about generic bit-sizes for timeline values?
We are now not too far from being able to support generic bit-sizes (e.g. a 64bit
Point3Dor a 16bitScalar) for all things backed by primitive types or arrays of primitive types, for two reasons:Chunks allow it, and therefore theChunkStorealso (mostly) allows it.I.e. one can easily imagine casting to different native types based on the current arrow datatype (likely tedious, but also probably straightforward in most cases).
Note that I'm only talking about bit-size transformations here, nothing beyond that, and definitely nothing that involves changes in semantics (so e.g.
Coloris likely out).Still, even that kind of limited change will likely have ripple effects across the entire stack, and those need to be planned for appropriately:
Looking further into the future, what about generic bit-sizes for timeline values?