Summary

This PR introduces a breaking change to dora-rerun that replaces the input-name-based visualization system with an explicit primitive-based metadata system. This change was motivated by real-world requirements from complex robotics applications, specifically the dora_quad port of the Peng quadrotor framework.

Key Changes

1. Core Architecture Change

• Before: Visualization type inferred from input names (e.g., input named "image" → image visualization)
• After: Explicit primitive field required in metadata (e.g., metadata: { "primitive": "image" })

2. Updated Dependencies

• Upgraded Rerun SDK from 0.23.3 to 0.24.0
• Updated Python dependency requirements

3. New Features

• Added 3D bounding box support with multiple formats
• Enhanced depth visualization with pinhole camera support
• Support for new visualization primitives

4. Migration Work

• Updated all node-hub packages to include primitive metadata:
  • opencv-video-capture → "primitive": "image"
  • dora-yolo → "primitive": "boxes2d"
  • dora-keyboard → "primitive": "text"
  • dora-distil-whisper → "primitive": "text"
  • dora-argotranslate → "primitive": "text"
  • dora-sam2 → "primitive": "masks"
  • dora-qwen2-5-vl → "primitive": "text"
  • dora-qwenvl → "primitive": "text"
  • dora-reachy2 → appropriate primitives for each output
  • dora-dav1d → "primitive": "image"
  • dora-rav1e → "primitive": "image"

5. Example Updates

• Fixed inconsistent use of -e flag in dataflow pip installs
• Successfully tested examples:
  • ✅ examples/rerun-viewer/dataflow.yml
  • ✅ examples/camera/dataflow_rerun.yml
  • ✅ examples/python-dataflow/dataflow.yml
  • ✅ examples/python-multi-env/dataflow.yml
• Updated but not tested (need specific hardware/dependencies):
  • 🔧 examples/keyboard/dataflow.yml (requires Linux/X11)
  • 🔧 examples/translation/*
  • 🔧 examples/reachy2-remote/dataflow_reachy.yml
  • 🔧 examples/lebai/graphs/dataflow_full.yml
  • 🔧 examples/av1-encoding/*

Breaking Changes

⚠️ All nodes sending data to dora-rerun must be updated to include the appropriate primitive metadata field. See the migration guide in the README for details.

NOTE: Last commit makes this soft-breaking now.

Benefits

• ✅ Flexible naming: Inputs can have descriptive names instead of type-based names
• ✅ Proper hierarchies: Entity paths correctly maintained in complex systems
• ✅ Multiple same-type inputs: Easy to have multiple cameras, sensors, etc.
• ✅ Extensibility: New primitives can be added without breaking existing code
• ✅ Better error messages: Clear reporting of missing/incorrect primitives

More details in this README.

Cons

• ❌ Breaking change: All nodes sending to dora-rerun must be updated
• ❌ More verbose: Requires metadata on every input
• ❌ Migration effort: Existing systems need code changes

Testing

• Tested on macOS with Python 3.11
• All tested examples work correctly with the new API
• Complex hierarchical visualizations verified with dora_quad integration
• I also have Ubuntu 22 with NVidia 4090 to further test