Description

This issue describes and lists the relevant PRs that allow subscriptions to receive messages (i.e. DynamicData) with descriptions/types constructed at runtime, and use that type to access fields of those messages without using message-specific compiled structs, as specified in ros-infrastructure/rep#358

Note: This does NOT include:

• Dynamic publishing: a feature where publishers can publish dynamic messages
  • (static pub -> dynamic sub works. But there is no notion yet of dynamic pub, so no way for dynamic pub -> static/dynamic sub yet.)
• Service/action support: And also no support for services or actions yet.
• Support for other middlewares other than FastRTPS

Also, this feature set only supports rmw_fastrtps currently.

Demoing This Feature

The status of this feature development is reflected in https://github.com/methylDragon/ros-type-introspection-prototype

⚠️⚠️⚠️ Take note that the repos file is modified to prevent the building of any middleware implementations other than FastDDS

You MUST not have sourced any other ROS 2 workspace or distros! The presence of any other middleware will trigger the building of rmw implementations, and since new rmw interfaces were defined, this will cause compilation failures.

git clone -b runtime_interface_reflection https://github.com/methylDragon/ros-type-introspection-prototype.git
cd ros-type-introspection-prototype
git submodule update --init --recursive

mkdir -p prototype_ws/src/ros2_repos
cd prototype_ws

vcs import src/ros2_repos < src/ros2.repos

rosdep install --from-paths src -y --ignore-src
source install/setup.bash
colcon build --packages-up-to dynamic_typesupport_examples ros2run

Then in two terminals:

source install/setup.bash
ros2 run dynamic_typesupport_examples static_pub

source install/setup.bash
ros2 run dynamic_typesupport_examples dynamic_sub_serialized

Feature Description

• A C unified serialization support library for run-time type reflection and construction, and utilities that support it
  • And the associated specific serialization support libraries
  • And C++ wrappers to manage memory for those C types (in rclcpp)
• A C library for manipulating the runtime type description message structs
• New rmw interfaces for:
  • Obtaining middleware-specific serialization support
  • Checking if a middleware supports:
    • Runtime type discovery
    • Taking message buffers as runtime type (i.e. DynamicData)
  • A new middleware agnostic runtime type rosidl_message_type_support_t implementation
• A new DynamicSubscription subscription class that supports a callback that takes dynamic data
  • Should support taking serialized messages and converting them into dynamic data for the callback
  • And also taking dynamic data directly from the middleware
• Necessary changes in rclcpp to allow the DynamicSubscription to work

Prototype serialization support libraries and implementations of this feature will be made for FastDDS in rmw, rcl, rclcpp, and rmw_fastrtps.

Implementation Considerations

See REP-2011 for the feature specification.
(The REP will have to be updated slightly since the implementation of this feature exposed certain necessary refinements.)

FastDDS was chosen for the prototype implementation because it is currently the default rmw for ROS 2.

Additionally, some other REP-2011 features have barebones/minimal implementations that were created to support this feature. As other subsets of REP-2011 get implemented (e.g. the type description message and its utilities), sections of this feature will need to be swapped out with those implementations, namely:

• The type description message's C struct and how it is used
• The usage of type hashing

Relevant PRs

The top level PRs need to be merged and built alongside each other. The demos repo (and setup instructions above) will ensure the proper branches are checked out for the build.

• Read the PRs alongside this branch, which provides the examples package:
  • https://github.com/methylDragon/ros-type-introspection-prototype/tree/runtime_interface_reflection
• Runtime Interface Reflection: type_description_interfaces_utils rcl_interfaces#155
• Runtime Interface Reflection: rosidl rosidl#728
• Runtime Interface Reflection: rosidl_dynamic_typesupport rosidl_dynamic_typesupport#1
  • After merging, merge: Add rosidl_dynamic_typesupport repo #1391
• Runtime Interface Reflection: rosidl_dynamic_typesupport_fastrtps rosidl_dynamic_typesupport_fastrtps#1
  • After merging, merge: Add rosidl_dynamic_typesupport_fastrtps repo #1392
• Runtime Interface Reflection: rmw rmw#340
  • ⚠️ Note: Merging this WILL break all rmw middleware implementations! (There are new interfaces that must be fulfilled, even if to put an unsupported warning) Be sure to merge the follow-on stub implementations
  • Runtime Interface Reflection: rmw_cyclonedds stubs for new RMW interfaces rmw_cyclonedds#447
  • Runtime Interface Reflection: rmw_connextdds stubs for new RMW interfaces rmw_connextdds#111
• Runtime Interface Reflection: rmw_fastrtps rmw_fastrtps#655
• Runtime Interface Reflection: rcl rcl#1025
• Runtime Interface Reflection: rclcpp rclcpp#2077
  • Depends on: Runtime Interface Reflection: Dynamic Typesupport Wrappers rclcpp#2134
• Runtime Interface Reflection: rmw_implementation rmw_implementation#215

Interim Stubs
These are so we can get in ABI stable backports (to tide us over the freeze)

• Dynamic Subscription (REP-2011 Subset): Stubs for rclcpp rclcpp#2165
• Implement deliver message kind rclcpp#2168

Stub Impl

• Fix delivered message kind typo rclcpp#2175
• Runtime Interface Reflection: rclcpp (Rebased) rclcpp#2176

TODOs

• Initial implementation
• Taking dynamic data directly
• Minor: Clarify if I should be calling it dynamic_type/dynamic_data or runtime_type/runtime_data
• type description utils
• Migrate rosidl_dynamic_typesupport to use type description
• BONUS: defer construction till build()
• rclcpp wrappers for dynamic typesupport structs
• Populate rmw impl virtual methods (so the other rmw implementations can build)
• Linting (I am deliberately leaving this till last because linting will cause the serialization support library interface and implementation PRs to become prohibitively long)

When this feature is considered more stable:

• Tests
• Separate serialization_support_lib (and its associate libraries) out into their own ros2 repositories
• Update ros2 examples

Additional Waves

• Dynamic Subscription (REP-2011 subset): BONUS: Allocator Refactor #1405

Additional notes/issues

const correctness and lifetime management of rosidl_message_type_support_t

The rosidl_message_type_support_t was, in all cases prior to this feature, populated and used as a compile-time static objects.

This feature makes it so that users can generate and populate their own type support structs, which means that they will have to manage its lifetime.

Furthermore, the const void * typesupport implementation of the struct means that runtime mutation of the typesupport implementation is not logically const correct even though it is physically const correct. This is troublesome because mutation of the typesupport is necessary for deferring the getting of dynamic types.

FastDDS Issues

• Sequences of nested messages WILL NOT WORK
  • Sequences and Arrays of Nested Structs Cause Segfault on Pub-Sub for Dynamic Types (with minimal examples) eProsima/Fast-DDS#2969
  • [15446] TypeObject for sequences/arrays/maps of nested types crashes eProsima/Fast-DDS#3296
• Minor (?): Double free on type destruction
  • Double Free for DynamicData for Sequences of Nested Structs (with minimal example) eProsima/Fast-DDS#2970

For Posterity

If you want to add new (C compatible) serialization support functionality, follow these steps!