Feature request

@azeey @IanTheEngineer @cottsay FYI

Feature description

This is a ticket to use Pybind11 for the CPython extension instead of manually writing code to interact with the CPython API.

Implementation considerations

Plan: https://docs.google.com/document/d/1BsTlxgzAkpX-5Q7FW6_Hyw1fszDEQef_OEvYmdnrBrE/edit?usp=sharing

• Replace custom CMake with Pybind11's pybind11_add_module (Use Pybind11's CMake code #667)
  • Investigate CMake issues in Windows Debug Use Pybind11's CMake code #667 (comment)
  • Fix windows warning reported in 👨‍🌾 Warning in windows packaging jobs "ignoring '/INCREMENTAL' due to '/LTCG' specification" #687, (Clean windows flags on _rclpy_pybind11 and _rclpy_action #688)
  • Remove code unsetting Py_DEBUG definition once all modules use Pybind11 (Copy windows debug fixes for pybind11 #681 (comment)) Use pybind11 for signal handling, and delete now unused rclpy_common, pycapsule, and handle code #814
• Convert each module to use Pybind11 with the existing C to Python API
  • _rclpy (Split rclpy module for easier porting to pybind11 #675 ✔️, rclpy_ok and rclpy_create_context to pybind11 #691 ✔️, Convert guard condition functions to pybind11 #692 ✔️, Convert timer functions to pybind11 #693 ✔️, Convert service info functions to pybind11 #694 ✔️, Convert publisher functions to pybind11 #695 ✔️, Convert subscription functions to pybind11 #696 ✔️, Convert clock and time functions to pybind11 #699 ✔️, Convert rclpy client functions to pybind11 #701 ✔️, Convert rclpy service functions to pybind11 #703 ✔️, Convert wait_set functions to pybind11 #706 ✔️ , Use Pybind11 for name functions #709 ✔️ , Convert serialize/deserialize to pybind11 #712 ✔️ , Convert init/shutdown to pybind11 #715 ✔️ , Convert names_and_types graph APIs to pybind11 #717 ✔️ , Pybind11-ify rclpy_get_node_parameters #718 ✔️ , [pybind11] Node Accessors #719 ✔️, Convert take API to pybind11 #721 ✔️ , Migrate qos event APIs to pybind11 #723 ✔️, Convert more node accessors to pybind11 #730 ✔️ , Pybind 11: count_subscribers and count_publishers #732 ✔️ , Convert last of pub/sub getters to pybind11 #733 ✔️, Convert logging mutex functions to pybind11 #735 ✔️, Convert QoS APIs to pybind11 #736 ✔️, Converting last of _rclpy.c to pybind11 #738 ✔️)
  • _rclpy_action (Use pybind11 for _rclpy_action #678)
  • _rclpy_handle (Use pybind11 for _rclpy_handle #668)
  • _rclpy_logging (Use Pybind11 to generate _rclpy_logging #659)
  • _rclpy_pycapsule (Update just pycapsule lib to use pybind11 #652)
  • _rclpy_signal_handler (Implement signal handling in Python #728)
    • skipping - uses C atomic types that can't be used with C++ due to p0943r1 Use pybind11 for signal handling, and delete now unused rclpy_common, pycapsule, and handle code #814
    • Actually, couldn't skip because it uses rclpy_common, so rewrote in Python.
  • _rclpy_common (Common library, not a module, started in Split rclpy module for easier porting to pybind11 #675 ✔️, Remove pybind11 from rclpy common #727 ✔️, Combine pybind11 modules into one #743) Use pybind11 for signal handling, and delete now unused rclpy_common, pycapsule, and handle code #814
• Replace checking rcl_ret_t with void throw_if_not_ok(rcl_ret_t ret, const std::string & extra_error_msg);
  • Or use RCLError(), like Clean up exceptions in _rclpy_action #685 does for _rclpy_action
• Refactor rclpy objects to use C++ classes one object at a time, leaf objects first
  • rcl_time_point_t -> bind struct Use py::class_ for rcl_time_point_t #737
  • rmw_service_info_t -> bind struct (Use py::class_ for rmw_service_info_t and rmw_request_id_t #748)
  • rcl_duration_t -> bind struct (claimed) Use py::class_ for rcl_duration_t #744
  • rmw_qos_profile_t -> bind struct (Use py::class_ for rmw_qos_profile_t #741)
  • rmw_request_id_t -> bind struct (Use py::class_ for rmw_service_info_t and rmw_request_id_t #748)
  • rcl_wait_set_t -> not sure if rclpy::WaitSet or bind struct (maybe tricky) (Convert WaitSet to use C++ Classes #769)
  • rclpy_client_t -> rclpy::Client (Convert Client to use C++ Class #739)
  • rclpy_service_t -> rclpy::Service (Convert Service to use C++ Class #747)
  • rcl_guard_condition_t -> rclpy::GuardCondition Convert Guardcondition to use C++ classes #772 (for H-turtle branch)
  • rcl_event_t -> rclpy::Event (Use py::class_ for rcl_event_t #750)
  • rclpy_publisher_t -> rclpy::Publisher (Convert Publisher and Subscription to use C++ Classes #756)
  • rclpy_subscription_t -> rclpy::Subscription (Convert Publisher and Subscription to use C++ Classes #756)
  • rcl_timer_t -> rclpy::Timer (Convert Timer to use a C++ Class #745)
  • rcl_clock_t -> rclpy::Clock (Convert Clock to use a C++ Class #749)
  • rcl_action_client_t -> rclpy::ActionClient (Convert ActionClient to use C++ classes #759)
  • rcl_action_goal_handle_t -> rclpy::ActionGoalHandle (Use py::class_ for rcl_action_goal_handle_t #751, Pybind11 action goal handle nitpicks #767)
  • rcl_action_server_t -> rclpy::ActionServer (Convert ActionServer to use C++ Classes #766)
  • rcl_node_t -> rclpy::Node (Convert Node and Context to use C++ Classes #771)
  • rcl_context_t -> rclpy::Context (Convert Node and Context to use C++ Classes #771)

Refactor order: Arrows point from a type that needs to be refactored before the type it points to. Types with no arrows pointing them can be done as soon as all methods using them are converted to pybind11.

digraph order_of_conversions {
rcl_time_point_t
rcl_wait_set_t
rmw_qos_profile_t
rmw_request_id_t
rmw_service_info_t
rcl_duration_t
action_client -> rcl_node_t
action_server -> rcl_node_t
rcl_action_goal_handle_t -> action_server
rcl_node_t -> context
subscription -> rcl_node_t
publisher -> rcl_node_t
service -> rcl_node_t
client -> rcl_node_t
timer -> context
timer -> clock
clock
guard_condition -> context
rcl_event_t -> publisher
rcl_event_t -> subscription
}