Skip to content

Component manager load node service queue overflows and fails to launch some nodes #2362

@xmfcx

Description

@xmfcx

Bug report

Required Info:

  • Operating System:
    • Ubuntu 22.04
  • Installation type:
    • binaries
  • Version or commit hash:
    • humble
  • DDS implementation:
    • CycloneDDS
  • Client library (if applicable):
    • rclcpp_components

Steps to reproduce issue

import launch
from launch_ros.actions import ComposableNodeContainer
from launch_ros.descriptions import ComposableNode
from launch_ros.actions import LoadComposableNodes


def generate_launch_description():
    container = ComposableNodeContainer(
            name='my_container',
            package='rclcpp_components',
            executable='component_container',
            namespace='',
            output='log',
    )

    loaders = []
    for i in range(30):
      
        node = ComposableNode(
            package='composition',
            plugin='composition::Talker',
            name='talker_' + str(i),
        )
        
        loaders.append(
          LoadComposableNodes(
                composable_node_descriptions=[node],
                target_container="my_container",
            )
        )
    
    return launch.LaunchDescription([container, *loaders])
  • Save this launch file to a file called composition_overload.py.
  • Source ROS 2 humble: source /opt/ros/humble/setup.bash
  • Run the launch file: ros2 launch composition_overload.py
  • Run ros2 component list

Expected behavior

To see all talkers loaded into the container (30 for this case):

$ ros2 component list
/my_container
  1  /talker_0
  2  /talker_1
  3  /talker_2
  4  /talker_3
  5  /talker_4
... (omitted for clarity)
 25  /talker_24
 26  /talker_25
 27  /talker_26
 28  /talker_27
 29  /talker_28
 30  /talker_29

Actual behavior

About 2/3 is loaded.

$ ros2 component list
/my_container
  1  /talker_0
  2  /talker_1
  3  /talker_2
  4  /talker_3
  5  /talker_4
  6  /talker_11
  7  /talker_12
  8  /talker_28
  9  /talker_20
  10  /talker_21
  11  /talker_22
  12  /talker_23
  13  /talker_24
  14  /talker_19
  15  /talker_25
  16  /talker_26
  17  /talker_27
  18  /talker_29

Additional information

@VRichardJP has found out that

loadNode_srv_ = create_service<LoadNode>(
"~/_container/load_node",
std::bind(&ComponentManager::on_load_node, this, _1, _2, _3));
was the cause of this issue.

Simultaneous calls fill up the QoS queue of the loadNode_srv_ service, which has 10 queue size by default.

Related issue:

Potential solutions

Solution A

Increase the default queue size for these load time critical services (loadNode_srv_, unloadNode_srv_) to a large number like 1000?
Pros:

  • This wouldn't change the interface
  • Probably higher numbers won't be needed, since I think this is the first time this issue is reported?
  • The service LoadNode.srv is a small service, so having a lot of items to the queue doesn't increase the computational burden a lot.

Cons:

  • If for some reason someone needs a larger queue, we would have to repeat this issue.
Solution B

Parameterize the queue size for loadNode_srv_, unloadNode_srv_.
Pros:

  • Can be set to any number by the user
  • With default values, it would act the same

Cons:

  • Now it takes a parameter, launch interface changes (backward compatible)

We are open to any other solutions, ideas and additional design considerations while implementing this to rolling.

cc. @mitsudome-r @isamu-takagi @wep21 @esteve

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions