I had an email discussion with @v4hn and have been thinking that child frames on attached objects would be useful for parametrizing manipulations. It would make it easy to code "Place the tip of this peg at the hole of that bearing". Sadly, the planning scene does not really support this, since all collision objects live directly in the world, have only one frame, and no relations between each other.

Below, I am proposing some changes to allow writing operations like "Put the screw driver tip on the screw head" more clearly.

I read through some of the code, and my understanding is this: In moveit_core/collision_world, Object is defined as a list of shapes. In moveit_core/robot_state, AttachedBody is defined as (essentially) an object with a parent link, a transformation from the parent link, and touch links (for which collisions are allowed). The moveit_msgs CollisionObject and AttachedCollisionObject correspond to these. planning_scene_interface deals with adding/removing objects to/from the world, and attaching/detaching objects to/from the robot (and keeping the world up to date).

The simplest options I see are:

A) Add a list of connected objects to the Object struct (and the AttachedBody class). When an object is attached to a robot link or moved, all connected objects are moved/attached as well. This is the simplest way, so I would choose this.

B) Add a list of child frames/transformations to Object. This doesn't allow for connecting objects (during assembly tasks), so it feels like a half-measure to me. It could be combined with A, but it requires at least one new method to get the transformation to a child frame, and I am not sure this would be systematic.

On a related note, I noticed that in the AttachableObject in Tesseract (corresponds to MoveIt's CollisionObject), VisualGeometry and CollisionGeometry are defined separately (the latter containing a flag for bullet). Since it was mentioned at the maintainer meeting that bullet's continuous collision checking for convex objects should be ported to MoveIt at some point, I would suggest to add those types as well. I also personally find them quite readable.

Lastly, it would be convenient if move_group_interface allowed the use of attached objects' object_id (and possibly their child links) as a valid end_effector_link. In that case, the target_pose would be transformed from the AttachedCollisionObject to the object's parent_link, and the constraint formulated for that. If I am seeing right, it would only need a change here.

Let me know if this sounds reasonable or if you can already see pitfalls that I am overlooking. Thanks in advance.