At the same time, we may revise how we differentiate between right and depth images in the database (they are saved under same field, only their type differentiate them):

Constraints:

1. Multi-cameras requires that all images across all cameras have same resolution
2. RGB images are stitched together (currently simplifying features extraction), though that could be changed (if so, that could remove constraint 1)
3. ...

That could help to support new workflows with RTAB-Map:

• (Thermal image) https://answers.ros.org/question/211785/how-to-generate-thermal-colored-map
• (Segmented image from CNN) How can I make certain color on the ground as an obstacle? rtabmap_ros#317 (comment)
• (Depth confidence image) http://official-rtab-map-forum.206.s1.nabble.com/Noisy-floor-with-ARKit-tp10530p10541.html
• (iOS missing surfaces) http://official-rtab-map-forum.206.s1.nabble.com/RTAB-vs-other-scanning-apps-for-iOS-td10615.html#a10703

Downstream changes to support this new feature:

• Revise all post-processing features using RGB and or depth images (RTAB-Map standalone, rtabmap-databaseViewer, rtabmap-export)
• Point cloud generation functions would need to be updated
• ROS1 and ROS2 wrappers would need to be updated
• Android/iOS code would need to be updated
• Add new ways to visualize all image types (e..g, ImageView)
• Add new options to generate point cloud data depending on the image layers we want and their effect. For example: I want to generate point cloud from only high confidence depth values and apply segmented class ID as color on it.