Integration of https://github.com/orbbec/OrbbecSDK_v2

• Currently only tested with Astra2 camera
• Would close:
  • Support for Orbbec cameras #887
  • Orbbec SDK V2 Integration ? #1564

Build with Orbbec SDK v2:

cd rtabmap/build
cmake -WITH_ORBBEC_SDK=ON -DOrbbecSDK_DIR=/opt/OrbbecSDK_v2.4.11/lib ..
make -j4

Remaining tasks to do:

• Add #ifdef ... for build without Orbbec SDK support
• Test on Windows
  • To make Orbbec SDK v2 produce the right timestamps for the images, it is necessary to modify windows registry following these instructions: https://github.com/orbbec/OrbbecSDK_v2/blob/main/scripts/env_setup/obsensor_metadata_win10.md
• Test on Mac OS X
  • Follow same instructions for linux to install Orbbec SDK v2 on Mac OS X
  • When launching the orbbec examples or rtabmap, it may be necessary to launch in sudo if you have USB permission error like this (see also this libusb issue): libc++abi: terminating due to uncaught exception of type ob::Error: uvc_open failed: [Path: 1-2-1.0, Return Code: -3]
  • To deploy RTAB-Map, move orbbec build/install/lib/*and build/install/include/* to /user/local prefix, so that fixup_bundle tool can find the library. Note that while the app works before packaging, after packaging it seems missing components loaded dynamically (ob::Error: Component 8 not found!). We may need to install manually the build/install/lib/extensions folder in the bundle. I'll keep this fix for another time since the Mac OS X port of orbbec SDK is not officially released.
• Enable Inter IMU publishing when selecting orbbec sdk (UPDATE: removed as default, put back "sync" approach used like the other cameras)
• Support sync IMU publishing
• As they are still supporting https://github.com/orbbec/OrbbecSDK (version 1), doublecheck if there is a way to distinguish between OrbbecSDK v1 and OrbbecSDK v2 in cmake. They don't provide a version config file, so we cannot just do find_package(OrbbecSDK 2) unfortunately. I've set WITH_ORBBEC_SDK=OFF by default for now to not quietly try to build against an installed v1. The best fix would be to fix it upstream in OrbbecSDK v2 repo by adding something similar to this in their cmake (maybe here where the OrbbecSDKConfig.cmake is generated.):

  include(CMakePackageConfigHelpers)
  write_basic_package_version_file(
       "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
       VERSION ${PROJECT_VERSION}
       COMPATIBILITY AnyNewerVersion
  )
  install(
     FILES
       "${CMAKE_CURRENT_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/${PROJECT_NAME}Config.cmake"
       "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
    DESTINATION
       ${INSTALL_CMAKE_DIR}
  )

  • Other option is to look in this header: https://github.com/orbbec/OrbbecSDK_v2/blob/main/src/shared/environment/Version.hpp.in
  • UPDATE: sent pull request to fix it upstream: Generating OrbbecSDKConfigVersion.cmake file orbbec/OrbbecSDK_v2#107 (I'll require version 2 here now, so people would need to get that PR if they want to use OrbbecSDK with rtabmap).