ORB-SLAM3 IMU support fixes by matlabbe · Pull Request #1612 · introlab/rtabmap

matlabbe · 2025-11-09T05:51:54Z

Follow-up of #1152 to support IMU modes.

Now using this ORB_SLAM3 fork: https://github.com/matlabbe/ORB_SLAM3

Online realsense D435i 640x480x30 IR-Depth mode + IMU
- Have to make non-stop circle motion at the beginning so that the tracker doesn't reset every 2 seconds.
Online realsense D435i 640x480x30 Stereo IR mode + IMU
- Have to make non-stop circle motion at the beginning so that the tracker doesn't reset every 2 seconds.

rtabmap-euroc_dataset tool working:

rtabmap-euroc_dataset        \
    --Rtabmap/PublishRAMUsage true       \
    --Rtabmap/DetectionRate 2       \
    --RGBD/LinearUpdate 0       \
    --RGBD/AngularUpdate 0  \
    --Odom/Strategy 5 \
    --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt  \
    --OdomORBSLAM/Inertial true \
    --Mem/STMSize 30 \
    --uwarn   \
    ~/Downloads/euroc/MH_01_easy

Need to doublecheck the conversion of ORB-SLAM pose in base_link. EDIT: confirmed:

Peek.2025-11-11.12-01.mp4

ROS2 example

# RGB-D mode
ros2 launch rtabmap_examples realsense_d435i_infra.launch.py odom_args:="\
    --OdomORBSLAM/Inertial false \
    --Odom/Strategy 5 \
    --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt"

# Stereo mode
ros2 launch rtabmap_examples realsense_d435i_stereo.launch.py odom_args:="\
    --OdomORBSLAM/Inertial false \
    --Odom/Strategy 5 \
    --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt"

# IMU-RGB-D mode
ros2 launch rtabmap_examples realsense_d435i_infra.launch.py odom_args:="\
    --OdomORBSLAM/Inertial true \
    --Odom/Strategy 5 \
    --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt"

# IMU-Stereo mode
ros2 launch rtabmap_examples realsense_d435i_stereo.launch.py odom_args:="\
    --OdomORBSLAM/Inertial true \
    --Odom/Strategy 5 \
    --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt"

Installation

~~Tested with this patch orbslam3_v1-0_release_rtabmap_fix.patch~~ EDIT: using this fork now:

git clone -b v0.9.4  https://github.com/stevenlovegrove/Pangolin.git 
cd Pangolin && \
  mkdir build && \
  cd build && \
  cmake .. && \
  make -j6
sudo make install

git clone https://github.com/matlabbe/ORB_SLAM3.git 
cd ORB_SLAM3
mkdir build && \
  cd build && \
  cmake .. && \
  make -j6

git clone https://github.com/introlab/rtabmap.git
cd rtabmap
export ORB_SLAM_ROOT_DIR=~/ORB_SLAM3
cd build && \
  cmake -DWITH_G2O=OFF -DWITH_ORB_SLAM=ON .. && \
  make -j6

Tip: It seems that ORB_SLAM3 requires some kind of motion to initialize correctly with the IMU. Note sure if it is hardware related, but here what I need to do to make it work with a D435i. If we don't move enough at the beggining, ORB_SLAM3 will keep re initializing like this:

Peek.2025-11-11.11-42.mp4

Here is an example of initial motion that will avoid ORB_SLAM3 to re initialize (at 16 seconds mark in this video) after we stop moving:

Peek.2025-11-11.11-44.mp4

… possible.

…u_fixes

liangyuan109 · 2025-12-29T02:51:19Z

hallo,i have a problem.
why am I still getting the following error even after installing rtabmap orbslam3 as described above?

[ERROR] [stereo_odometry-2]: process has died [pid 34992, exit code -11, cmd '/home/ly/workspace/rtabmap_orbslam3_ros2_rtabmap-ros2_project/install/rtabmap_odom/lib/rtabmap_odom/stereo_odometry --OdomORBSLAM/Inertial false --Odom/Strategy 5 --OdomORBSLAM/VocPath /home/ly/workspace/rtabmap_orbslam3_ros2_project/src/ORB_SLAM3/Vocabulary/ORBvoc.txt --ros-args -p depth_module.emitter_enabled:=0 --params-file /tmp/launch_params_y84z64fc -r imu:=/imu/data -r left/image_rect:=/camera/infra1/image_rect_raw -r left/camera_info:=/camera/infra1/camera_info -r right/image_rect:=/camera/infra2/image_rect_raw -r right/camera_info:=/camera/infra2/camera_info'].

liangyuan109 · 2025-12-29T03:12:00Z

could you list which versions of the libraries you used?

matlabbe · 2026-01-06T02:22:48Z

This was probably tested on ROS2 humble (ubuntu 22.04), which ROS2 version are you using?

liangyuan109 · 2026-01-07T02:56:08Z

This was probably tested on ROS2 humble (ubuntu 22.04), which ROS2 version are you using?

yes,now using ROS2 humble ubuntu 22.04. I tried thousands of times, but I still couldn't succeed.

matlabbe · 2026-01-12T02:08:10Z

Here is a minimal example to get rtabmap_ros built with ORB_SLAM3 support. Build this docker image with this Dockerfile:

FROM osrf/ros:humble-desktop-full

WORKDIR /root

# Dependency of Pangolin
RUN apt update && apt install libepoxy-dev && \
    apt-get clean && rm -rf /var/lib/apt/lists/
    
RUN git clone -b v0.9.4  https://github.com/stevenlovegrove/Pangolin.git && \
    cd Pangolin && \
    mkdir build && \
    cd build && \
    cmake -DBUILD_PANGOLIN_PYTHON=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TOOLS=OFF .. && \
    make -j6 && \
    make install && \
    cd ~ && \
    rm -rf Pangolin

RUN git clone https://github.com/matlabbe/ORB_SLAM3.git && \
    cd ORB_SLAM3 && \
    mkdir build && \
    cd build && \
    cmake .. && \
    make -j6 && \
    cd .. && \
    rm -r build

RUN rm /bin/sh && ln -s /bin/bash /bin/sh

RUN source /opt/ros/$ROS_DISTRO/setup.bash && \
    git clone https://github.com/introlab/rtabmap.git && \
    export ORB_SLAM_ROOT_DIR=~/ORB_SLAM3 && \
    cd rtabmap/build && \
    cmake -DWITH_G2O=OFF -DWITH_ORB_SLAM=ON .. && \
    make -j6 && \
    make install && \
    cd ~ && \
    rm -rf rtabmap

ENV LD_LIBRARY_PATH=/root/ORB_SLAM3/lib:/root/ORB_SLAM3/Thirdparty/g2o/lib:$LD_LIBRARY_PATH

RUN source /opt/ros/$ROS_DISTRO/setup.bash && \
    mkdir -p ros2_ws/src && \
    cd ros2_ws && \
    git clone https://github.com/introlab/rtabmap_ros.git src/rtabmap_ros && \
    export MAKEFLAGS="-j6" && \
    rosdep update && \
    apt-get update && \
    rosdep install --from-paths src --ignore-src -r -y -t build_export -t test -t build -t buildtool_export -t buildtool --skip-keys="rtabmap" && \
    apt-get clean && rm -rf /var/lib/apt/lists/ && \
    colcon build --install-base /opt/ros/humble --merge-install --cmake-args -DCMAKE_BUILD_TYPE=Release && \
    cd && \
    rm -rf ros2_ws

# For the sake of the example, we need realsense2 driver
RUN source /opt/ros/$ROS_DISTRO/setup.bash && \
    apt-get update && \
    apt install -y \
    ros-humble-realsense2-camera \
    ros-humble-imu-filter-madgwick && \
    apt-get clean && rm -rf /var/lib/apt/lists/

ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/ros/humble/lib/x86_64-linux-gnu

RUN cd ~/ORB_SLAM3/Vocabulary && tar -xzf ORBvoc.txt.tar.gz

So from a terminal, copy this file under a file named Dockerfile in an empty directory (e.g., rtab_orbslam3_ros2/Dockerfile):

cd  rtab_orbslam3_ros2
docker build -t rtab_orbslam3_ros2 .

Try it with a D435i camera. Note that the IMU may not be accessible from inside the container, we have to add cgroup when launching docker. To find cgroup for the IMU (the number is 10 on my computer):

ls -l /dev | grep HID
crwxrwxrwx   1 root    root     10,   121 Jan 11 17:32 HID-SENSOR-2000e1.4.auto

Enable host display access:

xhost +local:root

Launch the example (adjust the cgroup with the number above if needed):

docker run -it --rm \
    --net=host \
    --privileged \
    -e DISPLAY=$DISPLAY \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    --device-cgroup-rule 'c 10:* rmw' \
    rtab_orbslam3_ros2 \
    ros2 launch rtabmap_examples realsense_d435i_infra.launch.py odom_args:="\
        --OdomORBSLAM/Inertial false \
        --Odom/Strategy 5 \
        --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt"

liangyuan109 · 2026-01-12T08:57:35Z

Here is a minimal example to get rtabmap_ros built with ORB_SLAM3 support. Build this docker image with this Dockerfile:

FROM osrf/ros:humble-desktop-full

WORKDIR /root

# Dependency of Pangolin
RUN apt update && apt install libepoxy-dev && \
    apt-get clean && rm -rf /var/lib/apt/lists/
    
RUN git clone -b v0.9.4  https://github.com/stevenlovegrove/Pangolin.git && \
    cd Pangolin && \
    mkdir build && \
    cd build && \
    cmake -DBUILD_PANGOLIN_PYTHON=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TOOLS=OFF .. && \
    make -j6 && \
    make install && \
    cd ~ && \
    rm -rf Pangolin

RUN git clone https://github.com/matlabbe/ORB_SLAM3.git && \
    cd ORB_SLAM3 && \
    mkdir build && \
    cd build && \
    cmake .. && \
    make -j6 && \
    cd .. && \
    rm -r build

RUN rm /bin/sh && ln -s /bin/bash /bin/sh

RUN source /opt/ros/$ROS_DISTRO/setup.bash && \
    git clone https://github.com/introlab/rtabmap.git && \
    export ORB_SLAM_ROOT_DIR=~/ORB_SLAM3 && \
    cd rtabmap/build && \
    cmake -DWITH_G2O=OFF -DWITH_ORB_SLAM=ON .. && \
    make -j6 && \
    make install && \
    cd ~ && \
    rm -rf rtabmap

ENV LD_LIBRARY_PATH=/root/ORB_SLAM3/lib:/root/ORB_SLAM3/Thirdparty/g2o/lib:$LD_LIBRARY_PATH

RUN source /opt/ros/$ROS_DISTRO/setup.bash && \
    mkdir -p ros2_ws/src && \
    cd ros2_ws && \
    git clone https://github.com/introlab/rtabmap_ros.git src/rtabmap_ros && \
    export MAKEFLAGS="-j6" && \
    rosdep update && \
    apt-get update && \
    rosdep install --from-paths src --ignore-src -r -y -t build_export -t test -t build -t buildtool_export -t buildtool --skip-keys="rtabmap" && \
    apt-get clean && rm -rf /var/lib/apt/lists/ && \
    colcon build --install-base /opt/ros/humble --merge-install --cmake-args -DCMAKE_BUILD_TYPE=Release && \
    cd && \
    rm -rf ros2_ws

# For the sake of the example, we need realsense2 driver
RUN source /opt/ros/$ROS_DISTRO/setup.bash && \
    apt-get update && \
    apt install -y \
    ros-humble-realsense2-camera \
    ros-humble-imu-filter-madgwick && \
    apt-get clean && rm -rf /var/lib/apt/lists/

ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/ros/humble/lib/x86_64-linux-gnu

RUN cd ~/ORB_SLAM3/Vocabulary && tar -xzf ORBvoc.txt.tar.gz

So from a terminal, copy this file under a file named Dockerfile in an empty directory (e.g., rtab_orbslam3_ros2/Dockerfile):

cd  rtab_orbslam3_ros2
docker build -t rtab_orbslam3_ros2 .

Try it with a D435i camera. Note that the IMU may not be accessible from inside the container, we have to add cgroup when launching docker. To find cgroup for the IMU (the number is 10):

ls -l /dev | grep HID
crwxrwxrwx   1 root    root     10,   121 Jan 11 17:32 HID-SENSOR-2000e1.4.auto

Enable host display access:

xhost +local:root

Launch the example (adjust the cgroup with the number above if needed):

docker run -it --rm \
    --net=host \
    --privileged \
    -e DISPLAY=$DISPLAY \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    --device-cgroup-rule 'c 10:* rmw' \
    rtab_orbslam3_ros2 \
    ros2 launch rtabmap_examples realsense_d435i_infra.launch.py odom_args:="\
        --OdomORBSLAM/Inertial false \
        --Odom/Strategy 5 \
        --OdomORBSLAM/VocPath ~/ORB_SLAM3/Vocabulary/ORBvoc.txt"

It's working properly now, thank you so much!

Working IMU_RGBD and IMU_STEREO. Changing Inter IMU base frame is now…

821ff6c

… possible.

matlabbe mentioned this pull request Nov 10, 2025

ORBSLAM3+RTABMAP introlab/rtabmap_ros#1378

Open

matlabbe added 3 commits November 11, 2025 12:04

Output error msg when vocabulary path is wrong.

0872fca

Merge branch 'master' of github.com:introlab/rtabmap into orbslam3_im…

22f0ac8

…u_fixes

Showing full local feature map

58d74c4

matlabbe merged commit e612d10 into master Nov 11, 2025
7 checks passed

matlabbe deleted the orbslam3_imu_fixes branch November 11, 2025 22:09

matlabbe mentioned this pull request Jan 6, 2026

How use external odometry? introlab/rtabmap_ros#1382

Closed

This was referenced Jan 12, 2026

Outdoor mapping problem #1484

Closed

Updating orbslam3 v1 support #1152

Merged

matlabbe mentioned this pull request Mar 8, 2026

Best VIO combination with RTAB introlab/rtabmap_ros#1403

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ORB-SLAM3 IMU support fixes#1612

ORB-SLAM3 IMU support fixes#1612
matlabbe merged 4 commits intomasterfrom
orbslam3_imu_fixes

matlabbe commented Nov 9, 2025 •

edited

Loading

Uh oh!

Uh oh!

liangyuan109 commented Dec 29, 2025

Uh oh!

liangyuan109 commented Dec 29, 2025

Uh oh!

matlabbe commented Jan 6, 2026

Uh oh!

liangyuan109 commented Jan 7, 2026

Uh oh!

matlabbe commented Jan 12, 2026 •

edited

Loading

Uh oh!

liangyuan109 commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

matlabbe commented Nov 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Installation

Uh oh!

Uh oh!

liangyuan109 commented Dec 29, 2025

Uh oh!

liangyuan109 commented Dec 29, 2025

Uh oh!

matlabbe commented Jan 6, 2026

Uh oh!

liangyuan109 commented Jan 7, 2026

Uh oh!

matlabbe commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

liangyuan109 commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matlabbe commented Nov 9, 2025 •

edited

Loading

matlabbe commented Jan 12, 2026 •

edited

Loading