• 2025-12-05: Codes are released.
• 2025-07-17: The arxiv paper is online.

Please create the virtual environment with python3.7 and a 1.13.1 build of PyTorch. The code has been tested on Ubuntu22.04 and CUDA 11.6. The environment can be created like:

conda create -n remix python=3.7
conda activate remix

Install PyTorch:

pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116

Then you can install the dependencies:

pip install -r requirements.txt

cd thirdparty/NumpyMarchingCubes
python setup.py install
cd ../../

Notes: If you have networks issues when installing libraries like tiny-cuda-nn, please try build from source following the original repository

1.Download the example data from google drive.

2.Move it into your local directory and unzip the data. Remember to change the datadir in the configs/study_example.yaml to your local path.

3.Run the demo.py using the example data for a quick start.

python run.py --config ./configs/BS3D/study_example.yaml

We first introduce the preparation of two indoor large-scale SLAM datasets (BS3D and uHumans2). Basically, we organize the data like most SLAM datasets (e.g. ScanNet). Then, we introduce other common datasets (ScanNet, Replica, TUM RGB-D).

Download the raw sequence data from the official websites, and associate the RGB-D pose data via corresponding timestamps. Here, we take the BS3D dataset as an example. You can use the similar procedure for other datasets.

1.The raw data of large-scale benchmark BS3D can be obtained from this link. Please download the specific sequences in the config file.

2.Use the command to associate RGB-D images and output the organized data.

python preprocess/preprocess.py /PATH/BS3D/SCENE_NAME

We have uploaded the proprocessed sequences to the hugging face. You can just download the preprocessed datasets (BS3D and uHumans2). First, you need to install huggingface_hub

pip install -U huggingface_hub

Download the datasets using the following command. If you encouter problems of connecting to the hugging face, please search the solutions on your own. Remember to change the local-dir to the directory you want. The data takes up to about 27GB.

hf download --repo-type dataset Kevin1804/RemixFusion --local-dir /media/lyq/temp/test/

The data structure for BS3D is like this:

BS3D/
├── cafeteria/                
│   ├── depth/               # Folder containing depth images
│   ├── color/               # Folder containing RGB images
│   ├── all_poses.npy        # All camera extrinsics [N,4,4]
│   ├── poses.txt            # All camera extrinsics [N,8(t,tx,ty,tz,qx,qy,qz,qw)]
│   ├── mesh.ply             # GT LiDAR mesh (w/o color)
│   ├── cafeteria_gt.ply     # GT reconstructed mesh using GT poses
│   ├── cameras.json         # Camera information
│   ├── calibration.yaml     # Calibration details
│   ├── color.txt            # All RGB files path
│   ├── depth.txt            # All Depth files path

The data structure for uHumans2 is like this:

uHumans2/
├── apartment/                
│   ├── depth/               # Folder containing depth images
│   ├── color/               # Folder containing RGB images
│   ├── poses.txt            # All camera extrinsics [N,8(t,tx,ty,tz,qx,qy,qz,qw)]
│   ├── cameras.json         # Camera information
│   ├── associations.yaml    # Association of RGB-D raw data
│   ├── color.txt            # All RGB files path
│   ├── depth.txt            # All Depth files path

Please follow the procedure on the official ScanNet website, and extract RGB-D frames from the .sens file using the code.

The data structure for ScanNet is like this:

ScanNet/
├── scene0xxx_0x/
│   ├── frames/
│   │   ├── color/
│   │   │   ├── 0.jpg
│   │   │   ├── 1.jpg
│   │   │   └── ...
│   │   ├── depth/
│   │   │   ├── 0.png
│   │   │   ├── 1.png
│   │   │   └── ...
│   │   ├── pose/
│   │   │   ├── 0.txt
│   │   │   ├── 1.txt
│   │   │   └── ...
│   │   ├── intrinsic/
│   │   │   └── intrinsic_depth.txt
│   └── scene0xxx_0x.txt
│   └── scene0xxx_0x_vh_clean_2.ply
└── ...

Download the data using the command (provided by NICE-SLAM) and the data is saved into the ./data/Replica folder.

bash scripts/download_replica.sh

The mesh for evaluation is saved as $OUTPUT_FOLDER/mesh/final_mesh_eval_rec.ply, where the unseen regions are culled using all frames.

Download the data using the command and the data is saved into the ./data/TUM folder

bash scripts/download_tum.sh

For other datsets (e.g. FastCaMo, FastCaMo-large and self captured datasets), please download the data and prepare the data following the procedure mentioned above.

In this section, we introduce how to run RemixFusion on different datasets. After you have prepared the datasets according to the instructions above, you can run the following commands to try RemixFusion on the specific sequence.

Please change the datadir in the config/SEQUENCE.yaml to the root of the processed BS3D sequence. Customize the --config to the sequence you want to try.

python run.py --config ./configs/BS3D/foobar.yaml

Please change the datadir in the config/SEQUENCE.yaml to the root of the processed uHumans2 sequence. Customize the --config to the sequence you want to try.

python run.py --config ./configs/uhumans/apartment.yaml

Please change the datadir in the config/SEQUENCE.yaml to the root of the processed Replica sequence. Customize the --config to the sequence you want to try.

python run.py --config ./configs/Replica/room0.yaml

Please change the datadir in the config/SEQUENCE.yaml to the root of the processed ScanNet sequence. Customize the --config to the sequence you want to try.

python run.py --config ./configs/ScanNet/scene0169.yaml

Please change the datadir in the config/SEQUENCE.yaml to the root of the processed TUM RGB-D sequence. Customize the --config to the sequence you want to try.

python run.py --config ./configs/Tum/fr1_desk.yaml

Please change the datadir in the config/SEQUENCE.yaml to the root of the processed FastCaMo-synth sequence. Customize the --config to the sequence you want to try.

python run.py --config ./configs/Fast_syn/apartment_1.yaml

To evaluate the average trajectory error, you can just check the output directory which contains the trajectory results. You can also run the command below with the corresponding pose file. An example is like this:

python tools/eval_ate.py --est /PATH/YOUR_RESULTS/***.npy --gt /PATH/YOUR_DATASET/***.npy

The evaluation of reconstructed mesh consists of two steps.

1. (Step1) We cull the GT meshes using the ground-truth camera poses and depth information. Since the GT meshes are reconstructed from LiDAR scans, they contain large areas that were not observed in the RGB-D images.

bash tools/mesh_cull_gt.sh

2. (Step2) Use the estimated camera poses and GT depth images to cull the reconstructed mesh. GT depth images are used because we observe that some methods perform bad in depth rendering. Use the rendered depth for culling the mesh will cause severe empty holes. Frustum-based culling, used by previous methods, is not adopted here, as it will reserve a large amount of noisy geometry in empty space.

bash tools/mesh_cull_est.sh

3. (Step3) Use the following command to evaluate the culled mesh and GT mesh.

bash tools/mesh_eval.sh

Note: If you only want to evaluate one specific scene, you can modify the command in the above script.

To evaluate the rendering results, take the following command as an example to output the rendering metrics. Remember to change the save_ckpt=True (default: False) for rendering.

bash tools/rendering.sh

Parts of the codes are modified from Co-SLAM, NICE-SLAM, GS-ICP SLAM, SplaTAM. Great Thanks to the authors.

If you find our work useful in your research, please consider giving a star ✨ and citing the following paper:

@article{lan2025remixfusion,
  title={RemixFusion: Residual-based Mixed Representation for Large-scale Online RGB-D Reconstruction},
  author={Lan, Yuqing and Zhu, Chenyang and Zhi, Shuaifeng and Zhang, Jiazhao and Wang, Zhoufeng and Yi, Renjiao and Wang, Yijie and Xu, Kai},
  journal={ACM Transactions on Graphics},
  volume={45},
  number={1},
  pages={1--19},
  year={2025},
  publisher={ACM New York, NY}
}