a. Create a conda virtual environment and activate it.

conda create -n resworld python=3.8 -y
conda activate resworld

b. Install PyTorch and torchvision following the official instructions.

pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

c. Install mmcv-full, mmdet and mmseg.

pip install mmcv-full==1.4.0
pip install mmdet==2.14.0
pip install mmsegmentation==0.14.1

d. Install mmdet3d

git clone https://github.com/open-mmlab/mmdetection3d.git
cd /path/to/mmdetection3d
git checkout -f v0.17.1
python setup.py develop

e. Install nuscenes-devkit.

pip install nuscenes-devkit==1.1.9

GeoBEV
├── data
│   ├── nuscenes
│   │   ├── lidarseg
│   │   ├── maps
│   │   ├── samples
│   │   ├── samples_point_label
│   │   ├── sweeps
│   │   ├── v1.0-test
|   |   ├── v1.0-trainval
│   │   ├── vad_nuscenes_infos_temporal_train.pkl
│   │   ├── vad_nuscenes_infos_temporal_val.pkl

a. Download nuScenes 3D detection data HERE and unzip all zip files.

b. Download the train file and val file generated by VAD.

c. Download nuScenes-lidarseg annotations HERE and put it under data/nuscenes/. Create depth label utilized by GeoBEV from point cloud by running:

python tools/generate_point_label.py

Download the backbones pretrained by GeoBEV HERE and put it under ckpts/. Then train the ResWorld model following:

bash tools/dist_train.sh projects/configs/resworld/resworld_config.py 4

bash tools/dist_test.sh projects/configs/resworld/resworld_config.py work_dirs/resworld_config/epoch_12_ema.pth 4 --eval bbox

The model after training is available HERE.

This project is not possible without multiple great open-sourced code bases. We list some notable examples below.

• BEVDet
• VAD
• SSR