This repository provides a streamlined pipeline for converting videos into 3D Gaussian Splatting models using COLMAP for Structure-from-Motion (SfM) and Speedy-Splat for model training.

-> Blog post where I have shared my experience.

• FFmpeg
• COLMAP
• Speedy-Splat dependencies
• Python 3.10+
• CUDA-enabled GPU (recommended) (CUDA Toolkit)

1. Clone this repository:

git clone --recursive https://github.com/nannigalaxy/video-3d-reconstruction-gsplat.git
cd video-3d-reconstruction-gsplat

2. Make scripts executable:

chmod +x *.sh

Run the complete pipeline using:

./video_to_gsplat.sh <fps> <input_video_path> <sfm_output_dir> <gsplat_output_dir_path>

• fps: Frame rate for video extraction (e.g. 20)
• input_video_path: Path to input video file
• sfm_output_dir: Directory for COLMAP output
• gsplat_output_dir_path: Directory for final 3D Gaussian Splatting model

./video_to_gsplat.sh 20 ./input.mp4 ./sfm_output ./gsplat_model

The pipeline consists of three main steps:

1. Frame Extraction: Extracts frames from input video at specified FPS
2. Structure from Motion: Uses COLMAP to perform SfM and generate camera parameters
3. 3D Gaussian Splatting: Trains a Speedy-Splat model using the processed data

ffmpeg -i <input_video> -vf "fps=<target_fps>" "frames/frame_%04d.png"

./colmap_undistorted_sfm_export.sh <input_images> <output_dir> [--sequential] [--enable_gpu]

./train_speedy_splat.sh <sfm_input_dir> <gsplat_output_dir_path>

Key Speedy-Splat training parameters:

• Resolution: 1024
• Iterations: 10000
• Position Learning Rate: 0.001 (initial) to 0.0001 (final)
• Feature Learning Rate: 0.0001
• Densification Interval: 1000
• Checkpoint Interval: 1000

sfm_output_dir/
├── database.db            # COLMAP feature database
├── sparse/               # Sparse reconstruction
│   └── 0/               # First reconstruction
│       ├── cameras.bin  # Camera parameters
│       ├── images.bin   # Image parameters
│       └── points3D.bin # 3D point cloud
├── undistorted/         # Undistorted images
│   ├── images/         # Undistorted input images
│   └── sparse/         # Undistorted reconstruction
└── custom_export/      # Exported formats
    ├── scene.ply       # Point cloud in PLY format
    ├── cameras.txt     # Camera parameters in text
    ├── images.txt      # Image parameters in text
    └── points3D.txt    # 3D points in text format

gsplat_output_dir/
├── checkpoints/        # Training checkpoints
│   ├── iteration_2000/
│   ├── iteration_4000/
│   └── iteration_final/
└── point_cloud/       # Point clouds
    ├── iteration_4000/
    └───── point_cloud.ply

A browser-based frontend for uploading videos, monitoring progress, and viewing results interactively.

pip install fastapi uvicorn python-multipart websockets

python3 -m uvicorn webapp.app:app --host 0.0.0.0 --port 8000

Open http://localhost:8000 in your browser. Upload a video, set FPS, and watch the pipeline run with real-time progress updates. When done, view the 3D Gaussian Splat directly in the browser.

docker build -t gsplat-web .

docker run -d --gpus all -p 8000:8000 -v $(pwd)/jobs:/app/jobs gsplat-web

1. Provision a Vultr Cloud GPU instance (A40/L40S recommended, Ubuntu 22.04, 8+ vCPU, 32GB RAM, 100GB SSD)

2. Install Docker + NVIDIA Container Toolkit:

   curl -fsSL https://get.docker.com | sh
   distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
     sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
   curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
     sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
     sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
   sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
   sudo nvidia-ctk runtime configure --runtime=docker
   sudo systemctl restart docker

3. Clone, build, and run:

   git clone --recursive https://github.com/nannigalaxy/video-3d-reconstruction-gsplat.git
   cd video-3d-reconstruction-gsplat
   docker build -t gsplat-web .
   docker run -d --gpus all -p 80:8000 -v /data/jobs:/app/jobs --restart unless-stopped gsplat-web

4. Access via http://<vultr-public-ip>/

5. Firewall: Allow ports 80, 443 (HTTPS), and 22 (SSH) only

You can view the trained Gaussian Splatting model using the SIBR viewer, found in speedy-splat dir or pull from source. Need to build this to use, follow its instructions.

The PLY files (found in gsplat_output_dir/point_cloud/ directory) can be viewed using various 3D visualization tools:

• Blender (using File > Import > PLY)
• MeshLab
• CloudCompare
• ParaView

1. CUDA/GPU Errors:

   • If COLMAP fails with CUDA errors, try using --disable_gpu option
   • Ensure CUDA drivers are up to date
   • Try reducing the maximum image resolution if GPU memory is insufficient

2. Feature Matching Failures:

   • Increase image overlap by extracting more frames (higher FPS)
   • Try adjusting lighting or camera exposure for better feature detection

3. Memory Management:

   • For high-resolution videos (2K+), reduce FPS to 8-15
   • Consider downscaling input images using: ffmpeg -i input.mp4 -vf "scale=1920:-1" output.mp4

1. Training Stability:

   • If training diverges, try reducing position_lr_init to 0.0005
   • Increase densification_interval to 2000 for complex scenes
   • Use checkpoint_iterations=500 for more frequent saves

2. Output Quality:

   • Poor reconstruction might indicate insufficient camera coverage
   • Try increasing iterations to 30000-50000 for complex scenes
   • Adjust percent_dense parameter (0.6-0.9) based on scene complexity

3. Common Error Messages:

   • "CUDA out of memory": Reduce batch size or image resolution
   • "Invalid camera parameters": Check COLMAP reconstruction quality
   • "Missing sparse/0": Ensure COLMAP pipeline completed successfully

MIT License - See LICENSE file for details

Licensing Notice

This repository includes a submodule or dependency on the Gaussian Splatting project, which is licensed separately under a non-commercial, research-only license by Inria and MPII.
Please refer to Gaussian-Splatting-LICENSE for full details.
This code cannot be used for commercial purposes without explicit permission from the original authors.

This project builds on:

• Project Page
• Paper (arXiv)
• Original Repository

• Project Page
• Paper
• Original Repository