The SlicerMorph Photogrammetry extension provides a complete workflow for reconstructing 3D models from photographs or video within 3D Slicer. The extension includes four specialized modules that work together to transform your images into textured 3D models.
| Module | Description |
|---|---|
| PhotoMasking | Semi-automatic image masking using SAM (Segment Anything Model). Removes backgrounds from photographs to prepare them for 3D reconstruction. |
| VideoMasking | Video-based masking using SAMURAI. Extracts frames from video and tracks/masks objects automatically across all frames. |
| ODM | 3D reconstruction engine using OpenDroneMap/NodeODM. Converts masked images into textured 3D models. |
| ClusterPhotos | AI-powered image clustering using Vision Transformers. Organizes large photo collections into similar groups for efficient batch masking. |
- (Optional) Use ClusterPhotos to organize photos by viewing angle
- Use PhotoMasking to remove backgrounds from photographs
- Use ODM to reconstruct the 3D model
- Use VideoMasking to extract and mask frames from video
- Use ODM to reconstruct the 3D model
If you use the Photogrammetry extension in a scientific publication (conference abstract, preprint, journal paper, etc.), please cite:
For details about photographing specimens using a low-cost setup and using ArUco markers for physical scale, see:
Zhang and Maga (2023) An Open-Source Photogrammetry Workflow for Reconstructing 3D Models
There are no prerequisites if you are using MorphoCloud On Demand. All necessary libraries are preloaded.
We recommend using MorphoCloud On Demand for the best experience:
- GPU Acceleration: NVIDIA A100 GPUs significantly speed up both masking and reconstruction
- Typical runtime: 60-70 minutes for the sample data workflow on MorphoCloud
To run locally, you'll need:
- Docker
- NVIDIA Container Toolkit (for GPU support)
- Admin access to your computer
Note: Due to Docker installation complexities, the Photogrammetry extension is currently only available in the Slicer Extension Catalogue for Linux. Again, we suggest running the Photogrammetry extension in MorphoCloud On Demand using g3.xl flavor for best performance.
Unprocessed photographs from 15 mountain beavers used in Zhang and Maga, 2022
Single specimen (UWBM 82409) used in tutorials: https://app.box.com/shared/static/z8pypqqmel8pv4mp5k01philfrqep8xm.zip
Watch the Photogrammetry video tutorial on YouTube
- PhotoMasking User Guide - Image masking with SAM
- VideoMasking User Guide - Video frame extraction and masking with SAMURAI
- ODM User Guide - 3D reconstruction with NodeODM
- ClusterPhotos User Guide - AI-powered image organization
The Photogrammetry extension is supported by grants (DBI/2301405, OAC/2118240) from the National Science Foundation to AMM (Seattle Children's Research Institute).
The Photogrammetry extension uses the following open-source projects:
- Segment Anything Model (SAM) - for segmenting foreground objects in photographs
- pyODM from the OpenDroneMap project - for stereophotogrammetric reconstruction of 3D models
- SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory - for segmenting foreground objects from video
- Vision Transformer (ViT-large) - for image clustering in ClusterPhotos
We thank these groups for making their tools publicly available.