🚧 This repository is currently under construction, and we are continually developing and refining the method. Check back soon for updates! 🚧

🚧 We will provide updated complete version of code and checkpoints upon paper acceptance 🚧

This repository contains the psudocode for our paper:

PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation

PPS-Ctrl is a image translation framework that combines Stable Diffusion and ControlNet, guided by a Per-Pixel Shading (PPS) map — a physics-informed representation capturing surface-light interactions. Unlike prior sim-to-real approaches that condition on depth maps, PPS provides a more faithful and geometrically consistent structural prior, enabling better texture realism and structure preservation in endoscopy image translation.

We recommend Python 3.9 with PyTorch ≥ 2.0 and the HuggingFace diffusers library.

conda create -n ppsctrl python=3.9
conda activate ppsctrl
pip install -r requirements.txt

Download the following datasets:

SimCol3D

C3VD

Colon10K

Precompute PPS maps:

python utils/compute_pps.py --depth_dir path/to/depth --output_dir path/to/pps

We provide the partial pseudo-code for the paper in the repository.

@article{xiong2025pps,
  title={PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation},
  author={Xiong, Xinqi and Beltran, Andrea Dunn and Choi, Jun Myeong and Niethammer, Marc and Sengupta, Roni},
  journal={arXiv preprint arXiv:2504.17067},
  year={2025}
}

This work builds on Stable Diffusion, ControlNet, Hugging Face Diffusers and PPSNet. We thank the maintainers for their open-source contributions.