"Simplicity is prerequisite for reliability." --- Edsger W. Dijkstra

AnyDepth: Depth Estimation Made Easy

Zeyu Ren^1*, Zeyu Zhang^2*†, Wukai Li², Qingxiang Liu³, Hao Tang^2‡

¹The University of Melbourne, ²Peking University, ³Shanghai University of Engineering Science

^*Equal contribution. ^†Project lead. ^‡Corresponding author.

Paper | Website | Model | HF Paper

If you find our code or paper helpful, please consider starring ⭐ us and citing:

@article{ren2026anydepth,
  title={AnyDepth: Depth Estimation Made Easy},
  author={Ren, Zeyu and Zhang, Zeyu and Li, Wukai and Liu, Qingxiang and Tang, Hao},
  journal={arXiv preprint arXiv:2601.02760},
  year={2026}
}

We present AnyDepth, a simple and efficient training framework for zero-shot monocular depth estimation. The core contribution is the Simple Depth Transformer (SDT), a compact decoder that achieves comparable accuracy to DPT while reducing parameters by 85%--89%.

Key Features:

• Single-Path Fusion: Fuse-then-reassemble strategy avoids multi-branch cross-scale alignment
• Weighted Fusion: Learnable fusion of 4-layer ViT features with cls token readout
• Spatial Detail Enhancer (SDE): Depthwise convolution for local spatial modeling
• DySample Upsampling: Two-stage learnable upsampling (H/16 -> H/4 -> H)
• Lightweight: Only ~5-13M parameters for the decoder

SDT has no additional dependencies beyond PyTorch. Simply replace DPT or other decoders with SDT in your existing codebase.

import torch
from sdt_head import SDTHead

# in_channels: ViT-S=384, ViT-B=768, ViT-L=1024
# extract layers: [2,5,8,11] for ViT-S/B, [4,11,17,23] for ViT-L
head = SDTHead(
    in_channels=in_channels,
    fusion_channels=fusion_channels,
    n_output_channels=1,
    use_cls_token=True
)

We provide the training splits (369K samples) in the datasets/ folder. To prepare the data:

1. Hypersim & Virtual KITTI: Follow the instructions from Lotus to download and prepare these datasets.

2. IRS: Follow the official instructions at IRS.

3. BlendedMVS: Follow the official instructions at BlendedMVS.

4. TartanAir: Follow the official instructions at TartanAir.

Key Difference: DPT uses a reassemble-fusion strategy (per-layer reassembly + cross-scale fusion), while SDT uses a fusion-reassemble strategy (fuse tokens first, then single-path upsampling).

Metric: AbsRel % (lower is better)

We fine-tune on Hypersim and Virtual KITTI with depth foundation models (DAv2, DA3, VGGT).

Metric: AbsRel % (lower is better). The encoder used pre-trained weights, and the decoder was randomly initialized.

SDT has been tested on Jetson Orin Nano (4GB):

• DINOv2 / DINOv3
• Depth Anything V2
• Depth Anything 3
• VGGT
• DySample
• DPT

This work is licensed under CC BY-NC-SA 4.0.