Skip to content

HKUST-C4G/DomainShuttle

Repository files navigation

DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation

Nan Chen† ·Yiyang Cai† · Rongchang Xie · Junwen Pan · Cheng Chen · Weinan Jia · Zhuowei Chen ·

Wen Zhou‡ · Zhenbang Sun · Wenhan Luo

Equal Contribution. Project Leader Corresponding Author

TL; DR: We propose DomainShuttle, an open-domain subject-driven text-to-video method that flexibly handles both in-domain fidelity and cross-domain editability by decoupling reference and video features, modeling domain attributes, and learning intrinsic subject representations.

## 🔥 Latest News

📖 Jun 23, 2026: We release the technical report.

🔥 Jun 23, 2026: We release the unofficial DomainShuttle implementation weights, inference code, and project page.

📑 Todo List

  • [✅] Inference code
  • [✅] checkpoint
  • [✅] Technical report

Quick Start

🛠️Installation

1. Create a conda environment and install pytorch

conda create -n DomainShuttle python=3.10
conda activate DomainShuttle 
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

bash build_env_conda.sh

🧱Model Preparation

Models Download Link Notes
Wan2.2-T2V-A14B 🤗 Huggingface Base model
DomainShuttle 🤗 Huggingface Our 14B weights

Download models using huggingface-cli:

mkdir models/Diffusion_Transformers
hf download CNcreator0331/DomainShuttle_weight --local-dir ./models/Diffusion_Transformers/Wan2.2-DomainShuttle-A14B
hf download Wan-AI/Wan2.2-T2V-A14B --local-dir ./checkpoints/Wan2.2-T2V-A14B
mv ./checkpoints/Wan2.2-T2V-A14B/google  ./models/Diffusion_Transformers/Wan2.2-DomainShuttle-A14B/
mv ./checkpoints/Wan2.2-T2V-A14B/Wan2.1_VAE.pth ./models/Diffusion_Transformers/Wan2.2-DomainShuttle-A14B/
mv ./checkpoints/Wan2.2-T2V-A14B/configuration.json ./models/Diffusion_Transformers/Wan2.2-DomainShuttle-A14B/
mv ./checkpoints/Wan2.2-T2V-A14B/models_t5_umt5-xxl-enc-bf16.pth ./models/Diffusion_Transformers/Wan2.2-DomainShuttle-A14B/

The directory should be organized as follows.

models/Diffusion_Transformers/
└── Wan2.2-DomainShuttle-A14B/
    ├── google/
    ├── high_noise_model/
    ├── low_noise_model/
    ├── configuration.json
    └── models_t5_umt5-xxl-enc-bf16.pth
    └── Wan2.1_VAE.pth
    

🔑 Quick Inference

This unofficial inference script can run 480p/720p inference on GPU.

14B inference

bash run_wan22_domainshuttle.sh

We provide some test cases in test_case folder. You can also try our model with your own data. You can change the reference image in the json file.

📚 Citation

If you find our work useful in your research, please consider citing.

📜 License

The models in this repository are licensed under the Apache 2.0 License. We claim no rights over the your generated contents, granting you the freedom to use them while ensuring that your usage complies with the provisions of this license. You are fully accountable for your use of the models, which must not involve sharing any content that violates applicable laws, causes harm to individuals or groups, disseminates personal information intended for harm, spreads misinformation, or targets vulnerable populations.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors