Official implementation of SemFi in Semantic Frame Interpolation.

Yijia Hong^1,2, Jiangning Zhang^3,4, Ran Yi¹, Yuji Wang¹, Weijian Cao⁴, Xiaobin Hu⁴, Zhucun Xue³, Yabiao Wang⁴, Chengjie Wang⁴, Lizhuang Ma¹

¹Shanghai Jiao Tong University, ²Shanghai Innovation Institute, ³Zhejiang University, ⁴Tencent YouTu Lab

Our code is based on DiffSynth-Studio, you can skip this part if you have already installed its environment.

The code has been tested on Python 3.10.9 and Pytorch 2.6.0.

git clone https://github.com/hyj542682306/Semantic-Frame-Interpolation
cd Semantic-Frame-Interpolation
pip install -r requirements.txt

1. Please download the Wan2.1-I2V-14B-720P model from HuggingFace or ModelScope and save it to the models directory.
2. Download LoRA’s checkpoint from Google Drive or HuggingFace and save it to the ckpts directory.

After downloading, the files related to pre-trained models are structured like this:

./ckpts
└──semfi.ckpt
./models
└──Wan2.1-I2V-14B-720P
   └──...

You can directly run wan_fi_demo.sh to generate the example output:

sh wan_fi_demo.sh

The generated result will be saved to videos.mp4.

Or you can customize the parameters in the run shell to perform your own tests with the following parameter explanations:

• --lora: Path to the pre-trained lora checkpoint.
• --prompt: Prompt to generate.
• —-negative_prompt: Negative prompt to generate.
• —-st_img: Path to the first frame.
• —-ed_img: Path to the last frame.
• —-height: Height of the generated video.
• —-width: Width of the generated video.
• —-frame: Number of the interpolated frames.
• —-output: Output path for the generated video.
• —-num_inference_step: Diffusion denoise steps.
• —-seed: Seed.
• —-fps: FPS of the generated video.

Our project benefits from the amazing open-source projects:

• DiffSynth-Studio

• Wan2.1

• Open-Sora

We would like to thank the authors of these projects for their contributions to the community.

If you find our work useful in your research, please cite:

@misc{hong2025semanticframeinterpolation,
      title={Semantic Frame Interpolation}, 
      author={Yijia Hong and Jiangning Zhang and Ran Yi and Yuji Wang and Weijian Cao and Xiaobin Hu and Zhucun Xue and Yabiao Wang and Chengjie Wang and Lizhuang Ma},
      year={2025},
      eprint={2507.05173},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2507.05173}, 
}