The benchmark evaluation code for Universe-1.
hf download dorni/Verse-Bench --local-dir ./verse_bench| Models | 🤗 Hugging Face |
|---|---|
| VerseBenchModels | VerseBenchModels |
download the pretrained evaluation models into ./models
pip install -r requirements.txtWe tested the code on Python 3.10, PyTorch 2.6.0, and CUDA 12.4 on Ubuntu 22.04 LTS.
export MODELS_PATH=models
python calculate_metrics.pyOptional arguments:
--input_dir # Your data for evaluation, the file names must match the names in our Verse-Bench datasets. Each test case must contains both .mp4 and .wav.
#(default: ./mini_testset for quick test.)
--models_path # Path to the models. (default: ./models)