Monocular human motion capture systems often suffer from noisy or incomplete predictions due to occlusions, poor visibility, or ambiguous poses. To address these limitations, we propose DREAMoR: a diffusion-based motion prior framework for reconstructing physically plausible human motion from corrupted sequences. Our method first learns a latent space of motion transitions using a MotionVAE trained on clean data. We then train a diffusion model in this latent space to capture the distribution of realistic motion transitions, conditioned on previous frames. At inference time, we use multistep DDIM-style denoising and score distillation to optimize the latent sequence, ensuring that the resulting motion both aligns with the noisy input and adheres to the learned motion prior. Experiments on AMASS show that DREAMoR outperforms prior methods in recovering occluded joints and produces smoother, more realistic motion. Our ablation studies further highlight the effectiveness of latent diffusion priors for motion refinement.

DREAMoR supports three main functionalities:

1. Training the Diffusion-based Motion Prior: Learn a realistic motion prior using a latent diffusion model
2. Fitting on AMASS Datasets: Apply DREAMoR to recover plausible human motion from corrupted sequences
3. Comprehensive Evaluation: Both qualitative visualization and quantitative metrics to assess reconstruction quality

• Datasets
• Body_models
• Pretrained models
• Checkpoints

Before training, get dataset and bodymodels and pretrained motionVAE:

• Datasets
• Body_models
• Pretrained models

Your directory structure should look like this:

My Drive/
└── DREAMoR/
    ├── body_models/      # body models SMPLH and VPoser
    ├── datasets/         # Datasets
    ├── out/              # Output directory (for checkpoints, logs, etc.)
    └── pretrained_models/ # Pretrained MotionVAE

body_models, pretrained_models/ and datasets/ folders are shared folders.

1. Go to Google Colab
2. Click the GitHub tab
3. Search for our humor repo
4. Select the repository and the correct branch
5. Open the notebook: train.ipynb

1. In the opened notebook, uncomment the first three cells
2. Run those cells to install all dependencies
3. After installation, Colab may prompt you to Restart Runtime — accept then
4. Choose a training config from the configs directory (e.g. configs/colab_trian_motion_vae.yaml or configs/colab_train_diffusion_transformer.yaml)
5. Then, run the rest of the notebook to begin training

• Output checkpoints and logs are saved in: My Drive/DREAMoR/out/
• To modify more config settings, refer to config_new.py

To train DREAMoR locally:

1. Install the required dependencies
2. Modify the configuration files in the configs directory to match your local paths
3. Run the training notebook: train.ipynb

Make sure to adjust the paths both in config file and notebook for:

• Dataset location
• Body model files
• Pretrained MotionVAE
• Output directories

The training process is identical to the Colab version, but leverages your local hardware.

To run motion reconstruction on corrupted sequences:

1. Download the required pretrained models:

   • DREAMoR diffusion model
   • MotionVAE encoder/decoder
   • Body models (SMPL+H and VPoser)

2. Configure the fitting parameters in the fitting config file:

   • Input data paths
   • Corruption settings
   • Optimization parameters

3. Run the fitting process: fit.ipynb

The fitting process uses the pretrained DREAMoR model to optimize corrupted motion sequences, producing physically plausible reconstructions.

Get result npz files after fitting, then you can calculate the evaluation metrics and visualize the results.

To visualize the optimization results:

python dreamor/fitting/eval_fitting_3d.py --results ./out/your_experiment_name/results_out --out ./out/your_experiment_name/eval_out --qual --viz-observation

To compute evaluation metrics:

python dreamor/fitting/eval_fitting_3d.py --results ./out/your_experiment_name/results_out --out ./out/your_experiment_name/eval_out --quant

The most relevant quantitative results will be written to eval_out/eval_quant/compare_mean.csv.

This code is based on the:

• HuMoR

Thanks to the authors for their work and for making their code available!!