This instruction is for creating the Motion Representation with the Tomato format. The tomato format is extended from the H3D format and is different from it. We name it Tomato Representation for convenience. For detailed ablation on motion representation design choice, please refer to Appendix B.1 in the paper.
Download SMPL+H, SMPLX, DMPLs.
Download SMPL+H mode from SMPL+H (choose Extended SMPL+H model used in the AMASS project), DMPL model from DMPL (choose DMPLs compatible with SMPL), and SMPL-X model from SMPL-X. Then, please place all the models under ./body_model/. The ./body_model/ folder tree should be:
./body_models
├── dmpls
│ ├── female
│ │ └── model.npz
│ ├── male
│ │ └── model.npz
│ └── neutral
│ └── model.npz
├── smplh
│ ├── female
│ │ └── model.npz
│ ├── info.txt
│ ├── male
│ │ └── model.npz
│ └── neutral
│ └── model.npz
├── smplx
│ ├── female
│ │ ├── model.npz
│ │ └── model.pkl
│ ├── male
│ │ ├── model.npz
│ │ └── model.pkl
│ └── neutral
│ ├── model.npz
└───────└── model.pklDownload Motion-X datasets
Please follow the instruction of Motion-X to download the SMPL-X data with the dimension of 322. Put the motion data in folder ./data/motion_data/smplx_322.
(1) get joints positions
python raw_pose_processing.py
(2) get Representation
python motion_representation.py
(3) (a) visualization for checking. If you want to check the joint visualization (The input shape is b * frame * 52 * 3, which should be under folder new_joints), then you run the following line.
python plot_3d_global.py
(3) (b) visualization for checking. If you want to check the 623-dim visualization (The input shape is b * frame * 623, which should be under the folder new_joints_vecs), then you run the following line.
python plot_feature.py
If you use the tomato format, please consider to cite us as:
@article{humantomato,
title={HumanTOMATO: Text-aligned Whole-body Motion Generation},
author={Lu, Shunlin and Chen, Ling-Hao and Zeng, Ailing and Lin, Jing and Zhang, Ruimao and Zhang, Lei and Shum, Heung-Yeung},
journal={arxiv:2310.12978},
year={2023}
}