Paper | Videos

This repository contains the experimental code for the paper Bimanual Catching Skill Learning, presented at ICRA 2025. The implementation is based on the IsaacGymBenchmarks framework and utilizes a custom-built rl_games_twk library for the heterogeneous-agent reinforcement learning (HARL) version of rl_games.

We highly recommend using a Conda environment for setup.

Create a dedicated Conda environment for this project:

conda create --name bicatch python=3.8 -y
conda activate bicatch

Download the Isaac Gym Preview 4 release from the official website, then follow the installation instructions in the documentation. Ensure Isaac Gym runs correctly by testing an example in the python/examples directory, such as:

python examples/joint_monkey.py

If any issues arise, refer to the troubleshooting guide in the Isaac Gym Preview 4 installation instructions.

Once Isaac Gym is installed and verified, install this repository:

pip install -e .

Modify the get_axis_params() function in isaacgym/python/isaacgym/torch_utils.py:

🔹 Change dtype=np.float → dtype=float

# Before:
dtype=np.float

# After:
dtype=float

This prevents compatibility issues with newer NumPy versions. 🚀

Refer to the following configuration files for training and evaluation of the Bimanual Catch Skill.

Some parameters may be duplicated due to the complexity of the code structure.

• isaacgymenvs/cfg/config.yaml

  • Defines the task name, PBT usage, number of training environments, max iterations, etc.

• isaacgymenvs/cfg/train/BimanualDexCatchUR3AllegroPPO.yaml

  • Specifies whether to use single-agent or multi-agent training, along with other training parameters.
  • For single-agent training: Set params.algo.name: a2c_continuous
  • For multi-agent training: Set params.algo.name: a2c_multi_agent

• isaacgymenvs/cfg/task/BimanualDexCatchUR3Allegro.yaml

  • Multi-agent parameters: {uniform test setting, alpha value, alpha decay params, etc.}
  • Evaluation parameters (e.g., controlled experiments)

This implementation supports execution via CLI (Command Line Interface) but is optimized for use in IDEs like PyCharm.
Adjust the required parameters in the Configs section and run:

python train.py

Adjust the required parameters in the Configs section and run:

python test.py

You can download all the pretrained models used in the experiments. After downloading, extract the model folders to the following path:

isaacgymenvs/runs

If objects unexpectedly reset during testing, modify the following setting:

• Open cfg/config.yaml
• Set sim_device: cpu

This should resolve the issue.

The Multi(Heterogeneous)-Agent RL implementation is located in:

📂 rl_games_twk/algos_torch/a2c_multi_agent.py

Please cite this work as:

@INPROCEEDINGS{kim2025learning,
  author    = {Taewoo Kim and Youngwoo Yoon and Jaehong Kim},
  title     = {Learning Dexterous Bimanual Catch Skills through Adversarial-Cooperative Heterogeneous-Agent Reinforcement Learning},
  booktitle={International Conference on Robotics and Automation (ICRA)},
  year      = {2025}
}