Christopher Agia¹, Rohan Sinha¹, Jingyun Yang¹, Zi-ang Cao¹, Rika Antonova¹, Marco Pavone^1,2, Jeannette Bohg¹

¹Stanford University, ²NVIDIA Research

The official code repository for "Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress," presented at CoRL 2024. For a brief overview of our work, please refer to our project page. Further details can be found in our paper available on arXiv.

This repository implements failure detection for generative robot policies in two stages:

1. 🏗️ Generating offline datasets of generative policy rollouts.
2. 📊 Evaluating failure detectors on these offline datasets.

Important note: The first stage is optional, as we release (for download) the datasets associated with our published results (see Download Datasets), on which existing failure detectors can be evaluated and new failure detectors can be developed. However, if you would like to generate your own custom datasets, we also release an official set of diffusion policy checkpoints for each simulation environment we study (see Download Policies). Lastly, we note that this repository does not support policy training; we refer to the EquiBot project for training.

• The implementation of the Sentinel runtime monitor and a suite of failure detection baselines.
• Three simulated mobile manipulation environments: Box Closing, Object Covering, Cloth Folding (+ PushT).
• Dataset generation and evaluation scripts to accompany the above algorithms and environments.

SLURM: Most of our launch (.sh) scripts support parallelization on a cluster managed by SLURM, but will otherwise default to sequentially processing jobs.

This repository is tested on Ubuntu 20.04 with Python 3.10.13.

Python packages are primarily managed through Poetry. Follow the installation procedure below to get setup:

# Install pyenv.
curl https://pyenv.run | bash 
exec $SHELL           # Restart shell for path changes to take effect.
pyenv install 3.10.13 # Install Python version.

# Clone repository.
git clone https://github.com/agiachris/sentinel.git
cd sentinel

# Install Poetry: see https://python-poetry.org/docs/#installation
# Install dependencies and activate virtual environment.
poetry install
poetry shell

# Install extra pip dependencies.
pip install -r requirements.txt

# Install PyTorch3D prebuilt wheel: see https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md
# E.g., for Python 3.10, CUDA 11.8, PyTorch 2.0.0 
pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py310_cu118_pyt200/download.html

Use poetry shell to activate the virtual environment in the current shell.

Run the following command to download our official evaluation datasets (requires ~319Gb of disk space):

# cd <repository_root>; requires gdown.
bash scripts/download/download_datasets.sh

Modify download_datasets.sh as needed to download only a subset of the datasets.

Run the following sequence commands to evaluate all failure detectors over the official datasets:

# PushT environment data.
bash scripts/eval_detector/evaluate_ood_detectors_pusht.sh
# Simulated mobile manipulation environments data.
bash scripts/eval_detector/evaluate_ood_detectors_mobile.sh
# Real-world mobile manipulation environment data.
bash scripts/eval_detector/evaluate_ood_detectors_real.sh

Modify evaluate_ood_detectors_<env>.sh as needed to evaluate only a subset of the failure detectors.

export OPENAI_API_KEY=...       # If evaluating GPT.
export ANTHROPIC_API_KEY=...    # If evaluating Claude.
export GOOGLE_API_KEY=...       # If evaluating Gemini.

Notebooks for rendering figures/plots of failure detection results can be found in ./notebooks/final_results.

Run the following command to download our official diffusion policy checkpoints (requires ~82Gb of disk space):

# cd <repository_root>; requires gdown.
bash scripts/download/download_checkpoints.sh

Modify download_checkpoints.sh as needed to download only a subset of the policy checkpoints. Policy checkpoints will be unzipped to ./logs/bc/train.

Run the following sequence commands to generate datasets of policy rollouts:

# PushT environment.
bash scripts/data_generation/compute_rollout_actions_pusht.sh
# Simulated mobile manipulation environments.
bash scripts/data_generation/compute_rollout_actions_mobile.sh
# Real-world mobile manipulation environment.
bash scripts/data_generation/compute_rollout_actions_real.sh

Modify compute_rollout_actions_<env>.sh as needed to create new distribution shifts, store specific policy rollout data, and control other policy rollout settings (e.g., policy execution horizon).

Sentinel is offered under the MIT License agreement. If you find Sentinel useful, please consider citing our work:

@inproceedings{AgiaSinhaEtAl2024,
    title     = {Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress},
    author    = {Agia, Christopher and Sinha, Rohan and Yang, Jingyun and Cao, Zi-ang and Antonova, Rika and Pavone, Marco and Bohg, Jeannette},
    year      = {2024},
    booktitle = {8th Annual Conference on Robot Learning}
}

The simulation environments and generative policies used in this project were adapted from EquiBot.