Overview • Environment • Dataset • Start • License • Citation

This is the official implementation of the following paper:

Unifying Unsupervised Graph-Level Out-of-Distribution Detection and Anomaly Detection: A Benchmark

UB-GOLD provides a fair and comprehensive platform to evaluate 18 existing unsupervised GLAD and GLOD works on 4 types of datasets and facilitates future GLAD/GLOD research.

Before you begin, ensure that you have Anaconda or Miniconda installed on your system. This guide assumes that you have a CUDA-enabled GPU.

Required Dependencies :

• torch>=2.0.1
• torch_geometric>=2.4.0
• python>=3.8
• scikit-learn>=1.0.2
• networkx>=2.6.3
• rdkit>=2023.3.1
• dgl>=2.1.0
• pygcl>=0.1.2

You can easily create a virtual environment with all the required libraries for this project by running:
conda env create -n UBGOLD -f environment.yml

If errors occur during the above step, you may also install the dependencies using pip. Before using pip, it is strongly recommended to manually select the appropriate PyTorch version (compatible with your Python and CUDA versions). In this example, the Python version is 3.8.10 and the CUDA version is 11.8. Thus, the manually downloaded PyTorch and its extension packages are as follows:

• torch-2.1.0+cu118-cp38-cp38-linux_x86_64.whl
• torch_scatter-2.1.2+pt21cu118-cp38-cp38-linux_x86_64.whl
• torch_sparse-0.6.18+pt21cu118-cp38-cp38-linux_x86_64.whl
• torchvision-0.15.1+cu118-cp38-cp38-linux_x86_64.whl

If you use a different CUDA version, please refer to the PyTorch and Pyg websites for the appropriate versions.

For additional torch_geometric extensions, you can download them from this link.
For PyTorch packages, users in China can download them from the Nanjing University mirror site.

After manually configuring the PyTorch-related packages, run the following command to install the remaining Python libraries:
pip install -r requirements.txt

├── ...
├── benchmark (Benchmark execution)
│   ├── Source (Default hyperparameters)
│   ├── my_random_search.py
│   ├── mymain.py
│   ├── ...
├── results (Save the results)
├── data (Raw datasets)
├── GAOOD (Methods)
│   ├── nn
│   ├── detector
│   ├── ...
├── dataloader (Data preprocessing pipeline)
│   ├── ...
├── ...

The data used in UB-GOLD should be downloaded in './data', and they can be downloaded from the following sources:

1. Intrinsic Anomaly: TOX21
   • Tox21_p53, Tox21_HSE, Tox21_MMP,Tox21_PPAR-gamma
2. Inter-Dataset Shift & Class-based Anomaly: TUDataset
   • COLLAB, IMDB-BINARY, REDDIT-BINARY, ENZYMES, PROTEINS
   • DD, BZR, AIDS, COX2, NCI1, DHFR
3. Inter-Dataset Shift: OGB
   • BBBP, BACE, CLINTOX, LIPO, FREESOLV
   • TOXCAST, SOL, MUV, TOX21, SIDER
4. Intra-Dataset Shift: DrugOOD
   • IC50 (SIZE, SCAFFOLD, ASSAY)
   • EC50 (SIZE, SCAFFOLD, ASSAY)
5. Intra-Dataset Shift: GOOD
   • GOODHIV
   • GOODZINC

git clone https://github.com/UB-GOLD/UB-GOLD.git
cd ./UBGOLD

With Default Hyperparameters

The main experiments:

python benchmark/mymain.py 

OR

cp ./benchmark/Source/GOOD-D.sh .

bash GOOD-D.sh

The near-far OOD detection experiments:

python benchmark/near_far_ood.py 

OR

cp ./benchmark/Source/near_far_ood.sh .

bash near_far_ood.sh

The perturbation OOD detection experiments:

python benchmark/per_ood.py 

OR

cp ./benchmark/Source/per_ood.sh .

bash per_ood.sh

With Optimal Hyperparameters through Random Search

python benchmark/my_random_search.py

OR

cp ./benchmark/Source/search.sh .

bash search.sh

This part lists all the methods we include in this codebase. We support 16 popular methods for anomaly detection and OOD detection.

If you find our repository useful for your research, please consider citing our paper:

@article{wang2024unifying,
  title={Unifying Unsupervised Graph-Level Anomaly Detection and Out-of-Distribution Detection: A Benchmark},
  author={Wang, Yili and Liu, Yixin and Shen, Xu and Li, Chenyu and Ding, Kaize and Miao, Rui and Wang, Ying and Pan, Shirui and Wang, Xin},
  journal={arXiv preprint arXiv:2406.15523},
  year={2024}
}

UB-GOLD has completed the approaches related to unsupervised Graph detection for Tables 1 and 2. As for some special scenarios in Tables 3-5, we will gradually adapt them to this benchmark.