This repo contains the supported code and configuration files to reproduce semantic segmentation results of LIT. It is based on mmsegmentation.

1. Make sure you have created your environment with our provide scripts. We recommend you create a new environment for experiments with semantic segmentation.

   # Suppose you already have an env for training LIT on ImageNet.
   conda create -n lit-seg --clone lit

2. Next, please refer to get_started.md for mmsegmentation installation.

3. Prepare ADE20K dataset.

   # Within this directory, do
   ln -s [path/to/ade20k] data/

4. Download our pretrained weights on ImageNet and move the weights under pretrained/.

# single-gpu testing
python tools/test.py <CONFIG_FILE> <SEG_CHECKPOINT_FILE> --eval mIoU

# multi-gpu testing
tools/dist_test.sh <CONFIG_FILE> <SEG_CHECKPOINT_FILE> <GPU_NUM> --eval mIoU

For example, to evaluate a Semantic FPN model with a lit-ti backbone, run:

tools/dist_test.sh configs/lit/lit_ti_fpn_r50_512x512_80k_ade20k.py [path/to/checkpoint] 1 --eval mIoU

To train a detector with pre-trained models, run:

# single-gpu training
python tools/train.py <CONFIG_FILE> --options model.pretrained=<PRETRAIN_MODEL> [model.backbone.use_checkpoint=True] [other optional arguments]

# multi-gpu training
tools/dist_train.sh <CONFIG_FILE> <GPU_NUM> --options model.pretrained=<PRETRAIN_MODEL> [model.backbone.use_checkpoint=True] [other optional arguments] 

For example, to train a Semantic FPN model with a lit-ti backbone and 8 gpus, run:

tools/dist_train.sh configs/lit/lit_ti_fpn_r50_512x512_80k_ade20k.py 8 --options model.pretrained=<PRETRAIN_MODEL>

If you use this code for a paper please cite:

@article{pan2021less,
  title={Less is More: Pay Less Attention in Vision Transformers},
  author={Pan, Zizheng and Zhuang, Bohan and He, Haoyu and Liu, Jing and Cai, Jianfei},
  journal={arXiv preprint arXiv:2105.14217},
  year={2021}
}