This is an official pytorch implementation of the 2025 IEEE Transactions on Circuits and Systems for Video Technology paper:

AVLTrack: Dynamic Sparse Learning for Aerial Vision-Language Tracking
(accepted by IEEE Transactions on Circuits and Systems for Video Technology, DOI: 10.1109/TCSVT.2025.3549953)

The paper can be downloaded from IEEE Xplore

The models and raw results can be downloaded from [GitHub] and [BaiduYun].

The tracking demos are displayed on the Bilibili.

• DSL and MLP in backbone

• python==3.8.18
• torch==1.13.0
• torchvision==0.14.0
• torchaudio==0.13.0
• timm==0.9.10
• pip install transformers==4.12.5 -i https://pypi.tuna.tsinghua.edu.cn/simple/

It should be noted that the above pretrained model is trained on an Ubuntu 18.04 server with multiple NVIDIA RTX A100 GPUs. For WebUAV-3M, we recommend the official evaluation toolkit.

Run the following command to set paths for this project

python tracking/create_default_local_file.py --workspace_dir . --data_dir ./data --save_dir ./output

After running this command, you can also modify paths by editing these two files

lib/train/admin/local.py  # paths about training
lib/test/evaluation/local.py  # paths about testing

Training datasets download

• LaSOT: [Link]
• GOT-10k: : [Link]
• COCO: : [Link]
• GOT-10k: : [Link]

Put the tracking datasets in ./data. It should look like this:

${PROJECT_ROOT}
 -- data
     -- lasot
         |-- airplane
         |-- basketball
         |-- bear
         ...
     -- got10k
         |-- test
         |-- train
         |-- val
     -- coco
         |-- annotations
         |-- images
     -- trackingnet
         |-- TRAIN_0
         |-- TRAIN_1
         ...
         |-- TRAIN_11
         |-- TEST

• Download pre-trained [MAE ViT-Base weights] and put it to $PROJECT_ROOT$/AVLTrack/lib/models/pretrained_models.
• Download bert-base-uncased pre-trained weights [Baidu Yun] or [Gitcode] or [Hugging Face] and put them to $PROJECT_ROOT$/AVLTrack/lib/models/pretrained_models/bert-base-uncased.

1.Training with one GPU.

cd /$PROJECT_ROOT$/AVLTrack
CUDA_VISIBLE_DEVICES=0 python tracking/train.py --script ostrack --config abavit_patch16_224_ep300 --save_dir ./output --mode single --nproc_per_node 1

2.Training with multiple GPUs.

cd /$PROJECT_ROOT$/AVLTrack
CUDA_VISIBLE_DEVICES=0,1 python tracking/train.py --script ostrack --config abavit_patch16_224_ep300 --save_dir ./output --mode multiple --nproc_per_node 2

Before training, please make sure the data path in local.py is correct.

Download the model AVLTrack, extraction code: 43xv. Add the model to $PROJECT_ROOT$/AVLTrack/output/checkpoints/train/.

python tracking/test.py --tracker_name ostrack --tracker_param abavit_patch16_224_ep297 --dataset webuav3m --threads 2 --num_gpus 2
python tracking/analysis_results.py

Before evaluation, please make sure the data path in local.py is correct.

python tracking/profile_model.py --script ostrack --config levit_256_32x4_ep300

Considering that WebUAV-3M is the only UAV vision-language tracking dataset, we additionally construct vision-language tracking datasets DTB70-NLP, UAV20L-NLP, UAVDT-NLP, and VisDrone2019-SOT-test-dev-NLP based on the vision-only aerial dataset DTB70, UAV20L, UAVDT, and VisDrone2019-SOT-test-dev. Note that DTB70-NLP and UAV20L-NLP are non-overlapping with the training set. Therefore, to better evaluate the model generalization, DTB70-NLP and UAV20L-NLP are only used for testing.

The code based on the OSTrack, All-in-One, and A-ViT.

We would like to express our sincere thanks to the contributors.

If you find this work useful for your research, please cite the following papers:

@ARTICLE{10220112,
  author={Yuanliang Xue,Bineng Zhong,Guodong Jin,Tao Shen,Lining Tan,Ning Li,Yaozong Zheng},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={AVLTrack: Dynamic Sparse Learning for Aerial Vision-Language Tracking}, 
  year={2025},
  doi={10.1109/TCSVT.2025.3549953}}

If you have any questions about this work, please contact with me via xyl_507@outlook.com