This repository contains the official PyTorch implementation of AdaMSS (Adaptive Multi-Subspace Approach), a parameter-efficient fine-tuning (PEFT) method designed to improve the expressiveness–efficiency trade-off in fine-tuning of large models.

git clone https://github.com/jzheng20/AdaMSS.git
cd AdaMSS
conda create -n adamss python=3.12.2
conda activate adamss
pip install -r requirements.txt

If you find this work or code useful, please consider citing:

@inproceedings{zheng2025adamss,
  title={AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning},
  author={Zheng, Jingjing and Lu, Wanglong and Dong, Yiming and Ji, Chaojie and Cao, Yankai and Lin, Zhouchen},
  booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
  year={2025},
}

We are actively working on releasing AdaMSS as a standalone PEFT package, with a unified and user-friendly API compatible with the HuggingFace ecosystem. Please check here for more details of peft integration: AdaMSS-PEFT