An Interactive Framework for Implementing Privacy-Preserving Federated Learning: Experiments on Large Language Models
The GLUE dataset learning process is using Transformers library and is adopted from run_glue.
The Federated learning enviroment is using Flower.
Because of an existing problem with flwr[simulation]==1.12.0 when using local DP, following steps should be done:
-
Copy localdp_fixed_mod.py in flwr/client/mod
-
Use differential_privacy.py instead of flwr/common/differential_privacy.py
pip install -r requirement.txt
To run the experiments in the paper run:
./script.sh
To calculate required noise for target Epsilon, we used our previous work FSRDP Accountant.
Python ./noise_calculation/get_noise.py
target_epsilons and dataset_size_list is configurable in get_noise.py file.
python federated.py \
--model_name_or_path google-bert/bert-base-cased \
--max_seq_length 128 \
--task_name SST2 \
--partition_policy Linear \
--per_device_train_batch_size 550 \
--learning_rate 2e-5\
--output_dir /tmp/SST2/
- Model_name is the based model.
- task_name is the dataset which can be (SST2, QNLI, or QQP).
- Parition_policy can be (Iid, Linear, Square, or Exp).
Please cite our papar if you find our repo helpful.
@misc{ahmadi2025interactiveframeworkimplementingprivacypreserving,
title={An Interactive Framework for Implementing Privacy-Preserving Federated Learning: Experiments on Large Language Models},
author={Kasra Ahmadi and Rouzbeh Behnia and Reza Ebrahimi and Mehran Mozaffari Kermani and Jeremiah Birrell and Jason Pacheco and Attila A Yavuz},
year={2025},
eprint={2502.08008},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2502.08008},
}