Skip to content

paulyoussef/deed

Repository files navigation

Has this Fact been Edited? Detecting Knowledge Edits in Language Models

Environment

We use the mamba package manager. Install Miniforge

mamba env create -f environment.yml
conda activate deed

Detecting Knowledge Edits

  • The code for editing, extracting the representations, and detection can be found under ./deed/.

  • Here is an example for editing and extracting the representations with ROME and GPT-J: python deed.py --model gpt-j-6B --editing_method ROME --instances_per_class 1000 --starting_index_other 1010 --results_dir ./results-1k/ --gpu 0 --paraphrased_prompts --ds zsre

  • Only MEND, ROME, and MEMIT are supported. For MALMEN, please refer to the original implementation.

  • For MEND, we used the hypernetworks from ROME

  • For evaluation, you can run this script (after adapting the direcotries): python eval.py --classifier ada --setting both

Acknowledgement

This repository is based on EasyEdit ACL2024 (forked on Feb 6th 2024)

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors