This is the relevant code for the paper WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing.
conda create -n wilke python=3.10
pip install -r requirements.txt
To achieve the baseline (KE & MEND), download the necessary open source model (Please refer to ROME).
- Download related KE (for GPT2-XL):
wget https://rome.baulab.info/data/weights/efk-1tok-gpt2-xl.pt -P baselines/efk/weights
- Download related MEND (for GPT2-XL & GPT-J):
wget https://rome.baulab.info/data/weights/mend-10tok-gpt2-xl.pt -P baselines/mend/weights
wget https://rome.baulab.info/data/weights/mend-10tok-gpt-j-6b.pt -P baselines/mend/weights
To achieve the baseline (ROME&MEMIT), download the necessary open source data (Please refer to ROME).
wget -r -np -nH --cut-dirs=2 https://rome.baulab.info/data/stats -P data
wget -r -np -nH --cut-dirs=2 -A "*" https://rome.baulab.info/data/dsets/ -P data
Based on the CounterFact dataset, we filter the data that would cause toxicity flash when editing GPT2-XL and GPT-J, respectively. For details, see data/
.
You can use WilKE to perform 1024 steps of continuous knowledge editing on GPT2-XL with the following command, and you can modify --alg_name
and --model_name
to use other knowledge editing methods and language models.
python3 -m experiments.evaluate --alg_name=WilKE --model_name=gpt2-xl --hparams_fname=gpt2-xl.json --ds_name=counterfact --dataset_size_limit=2048 --edit_times=1024 --skip_generation_tests
You can then use the experiments/summarize*
jupyter notebook files to summarize the results of your runs.
Our code is built based on ROME and MEMIT, so we would like to thank the original authors.
@article{hu2024wilke,
title={WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing},
author={Hu, Chenhui and Cao, Pengfei and Chen, Yubo and Liu, Kang and Zhao, Jun},
journal={arXiv preprint arXiv:2402.10987},
year={2024}
}