alrope123 / rethinking-demonstrations Goto Github PK

There is no test.py

I tested the following code on Google Colab with an A100 (40G) but got out of memory, may I ask for how many memory does the following code needs?

!python test.py --gpt2 channel-metaicl --method direct --out_dir out/gpt2-large --do_zeroshot --use_demonstrations --k 16 --seed 100,13,21,42,87 --dataset glue-wnli_random

Hi! I want to use the fairseq 13B model disscussed in your paper. Could you tell me what should I do?

Hello,

I am not able to find test.py file in the main branch of GitHub.

I correctly set up the environment and it's running for zero_shot.

But, I am running the following command:
python test.py --dataset tweet_eval-hate --gpt2 gpt2-large --method direct --out_dir out/gpt2-large --use_demonstrations --k 16 --seed 100,13,21,42,87

However, I am getting this error:

10/19/2022 15:15:53 - INFO - __main__ - Namespace(checkpoint=None, dataset='tweet_eval-hate', do_zeroshot=False, global_step=None, gpt2='gpt2-large', is_null=False, k=16, log_file=None, method='direct', out_dir='out/gpt2-large', seed='100,13,21,42,87', split='test', task=None, test_batch_size=64, unseen_domain_only=False, use_calibration=False, use_demonstrations=True, use_random_english_words=False)
Traceback (most recent call last):
  File "test.py", line 274, in <module>
    main(logger, args)
  File "test.py", line 44, in main
    assert args.global_step is not None
AssertionError

Can you please help me with this error?

I got the following message after running the code to generate random labels:

python create_data.py --variant random --dataset commonsense_qa