Code and Data associated with the INLG'22 Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT paper

std_davinci has analogies generated by the Davinci model for STD concepts.

saqa.txt has Target, Source, and Explanation extracted from analogies in saqa_full.txt. Explanations were used for automatic evaulation for precision since they genereally contain the core part of the analogies, while the full analogies were used for human evaluation for clarity. Target and source is also present in saqa_concepts.txt.

saqa_<model> and saqa_<model>_src contain analogies generated by the MODEL for SAQA dataset concepts in the no_src and wsrc settings.

amt_res and amt_src_res contain the results of human evaluation of no_src and wsrc analogies.

non_analogies contains text generated by no_anlgy prompts.

plm_generator.py can be used to generate analogies. Prompt type, temperature, and model should be provided as command line arguments.