joeljang / negated-prompts-for-llms Goto Github PK

Thank you for sharing this interesting work.
As I looked at the prompts used in the work, I noticed almost in all cases the negations simply replace the word 'correct' with 'incorrect'. I wonder whether you also tested other negation instructions, and would the results still be the same?
For example, switching instructions and questions so that the instructions are directly followed by the answers:

Original:

Generate the incorrect answer to the following question. Question: Astronauts weigh more on Earth than they do on the moon because Answer is

Inverse:

Question: Astronauts weigh more on Earth than they do on the moon because what?
Generate an incorrect answer to the above question. The answer is

For an AR language model, the current prompt "Astronauts weigh more on Earth than they do on the moon because" seems to me a little misleading that the model may tend to simply complete the sentence based on its knowledge regardless of the instruction. Maybe (for now I don't have an OpenAI API to verify this) using a different prompt structure will vary the results?

Any discussion by anyone is welcome.