Analysis of In-The-Wild Jailbreak Prompts on LLMs

• We are going to try to figure out how people are using Large Language Models (LLMS) like ChatGPT. Specially we are looking into ethical use of these LLMs. We will be working on the "In-The-Wild Jailbreak Prompts on LLMs" dataset which contains 6,387 prompts from four platforms (Reddit, Discord, websites, and open-source datasets) during Dec 2022 to May 2023. Among these prompts, 666 prompts are jailbreak prompts. These are mainly prompts that go against the terms of service of the LLM vendors. So, we will analyze this data and see what kind of unethical use people are doing with these LLMs.

• Manas Tiwari (mtiwari), Tokey Tahmid (ttahmid), Tushar Panumatcha(tpanumat), Andy Zeng (azeng2)