Repository for the paper "The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning", including 1.84M CoT rationales extracted across 1,060 tasks"

Paper Link : https://arxiv.org/abs/2305.14045

Overview of CoT Collection.

You could access CoT Collection via huggingface datasets library as follows:

from datasets import load_dataset

dataset = load_dataset("kaist-ai/CoT-Collection")

You could access CoT-T5 trained on CoT Collection via huggingface transformers library as follows:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("kaist-ai/CoT-T5-11B")

model = AutoModelForSeq2SeqLM.from_pretrained("kaist-ai/CoT-T5-11B")

Note that we have a 11B version and a 3B version of CoT-T5!

We'll currently refactoring our code as well! We'll upload it soon:)

sh scripts/{subset}.sh # flan, sni, t0, t0p, additional

Rationales for CoTCollection can be obtained by running each script corresponding to the data-subset of CoTCollection.

On the first run, you will be asked to provide api keys for OpenAI.
These keys will be saved as locally saved as api.json and be reused for future runs.

Results,

• resutls for each instance during the augmentation process will be saved under "CoT_Rationale_Augmentation/outputs/{subset}/{model_name}/rat/temp_{temperature}/"
• after all augmentation process is finished the merged data can be found at "data_extraction/data/{split}/{subset}/codex_rationale_{split}_{phase}.json"

CoT Collection is only for non-commercial use and is subject to OpenAI's Terms of Use for the generated data. If you suspect any violations, please reach out to us.

If you find this useful, please consider citing our paper:

@article{kim2023cot,
  title={The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning},
  author={Kim, Seungone and Joo, Se June and Kim, Doyoung and Jang, Joel and Ye, Seonghyeon and Shin, Jamin and Seo, Minjoon},
  journal={arXiv preprint arXiv:2305.14045},
  year={2023}
}

For any questions about the implementation or content of the paper, you could contact me via the following email:)

[email protected]