This is the official repository for the paper "Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System". In this paper, we introduce the Self-Controlled Memory (SCM) system to unleash infinite-length input capacity for large-scale language models. Our SCM system is composed of three key modules: the language model agent, the memory stream, and the memory controller.

• [2023-12-14] Newest version of the paper is released. Check it out in SCM_Paper_Draft.pdf!
• [2023-4-26] We released our first version paper, codes. Check it out!

Our SCM system can be integrated with any LLMs to enable them to process ultra-long texts without any modification or fine-tuning.

In config directory, copy apikey.txt.template to apikey.txt, put your openai apikey in it, support multiple keys. This file is ignored to protect privacy.

Note: If the ChatGPT service is unavailable in your area, please utilize proxy settings. Copy config\api_config.template.json to config\api_config.json and config http_proxy as you like.

The key requirements are as below:

• python 3.8+
• openai 0.27.0+
• gradio 3.27.0+

Use conda to create environment.

conda create -n scm python=3.8 -y
conda activate scm

You can install the requirements by running:

pip install -r requirements.txt

Default agent model use text-davinci-003.

You can specify model by --model_name, current support model list:

• text-davinci-003
• gpt-3.5-turbo

Run this command, chat with model. Chat logs are recorded in logs\log.txt.

python dialogue_demo.py

Functional command during dialogue, these operations will be silently done, you can see them in the log output:

• reset or 清空: clear dialogue history.
• export or 导出: save the dialogue history to files.
• roll back or 回滚: pop previous turn dialogue.

Take the shortest book The Old Man and the Sea, whose content cost 34k tokens, as a demo example:

python book_summary.py   --book_files data/book/EnglishBook/The_Old_Man_and_the_Sea.txt

Take the blockchain meeting as example, whose content cost 37k tokens, as a demo example:

python meeting_summary.py --meeting_ids 26231372_区块链技术的应用前景

Data files are in data folder.

Evaluation dataset statistics. 2M means 2 miillion token count.

• Chinese Long-term Dialogue QA results
• English Long-term Dialogue QA results
• Book Summarization results
• Meeting Summarization Results

More running records and summary details are zipped to history.zip. You can download it from Google Drive Link and Baidu Netdisk. call_embedding_history.json and call_func_history.json store the openai cache log and are zipped in logs.zip. You can download it from Google Drive Link and Baidu Netdisk.

we will assess the efficacy of our system on more open-source models that possess single-turn instruction comprehension capability.

Our system has the capability to attach to any LLMs, which may be prone to factual errors, delusions, toxic language, and malicious responses. Consequently, we restrict the usage of our system to academic research purposes for now.

If you find our work is helpful, please cite as:

@article{liang2023unleashing,
      title={Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System}, 
      author={Xinnian Liang and Bing Wang and Hui Huang and Shuangzhi Wu and Peihao Wu and Lu Lu and Zejun Ma and Zhoujun Li},
      year={2023},
      eprint={2304.13343}
}

We welcome contributions and suggestions!