This repository attempts to combine the Dreamer reinforcement learning algorithm with a language modeling objective to determine if encoded world models can significantly improve text coherency, accuracy, and most importantly, soul.
No guarantees it'll work. But screw it, we ball.
pip3 install -r requirements.txt
- Adapt Dreamer-v1 model to text generation
- Set up configs
- Implement trainer
- Implement tokenizing
- Add huggingface
datasets
support - Implement DreamerV2 algorithm
- Implement DreamerV3 algorithm
- And more...