This is a simple example of using mojo to serve gpt2 and gpt3.5 models.
You can also read tech article from here
- Install dependencies
python3 -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install -r requirements.txt
- check mojo
mojo --version
- Run mojo
mojo gpt2.mojo
I use transformers library to load the model and tokenizer. so you don't need to set api key. https://huggingface.co/docs/transformers/en/model_doc/gpt2
mojo gpt2.mojo
I use openai's gpt3.5 model.
You need to set api key in .env
file.
copy .env.copy .env
echo "OPENAI_API = 'your api key'" >> .env
https://platform.openai.com/api-keys
mojo gpt35.mojo
You can also run the same model with python.
python gpt2.py
python gpt35.py
You can see mojo is a little faster than python. But the difference is not so big.