This is a simple example of using mojo to serve gpt2 and gpt3.5 models.

You can also read tech article from here

1. Install dependencies

   python3 -m venv .venv
   source .venv/bin/activate
   pip install --upgrade pip
   pip install -r requirements.txt
   

2. check mojo

   mojo --version
   

3. Run mojo

   mojo gpt2.mojo
   

I use transformers library to load the model and tokenizer. so you don't need to set api key. https://huggingface.co/docs/transformers/en/model_doc/gpt2

mojo gpt2.mojo

I use openai's gpt3.5 model. You need to set api key in .env file.

copy .env.copy .env
echo "OPENAI_API = 'your api key'" >> .env

https://platform.openai.com/api-keys

mojo gpt35.mojo

You can also run the same model with python.

python gpt2.py

python gpt35.py

You can see mojo is a little faster than python. But the difference is not so big.