๐ A Robust Open Source AI studio. A Universal API for inferencing 100+ LLMs(OpenAI, Azure, Cohere, Anthropic, HuggingFace, Replicate, Stable Diffusion).
Not supported (yet): images, audio, files, fine-tunes, moderations
Missing studio can be deployed in a variety of ways. It is deployable on bare metal, or in dockerized environments.
- Buf CLI is required for building proto.
To start missing studio server, simply run the following command:
make compose-dev
Your AI Gateway is now running on http://localhost:8080 ๐ฅ
curl \
--header "Content-Type: application/json" \
--header "x-ms-provider: openai" \
--header "Authorization: Bearer {{OPENAI_API_KEY}}" \
--data '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"who are you?"}]}' \
http://localhost:8080/v1/chat/completions
grpcurl \
-d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"hi"}]}' \
-H 'x-ms-provider: openai' \
-H 'Authorization: Bearer {{OPENAI_API_KEY}}' \
-plaintext localhost:8080 llm.v1.LLMService.ChatCompletions
- ๐ Universal API - Call every LLM API like it's OpenAI
- AI Gateway for Security, Reliability and Observability
- Load balancing across multiple provider and models
- Atomatic Retries with exponential backoff
- Rate limiting
- Caching
- Fallbacks
- Monitoring
- Alerting
- Analytics
- AI Studio
- AI Agents
- AI Workflow builder
- OSS AI Models Inferancing
- Serving model api at Scale on Kubernetes ๐ฆ๏ธ
- Building dev tools (CLI, SDK, API Client)
AI studio is an open-source project, and contributions are welcome. If you want to contribute, you can create new features, fix bugs, or improve the infrastructure.
It's still very early days for this so your mileage will vary here and lots of things will break. But almost any contribution will be beneficial at this point. Check the current Issues to see where you can jump in!
If you've got an improvement, just send in a pull request!
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'feat(module): add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request
If you've got feature ideas, simply open a new issues!
Please refer to the CONTRIBUTING.md file in the repository for more information on how to contribute.
AI Studio is Apache 2.0 licensed.