Integration with Ollama about skyvern HOT 9 CLOSED

We're going to release some LLM router changes really soon (testing's almost complete!!!)

We used LiteLLM to implement a configurable, easily extensible LLM manager. I just checked it out and it has support for Ollama (https://litellm.vercel.app/docs/providers/ollama).

We'd appreciate it if you could open a PR to add support for Ollama after we merge the LLM router changes. I'll let you know when that PR is merged.

from skyvern.

Our first release with the brand new LLM module is out. Feel free to check out the release notes for more details: https://github.com/Skyvern-AI/skyvern/releases/tag/0.1.1

from skyvern.

#76

from skyvern.

Closing this as litellm supports ollama https://docs.litellm.ai/docs/providers/ollama

from skyvern.

@suchintan apologies for commenting on a closed PR, but wondering how do we use this to use local (Ollama) models? Can you provide a simple example...thanks!

from skyvern.

@arsaboo #242

I closed this a little bit hastily -- we would love a contribution in #242 -- just need to add a new model provider type in the code and LiteLLM would take care of the rest

from skyvern.

Thanks....will keep an eye on that. Hopefully, we can get it soon.

from skyvern.

@arsaboo If you're open to writing a little bit of code -- we'd love some help here! :)

from skyvern.

@suchintan I will take a look at this, but I think it is above my pay grade 😜

I would love to help you implement this (testing or docs). Given how expensive GPT4 is, local models are a must. I have a very capable device that can run large models and would love to test this with local models. Litellm devs are very active on GH, and you may have some luck with them.

from skyvern.