How to load third party models? about llm-workflow-engine HOT 2 CLOSED

I've added Ollama support via a provider plugin: https://github.com/llm-workflow-engine/lwe-plugin-provider-chat-ollama

Also Anyscale support: https://github.com/llm-workflow-engine/lwe-plugin-provider-chat-anyscale

This now gives access to a wide range of OSS LLMs.

from llm-workflow-engine.

Besides the default OpenAI Chat plugin, these are the following provider plugins that have been implemented:

https://github.com/orgs/llm-workflow-engine/repositories?q=lwe-plugin-provider-&type=all&language=&sort=name

I suspect https://github.com/llm-workflow-engine/lwe-plugin-provider-huggingface-hub could run local custom models, but I've only tried it with a few of the more popular hosted models on Hugging Face.

Writing a new provider plugin is fairly simple, if someone wanted to, for example, run Llama locally. Best documentation for how to do that is reading the code of existing plugins, and asking here.

from llm-workflow-engine.