Support additional chat LLMs about ontogpt HOT 7 OPEN

h/t @justaddcoffee https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

from ontogpt.

• I fully support the direction we are heading in, and I am intrigued to see which option we will choose from the several that Harry has proposed.
• I would like to provide some background idea why I support this direction. Currently, I cannot use ontoGPT in my organization as it is due to security concerns around external APIs like OpenAI's. To address this issue, I suggest exploring the possibility of using open-source, local LLMs and their APIs as an alternative to OpenAI's API. This alternative could benefit other users who want to minimize operating costs and speed up processing using their local GPU machines, I believe. Btw, this is not an urgent requirement at all. I just wish to share my mere suggestions. Thanks (CC: @cmungall).

from ontogpt.

Just FYI. I actually check with my peers about the use of any open-source LLMs in my organization and they suggested the use of instruction-finetuned LLMs: Flan-T5 and Dolly v2. I wonder if ontoGPT could also additionally suppose these LLMs in the future, if you are okay. Thanks.

from ontogpt.

Needs #93

from ontogpt.

Notes for later reference:

• FLAN-T5: it's on Huggingface Hub , so that's probably the best way to use it through langchain. See https://colab.research.google.com/drive/19Zr9PBgPX3Q8B8yHXYFatCodnNdqP8fg for example
• Dolly-v2: same deal re Huggingface; see potentially outdated example here: langchain-ai/langchain#2928

Supporting HF Hub opens up access to many models...

from ontogpt.

Currently, I cannot use ontoGPT in my organization as it is due to security concerns around external APIs like OpenAI's. To address this issue, I suggest exploring the possibility of using open-source, local LLMs and their APIs as an alternative to OpenAI's API.

I have a similar use case, however, could it ever be abstracted even further, with an option to use in-house LLM services? We are currently working on such a ChatGPT alternative, and it would be amazing to simply plug in that API for ontoGPT...

from ontogpt.

Sure! I'm going to open up another issue along those lines.

from ontogpt.