[BUG]: Ollama models not loading with http://172.17.0.1:11434 on Debian 12 about anything-llm HOT 4 CLOSED

The same problem on MacOS Desktop version v1.5.7

from anything-llm.

The connector does work, you just are using the wrong URL or Ollama is actually not running on the host.
assuming both are on the same machine and you _confirmed that ollama serve is running.

If on Docker:

• you need to specific the host machine IP or you can use HTTP://host.docker.internal to access the host machine
  However, on Ubuntu/Linux docker the above URL does not work and you need to use 172.17.0.1:11434. If this is not your situation you dont need to use this URL. This also presumes you have not already tried to boot Ollama with the proper ENVs for ollama to be able to listen on the correct loopback. 172.17.0.1 = 0.0.0.0

On desktop:
HTTP://127.0.01:11434

Things to read about this:
https://docs.useanything.com/ollama-models-not-loading
https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md

from anything-llm.

After following the documentation at https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md I managed to resolve the issue by binding Ollama to 0.0.0.0 and using 172.17.0.1:11434 as base URL.

Thank you. For future problems I'll take a closer look at the github before creating a new issue!

from anything-llm.

@Mattias-Code Thanks for the reply + resolution. Also no big deal, we get Ollama connection issues about 5-10 times a week because of that caveat on address binding Ollama has - you are not alone!

from anything-llm.