No models shown in the 'Choose a model' dropdown about alpaca-turbo HOT 19 CLOSED

Alpaca Turbo is unable to load model on distant computer.

The problem is that the request IP address is set to "localhost" by default instead of the real current IP address of the alpaca server.

To solve the problem you can replace the following command const tt_socketUrl="http://localhost:5000" by const tt_socketUrl=window.location.href in the /templates/main.f729fcdb88c6ef0d.js file.

from alpaca-turbo.

Thanks @RocaroDev93, that fixed it.

from alpaca-turbo.

@JosephSBoyle which model are you using ? Because it gets stuck on loading even in localhost

from alpaca-turbo.

@Cabanera try this 7B one:
https://huggingface.co/Pi3141/alpaca-native-7B-ggml/blob/main/ggml-model-q4_0.bin

My understanding is that the upstream llama.cpp project changed their binary files and so some of the newer binaries don't work... This one works for me though:)

from alpaca-turbo.

I have this message when running it,

Address already in use
Port 5000 is in use by another program. Either identify and stop that program, or start the server with a different port.

How do I change that port to something else?

from alpaca-turbo.

I have this message when running it,

Address already in use Port 5000 is in use by another program. Either identify and stop that program, or start the server with a different port.

How do I change that port to something else?

Try restarting. If that doesn't work, search api.py for 5000 and replace it with something else. Bear in mind this will change the last part of the url you're accessing in your browser.

from alpaca-turbo.

@krravi55 To run the server with a different port, just duplicate the file api.py and rename it app.py. Then you can run the server with flask command:

1. Copy and rename api.py file as app.py
2. Open a terminal
3. Activate conda env with command conda activate alpaca_turbo
4. Run the server using the command flask run --port=<YOUR-PORT-NUMBER> → Ex: flask run --port=8080

from alpaca-turbo.

I have this message when running it,
Address already in use Port 5000 is in use by another program. Either identify and stop that program, or start the server with a different port.
How do I change that port to something else?

Try restarting. If that doesn't work, search api.py for 5000 and replace it with something else. Bear in mind this will change the last part of the url you're accessing in your browser.

I tried to set the parameter port of the function run in the api.py file but the server wasn't reachable on the selected port. The only solution I used that works was running the server with the command flask and setting the port with the command. But running a Flask server requires a app.py file in the directory. The api.py file is a app.py but the developper change its name. So renaming this file make the Flask server runable with the command flask

from alpaca-turbo.

Running the flask command worked! I tried searching for all instanced of "5000" in the folder and changed the port which were in two files and a javascript file but yet it somehow stuck to 5000. Not sure where that is coming from.

from alpaca-turbo.

@ViperX7, would you accept a PR with this change:

To solve the problem you can replace the following command const tt_socketUrl="http://localhost:5000" by const tt_socketUrl=window.location.href in the /templates/main.f729fcdb88c6ef0d.js file.

It isn't breaking and allows users to run inference from other devices.

from alpaca-turbo.

So you are removing a hardcoded URL value with getting the URL of the current window. But, still how do you ask flask to run it from a particular port?

from alpaca-turbo.

The hardcoded "localhost" doesn't change anything for the port problem.

If you want to ask the Flask serve being run wirh a different port use the following command:

Flask command: flask run --port 8080

For example if you want to run it on the 8080 port.

But before using this command, you need to activate the alpaca_turbo conda venv created when you followed the repo tutorial.

And also, the Flask server need to have a app.py file in the directory but it doesn't exist. The project developper have created the required file but the name was not app.py but api.py. If you rename the api.py to app.py, you'll be able to run the Flask command and so run the server on a different port.

But instead of renaming the api.py file to app.py, I suggest you to create a copy of the api.py file and rename the copy file to app.py. So you keep the project structure and you keep all other features that are dependent to the project structure like running the project on a Docker.

from alpaca-turbo.

I got the site to load. Now, I have this

n/chat/gpt4all-lora-quantized-OSX-m1 ; exit;
main: seed = 1680883907
llama_model_load: loading model from 'gpt4all-lora-quantized.bin' - please wait ...
llama_model_load: failed to open 'gpt4all-lora-quantized.bin'
main: failed to load model from 'gpt4all-lora-quantized.bin'

Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

from alpaca-turbo.

For anyone else still having the problem where the models won't load appear in the drop-down; the problem is that the models .bin file is missing from the /models/ directory. Goto the models dir, then wget https://huggingface.co/Pi3141/alpaca-native-7B-ggml/blob/main/ggml-model-q4_0.bin, then goto the primary dir, and run docker-compose up.

Then goto localhost:5000, the ggml-model should appear in the dropdown (or whatever model you put in the models directory), click change and hope it finishes loading.

from alpaca-turbo.

@aalbrightpdx the model that you linked wont work checkout the anouncement channel on discord for the list of supported models

from alpaca-turbo.

@ViperX7, would you accept a PR with this change:

To solve the problem you can replace the following command const tt_socketUrl="http://localhost:5000" by const tt_socketUrl=window.location.href in the /templates/main.f729fcdb88c6ef0d.js file.

It isn't breaking and allows users to run inference from other devices.

I would love to but please wait for the next release should be on monday then you can add a pr if required

from alpaca-turbo.

If I followed the directions in the windows installer video (I'm using Linux by the way so this was slightly confusing), then the correct steps would be:

1. Goto the /models/ directory
2. wget https://huggingface.co/Pi3141/alpaca-7b-native-enhanced/resolve/main/ggml-model-q4_1.bin
3. goto the primary directory
4. run docker-compose up
5. goto https://localhost:5000
6. click the drop down, select the model
7. click change
8. hope that the loading completes

During the video, ViperX7 specifically states that the import part is the Pi3141, therefore I would assume that all of the models in the https://huggingface.co/Pi3141/ url would probably also work, assuming your system can support them.

from alpaca-turbo.

Thank you guys. It was the problem with not downloading the correct model. Now it's working. I am new to this am just exploring LLM's.

What is the advanced mode in the interface?

from alpaca-turbo.

window.location.href

New v0.6 update.

This is still the valid solution for me. Though the name of the and format of the variable changed, it is still the second SocketURL that you have to change. Search for 7887, the new port and replace the second instance. However, the prompts don't return an answer for some reason. As seen from the debug screen the prompt is input, but processing does not start.
nt={production:!1,apiUrl:window.location.href,socketUrl:window.location.href}

from alpaca-turbo.