No model is shown in web-ui with docker about alpaca-turbo HOT 9 OPEN

With another model

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

i've just tried with conda, the models are showing but the error for each one is the following (paste just one of them but is the same ) :

[
'bin/main',
'-i',
'--seed',
'888777',
'-ins',
'-t',
'4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
 '--repeat_last_n',
 '64',
'--repeat_penalty',
'1',
 '--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR
[

with another model the previous error disappears but in webui it remains stuck in loading. And the log fle is the following:

[
'bin/main',
'-i',
'--seed',
 '888777',
'-ins',
'-t',
 '4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
'--repeat_last_n',
'64',
'--repeat_penalty',
'1',
'--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]

Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

That "Working" complete in 0 sec seems suspicious.

from alpaca-turbo.

what command you used to run alpaca turbo using Docker

from alpaca-turbo.

docker-compose up

from alpaca-turbo.

send me the output of

can you get inside the container using
docker-compose exec alpaca /bin/bash

and show me the output of
ls /app/models/

from alpaca-turbo.

send me the output of

can you get inside the container using docker-compose exec alpaca /bin/bash

and show me the output of ls /app/models/

ls /app/models/
ggml-alpaca-7b-q4.bin ggml-model-q4_0.bin ggml-model-q4_1.bin

from alpaca-turbo.

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

from alpaca-turbo.

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

i've just tried with conda, the models are showing but the error for each one is the following (paste just one of them but is the same ) :

[
'bin/main',
'-i',
'--seed',
'888777',
'-ins',
'-t',
'4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
 '--repeat_last_n',
 '64',
'--repeat_penalty',
'1',
 '--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR
[

from alpaca-turbo.

Same issue, the model sits inside docker container but the webui is not showing it

from alpaca-turbo.

I had the same issue.
For me the problem was I was running Docker on my server (network machine).
The way I got it working is by getting the source code and editing it and then rebuilding.

Load the ui folder into your code editor.
Go to src/app/environments/environment.prod.ts and change the https site to http://(network machine ip):5000

Then go to angular.json find and change the config as bellow:

"configurations": {
            "production": {
              "fileReplacements": [
                {
                  "replace": "src/app/environments/environment.ts",
                  "with": "src/app/environments/environment.prod.ts"
                }
              ],
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "500kb",
                  "maximumError": "1mb"
                },
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "2kb",
                  "maximumError": "4kb"
                }
              ],
              "outputHashing": "all"
            },

Then

npm install
npm run build

Go into your docker folder where you have your project (alpaca-turbo folder), into templates , create a folder called whatever you like and move the contents of templates to it.
Then copy everything from /dist/angular-turbo where you built the angular ui to alpaca-turbo/templates .

Now you should see your .bin files

Edit:
Or even easier , you can do what this guy said
#39 (comment)

from alpaca-turbo.