Unable to run TGI following the instructions on the readme about text-generation-inference HOT 5 OPEN

Hey @xianbaoqian, could you share a bit about what is the exact problem you're seeing? I see two problems above, and I'm unsure of which one is affecting you, I see this one first:

(base) [email protected]:$ conda conda create -n text-generation-inference python=3.11
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'conda' (choose from 'activate', 'deactivate', 'clean', 'compare'
, 'config', 'create', 'env', 'export', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninst
all', 'rename', 'run', 'search', 'update', 'upgrade', 'content-trust', 'doctor', 'repoquery', 'build', 'develop',
'metapackage', 'render', 'index', 'debug', 'inspect', 'convert', 'skeleton')

Here it seems like you're calling conda twice.

I'm happy to help out, but it would be helpful if the formatting of the issue was easier to parse 🤗 thanks!

from text-generation-inference.

Hi @LysandreJik the problem is that after conda activate I still can't use text-generation-launcher as the binary file can't be found:

(base) [email protected]:$ conda activate text-generation-inference
(text-generation-inference) [email protected]:$ text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-
v0.2
bash: text-generation-launcher: command not found

from text-generation-inference.

I updated the style (surrending it with a code block) to make it easier to read. The original format is indeed a bit misleading!

from text-generation-inference.

Hey @xianbaoqian, thanks for clearing the style. There is definitely an issue in the code you run:

$ conda create -n text-generation-inference python=3.11                                  
$ conda activate text-generation-inference
$ text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-v0.2
bash: text-generation-launcher: command not found

Here you're only creating a new conda environment and activating it. You named it text-generation-inference, but that doesn't mean that it's installed.

If you took the example from this part of the README, it seems that you forgot this line:

+ BUILD_EXTENSIONS=True make install # Install repository and HF/transformer fork with CUDA kernels
text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-v0.2

Beware: it's a long installation. Using TGI with the docker image is recommended as much faster.

from text-generation-inference.