LLM_Water is an application mainly based on LangChain_ChatGLM. It mainly focus on the QA function, based on knowledge base about water conservancy. For now, there aren't enough changes from Langchain_ChatGLM. So if you'd like to learn about how it works logically, I suggest you click the open link above. This README file is mainly focus on the way to use it and the work I need to go on.

Make sure your Python version is at least higher than 3.8.

python --version
# Python 3.10.11

If your version is less than Python 3.8, you need to update it or create a virtual environment by Anaconda.

conda create -p /your_path/env_name python=3.8

# activate
conda activate /your_path/env_name

# install
git clone https://github.com/imClumsyPanda/langchain-ChatGLM.git
cd langchain-ChatGLM

Install packages.
```python
pip install -r requirements.txt
# The packages offered by LangChain to deal with unstructed text
pip install "unstructured[local-inference]"
pip install "detectron2@git+https://github.com/facebookresearch/[email protected]#egg=detectron2"
pip install layoutparser[layoutmodels,tesseract]

Milvus can be used to replace FAISS as the vector store.

Here are the ways to install docker and docker-compose. And if you have podman on your server, you can use podman and podman-compose to replace docker.

Get docker-compose file.

wget https://github.com/milvus-io/milvus/releases/download/v2.2.10/milvus-standalone-docker-compose.yml -O docker-compose.yml

Run.

sudo docker-compose up -d
sudo docker-compose ps
sudo docker-compose down

If you'd like to run with podman, use this.

mv docker-compose.yml podman-compose.yml
sudo podman-compose up -d
sudo podman-compose ps
sudo podman-compose down

First, check configs/model_config.py to make sure your server is capable to run.

Besides, embedding model also needs about 3GB GPU Memory, or you can change it to CPU mode. If you find out that your GPU Memory is not capable to run, please change the args in configs/model_config.py and models/loader/args.py.

#model_config.py
LLM_MODEL = "chatglm-6b"

#args.py
parser.add_argument('--model', type=str, default='chatglm-6b', help='Name of the model to load by default.')

And if you'd like to run these model locally, you can follow the steps below to download the models.

# install git-lfs
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
sudo yum install git-lfs
git lfs install

# download the model
git clone https://huggingface.co/THUDM/chatglm-6b

If the speed is too slow to download from huggingface, try this.

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/THUDM/chatglm-6b

Then download the download the checkpoints from here. And change the llm_model_dict path and the embedding_model_dict path in configs/model_config.py.

Run the demo.

python cli_demo.py

Run the webui.py

python webui.py

#output
Running on local URL:  http://0.0.0.0:7861

To create a public link, set `share=True` in `launch()`.

You can open the link if you are running loaclly. Or if you are running on a server, you need to open the port 7861 using the firewall. And you can enter the link and find out the page below. Just follow the instruction step by step. And you can get an easy QA application based on knowledge base. If you are a member of our team, you can read this file to get a detailed instruction.

The documents you upload will be stored in content folder. And the processed index for faiss will be stored in vector_store.

Create and combine Agents and Chains/Tools Test the accuracy when there are a large amount of documents. Accelerate the generation of ChatGLM