Giter VIP home page Giter VIP logo

elasticsearch-langchain-chatglm2's Introduction

ElasticSearch-Langchain-Chatglm2

项目介绍

langchain-ChatGLM项目启发,由于Elasticsearch可实现文本和向量两种方式混合查询,且在业务场景中使用更广泛,因此本项目用Elasticsearch代替Faiss作为知识存储库,利用Langchain+Chatglm2实现基于自有知识库的智能问答。

本项目希望抛砖引玉,能够帮助大家快速地做技术验证和技术路线选取。

默认使用的embedding模型为moka-ai/m3e-large

目前仅支持上传 txt、docx、md等文本格式文件。

默认使用余弦距离计算文本相似性。

使用方式

修改配置文件

修改配置文件config.ini,配置Elasticsearch链接

模型可修改为本地路径

运行web demo

执行web.py

python web.py

Demo讲解

文档交互模块:

ES插入时文档交互模块会显示插入是否成功,或抛出异常内容;问答时,文档交互模块会展示查询到的内容,包括文档来源,文档内容和分数

查询设置模块:

近似查询:Approximate kNN

混合查询:Combine approximate kNN with other features

精确查询:Exact, brute-force kNN

top_k:取最相关的k个文本

knn_boost:knn score所占比例

Docker 部署

打包docker镜像

docker build -f docker/Dockerfile -t es-chatglm:v1.0 .

启动docker容器

docker run --gpus "device=0" -p 8000:8000 -it es-chatglm:v1.0 bash

To do

  1. 多文档类型支持
  2. 文档切分可配置
  3. docker部署Elasticsearch教程

引用及感谢

  1. THUDM/chatglm2-6b
  2. moka-ai/m3e-large
  3. LangChain
  4. langchain-ChatGLM

联系方式

[email protected]

欢迎沟通交流!

elasticsearch-langchain-chatglm2's People

Contributors

imagist486 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.