- 🎯 喜欢python、transformers、nlp、pytorch
yuanzhoulvpi2017 / documentsearch Goto Github PK
View Code? Open in Web Editor NEW基于sentence transformers和chatglm实现的文档搜索工具
License: Apache License 2.0
基于sentence transformers和chatglm实现的文档搜索工具
License: Apache License 2.0
您好,想請問目前支援 pdf 和 docx 外的格式嗎?
像 pandas.Dataframe(), JSON or text 之類的?
謝謝🙏
知识库中的文件内容的Embedding是用chinese-roberta-wwm-ext向量化模型做的
而输入问题的Embedding是用THUDM/chatglm-6b对话大模型做的
两者之间计算相似度合理吗?为什么不统一用一个模型做?
为什么 我换成 sber的取embedding 效果差很多,
大佬帮忙看看,执行命令 streamlit run web_ui.py --server.fileWatcherType none --server.port 8080
,项目启动后,提问时报错:
2023-04-27 11:40:16.159 Uncaught app exception
Traceback (most recent call last):
File "/root/anaconda3/envs/ds/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/root/DocumentSearch/web_ui.py", line 36, in <module>
output_str, output_df = kl.search_result(input_str)
File "/root/DocumentSearch/demo.py", line 252, in search_result
search_table_info = pd.concat(
File "/root/anaconda3/envs/ds/lib/python3.9/site-packages/pandas/util/_decorators.py", line 331, in wrapper
return func(*args, **kwargs)
File "/root/anaconda3/envs/ds/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 368, in concat
op = _Concatenator(
File "/root/anaconda3/envs/ds/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 425, in __init__
raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate
作者你好,我在colab上运行项目的时候出现了这样的问题,看代码中的cuda指定的是0也没有指定多个GPU。我不知道是什么原因导致的这个问题。
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be
incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA
to enable device-side assertions.
这种 和 直接用BM25检索本地的知识文档,效果有什么区别吗,,感觉这种就是稍微带点生成,
有的时候无法正确回答是不是因为一个chunk的信息不够导致的,谢谢
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.