quivrhq / quivr Goto Github PK

Before building quivr out further, I would like to look into this problem: https://twitter.com/bitfalls/status/1657736877007405056?s=20

Specifically, I fed it around 30 markdown documents, but it looks like it does not really follow context and has no real idea what is going on. Every article has a "status" at the top, and when I asked it to give me all active ones, it just gives me random ones, while asking for the status of one I know is active it tells me it is active. This shows me that it can check for status of a targeted ask, but cannot reason on the whole corpus of information. I wonder why this happens and if we can debug it, because unless this is resolved, Quivr is dangerously unreliable to use.

@StanGirard any ideas on what to do to test this and come to a satisfactory output?

As discussed here.

The ideal UX flow would be:

• on answer, add three buttons

  • 1. Build on answer
  • 2. New context answer
  • 3. Fork context

• "Build on answer" adds this question and answer to the DB for future queries.

• "New context answer" works as it does now, fresh context API call with only current raw-data vectors, previous questions and answers are ignored, even if there are some.

• "Fork context" allows you to branch off of a certain answer (even past ones) with a different query, like so: suppose there was Context 0 -> Question 1 -> Answer 1 -> Question 2 -> Answer 2. I can scroll up to Answer 1, click Fork Context, and the next query I write will pretend Question 2 and Answer 2 never happened. This allows the user to have branched context-persistent conversations with the API and the ability to fine tune responses from the past based on new prompt insights from the future. It is also very useful for prompt auditing and reporting.

I understand 3) is very hard to build right now as it also requires a separate UI element for the forked conversations so 1) and 2) would be good enough, especially 1) which is very important in longer-context plans.

Add the following options for vector store:

1. embedded weaviate
2. WCS
3. Weaviate docker compose

Even if we miss https:// from a URL, it fails with this error:

The browser shows "PLEASE WAIT " indefinetly. Any ideas? All dependencies installed correctly, ran table creation script in supabase successfully, tried credentials in secrets.toml with and without quotes.

Render Javascript when loading an URL

Looking for a contributor

errorquiver.docx

a dependency in realtime-py with web sockets messes the execution of requirements

Might be better than issues for discussing future plans / other libraries etc.

Stack End Goal:

• Python with FastAPI for the Backend
• NextJS for the WebAPP
• Electron + NextJS Desktop App

All future code will go toward this.

Looking for NextJS experts ;)

Vector Stores to implement:

• Weaviate
• Chromadb

LLM support

• GPT4all
• Llama
• Whisper alternative

Features

• Copy/Paste information #19
• Training brain with its answers #16
• Sources to output #11
• Self Hosting LLM and Embeddings #5
• History of answers #13
• Zip Ingestion

Please check the link, I can't join the discord channel.

Everything is fine, but this prompt does not support languages other than English

Allow users to copy paste content in a text.input_area

Looking for a contributor

What license is being used for this project?

Obsidian uses MIT License:

https://github.com/open-source-labs/obsidian/blob/master/LICENSE

Awesome project!!! it will help me to spend less time to read papers, and to acquire a lot key messages.
GPTCache maybe reduced cost of use makes this project more attractive.

Hi there, I would like to work on the website redesign for quivr.app. Could you please assign this issue to me? I have experience with website design and development with Next JS, and I am excited to contribute to this project. Thank you!

I know this project is very reliant on cloud services in it's current state and for some people that may be fine.

Equally it may also now be possible to run this application using fully self hosted services.

• Whisper - https://github.com/guillaumekln/faster-whisper
• GPT - https://github.com/nomic-ai/gpt4all

Is this considered in or out of scope for this project? Would you accept a PR that provided local support?

Could you please show how to deploy this project on Vercel for better access, (such as how to set the needed openai_api_key\supabase_service_key\supabase_url, etc, tried to set them into the env.vars but got 404), thanks!

Deploy the app to streamlit

Needs to load the API_KEY and all via the streamlit app.

It means adding a configuration box to ask the users for their API_KEYS instead of reading it from a secret

Hi,

would be great if you could add a link to the referenced document in the output.

thanks

can consume word file?

Hey,

Would it be a big deal to keep previous questions/answers visible and add a scroll bar?

thank you,

Not all knowledge are structured. It would be great to allow adding adhoc knowledge into our brain

Whisper API limits upload to 25MB. If it is case maybe you will need to chunk the audio file before processsing

Tried Demo first time, am I using it wrong?

Hi,

I was just trying to install this code and it failed for me because I did not have pycryptodome, so I was wondering if it would be good to add it to the list?

This was on a new clean install of Ubuntu 22.04

Joe

Add support for multiple URLs loading.

Looking for a contributor

Run the migration script on the Supabase database via the web interface

Sorry, I'm new to this stuff and this simple sentence is very beginner-unfriendly to me. Could you point me out where to find the web interface?

Add source to output messages

Looking for a contributor

Hey, your app is really great. a filter based on the file_name in metadata while calling ConversationalRetrievalChain would be great so that I can ask questions to selected documents only.

Individuals and organizations would need multiple brains depending on the use case. For example, I may create separate brains for programming, growth strategy, product management, philosophy, negotiation, healthcare innovation, politics, and artificial intelligence.

Each could require its own set documents, chunk size, chunk overlap, foundation model, and in the future, even its own data model. For example, each brain may require not just retrieval of context relevant to the query, but for the retrieved context to cover various types of data. For a financial advisor, depending on the query, it may make sense for the context to include some mix of news, market data, and applied reasoning data (equity research reports).

With more people using it, it would be great to have a discord server

The above error occurred when adding a website."

I got Quivr up and running, Windows 10, but am having issues when I try to upload a file. It seems it doesn't have permission to access the Temp folder? It should as far as I can tell.

PermissionError: [Errno 13] Permission denied: 'C:\Users\29158~1.1E1\AppData\Local\Temp\tmpl5t8vu99.txt'
Traceback:

File "C:\Users\Laptop\Documents\GitHub\quivr\venv\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
exec(code, module.dict)
File "C:\Users\Laptop\Documents\GitHub\quivr\main.py", line 66, in
file_uploader(supabase, openai_api_key, vector_store)
File "C:\Users\Laptop\Documents\GitHub\quivr\files.py", line 37, in file_uploader
file_processors[file_extension](vector_store, file)
File "C:\Users\Laptop\Documents\GitHub\quivr\loaders\txt.py", line 5, in process_txt
return process_file(vector_store, file, TextLoader, ".txt")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Laptop\Documents\GitHub\quivr\loaders\common.py", line 19, in process_file
documents = loader.load()
^^^^^^^^^^^^^
File "C:\Users\Laptop\Documents\GitHub\quivr\venv\Lib\site-packages\langchain\document_loaders\text.py", line 17, in load
with open(self.file_path, encoding=self.encoding) as f:

I think I followed all the instructions but once the streamlit runs I drag a PDF and when a click on Add to Database, this error is shown.
Any idea?

THANK YOU !!!

Allow quiver to use 100k context

Allow users to load a zip that will be de-zipped

Looking for a contributor

Have a recursive crawling over a website, pulls every other website linked in the website

Hi,

Should one be able to test this system with the free version of OpenAI?

I have managed to install all of the software but I received an error saying I had reached my OpenAI limit when trying to upload my first PDF.

I am currently exploring the alternative Anthropic models as well.

Thanks

Joe