Comments (5)
It seems like you're running into rate limiting issues due to OpenAI's API. You could request a rate limit increase from OpenAI if they allow it.
Also, currently, we utilize multithreading to build the leaf nodes. You can switch off multithreading, which will make it slower but should help avoid hitting the rate limits.
To make this change, update the following line in raptor/RetrievalAugmentation.py
:
raptor/raptor/RetrievalAugmentation.py
Line 219 in 2e3e83e
From:
self.tree = self.tree_builder.build_from_text(text=docs)
To:
self.tree = self.tree_builder.build_from_text(text=docs, use_multithreading=False)
from raptor.
Are you using the default OpenAI embeddings? Is there an error message or is the code stalling?
from raptor.
I have noticed that you have already added retry decorators, but the 429 response is still being triggered. Only some open-source embedding methods seem to work.
@Retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6))
def create_embedding(self, text):
text = text.replace("\n", " ")
return (
self.client.embeddings.create(input=[text], model=self.model)
.data[0]
.embedding
)
If I shortened the text, the embedding API would work. However, if the text gets longer, the following information must be shown:
2024-03-15 07:46:21,911 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,912 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,912 - Retrying request to /embeddings in 20.000000 seconds
2024-03-15 07:46:21,913 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,916 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,916 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,917 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,918 - Retrying request to /embeddings in 20.000000 seconds
2024-03-15 07:46:21,922 - Retrying request to /embeddings in 20.000000 seconds
2024-03-15 07:46:21,925 - Retrying request to /embeddings in 20.000000 seconds
2024-03-15 07:46:21,926 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,928 - Retrying request to /embeddings in 20.000000 seconds
2024-03-15 07:46:21,930 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,933 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
2024-03-15 07:46:21,934 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
from raptor.
Thanks, Bro, Follow your instrcution, I changed the line.
But it still works in short contexts. When dealing with longer ones, timeouts still occur again.
from raptor.
Strange! It works fine when I run the demo for the first time. But when I rerun the demo, an error occurred, the text in demo seems too long for raptor.
I can run it successfully after intercepting part of the content.
EmbedModel, QAModel and SummModel are all custom.
Follow this, the question are solved.
Update:
When switching off multithreading, building a tree from a story extracted from NarrativeQA datasets costs too long......
Is there any ideas to fix this problem? @parthsarthi03
from raptor.
Related Issues (20)
- NarrativeQA metrics
- TypeError: Cannot use scipy.linalg.eigh for sparse A with k >= N. Use scipy.linalg.eigh(A.toarray()) or reduce k. HOT 4
- Question about multiple texts HOT 3
- Question about experiment HOT 3
- Missing implied functionality `add to existing` HOT 3
- Support for Azure OpenAi HOT 2
- Return Citation HOT 2
- Constructing Layer Issue HOT 3
- Is it not possible to use in AZURE? HOT 1
- Adding new Document to the existing RAPTOR setup
- num_layers acts like max_num_layers HOT 2
- UMAP n_neighbors must be greater than 1 HOT 10
- Inquiring about Vector DB implementation HOT 2
- multi doc HOT 2
- There seems to be a bug in the demo code.
- Evaluation code HOT 1
- Support for seeing Created Tree ? HOT 4
- **Outdated Dependencies in requirements.txt Causing Conflicts** HOT 1
- Bug in chunk splitting HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from raptor.