dylanjcastillo / chatbot-code-interpreter Goto Github PK

Heya Dylan, nice repo. The current usage of chain_create.run in @post(path="/ask", name="ask") is a blocking operation (internally uses requests a sync library) which causes the event loop to be blocked and can reduce the throughput of your server.

Unlike FastAPI which automagically runs def routes in a thread pool, Litestar does not do so, one has to explicitly use sync_to_thread=True in the route handler @post(path="/ask", name="ask", sync_to_thread=True) to ensure it runs in a thread pool in a non blocking way.

Feel free to ignore if you are aware and for a valid reason did not decide to use this, just wanted to let you know 🙂.