Comments (12)
I guess I'd be open to additional API for this. Something along the lines of a get_weak()
method on the Pool
, which clearly documents the risk of leaking the connection and explicitly points to get()
as the preferred alternative.
from bb8.
In principle, the lifetime is there on purpose to make it harder to "leak" pooled connections. Why don't you keep a reference to the pool itself in each thread and keep PooledConnection
s scoped to the thread they originated in?
from bb8.
Hi @djc, thanks for the response! I think your suggestion won't work (at least the way I have set things up), because the thread originating the PooledConnection
is not the thread using the PooledConnection
.
I'll paste my relevant code below, which works with my branch of bb8
. It is based loosely on the sync/blocking interfaces of reqwest
and ldap3
. I'm not sure if it is sound, but it seems to work fine in some small testing. However, I understand your rationale for having pool be a ref, so feel free to close this if you like. Thanks!
impl<M: crate::ManageConnection> Pool<M> {
pub fn new(
builder: bb8::Builder<ManageConnectionContainer<M>>,
manager: M,
) -> Result<Self, BuildError<M::Error>> {
use futures::future::{Future, TryFutureExt};
let (tx, mut rx) = mpsc::unbounded_channel::<
oneshot::Sender<
Result<
bb8::PooledConnection<ManageConnectionContainer<M>>,
bb8::RunError<M::Error>,
>,
>,
>();
let (tx_b, rx_b) = oneshot::channel::<Result<(), BuildError<M::Error>>>();
let handle = std::thread::Builder::new()
.name("bb8-sync-runtime".into())
.spawn(move || {
let mut rt = match runtime::Builder::new()
.basic_scheduler()
.enable_all()
.build()
.map_err(BuildError::TokioIo)
{
Ok(rt) => rt,
Err(e) => {
tx_b.send(Err(e)).unwrap();
return;
}
};
rt.block_on(async move {
let pool = match builder
.build(ManageConnectionContainer(manager))
.map_err(BuildError::ManagedConnectionError)
.await
{
Ok(pool) => pool,
Err(e) => {
tx_b.send(Err(e)).unwrap();
return;
}
};
tx_b.send(Ok(())).unwrap();
while let Some(mut tx_get) = rx.recv().await {
let pool = pool.clone();
tokio::spawn(async move {
use std::task::Poll;
let fut = pool.get();
futures::pin_mut!(fut);
let res =
futures::future::poll_fn(|ctx| match fut.as_mut().poll(ctx) {
Poll::Ready(val) => Poll::Ready(Some(val)),
Poll::Pending => {
futures::ready!(tx_get.poll_closed(ctx));
Poll::Ready(None)
}
})
.await;
if let Some(res) = res {
let _ = tx_get.send(res);
}
});
}
});
})
.unwrap();
futures::executor::block_on(async move { rx_b.await.unwrap() })?;
Ok(Pool { tx, handle })
}
pub fn get(
&self,
) -> Result<bb8::PooledConnection<ManageConnectionContainer<M>>, bb8::RunError<M::Error>> {
let (tx, rx) = oneshot::channel();
self.tx.send(tx).ok().unwrap();
futures::executor::block_on(async move { rx.await.unwrap() })
}
}
from bb8.
What I was asking is, why do you need to get the PooledConnection
out of the pool on a different thread than the thread where you use it?
from bb8.
@djc In my example, it is because the Tokio runtime is working in a background thread. Even if I store the Pool
outside of that thread, I will need to send it across to the Tokio thread and send back a PooledConnection
, which won't work because of the 'static
lifetime requirement.
from bb8.
So you're saying the thread where you want to use the pooled connection is not managed by tokio, therefore you cannot get a connection from the pool directly on the thread where you want to use it?
from bb8.
Correct. In the code above, I am using combined oneshot
/mpsc
channels to get the PooledConnection
back to the current thread. This is similar to how reqwest
provides its blocking
API.
from bb8.
Isn't there some async pooling solution for whatever runtime is managing your thread?
from bb8.
I don't think I understand the question.
from bb8.
Why don't you run your worker thread/task (the one that needs the database connection) in the context of the tokio runtime?
from bb8.
I'm also interested in something like this. My use-case is that I have a trait along the lines of
#[async_trait]
pub trait FromRequest {
async fn from_request(req: &mut http::Request<hyper::Body>) -> Result<Self, Error>;
}
And would like to implement it for pooled connections in such a way that it would get the pool from request extensions and then grab a connection from there. However since the connection has a reference back to the pool, I cannot return it.
Making the trait generic over the lifetime of the request wouldn't work since its borrowed mutably leading to multiple mutable borrows when using the trait.
If the pooled connection was fully owned however, everything should work.
from bb8.
I've submitted #107 to address this 👀
from bb8.
Related Issues (20)
- Time taken to get a connection from bb8 redis pool is very high HOT 2
- Unable to use bb8-redis pool connection to execute `query_async` commands on redis HOT 3
- Why bb8-redis can't support redis features? HOT 3
- ACL support for redis HOT 2
- Error sink not called unless debugger is attached with breakpoints HOT 2
- Reentrant deadlock HOT 5
- Reaper default frequency conflicts with other settings
- Unable to connect to secure client using bb8-redis HOT 2
- TimedOut error when has_broken = true HOT 8
- add a tag for v0.8.1 HOT 2
- Expose internal connection type in some way HOT 2
- Support for pool creation without requiring initial connection HOT 2
- Unusually slow to give redis connections HOT 7
- Close pool and all connections? HOT 1
- Support for pool creation with concurrent connections creation HOT 5
- Attempt to subtract with overflow, new version 8.2 HOT 5
- Timeout Issue with bb8-redis 0.14.0 on GCP Memorystore Redis HOT 5
- Changelog HOT 1
- Remove async-trait in favor of impl trait and native async in impls HOT 6
- Support async behavior to determine if the connection is no longer usable in ManageConnection Trait HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bb8.