Giter VIP home page Giter VIP logo

Comments (12)

djc avatar djc commented on May 31, 2024 1

I guess I'd be open to additional API for this. Something along the lines of a get_weak() method on the Pool, which clearly documents the risk of leaking the connection and explicitly points to get() as the preferred alternative.

from bb8.

djc avatar djc commented on May 31, 2024

In principle, the lifetime is there on purpose to make it harder to "leak" pooled connections. Why don't you keep a reference to the pool itself in each thread and keep PooledConnections scoped to the thread they originated in?

from bb8.

kardeiz avatar kardeiz commented on May 31, 2024

Hi @djc, thanks for the response! I think your suggestion won't work (at least the way I have set things up), because the thread originating the PooledConnection is not the thread using the PooledConnection.

I'll paste my relevant code below, which works with my branch of bb8. It is based loosely on the sync/blocking interfaces of reqwest and ldap3. I'm not sure if it is sound, but it seems to work fine in some small testing. However, I understand your rationale for having pool be a ref, so feel free to close this if you like. Thanks!

impl<M: crate::ManageConnection> Pool<M> {
    pub fn new(
        builder: bb8::Builder<ManageConnectionContainer<M>>,
        manager: M,
    ) -> Result<Self, BuildError<M::Error>> {
        use futures::future::{Future, TryFutureExt};

        let (tx, mut rx) = mpsc::unbounded_channel::<
            oneshot::Sender<
                Result<
                    bb8::PooledConnection<ManageConnectionContainer<M>>,
                    bb8::RunError<M::Error>,
                >,
            >,
        >();
        let (tx_b, rx_b) = oneshot::channel::<Result<(), BuildError<M::Error>>>();

        let handle = std::thread::Builder::new()
            .name("bb8-sync-runtime".into())
            .spawn(move || {
                let mut rt = match runtime::Builder::new()
                    .basic_scheduler()
                    .enable_all()
                    .build()
                    .map_err(BuildError::TokioIo)
                {
                    Ok(rt) => rt,
                    Err(e) => {
                        tx_b.send(Err(e)).unwrap();
                        return;
                    }
                };

                rt.block_on(async move {
                    let pool = match builder
                        .build(ManageConnectionContainer(manager))
                        .map_err(BuildError::ManagedConnectionError)
                        .await
                    {
                        Ok(pool) => pool,
                        Err(e) => {
                            tx_b.send(Err(e)).unwrap();
                            return;
                        }
                    };

                    tx_b.send(Ok(())).unwrap();

                    while let Some(mut tx_get) = rx.recv().await {
                        let pool = pool.clone();
                        tokio::spawn(async move {
                            use std::task::Poll;

                            let fut = pool.get();

                            futures::pin_mut!(fut);

                            let res =
                                futures::future::poll_fn(|ctx| match fut.as_mut().poll(ctx) {
                                    Poll::Ready(val) => Poll::Ready(Some(val)),
                                    Poll::Pending => {
                                        futures::ready!(tx_get.poll_closed(ctx));
                                        Poll::Ready(None)
                                    }
                                })
                                .await;

                            if let Some(res) = res {
                                let _ = tx_get.send(res);
                            }
                        });
                    }
                });
            })
            .unwrap();

        futures::executor::block_on(async move { rx_b.await.unwrap() })?;

        Ok(Pool { tx, handle })
    }

    pub fn get(
        &self,
    ) -> Result<bb8::PooledConnection<ManageConnectionContainer<M>>, bb8::RunError<M::Error>> {
        let (tx, rx) = oneshot::channel();

        self.tx.send(tx).ok().unwrap();

        futures::executor::block_on(async move { rx.await.unwrap() })
    }
}

from bb8.

djc avatar djc commented on May 31, 2024

What I was asking is, why do you need to get the PooledConnection out of the pool on a different thread than the thread where you use it?

from bb8.

kardeiz avatar kardeiz commented on May 31, 2024

@djc In my example, it is because the Tokio runtime is working in a background thread. Even if I store the Pool outside of that thread, I will need to send it across to the Tokio thread and send back a PooledConnection, which won't work because of the 'static lifetime requirement.

from bb8.

djc avatar djc commented on May 31, 2024

So you're saying the thread where you want to use the pooled connection is not managed by tokio, therefore you cannot get a connection from the pool directly on the thread where you want to use it?

from bb8.

kardeiz avatar kardeiz commented on May 31, 2024

Correct. In the code above, I am using combined oneshot/mpsc channels to get the PooledConnection back to the current thread. This is similar to how reqwest provides its blocking API.

from bb8.

djc avatar djc commented on May 31, 2024

Isn't there some async pooling solution for whatever runtime is managing your thread?

from bb8.

kardeiz avatar kardeiz commented on May 31, 2024

I don't think I understand the question.

from bb8.

djc avatar djc commented on May 31, 2024

Why don't you run your worker thread/task (the one that needs the database connection) in the context of the tokio runtime?

from bb8.

davidpdrsn avatar davidpdrsn commented on May 31, 2024

I'm also interested in something like this. My use-case is that I have a trait along the lines of

#[async_trait]
pub trait FromRequest {
    async fn from_request(req: &mut http::Request<hyper::Body>) -> Result<Self, Error>;
}

And would like to implement it for pooled connections in such a way that it would get the pool from request extensions and then grab a connection from there. However since the connection has a reference back to the pool, I cannot return it.

Making the trait generic over the lifetime of the request wouldn't work since its borrowed mutably leading to multiple mutable borrows when using the trait.

If the pooled connection was fully owned however, everything should work.

from bb8.

davidpdrsn avatar davidpdrsn commented on May 31, 2024

I've submitted #107 to address this 👀

from bb8.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.