add_callback failed with error about caproto HOT 2 OPEN

Thanks for the detailed reply, thats' really helpful, and I tried to use async for to handle the received data, it works!

from caproto.

Thanks for reporting this. There's an underlying bug causing that message, but also an issue with the code you have provided.

Suggestions for your code

In your code, add_callback(f) happens and start returns almost immediately. That means that the closure f gets garbage collected (along with the context and everything else).

You need to keep that function alive for a time period, such as adding an await asyncio.sleep(5) to sleep for 5 seconds after you add the callback.

Alternatively, you might consider using the async with and async for syntax which is a bit more ergonomic than using callbacks. See an example here:

async def start():
    ctx = Context()
    (array_data,) = await ctx.get_pvs("{pv_name}")
    sub = array_data.subscribe()
    responses = []
    def f(sub, response):
        print("Received response from", sub.pv.name)
        responses.append(response)
    token = sub.add_callback(f)
    await asyncio.sleep(10)

or

async def start():
    ctx = Context()
    (array_data,) = await ctx.get_pvs("{pv_name}")
    responses = []
    async with array_data.subscribe() as sub:
        async for value in sub:
            print("Received response from", sub.pv.name, value)
            responses.append(value)

The bug you helped uncover (re-uncover?)

The conversion of the threaded client to asyncio was never fully vetted. The background of the caproto async client issue that needs resolving is:

1. CallbackHandler has a synchronous remove_callback method:

   caproto/caproto/asyncio/client.py

   Line 1907 in 57e0c56

   def remove_callback(self, token):

2. When your callback function gets garbage collected, remove_callback is automatically called as in here:

   caproto/caproto/asyncio/client.py

   Line 1888 in 57e0c56

   self.remove_callback(cb_id) # defined below inside the lock

3. However, the subclass Subscription reimplements (!) remove_callback as async (the horror...):

   caproto/caproto/asyncio/client.py

   Line 2122 in 57e0c56

   async def remove_callback(self, token):

4. This means that garbage-collected Subscription callbacks will call the subclassed async method but never await them

I'd have to dive deeper to figure out if reimplementing CallbackHandler to be async or decoupling Subscription from the superclass would be better.

from caproto.