Subscription backlog memory usage too high for PVs storing large arrays about caproto HOT 2 OPEN

Thanks for reporting this @vstadnytskyi-FDA.

I believe the source of this to be:

where

Would you mind lowering this number (to, say, 5) and re-running your test to verify that it's the source?

Continuing the thought process, assuming I'm right here:

This bounded-length list (specifically a deque with maxlen set) would mean it's not a "memory leak" per se; the real issue is that this can result in ballooning memory usage (up to approximately number_of_clients_monitoring * number_of_array_elements * sizeof(array_element) or in your test case 1 * 1000 * 1920 * 1080 * 4 (32-bit int) = around 8.3 gigabytes).

So, while the above may be splitting hairs - let me say that you identified a real deficiency in how caproto handles subscriptions for large arrays. I'm thinking of potentially one or more of the following:

1. Separate subscription backlog settings for arrays of a certain size
2. Allow subscription backlog settings to be configured on a per-ChannelData basis
3. Use the configuration mechanism from (2) to automatically select good settings for this backlog

from caproto.

I think 2 and 3 are the best option

from caproto.