Shared read-only state between objects with copy on write about persistent HOT 5 OPEN

This is somewhat similar to RelStorage's in-memory pickle state cache, which is shared by all Connections of a Storage, but operating on the unpickled data (and then of course copying it). I like the idea!

A challenge there is making such a shared cache effective with the different MVCC states that each Connection may be seeing. RelStorage has a complicated system of "checkpoints" it uses to accomplish this that works OK for short-lived transactions and Connections that don't drift too far apart from each other in terms of their MVCC state.

from persistent.

This cache would be keyed by oid + serial, so it would be orthogonal to MVCC. It would store Python objects, so there would be no additional deserialization overhead. Because the sharing would be at the object level, there would be memory savings, not just savings in loading object objects.

from persistent.

If we could store non-dicts as __dict__, then we could use immutable dicts as shared state and trigger copy on failed setitem (or on noticing non-dicts), requiring no change to persistent state metadata.

from persistent.

This cache would be keyed by oid + serial, so it would be orthogonal to MVCC.

Ah, I see. It helps that the current laughingly-misnamed "pickle cache" knows what (oid, serial) values it's going to be requesting; the RelStorage case just has to deal with arbitrary requests over time.

from persistent.

Nice idea!

from persistent.