Comments (5)
This is somewhat similar to RelStorage's in-memory pickle state cache, which is shared by all Connections of a Storage, but operating on the unpickled data (and then of course copying it). I like the idea!
A challenge there is making such a shared cache effective with the different MVCC states that each Connection may be seeing. RelStorage has a complicated system of "checkpoints" it uses to accomplish this that works OK for short-lived transactions and Connections that don't drift too far apart from each other in terms of their MVCC state.
from persistent.
This cache would be keyed by oid + serial, so it would be orthogonal to MVCC. It would store Python objects, so there would be no additional deserialization overhead. Because the sharing would be at the object level, there would be memory savings, not just savings in loading object objects.
from persistent.
If we could store non-dicts as __dict__
, then we could use immutable dicts as shared state and trigger copy on failed setitem (or on noticing non-dicts), requiring no change to persistent state metadata.
from persistent.
This cache would be keyed by oid + serial, so it would be orthogonal to MVCC.
Ah, I see. It helps that the current laughingly-misnamed "pickle cache" knows what (oid, serial) values it's going to be requesting; the RelStorage case just has to deal with arbitrary requests over time.
from persistent.
Nice idea!
from persistent.
Related Issues (20)
- pypy -m pip install persistent==4.6.0 fails with ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/home/travis/virtualenv/pypy2.7-7.1.1/include/site' HOT 3
- Segfault in persistent 4.6.0 HOT 4
- Install _compat.h HOT 5
- AssertionError clearing PersistentMap not added to connection.
- Two potential reference leaks
- 6.4.6 test test_inst_does_not_conform_to_IExtendedPickleCache fails with latest zope.interface
- RTD out of date, needs new webhook HOT 2
- PyPy: Sweeping the cache may need to call gc.collect()
- AttributeError: '_WeakValueDictionary' object has no attribute '_addr_to_oid' HOT 1
- Issue in macOS binary wheel for 4.6.4/3.8: Illegal Instruction (my fault, but needs a change to prevent in the future) HOT 4
- C/Python difference: Setting __class__ activates in C, doesn't activate in Python HOT 4
- Move Mac and manylinux builds to GitHub Actions HOT 3
- Move to the src/ layout? HOT 2
- CI breakage: docs builds fail on GHA (yes, it's repoze.sphinx.autointerface)
- RTD should publish the changelog
- Building a wheel fails on GHA for PyPy2 on Windows
- 4.9.0: pytest is failing HOT 1
- 4.9.0: sphinx warnings `reference target not found`
- macOS: symbol not found in flat namespace '_ffi_prep_closure' HOT 2
- The python 3.11 wheel for linux x86 is missing HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from persistent.