bitsplitter / startupdb Goto Github PK
View Code? Open in Web Editor NEWstartupDB is an Express middleware function implementing a high-performance in-memory database
startupDB is an Express middleware function implementing a high-performance in-memory database
The calculateMemoryFootprint function unpredictably crashes. It's unable to access startupDB[collection].data.
While testing the RAM usage of the startupDB, I've got in some how a negative RAM usage which is not possible to be. So please fix this bug.
usedBytesInMemory: -157573965
When a malformed url is passed, an exception is thrown that is not caught anywhere crashing the request and leaving the requester without a response.
Better would be to try Decoding an return a 400 error when it fails
Currently, the PATCH method only supports jsonpatch. I'd like to be able to merge my update with an existing document, only passing the new or updated properties instead of having to replace the entire document with a PUT.
If the PATCH body does not contain a patch array, apply Object.assign to it to merge the source and target objects.
In order to make startupDB more generic, allow to configure its memory management.
DB_CACHE_OVERHEAD_RATIO Determines the predicted memory footprint of JSON objects. The default value of 2 seems to work well in existing deployments but in more specific use-cases, another ratio might work better, for example if there are very deeply nested objects that have higher memory overhead.
DB_CACHE_FRACTION Determines the fraction of memory used for the DB cache. If the NodeJS-Express API that embeds StartupDB needs a larger fraction of available memory allocated to NodeJS via the โ-max-old-space-size parameter, this fraction needs to be configurable.
Due to the inaccuracy in memory footprint estimation, GC can draw the wrong conclusion about available memory and evict all collections, even the ones that are in the process of being read form disk.
When using the HEAD method to get the latest update information on a collection, the x-last-oplog-id header will return a wrong id when there are oplog files with different filename lengths.
Oplog files are named .json, when retrieving all files from the oplog directory, at least on Windows, files are return sorted, that is, alphabetically. That does not give us the most recent file when not all filenames are the same length.
As a developer, I would like to use the DELETE method in a similar way as the GET method.
So it should support the id and filter parameters just as the GET method does.
It should DELETE a single object matching the id or it should DELETE all objects matching the expression in the filter.
This combination does return a checkpoint but the data object within it is a filtered array instead of a filtered object.
Please add the following two DBcommands:
When an operation is performed on a collection that is among the least recently used collection, the GC may decide to delete it causing the application to fail because it expects the data to be available.
We need to 'touch' the collection before the actual transaction is written to keep the GC from interfering.
When accessing a resource that is not in memory yet, with multiple requests simultaneously, different errors are thrown, all boiling down to the resource not (completely) loaded in the in-memory cache.
When I call updateObjects without a payload, I get the following error:
STARTUPDB Error TypeError: Cannot read properties of undefined (reading 'id')
at addIdsToItemsThatHaveNone (/Users/jeroen/LatestCollectionServer/node_modules/startupdb/dist/server.js:117:19)
at dbUpdateObjects (/Users/jeroen/LatestCollectionServer/node_modules/startupdb/dist/server.js:569:5)
at async /Users/jeroen/LatestCollectionServer/dist/hooks/beforeGetCampaign.js:24:5
at async processMethod (/Users/jeroen/LatestCollectionServer/node_modules/startupdb/dist/server.js:692:24)
at async /Users/jeroen/LatestCollectionServer/node_modules/startupdb/dist/server.js:772:20
As a developer, I would like a more relevant error that helps me pinpoint the issue quicker.
Chalk library output on the console is hard to read when analysing raw log files from CI/CI pipeline.
I would prefer plain log files.
When a collection is being loaded (async) startupDB can kick in for another request and decides to remove the new collection since that has no lastAccessed timestamp yet which implies that it is the oldest collection around making it eligible for GC.
usedBytesInMemory
is updated with a wrong number when a collection loads into memory.
As a result, the garbage collection decides to evict collections for no good reason.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.