avdgrinten / d3b Goto Github PK

Use protobuf for the API and configuration files but use own data structures internally.

This file is a huge mess.

Many closures are currently managed through capturing lambdas and std::function. Manually allocating closure objects would require a smaller number of calls to the memory allocator.

Implement a command that allows the user to scan a complete storage.

Implement two-phase commit. This will enable us to support distributed transactions.

Add ability to remove existing documents.

As we do your own caching O_DIRECT should be slightly faster than kernel buffered IO.

Storage and view drivers should allow access to past revisions. This is an integral part for our transaction model and improves the ability to distribute the database across nodes.

Implement a LRU cache strategy.

PageCache currently uses arbitrarily much memory under heavy load.

Engine should log transactions to a write-ahead log. If a storage or view driver fails Engine should replay that log to the failing driver. Storage and view drivers should not deal with durability issues themselves.

Querying a view driver may return inconsistent data: The view's sequence() method can be while processQuery() is still running.

Views should queue transactions and queries and execute them sequentially in order to prevent this.

The same applies to sequence() and processFetch() for storage drivers.

Use TLS to secure API requests and responses.

Implement an API message to allows users to access the fetch() command.

Make sure all files are closed and all data is in consistent state after shutdown.

Use worker threads to read/write from disk. Use multiple threads to process queries.

Implement associative reduce functions by storing an intermediate value in each btree node.

Use binary search when comparisons are expensive (e.g. require executing java script code or loading data from another file).

Both data structures currently use synchronous read/write operations.

Use edge triggered epoll mode.

Implement API to configure cache size and other server-wide parameters.