For use with the bloom filter

Accept the first one or two and compare the times to decide which to respond with.

https://en.wikipedia.org/wiki/Lamport_timestamps

Right now weŕe just auto-declaring 10k size bloom filters. It'd be a lot wiser to keep metadata alongside the bloom filter whenever transmitting

It should use interfaces also. Right now it's extremely messy and not idiomatic. Moreover, I can improve LRU cache lookups from O(n) to O(logN) using a btree and a priority queue.

Seems to be an issue with time and ordering of values

Right now I'm mostly familiar with the Chord implementation, but there are others which could potentially be better suited for the task. I'm going to use a todo list here, but whenever an implementation is decided to be not sufficient, I'll tick it off and post an accompanying reason.

• Chord
• Kademlia
• Pastry
• Tapestry

It seems that each node in the distributed cache could send a bloom filter to every other node (although such a naive implementation probably isn't necessary, there could probably be some message passing through each node to other known nodes in the finger tables) notifying which hashed keys it most likely (there are known false positives with using a bloom filter) has.

http://www.eecs.harvard.edu/~michaelm/postscripts/im2005b.pdf

https://docs.basho.com/riak/kv/2.1.4/developing/usage/conflict-resolution/
http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf

http://www.eecs.harvard.edu/~michaelm/postscripts/im2005b.pdf

The compresed bloom filters section has more information on this.

To transfer the bloom filter over the the outgoing/incoming connections, we need to serialize and deserialize the data into an easily transferrable format.

Right now it's really inefficient and isn't compressing nearly as much space as it ought to.

Right now theyŕe two seperate entities, but they need to work in unison.

Right now it's not and I think that's causing issues during building/testing.

It should work opposite to the parser, essentially.

Currently the internal cache is created and never used. We want it to be used throughout the program, rather than each having its own storage system.

Oh how times are a changin

There's a growing number of magic numbers used throughout the codebase, so it's better I start configuration now rather than later.

With the priority queue, it should be alot simpler than previously though. Depending on how the heartbeat event loop is built, this may be baked into it.

Should be passed a peer and a desired item and that can be stored in the response container and shared between the requester and receiver.

The erlang interoperability should allow us to easily interface with C and ensure were using a fast implementation of fnv, murmur and Jenkins.

There should be a thin layer that handles creating a new connection processor per incoming socket.

The Connection processor needs to be a FSM which has at least these states:
unauth
authenticated (able to process incoming commands)

There shouldn't be any actual command processing included in this, as that will be part of another issue.

I've been developing on Windows (Xubuntu is a POS now) recently and have noticed weird build failures.

Essentially with COW (copy on write) I can make the underlying map thread-safe and only have to update on each write. This'll help improve performance and it'll cut down on the number of locks necessary to only write/delet operations rather than all CRUD operations.

https://en.wikipedia.org/wiki/Chord_%28peer-to-peer%29#Finger_table

https://www.perform.illinois.edu/Papers/USAN_papers/03AGB02.pdf
https://people.cs.umass.edu/~emery/classes/cmpsci691st/scribe/lecture17-byz.pdf
http://sce.uhcl.edu/goodwin/Ceng5334/downLoads/byzantine.pdf

The heartbeat should trigger every x milliseconds. If 10 or more successive heartbeats fail, mark the node as timed out and replace the peer with another. Honestly not a fan of this magic number.

We can heartbeat additional nodes once a minute (or so), but this should be a configurable option.

This can either be before or after the previous issue, but it makes sense for this to be before (if you're me at this exact specific time, maybe not in the future).

Right now we're sending a ton of data and there's a high likelihood of there being a lot of 0s in the data. RLE would help to cut down on the amount of data we're sending in.

Esesntially the event loop will be the backbone of key expirations & sending heartbeat updates
to remote nodes.

There's probably libraries for concurrent/atomic maps somewhere. If I want to keep with the entirely hand-written, then I can roll my own, I guess

http://www.ccs.neu.edu/home/amislove/publications/Glacier-NSDI.pdf

Info should be put into the peer data structure

We need a way for nodes to discover each other so they never get segmented.

Moreover, once a peer has been discovered, it needs to be placed into a backup list in the peer list which can be later used to retrieve backup peers in the case of a peer dying.

There's a good chance I have a REQUEST peers implemented, and if I don't, that should be how it's done. Just return a comma delimited list of IP:PORT which can be parsed out.

Peers should then be tested to see if they're connectable. If not, set their status to disconnected.

Probably targeting a Chord DHT (https://en.wikipedia.org/wiki/Chord_(peer-to-peer)

Basic functionality:

• Connect a node
• Disconnect from remote node
• Add value
• Get value
• Get peer(s?)

Just need a general idea of what information the peer struct will hold, some generic operations, &c.

It only needs to be simple.

This will mostly be used for the hashKey function in bloom.go, but it might serve useful in the future.