Improve database design about fabric HOT 3 CLOSED

@manish-sethi and I met this morning to discuss the DB design. @manish-sethi made many good points as to why the transactions should not be separated from the block bytes. These include

• Query performance: Much faster to read a block from the DB
• Storage performance: No need to calculate hash of every transaction when storing a block
• Size: Transaction data may actually be smaller than the hash

The plan is to create column families that will act as indexes and map transaction hashes to the block # and transaction index in the block. Calculating the transaction hash will be performed in a separate thread so that it does not slow down transaction processing.

from fabric.

One though about the column family that maps transactions hashes to block # and transaction index number. Currently, a transaction hash is not unique. You could send the same transaction in different blocks or the same block. To make transaction hashes unique, we need to do one of the following

1. Create a GUID and store that in the transaction
2. Transactions will need a sender. Maybe this is always unique due to our use of butterfly keys? Is every token unique?
3. Some other idea to make the transaction hash unique

@jeffgarratt If we do one of these, the transaction hash will be unique and that can be used as the ID.

from fabric.

Added a separate column family to RocksDB for maintaining a mapping between queries keys and (blockId,txIndex). When a new block is committed, the entries in this column families are made in a separate thread. The client query thread waits for index completion up to the block level that the client thread notices when the query arrives.

@srderson we can make this configurable to add entries into this column families if we prefer to have these only on the peer nodes and not on the validator.

Deferring the implementation of some of the queries to issue #73 because of missing information in proto structures as yet.

from fabric.