Change the default DB directory in openchain.yaml to the DB directory created during the vagrant up command. See https://github.com/openblockchain/obc-dev-env/issues/17

Currently, dependencies are listed in a file called Gomfile, but a developer must run 'go get' manually for each dependency. We need to

• Pick a dependency manager that supports versions
• Add all dependencies including versions

We need a util function to run a transaction on a chaincode and keep the state temporarily until commit
exec(ID, tx) -> hash of result
The ID is the transaction ID from begin() function on the Ledger

When deploying a chaincode, we need to define the validator set for that chaincode. The default is nil or 0, which means to use the configured set of validators.

Each consensus algorithm package should come with its own config.yaml file that defines implementation-related properties.

For instance, for PBFT:

• msg_timeout: time to wait for message
• block_timeout: time to wait for total ordered message (block)

The file will be read via the Consenter.Init() method and populate local variables accordingly.

The number of faulty nodes is dynamically determined, f=floor(n/3), where n is the number of VPs specified in the chaincode deployment context. Since we don't implement confidential chaincode yet, the default is all VPs.

See #108, #115 for more.

There needs to be a JSON RPC API on a peer for a client to send a transaction that will build and deploy a chaincode. The API will accept transaction parameters and respond with the result of the transaction once the transaction has been written to a block in the blockchain.

In the "Configuration" section, there is a reference to a config.yaml.

As far as I can tell, this reference is invalid.

Should be replaced by openchain.yaml.

Define and implement what we need to configure a set of validators. For now, we also need a statically defined leader rather than go through election, which will be later.

This relates to how/when a call to execute a particular chaincode is made.

I see 3 scenarios (there could be more):

1. A peer requests execution of a chain code - this is the simplest case
2. An external event is the trigger for chain code execution (the authority to initiate execution of the chain code does not reside with one peer) - for instance, time triggered
3. A set of peers need to sign (possibly at different times), and once all have signed off the chaincode executes

At the time of deploying a chaincode, we also need a way to specify rules for how/when the chaincode is to be executed.

The world state needs to be persisted in RocksDB. @manish-sethi and I are discussing and implementing this feature.

Planned implementation

• In the database, state is grouped by chaincode
• When modifying the state, an in memory map is made of all changes
• When a block is added (after consensus) the state is committed to the DB
• A hash of each modified chaincode state must be calculated
• A global hash of all chaincode hashes is calculated and stored in the block

@manish-sethi and I discussed possibly not storing the global hash in the block and only storing the hash of chaincodes that had been modified in the given block. @binhn said that this method would require too many comparisons across the network during consensus. The pro to this method is that the hash calculation per block would be faster and for private chaincodes, it would be possible to not store encrypted data on peers that do not have access. @binhn said he wanted the encrypted data to be on ALL nodes so that it would be distributed and recoverable in the event of a data loss across a large number of nodes.

@jeffgarratt and I were discussing this today.

When sending a deploy chaincode transaction, there must be both an ID and version defined for the chaincode.

Anyone calling the chaincode must reference the ID, and optionally, they can reference the version. We'd like to use the the same rules defined by NPM https://docs.npmjs.com/misc/semver

This would allow chaincode writers to update their chaincodes while allowing chaincode callers to choose the versions they will work with.

For example, an execute chaincode transaction would include an ID and a version such as >=1.0.2 <2.1.2

Run all api_test.go test cases. This will fail with the stack below. This is happening because the first test case locks the DB, and then in the same process in the second test case, it tries to destroy the DB. We either need to see if the lock can be released or use a different DB name for each test case.

17:18:24.898 [server] SetupTestConfig -> DEBU 001 setting Number of procs to 2, was 1
Error opening DB IO error: lock /tmp/OpenchainAPITestChain0/LOCK: No locks available
--- FAIL: TestServerOpenchain_API_GetBlockByNumber-2 (0.00s)
api_test.go:152: Error destroying chain0 DB IO error: lock /tmp/OpenchainAPITestChain0/LOCK: No locks available
api_test.go:158: Error destroying state0 DB IO error: lock /tmp/OpenchainAPITestState0/LOCK: No locks available
api_test.go:164: Error creating chain0: IO error: lock /tmp/OpenchainAPITestChain0/LOCK: No locks available
api_test.go:168: Chain 0 =>
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x467f55]

goroutine 9 [running]:
testing.func·006()
/opt/go/src/testing/testing.go:441 +0x181
github.com/openblockchain/obc-peer/openchain.(*ServerOpenchain).GetBlockByNumber(0xc208097df0, 0x7f98a3dcd230, 0xc20800b240, 0xc208097dc8, 0x1, 0x0, 0x0)
/opt/gopath/src/github.com/openblockchain/obc-peer/openchain/api.go:77 +0x2f5
github.com/openblockchain/obc-peer/openchain.TestServerOpenchain_API_GetBlockByNumber(0xc208138510)
/opt/gopath/src/github.com/openblockchain/obc-peer/openchain/api_test.go:173 +0x71b
testing.tRunner(0xc208138510, 0xf0a030)
/opt/go/src/testing/testing.go:447 +0xbf
created by testing.RunTests
/opt/go/src/testing/testing.go:555 +0xa8b

goroutine 1 [chan receive]:
testing.RunTests(0xb530d8, 0xf0a000, 0x16, 0x16, 0xc208112901)
/opt/go/src/testing/testing.go:556 +0xad6
testing.(*M).Run(0xc2080f0fa0, 0xc2080b6160)
/opt/go/src/testing/testing.go:485 +0x6c
github.com/openblockchain/obc-peer/openchain.TestMain(0xc2080f0fa0)
/opt/gopath/src/github.com/openblockchain/obc-peer/openchain/peer_test.go:48 +0x18c
main.main()
github.com/openblockchain/obc-peer/openchain/_test/_testmain.go:98 +0x1d1

goroutine 17 [select]:
google.golang.org/grpc/transport.(*http2Client).controller(0xc208140000)
/opt/gopath/src/google.golang.org/grpc/transport/http2_client.go:799 +0x61f
created by google.golang.org/grpc/transport.newHTTP2Client
/opt/gopath/src/google.golang.org/grpc/transport/http2_client.go:193 +0x158f

goroutine 18 [IO wait]:
net.(_pollDesc).Wait(0xc2080119c0, 0x72, 0x0, 0x0)
/opt/go/src/net/fd_poll_runtime.go:84 +0x47
net.(_pollDesc).WaitRead(0xc2080119c0, 0x0, 0x0)
/opt/go/src/net/fd_poll_runtime.go:89 +0x43
net.(_netFD).Read(0xc208011960, 0xc208138024, 0x9, 0x9, 0x0, 0x7f98a3dca158, 0xc208111bb8)
/opt/go/src/net/fd_unix.go:242 +0x40f
net.(_conn).Read(0xc20812c000, 0xc208138024, 0x9, 0x9, 0x0, 0x0, 0x0)
/opt/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7f98a3dcd098, 0xc20812c000, 0xc208138024, 0x9, 0x9, 0x9, 0x0, 0x0, 0x0)
/opt/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7f98a3dcd098, 0xc20812c000, 0xc208138024, 0x9, 0x9, 0xc208110004, 0x0, 0x0)
/opt/go/src/io/io.go:316 +0x6d
github.com/bradfitz/http2.readFrameHeader(0xc208138024, 0x9, 0x9, 0x7f98a3dcd098, 0xc20812c000, 0x0, 0xc200000000, 0x0, 0x0)
/opt/gopath/src/github.com/bradfitz/http2/frame.go:228 +0xa2
github.com/bradfitz/http2.(_Framer).ReadFrame(0xc208138000, 0x0, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/bradfitz/http2/frame.go:373 +0xf2
google.golang.org/grpc/transport.(_framer).readFrame(0xc20812a090, 0x0, 0x0, 0x0, 0x0)
/opt/gopath/src/google.golang.org/grpc/transport/http_util.go:453 +0x50
google.golang.org/grpc/transport.(*http2Client).reader(0xc208140000)
/opt/gopath/src/google.golang.org/grpc/transport/http2_client.go:729 +0x12d
created by google.golang.org/grpc/transport.newHTTP2Client
/opt/gopath/src/google.golang.org/grpc/transport/http2_client.go:199 +0x15e6

goroutine 19 [select]:
google.golang.org/grpc.(*ClientConn).transportMonitor(0xc2080b6160)
/opt/gopath/src/google.golang.org/grpc/clientconn.go:383 +0x28c
created by google.golang.org/grpc.Dial
/opt/gopath/src/google.golang.org/grpc/clientconn.go:188 +0x5c5

goroutine 33 [syscall, locked to thread]:
runtime.goexit()
/opt/go/src/runtime/asm_amd64.s:2232 +0x1

We need a utility function to hash an array of bytes or a string and return the hash value

Imagine the following chaincode

chaincode {
   var x;
   function setA(v) {
      x=v;
   }
   function getA() {
      return x;
   }
}

The setA function modifies the state and a call to this function must be recorded on the blockchain as a transaction. The getA function does not modify the state and therefore does not need to be recorded on the blockchain as a transaction. There is also no consensus necessary because we can assume that the client trusts the peer it is calling.

Requiring chaincode developers or client developers to understand this subtlety leads to a great deal of confusion as we have observed in other smart contract blockchain implementations. If you are familiar with Ethereum, you know the developer is provided both a sendTransaction and call API depending on how they want the invocation to be handled. Ideally, this would be abstracted away from developers, allowing for just one API to invoke a chaincode function.

Proposed solution

1. A client calls a chaincode function via the API on a non-validating peer.
2. The non validating peer must send the transaction to a validating peer. Each non-validating peer communicates with exactly 1 validating peer.
3. The validating peer sends the transaction to a leader for block packaging.
4. The block packaging peer runs all transactions it has received.
5. If the transaction modifies the state, it is placed in the block's array of transactions.
6. If the transaction does not modify the state, it is placed in a discarded transaction data structure in the block [1] that contains 3 pieces of information. 1) The transaction 2) The order in which the transaction was run (after element N in the transactions array) 3) The reason why the transaction was discarded, in this case, the fact that it did not modify the state. Other reasons could be 'error' or 'invalid transaction.'
7. The block packager sends the block to the validating peers
8. Most validating peers can ignore the discarded transactions. However, in the case of the validating peer from step 3, it recognizes the transaction that it sent in step 3.
9. The validating peer runs the transaction in the correct order and responds to the client with the return value
10. All validating peers remove the discarded transaction structure from the block before adding the block to their blockchain.

[1] A peer may disagree with the reason for a transaction being discarded. For example, the leader may say the transaction does not modify the state, but the validator thinks it does modify the state. This is the same problem as a block packager choosing to ignore a transaction. In existing blockchain implementations, this problem is solved by paying a fee to the block packager, giving them an incentive not to ignore transactions. Because there is no coin in Openchain, there must be a way for a peer to alert other peers that a transaction is being ignored and obtain consensus that the transaction should be run in a future block.

The implementation-specific "consensus algorithm" module (q.v. issue #70 for context) should have a receive(message) function as part of its interface.

For now at least, this message is a client request originating from the P2P layer, and passed to us by the generic consensus module.

This message will either be packaged into a block in the consensus algorithm module, or prompt the transmission of a consensus phase message (e.g. PRE-PREPARE, PREPARE, or COMMIT for PBFT).

state.go needs to provide API for fetching the entire state of the current block. This API will

• Capture a read only snapshot of the entire state so the peer can continue adding new blocks.
• Provide a stream of the read only snapshot so that a requesting peer can download the entire state.

When running transactions in a block, a peer may find that some transactions are invalid due to an error while running the transaction or other issue. Peers need to agree on these invalid transactions during consensus. Invalid transactions also help a client see that there transaction was successfully submitted, but that the network could not run their transaction.

The process is

• Leader places all transactions in the block transaction array and distributes the block.
• Peers run the transaction in the block. If a transaction cannot be run, it is placed in an invalid transaction array along with a reason why the transaction could not be run.
• During consensus, peers agree on the invalid transactions.
• Clients can inspect this array on the blockchain, to see if there was an issue with their transaction.

Basic VM framework for VM orchestration

• build image
• start image
• stop image

The 'Protobuf dependencies' section can be removed from the readme as these are setup by the obc-dev-env

Remove the state hash from a block's constructor. The state hash is automatically added when calling blockchain.AddBlock(). Leaving it in the constructor is confusing and not needed for normal use cases.

It would be helpful if the result (return value) of a transaction was stored in the block. This improves audibility and makes it easier for clients to read the result of a transaction that may not be stored in the state.

This requires

• A new results array to be added to block. The elements of this array are arrays of bytes.
• When a chaincode is run, the return value from the run function will be stored in the result array at the same index as the transaction in the transactions array.

The downside is that this increases the size of the blockchain.

Define the message structure for the consensus messages that will be passed between the "consensus" and the "consensus algorithm" modules. (q.v. issue #70 for context)

Need to change from hub.jazz.net/openchain-peer to github.com/openblockchain/obc-peer.

Peers need to store the last X state deltas between blocks in the database so another peer can catch up with the blockchain.

Example

• A peer requests a download of the entire state. See issue #43.
• While download the state, the blockchain continues to progress.
• The peer must now be able to ask for state deltas for subsequent blocks so that it can obtain state change information for private transactions that it may not have the ability to run.

X, the number of state deltas stored by a peer, should be configurable in openchain.yaml.

A requirement of Openchain is the concept of off-chain storage. Off-chain storage will allow chaincode developers to store large files outside of the blockchain while referencing the files from a transaction stored in the blockchain.

For example, there may be a case where a chaincode is a codified written contract or represents a portion of a written contract. In this case we would like to 'link' the chaincode to the contract. Another scenario is the case where a transaction may be associated with a signed document. The signed document may be very large and we do not wish to store it across peers, but instead would like to record a verifiable link to the document in the transaction.

The verifiable link to a file from a transaction contains 2 parts.

• A URL to the file.
• A hash of the file.

While the URL does not guarantee future retrieval of the file, the hash does allow one to verify whether a file is the original file associated with the transaction.

As mentioned by @jeffgarratt in an earlier discussion, the goal of off-chain storage should not be to build a file storage repository. There are many existing storage repositories that would work for our use case. The most obvious example may be Git, although there are many other systems.

The goal of off-chain storage is to

1. Allow a user to associate documents or arbitrary and potentially large files with a transaction.
2. Store the URL and the hash of the files with the transaction on the blockchain.

Proposal

There exists an API on peers to invoke a transaction. This transaction either deploys a new chaincode or invokes a function in a chaincode. This API should expose a parameter which is an array of one of more URLs which point to files that are to be associated with the contract. Here are the steps for associating a file with a transaction.

1. A client sends a transaction to a peer, specifying 0 or many URLs associated with the transaction.
2. The transaction is sent to the block packaging peer.
3. The block packaging peer performs a GET on each URL. It expects a 200 response code with a response body. If the response code is not a 200 or the response body is empty, the transaction is marked as invalid.
4. Assuming a valid transaction, the block packaging peer hashes the response body and stores this hash in the transaction.
5. The transaction is sent to all validating peers.
6. Each validating peer also performs a GET on the URL, checking for a 200 response code and response body. Assuming a valid response, they again hash the response body and compare it with the hash added to the transaction by the block packaging peer. Assuming a match, they will vote 'yes' during the consensus. A non-match would result in a 'no' vote.
7. Assuming a successful consensus, the validating peers add the block containing the transaction which in turn contains the URL and hash to their blockchain.

• The above proposal requires the existence of an external central storage repository.
• The API for sending a transaction could also accept a file directly. The peer providing the API would have to be configured with the external storage URL. The peer would then upload the file to the off-chain storage and add the URL to the transaction before sending the transaction to the block packaging peer.

Chaincode development needs to be easy from the point of view of a chaincode developer. We need to document the use cases from the chaincode developers point of view.

The blockchain needs to be persisted in RocksDB.

gRPC is missing from the GOM file. Without running go get google.golang.org/grpc the build will fail.

We need the following functions on ledger to manage the block transaction

• begin(ID) – begin transaction for block number ID
• commit(ID, proof) – write block and changes to ledger; write proof to audit
• rollback(ID) – remove all changes within transaction scope ID

We need a function to return the current temporary world state hash for consensus; that is, the hash of the state of the current block result. Consensus will call this function at the end of executing the block.

The database design needs to support two modes of operation.

1. Running transactions and consensus
2. Queries from clients

• Consensus will primarily be performed by validating peers. It involves
  • Updating the state
  • Calculating the state hash
  • Storing a block on the blockchain
  • Sending transaction results and state deltas to peers that do not have access to a private transaction
• Queries from clients
  • Get block by number
  • Get block by hash
  • Get transactions in a block
  • Get transaction by index in a block
  • Get transaction by hash
  • Get transaction by address
  • Get transaction by identity
  • Get chaincodes I have access to

Currently the database format is
Block number -> Block bytes
which is fast for consensus, but my be slow for queries. We could consider

Block number -> block hash
block hash -> block bytes` (where block bytes contains an array of transaction hashes
transaction hash -> transaction bytes

but this may be slower for consensus. However, this could reduce the overall size of the DB as we do not need to store duplicate transactions.

Should we consider two different database schemas, one optimized for validating peers and one optimized for non-validating peers?

Change chainlet to chaincode

The db and trie directories are no longer used and can be removed

The legal header needs to be added to blockchain_db.go and blockchain_db_test.go

To enable integration with various systems, we need an event infrastructure that any components may emit events, and external or internal components may consume the events.

We need the following:

1. Define what events we want to produce
2. How to emit the events
3. What mechanism to deliver the events

Events

The list of events may be added over time, but we may start with block-added, block-removed (rollback), chaincode-deployed.

Event-Stream

Events are emitted on a gRPC stream from VP, where any gRPC client can listen to. An NVP may be an event client, who will listen to events from the connected VP. The event client implementation on NVP follows the adapter pattern such that a event-adapter may be plugged in to deliver the events to external systems.

Event-Adapter

Event-Adapter implements gRPC stream receiver on one end and event delivery on the other end. Event delivery, such as webhooks, Apache Kafka or websocket, enables external systems to subscribe to interesting events.

We need an audit log to capture the "proofs" of the committed blocks on the ledger. The audit log should be a separate data structure from the ledger/blockchain, and be persisted in the datastore

Here's how the state hash calculation is currently implemented

1. Get list of chaincodes that modified their state
2. Hash all the state data for each chaincode that modified its state
3. Store the hash in the DB (chaincode ID -> hash)
4. Hash all chaincode hashes to get the global hash

This is obviously quite inefficient for generating the global hash if there are either a large number of chaincodes or if a chaincode's state contains a large number of key/value paris. Our current DB structure does allow fast lookup of key/value pairs.

Our goals are

1. Fast lookup of key/value pairs
2. Fast global hash calculation
3. Avoid storing the global state in memory. It may be too large

I hypothesize that Openchain networks are likely to have a small number of active chaincodes each with a large number of key/value paris, but the code must still performant in other situations such as in networks that contain a large number of chaincodes.

We are considering some form of a Merkle tree to accomplish our goals.

Create a consensus framework to process all consensus related messages from peer. Also provide access functions to the consensus algorithms.

openchain.yml contains the following line

RUN go install hub.jazz.net/openchain-peer && cp src/hub.jazz.net/openchain-peer/openchain.yaml ./bin

The repository and project name need to be fixed.

Opechain service exposes functions such as

GetBlockchainInfo
GetBlockByNumber
GetBlockCount

We want to be able to call these using JSON-RPC, for example from Postman or curl. We think gRPC exposes JSON-RPC, but need to investigate.

Our JSON-RPC API should be as simple as https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)

• Define content of a chainlet transaction - code + docker image file
• Store URLs and hashes on chain
• Send chainlet to validating peer for block packaging
• Ability for peer to build docker image and confirm hashes
• Ability for peer to run ‘test’ chainlet to ensure docker image is correct
• Store mapping of chaincode ID and transaction ID

"cd $GOPATH/src/openblockchain/obc-peer" should be "cd $GOPATH/src/github.com/openblockchain/obc-peer"