hermeznetwork / hermez-node Goto Github PK

https://github.com/hermeznetwork/idocs/issues/66

Right now we have GHA tests for go 1.13 and go.14

Considering that we are not writing a library but a server software, I think we can force a dependency of a particular go version. For that reason I think the tests for go 1.13 can be removed (this way there's less noise in the PR test results).

Specification: https://idocs.hermez.io/#/spec/architecture/statedb

In the last changes of the API server spec, we remarked the importance of having endpoints consuming only a single DB to ensure the feasibility of a micro service architecture.

I think that with this in mind, each (DB) module should have it's endpoints, instead of having a module with all of them.

The method should:

• Looks at the last forged common.Batch.ForgeL1TxsNum
  - If it's nil returns an empty array of L1Txs
  - If it's not nil, returns the associated L1Txs

Will be used by Synchronizer & Coordinator

Depends on the decision about the key-value database to be used.
Depends on #49

Use cusnarks prover with Snarkjs trusted setup

I've just pushed a commit into a non master branch and it triggered the go test GHA.
IMO this action should run only when the PR is created

Implement a 'mock' ethClient, ideally would allow:

• to pre-define the blocks & transactions in each block
• call the methods from ethClient, but returning the predefined blocks & transactions

In this way, we will not depend on the blockchain to do part of the tests, and we will be able to tweak and force edge cases in the tests.

Logrus lib: https://github.com/sirupsen/logrus
Tendermint log approach: https://github.com/tendermint/tendermint/tree/master/libs/log

An option could be to use logrus, adapting it to print the module/file.go#line_num when log.Error(err) is called (without the /home/user/path/ in the printed string, to facilitate the development & debugging.

Update: zap looks good https://github.com/uber-go/zap

Some of the strings start with TxType... and others don't. This looks ugly and inconsistent and makes me sad

This will be used by Synchronizer & BatchBuilder & TxSelector

func (s *StateDB) ProcessTxs(l1usertxs, l1coordinatortxs []*common.L1Tx, l2txs []*common.L2Txer) (*common.ZKInputs, []*common.ExitTreeLeaf, error)

update zkInputs struct to the last circuit version

Add the endpoint handlers following the spec of the API. This will be prety much useless until there is a DB implemented, at least the interfaces

Depends on #38

When adding & updating leafs need the ProcessorProof, also need to a method to get VerifierProof

Current process is : Snarkjs computes a witness for a given circuit, writes it to file. The prover reads this file and computes the proof.

If the number of constraints is large, the reading and writing the witness may take several 10s of seconds. It would be better to have the witness generator and the prover share the witness via shared memory.

Revisit common structs to ensure consistency with last DB changes. In particular common.Batch should update:

	OldStateRoot     Hash
	NewStateRoot     Hash
	OldNumAccounts   int
	NewNumAccounts   int

but it's possible that other structs are outdated as well.

Edit: Other outdated structs:

• Coordinator

Spec: https://idocs.hermez.io/#/spec/architecture/price-updater

Spec: https://idocs.hermez.io/#/spec/architecture/batch-builder

Systems with 64M - 128M constraints do not work.

Spec: https://idocs.hermez.io/#/spec/architecture/batch-builder?id=exit-process

Depends on #46

Probably it will make sense to add a AuxToIdx in PoolL2Tx, and a AuxFromIdx in L1Tx.
When the tx-selector processes an L1Tx with FromIdx == 0, it will set AuxFromIdx.
When the tx-selector processes a PoolL2Tx with ToIdx == 0, it will set AuxToIdx.
In the batchbuilder, when L1Tx with FromIdx == 0, it uses AuxFromIdx to create the account
In the batchbuilder, when PoolL2Tx with ToIdx == 0,, it uses AuxToIdx to do the transfer

Verify that new snarkjs is able to cope with large number of constraints

In order to store float16 type to the DB, this should include the scanner + valuer interfaces.

Some examples:

• https://husobee.github.io/golang/database/2015/06/12/scanner-valuer.html
• http://jmoiron.net/blog/built-in-interfaces/

The tx that creates an account has the idx set to 0, and therefore it can not be linked with current SQL design

Analyze performance for cusnarks and main zkproof systems available in different platforms.

Latest (quite old) cusnarks numbers can be found in https://github.com/iden3/cusnarks/blob/master/profiling/cusnarks_profiling.xls

• L2DB
• HistoryDB
• State DB

Current design for account view include total txs, the API should provide this information

We have lots of columns that natively are bytes arrays of fixed length (hashes, ethereum addresses). It would be nice to know what performs better on Postgres:

• Bytea
• Char

In particular, the relevant metrics would be:

• Read speed for queries of one element
• Read speed for queries of many element
• Write speed for inserts of one element
• Write speed for inserts of many element
• Effective used disk space

Connect the API endpoints with the DB

https://idocs.hermez.io/#/spec/zkrollup/README?id=l2a-idl2

• Needs to implement also the PoolL2Tx parsers to bytes and then to BigInt to perform the signature.
• Needs the compatible construction with javascript version of transaction hash to be signed
• Needs to implement the signature verification for the PoolL2Tx

Add Infow, Errorw, etc.

Should we start defining the DB interfaces?

Although this was scheduled for later, I think it's a good idea to have a first iteration to facilitate the development of modules that are more advanced in terms of specification and that are already being developed.

depends on #35

• L1Txs data
• L2Txs data
• StateTree leaf updates (when transfering, making diposits, etc)
• ExitTree data

• Add txs (L1 & L2) tests for StateDB
• Add batch option for test tx lang