coinbase / mesh-cli Goto Github PK

TL;DR

One of the tests performed by the rosetta-cli check tool is for the nonexistence of duplicate transaction hashes, however that is troublesome for Decred because we have a transaction reversed on-chain and then re-mined (with the exact same hash) on a different mainchain block.

Thus we need to make the check for duplicate transaction hashes either optional or remove it entirely.

Details

On Decred we have a mechanic that is part of our Proof-of-Stake system where voters (the PoS validators) can disapprove a specific subset of the transactions (the so called "regular transactions") contained on the block they're voting on, undoing the changes in the utxo set introduced by those transactions, effectively "reversing" them.

This always happens on the very next block of any arbitrary block. In other words, the votes on block n+1 indicate whether the regular transactions of block n are approved. We only know whether block n is approved by the PoS validators ("voters" or "stakeholders" in Decred jargon) once block n+1 is mined by the Proof-of-Work miners, including the necessary vote transactions.

The reversed transactions of block n are then eligible for re-inclusion on a future block (starting at block n+1).

In our Rosetta implementation, we represent such transactions (txs of a block that was voted as disapproved) as operations with type reversed and negative amount at block n+1. That is, if a child block disapproves its parent, the Rosetta block n+1 will have a copy of the regular transactions of block n but with a reversed operation and negative amounts.

This currently causes the check tool to fail due to the transaction hash appearing a second time on a block.

To Reproduce
testnet block 460496 was reversed. Thus the c6e001 transaction appears on our Rosetta implementation as reversed on the next block (block 460497):

  {                                                                             
   "transaction_identifier": {                                                  
    "hash": "c6e001e8e0efb435f16aa1bbadcc24ed971692aaf38e2e7922eceb7bbca5e9bb"  
   },                                                                           
   "operations": [                                                              
    {                                                                           
     "operation_identifier": {                                                  
      "index": 0                                                                
     },                                                                         
     "type": "credit",                                                          
     "status": "reversed",                                                      
     "account": {                                                               
      "address": "TsT1FghZK4aZzo3VZMErD3rCdaC1D3XAjMJ"                          
     },                                                                         
     "amount": {                                                                
      "value": "-3347902131",                                                   
      "currency": {                                                             
       "symbol": "DCR",                                                         
       "decimals": 8                                                            
      }                                                                         
     },                                                                         
     "metadata": {                                                              
      "output_index": 0,                                                        
      "script_version": 0                                                       
     }                                                                          
    },                                              

Propposed Solutions

Either entirely disable duplicate tx detection in the check tool or make it optional so that blockchains that support multiple transactions with the same hash can be accounted for.

Is your feature request related to a problem? Please describe.
block_concurrency is changed to sync_concurrency in v0.4.1

Describe the solution you'd like
include config breaking changes in release note

Describe the bug
Installing on Mac OS X with latest go 1.14.4 results in this error:

# github.com/dgraph-io/badger/pb
../../go/src/github.com/gdraph-io/badger/pb/pb.pb.go:23:11: undefined:
proto.ProtoPackageIsVersion3

To Reproduce
go get github.com/coinbase/rosetta-cli

Expected behavior
Rosetta-cli to install

Additional context
Protobufs need regen for go v1.14.x

Expand rosetta-cli's testing capabilities with fuzzing of rosetta server endpoints. Some projects which may be useful:

https://github.com/dvyukov/go-fuzz
https://github.com/google/gofuzz

Describe the bug

Mistakes are easy to make in the JSON-inside-JSON workflow syntax in the construction configuration. When these mistakes happen, you get a generic error about the configuration file being invalid, but no context information or details.

To Reproduce

In an array or object inside one of the input strings, add a trailing comma like:

{
  "type": "set_variable",
  "input": "{\"network\":\"mainnet\",\"blockhain\":\"libra\",}",
  "output_path": "transfer.network"
}

and run rosetta-cli check:construction.

Expected behavior

It should tell me what specific kind of error occurred, and at least what line the error was found. It would be nice if it could print the offending line with an ascii arrow pointing to the place where parsing failed.

Replaces #86
Closes #119

Motivation

The recently added Construction API testing (#81) does not support multi-currency sends, on-chain address generation, testing non-transfer transactions, or testing account/utxo transfers on the same network. This makes it impossible to test the Construction API of many projects implementing the Rosetta API.

Solution

We need to redesign the constructor to allow for the generic execution of a core set of primitives (generate_key, save_address, derive, set_variable, find_account, find_coin, print_message, wait_for_funds, math, random_string) that can be stitched together into scripts (with some shared state). These "scripts" should be called scenarios and it should be possible to combine multiple scenarios together into a workflow where all execution state is shared. At the end of each scenario execution it should be possible to execute a transaction which will allow multiple transactions to be broadcasted as part of a workflow. The constructor will manage the execution of these workflows (instantiated as jobs) using a new storage backend called JobStorage.

Preview

Workflow
  []Scenario
    []Action

Job (instance of a plan)
  Identifier
  State
  Index
  []Scenario
  Status [ready, broadcasting, failed, complete]

  Create(Plan) Job
  Intent() [run after execute] -> sets status to broadcasting

Worker (executes job)
  Helper (fetcher, balanceStorage, coinStorage, broadcastStorage, keyStorage)
    Derive
    LockedAddresses
    AccountBalance
    CoinBalance
    StoreAddress

  processScenario(State, []Actions) -> NewState, Error
  Process(job) -> updatedJob, intent, error


Constructor (creates jobs, coordinates worker, manages Construction API Flow)
  Helper (broadcastStorage, keyStorage, flowStorage)
    ConstructionAPIFlow (-Submit)
    Broadcast()
    Sign()
    JobsProcessing() -> Count
    JobsReady() []jobs
  Handler
    AddressCreated
    TransactionCreated

  Loop:
    var job
    var intent
    for [
      ready jobs...,
      (new jobs excluding create_account and request_funds as long as concurrency lower than required)plans...,
      request funds (if no broadcasting jobs and broadcast completed hasn't been updated since start)
    ]:
      Worker.Process(Job) -> updatedJob, Intent, error
      if errInsufficientAccounts:
        run "create_account" then try jobs again (as long as not existing create account)
      if errInsufficientFunds, errUnsatisfiable:
        continue
      if err != nil:
        THROW ERROR

      job = updatedJob
      intent = intent
      break

    jobID = JobStorage.Update(job) -> only assign identifier and store once successful at least once/only update if successful processing
    if intent:
      networktransaction, transactionidentifier = CreateTransaction(intent) [existing]
      Broadcast(network transaction, intent, transactionIdentifier, jobID)

  Errors:
    ErrUnsatisfiable -> if some action can't be completed because of lack of resources, but not error
    ErrInsufficientFunds -> if all ready fail + no in progress + creating all new jobs fails (don't try create account and request funds), request funds
    ErrInsufficientAccounts -> means we should create a new account, returned when we don't max out (only some addresses are creatable)
  Methods:
    CreateTransaction(Intent) Broadcast, error

Todos

• Add new constructor package to rosetta-sdk-go
  • Add support for basic job (processing coinbase/mesh-sdk-go#104)
  • Add primitives
    • generate_key (coinbase/mesh-sdk-go#104)
    • save_address (coinbase/mesh-sdk-go#109)
    • derive (coinbase/mesh-sdk-go#104)
    • set_variable (coinbase/mesh-sdk-go#104)
    • find_account
    • find_coin
    • find_balance (replaces all other find calls) (coinbase/mesh-sdk-go#112)
    • print_message (coinbase/mesh-sdk-go#109)
    • wait_for_funds (just use find_account with a "wait")
    • math (coinbase/mesh-sdk-go#109)
    • random_string (coinbase/mesh-sdk-go#109)
  • Implement coordinator (coinbase/mesh-sdk-go#114)
  • make all calls during execution loop transactional + make BroadcastComplete accept a dbtx as an arg (like BlockWorker) -> coinbase/mesh-sdk-go#117
• Add JobStorage to rosetta-sdk-go/storage (coinbase/mesh-sdk-go#118)
  • allow for viewing the number of completed jobs of a certain workflow (considered an end condition) -> coinbase/mesh-sdk-go#118
• replace rosetta-cli/pkg/constructor with rosetta-sdk-go/constructor (#120)
• add constructor end conditions (i.e. stop after x transfers) (#122)
  • add bitcoin configuration file example (#122)
• err txn too big (coinbase/mesh-sdk-go#125)
• populate requested pubkeys (coinbase/mesh-sdk-go#133)
• option to return funds to a single address when ending (#119)
  • could be a reserved workflow ("shutdown") that is called after end condition is met (instead of just halting)
• (Post-Release) Allow for re-processing based on the result from SuggestedFee (coinbase/mesh-sdk-go#144)
  • add a method to the worker that allows for "dry running" a preprocess + metadata to get the suggested fee...this can then be injected into the state for later construction
  • allow for saving certain currencies from suggested fee as variables, then rewriting operations (triggers a re-run of construction flow)
  • if suggested fee is too high, allow for aborting (account may not have enough amount)
  • allow for a fee tolerance to continue (i.e. if fee within X range of Y, continue)

Is your feature request related to a problem? Please describe.
When syncing up a node, it would be useful to report (if available) node's syncing overall status. Since it could happen that certain endpoints would not return any valid data until the sync process has finished.

Describe the solution you'd like
Maybe having on NetworkStatusResponse a field like:

type SyncStatus struct {
        TargetIndex     int64
        CurrentIndex    int64              
	CurrentStage    string // different sync stages, could also be added to /network/options as a new field
}

Is your feature request related to a problem? Please describe.
Nearly all logging in the rosetta-cli uses the standard log package. Because of this, logs don't provide any context and don't easily allow for configuring a log level. We should use structured logs

Describe the solution you'd like
We should overhaul the logging in rosetta-cli to use structured logging with log15 or zap.

when we call "/block" request for genesis block(index:0), what the "parent_block_identifier" should we return in response, for the schema description this field is required and it cannot pass the cli check.

Is your feature request related to a problem? Please describe.

We would like to run check:construction in our CI server and it would be great to have some extra features that will make it easier for us.

Describe the solution you'd like

1. Run N amount of transfers. It might be good enough to just run a couple fo transfers to check everything is fine, not to run it until the funds are drained (I mean, there is no more to pay for the fees)
2. Be able to define an address where the funds will be returned (so we don't drain test wallets.

Describe alternatives you've considered

re 2 we might use change_scenario, will that work?

Additional context

We will likely run this using github actions

Is your feature request related to a problem? Please describe.
When a temporary directory is used in rosetta-cli check, it does not print the location of the temporary directory. This makes it impossible to view log files printed to this directory.

Describe the solution you'd like
Print out location of data directory used by rosetta-cli check.

Describe the bug

Check failed: reconciliation error
Searching for block with missing operations...hold tight
Using temporary directory /var/folders/d2/g75hyjnx42q4kp60zqyyfw4c0000gn/T/rosetta-cli538868630
2020/07/27 15:37:08 Syncing 24-274
2020/07/27 15:38:56 Finished syncing 24-274
Unable to find missing ops in block range 24-274, now searching 0-24
Using temporary directory /var/folders/d2/g75hyjnx42q4kp60zqyyfw4c0000gn/T/rosetta-cli184658941
2020/07/27 15:38:57 Syncing -226-24

When trying to find the block where there was a missing operation, the cli iterates through negative blocks. We should restrict this to block heights >= 0

Is your feature request related to a problem? Please describe.
When it is not possible to extract the balance for aCurrency from an /account/balance response (typically when trying to reconcile), the rosetta-cli returns a vague error:

Check failed: could not extract amount for &{Symbol:Coin Decimals:10 Metadata:map[]}

Describe the solution you'd like
The error should indicate that it was thrown because it was not possible to extract the balance of a specific Currency from an implementation's /account/balance response.

For context, this error is thrown whenever an implementation does not return a balance for a Currency that it referenced in an Operation. The reconciler attempts to compare the computed balance from operations with the balance returned by the node but cannot do so because it is not returned in the /account/balance response.

Describe the bug
check:data requires some configuration from the construction object to be present.

To Reproduce

1. Run check:data with a known working config file.
2. Remove:

{
  "construction": {
    "end_conditions": {
      "create_account": 0
    },
    "workflows": [
      {
        "name": "create_account",
        "concurrency": 1
      },
      {
        "name": "request_funds",
        "concurrency": 1
      }
    ]
  }
}

3. Run check:data
   Expected behavior
   check:data should run without any construction configuration.

Additional context
v0.5.0 4b2226853f91b4dc5ba7a18c2e3db730165b5232

Describe the bug
When running rosetta-cli check:data with a configuration that limits to a small range deep into the Cardano mainnet chain, the test summary reports success, however, an error is first thrown:

Error: context canceled: unable to lookup live balance
+--------------------+--------------------------------+--------+
|  CHECK:DATA TESTS  |          DESCRIPTION           | STATUS |
+--------------------+--------------------------------+--------+
| Request/Response   | Rosetta implementation         | PASSED |
|                    | serviced all requests          |        |
+--------------------+--------------------------------+--------+
| Response Assertion | All responses are correctly    | PASSED |
|                    | formatted                      |        |
+--------------------+--------------------------------+--------+
| Block Syncing      | Blocks are connected into a    | PASSED |
|                    | single canonical chain         |        |
+--------------------+--------------------------------+--------+
| Balance Tracking   | Account balances did not go    | PASSED |
|                    | negative                       |        |
+--------------------+--------------------------------+--------+
| Reconciliation     | No balance discrepencies were  | PASSED |
|                    | found between computed and     |        |
|                    | live balances                  |        |
+--------------------+--------------------------------+--------+

To Reproduce
https://github.com/input-output-hk/cardano-rosetta/actions/runs/235202551/workflow

Expected behavior
Either not to experience the error, or to see an item in the summary with a status of FAILED

Additional context

Describe the bug

Error: unknown flag: --start

To Reproduce

./rosetta-cli --configuration-file default.json --start 25554414 check:data

Please help me know how to use --start flag

When re-running this command, it will start where it left off if you specify
some data directory. Otherwise, it will create a new temporary directory and start
again from the genesis block. If you want to discard some number of blocks
populate the --start flag with some block index. Starting from a given index
can be useful to debug a small range of blocks for issues but it is highly
recommended you sync from start to finish to ensure all correctness checks
are performed.

Describe the solution you'd like
I understand that showing loaded configuration is clear and nice in check operations, but I personally think it is a little bit too much to show it every time for just viewing a block or an account status. It just print out too much which make me kind of lost focus to the data i am querying.
It would be nice if loaded configuration can be hided in view operations and maybe only be shown with a verbose flag.

Is your feature request related to a problem? Please describe.
currently every single command need to set --configuration-file flag to run

Describe the solution you'd like
similar to kubectl or any other command line tool with multiple environments(contexts), add a command like set configuration-file to set a default configuration to use when --configuration-file flag is not present.

Automatically test any Construction API implementation (the equivalent of the check command for the Data API).

• Add KeyStorage Backend (#67)
• Add intent parsing support (coinbase/mesh-sdk-go#78)
• Add scenario parser to /internal(#69)
• Support loading check:construction settings and scenarios from config file + cleanup commands (#70)
• StatefulSyncer refactor (#74)
• Get transaction depth (when first seen)...useful for confirming txs in Construction API (#76)
• CoinStorage stateful UTXO coin tracking as BalanceWorker (#77)
• Modify parser to assert successful intent [useful for confirming on-chain] (coinbase/mesh-sdk-go#83)
• Confirm Operation in CoinStorage was successful before modifying store (#78)
• Cleanup configuration options + default timeout (#79)
• Add BroadcastStorage for tracking transaction broadcast, address locking, and confirming transactions at certain depth (calls handler + unlocks address) (#80)
  • block more broadcasts until synced tip == current_block_identifier (on restart)
• Construction API tester (confirmation depth config parameter) (#81 )
  • allow change address for UTXO (so can get more addresses testing at once...could just reuse sender eventhough not supposed to)
  • this also checks if all operations in intent were successful on-chain (modify rosetta-sdk-go/parser to allow for specifying success must be confirmed -> coinbase/mesh-sdk-go#83)

Is your feature request related to a problem? Please describe.
When checking the balance reconciliation, the rosetta-cli has only two possible working patterns. One pattern is checking the active addresses related to the operations in new blocks, the other one is checking the inactive addresses randomly selected from previous records. But even after a very long time, the rosetta-cli only randomly chooses a limited subset of the addresses, it cannot cover all possible addresses.

Describe the solution you'd like
Maybe we could add a new --all-address-check flag to the check command to check them all. It will start the full checking process after the rosetta-cli check runs to the tip.

Is your feature request related to a problem? Please describe.
The BlockStorage backend that handles persistence of synced blocks (and is used to provide orphaned blocks during re-orgs) is very tightly coupled to Data API validation. It includes (in addition to block persistence) hash persistence (transaction hash duplicate checks) and balance tracking.

To support automated Construction API testing, we also need to make use of a block persistence layer but don't want or need any of the additional balance tracking logic. We may also want to add more modules to the block syncing loop and don't want to continue overloading this extraction (like tracking UTXOs).

Describe the solution you'd like
I propose:

• greatly simplifying the BlockStorage interface (just to handle the storage/retrieval of blocks and keeping track of sync progress across restarts)
• adding a new package called StatefulSyncer that exposes the same Handler interface as the syncer package (except that BlockRemoved returns the actual block removed instead of just the identifier)
• adding a new BalanceStorage backend that contains all balance tracking logic formerly in BlockStorage
• adding a new tx querying functionality that allows for querying for the existence of certain hashes (and hashes at a certain depth)
• adding a utils.go file that contains all common encoding/decoding logic in /internal/storage

Describe the bug
rosetta-cli v0.4.0 panic on resuming check:data with an existing data directory:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x44f1b12]

goroutine 68 [running]:
github.com/coinbase/rosetta-cli/internal/tester.(*DataTester).StartReconciler(0x0, 0x488dec0, 0xc000020ec0, 0x0, 0x0)
	/Users/yutong/code/mygo/rosetta-cli/internal/tester/data.go:272 +0x22
github.com/coinbase/rosetta-cli/cmd.runCheckDataCmd.func2(0x0, 0x0)
	/Users/yutong/code/mygo/rosetta-cli/cmd/check_data.go:134 +0x3f
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0084b3e60, 0xc0084a38a0)
	/Users/yutong/code/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/Users/yutong/code/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x66

To Reproduce
Steps to reproduce the behavior:

• start check:data with none empty data_directory, bootstrap_balances and interesting_accounts in config file (not sure if bootstrap_balances and interesting_accounts matter here)
• cancel the execution
• then resume with same command

Expected behavior
no panic

Is your feature request related to a problem? Please describe.
If rosetta-cli check can be configurable in a way to turn on/off certain checks it will help to identify which features are working or not instead of having a general error output. This could also include using or not the rosetta's asserter for instance.

Describe the solution you'd like
Adding a configuration file which specifies all the features to check.

Describe the bug
When running rosetta-cli check with bootstrap balances enabled, there is a bug where the cli counts the computed balance as the balance in the bootstrap file, instead of the balances which get calculated from the operations. This returns a missing balance-changing operation detected error.

When running the check without bootstrap balances enabled, the check passes. The problem is isolated to the bootstrap-balances flag.

Describe the bug
Got the following rosetta-cli validator error when populating the metadata field in AccountIdentifier:

$ ./rosetta-cli check --server-url "https://localhost/chainweb/0.0/development/rosetta" --data-dir ./logs/ --log-balance-changes --log-reconciliations
Initialized reconciler with 0 previously seen accounts
2020/07/08 17:39:17 waiting to start intactive reconciliation until a block is synced...
2020/07/08 17:39:17 waiting to start intactive reconciliation until a block is synced...
2020/07/08 17:39:17 waiting to start intactive reconciliation until a block is synced...
2020/07/08 17:39:17 waiting to start intactive reconciliation until a block is synced...
2020/07/08 17:39:17 Syncing 0-13
2020/07/08 17:39:17 Adding block &{Index:0 Hash:TZIFpwYZ8_V4LCQARxIIAtw3btXU35JdMAvrlh_uJSA}
Check failed: gob: type not registered for interface: map[string]interface {}: unable to sync to 13

To Reproduce
Hardware:

• macOS High Sierra
• go version go1.14.3 darwin/amd64

Seems to be crashing on the genesis block. So to reproduce, run the validator against a node that returns a (genesis) block with different JSON structures in AccountIdentifier.metadata field. Since this field is optional, omitting this field from AccountIdentifier "fixed" the error.
Here are some examples of different AccountIdentifier that were included in the block that caused the error (full block JSON found in Additional context section):

           "account": {
              "address": "allocation02",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation02"
                }
              }

            "account": {
              "address": "sender09",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "c59d9840b0b66090836546b7eb4a73606257527ec8c2b482300fd229264b07e6"
                  ]
                }
              }

          "account": {
              "address": "multi-02-03-04-any",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-any",
                  "keys": [
                    "2d70aa4f697c3a3b8dd6d97745ac074edcfd0eb65c37774cde25135483bea71e",
                    "3a9dd532d73dace195dbb64d1dba6572fb783d0fdd324685e32fbda2f89f99a6",
                    "43f2adb1de192000cb3777bacc7f983b6614fd9c1715cd44cd484b6d3a0d34c8"
                  ]
                }
              }

Expected behavior
Expected block to be added without this error

Check failed: gob: type not registered for interface: map[string]interface {}: unable to sync to 13

Additional context
We populated the AccountIdentifier's metadata field because the docs state that:

Blockchains that utilize a username model (where the address is not a derivative of a cryptographic public key) should specify the public key(s) owned by the address in metadata.

Source: https://www.rosetta-api.org/docs/models/AccountIdentifier.html#properties

Since it's optional, when we disabled populating AccountIdentifier's metadata field, this block was added fine.

I've included the full BlockResponse returned for the genesis block in /block:

{
  "block": {
    "transactions": [
      {
        "transaction_identifier": {
          "hash": "48T0LjAnSFpFWxvvaPV-_6E-CjDAPhWYUFWbvyf2lFs"
        },
        "operations": []
      },
      {
        "transaction_identifier": {
          "hash": "XGPEQDk5PIvQkpq0GGkgNTmo-mjki63ZPgER_kovxq4"
        },
        "operations": []
      },
      {
        "transaction_identifier": {
          "hash": "SB3W5ELizk9xzSVZOL_wlznU68yiHOC9pYHkxpU_0go"
        },
        "operations": []
      },
      {
        "transaction_identifier": {
          "hash": "kfLwf6kAstEgsCKbvO8tva3gZKAYx3OGGa6XEDLiOa0"
        },
        "operations": []
      },
      {
        "transaction_identifier": {
          "hash": "XNA6lOJ1W-7XYv5Yb4Aql74oxYI7OJqjOQT5oY48q-g"
        },
        "operations": []
      },
      {
        "transaction_identifier": {
          "hash": "3Sc2TwteIiLy7HRmy5H31N6MaHo3mjPjjQA_D7yUDgs"
        },
        "operations": [
          {
            "status": "Successful",
            "amount": {
              "value": "0",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 0
            },
            "account": {
              "address": "allocation02",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation02"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "0",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 1
            },
            "account": {
              "address": "allocation01",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation01"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "0",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 2
            },
            "account": {
              "address": "allocation00",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation00"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "0",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 3
            },
            "account": {
              "address": "allocation-test02",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation-test02"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "0",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 4
            },
            "account": {
              "address": "allocation-test01",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation-test01"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          }
        ]
      },
      {
        "transaction_identifier": {
          "hash": "ckb2fg5eIythKu9aSwNHHZYlYV6WTu4iXiFCjg9XQKY"
        },
        "operations": [
          {
            "status": "Successful",
            "amount": {
              "value": "190000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 0
            },
            "account": {
              "address": "sender09",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "c59d9840b0b66090836546b7eb4a73606257527ec8c2b482300fd229264b07e6"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "180000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 1
            },
            "account": {
              "address": "sender08",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "63b2eba4ed70d4612d3e7bc90db2fbf4c76f7b074363e86d73f0bc617f8e8b81"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "170000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 2
            },
            "account": {
              "address": "sender07",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "4c31dc9ee7f24177f78b6f518012a208326e2af1f37bb0a2405b5056d0cad628"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "160000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 3
            },
            "account": {
              "address": "sender06",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "5ffc1f7fef7a44738625762f75a4229454951e03f2afc6f81309c0c1bdf9ee6f"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "150000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 4
            },
            "account": {
              "address": "sender05",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "f09d8f6394aea425fe6783d88cd81363d8017f16afd3711c575be0f5cd5c9bb9"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "140000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 5
            },
            "account": {
              "address": "sender04",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "2d70aa4f697c3a3b8dd6d97745ac074edcfd0eb65c37774cde25135483bea71e"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "130000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 6
            },
            "account": {
              "address": "sender03",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "43f2adb1de192000cb3777bacc7f983b6614fd9c1715cd44cd484b6d3a0d34c8"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "120000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 7
            },
            "account": {
              "address": "sender02",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "3a9dd532d73dace195dbb64d1dba6572fb783d0fdd324685e32fbda2f89f99a6"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "110000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 8
            },
            "account": {
              "address": "sender01",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "6be2f485a7af75fedb4b7f153a903f7e6000ca4aa501179c91a2450b777bd2a7"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "100000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 9
            },
            "account": {
              "address": "sender00",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "368820f80c324bbc7c2b0610688a7da43e39f91d118732671cd9c7500ff43cca"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "123400000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 10
            },
            "account": {
              "address": "multi-02-03-04-any",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-any",
                  "keys": [
                    "2d70aa4f697c3a3b8dd6d97745ac074edcfd0eb65c37774cde25135483bea71e",
                    "3a9dd532d73dace195dbb64d1dba6572fb783d0fdd324685e32fbda2f89f99a6",
                    "43f2adb1de192000cb3777bacc7f983b6614fd9c1715cd44cd484b6d3a0d34c8"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "101000000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 11
            },
            "account": {
              "address": "multi-00-01",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "368820f80c324bbc7c2b0610688a7da43e39f91d118732671cd9c7500ff43cca",
                    "6be2f485a7af75fedb4b7f153a903f7e6000ca4aa501179c91a2450b777bd2a7"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "150000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 12
            },
            "account": {
              "address": "e7f7634e925541f368b827ad5c72421905100f6205285a78c19d7b4a38711805",
              "metadata": {
                "currentOwnership": {
                  "pred": "keys-all",
                  "keys": [
                    "e7f7634e925541f368b827ad5c72421905100f6205285a78c19d7b4a38711805"
                  ]
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "100000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 13
            },
            "account": {
              "address": "allocation02",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation02"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "100000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 14
            },
            "account": {
              "address": "allocation01",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation01"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          },
          {
            "status": "Successful",
            "amount": {
              "value": "100000000000000000",
              "currency": {
                "symbol": "KDA",
                "decimals": 12
              }
            },
            "operation_identifier": {
              "index": 15
            },
            "account": {
              "address": "allocation00",
              "metadata": {
                "currentOwnership": {
                  "keysetref": "allocation00"
                }
              }
            },
            "type": "TransferOrCreateAcct"
          }
        ]
      }
    ],
    "block_identifier": {
      "hash": "TZIFpwYZ8_V4LCQARxIIAtw3btXU35JdMAvrlh_uJSA",
      "index": 0
    },
    "parent_block_identifier": {
      "hash": "TZIFpwYZ8_V4LCQARxIIAtw3btXU35JdMAvrlh_uJSA",
      "index": 0
    },
    "timestamp": 1563388117613
  }
}

Is your feature request related to a problem? Please describe.
It is currently a manual and confusing process to find the source of inactive reconciliation errors (occur when a balance-changing operation is missing). To debug an inactive reconciliation error, you must create a JSON file containing all account-currency pairs that should be checked for missing ops on each block (using the --interesting-accounts flag) and then you must provide a start height to look for the source of the balance discrepancy (using the --start flag).

Aside from the difficulty of setting up the cli to find the issue, debugging also requires a knowledge of the different reconciliation strategies (i.e. what is an inactive vs active reconciliation error and why are they different?). If we find inactive errors automatically, it will allow for much clearer logging and the cli will require less learning to use effectively.

TODO in code: https://github.com/coinbase/rosetta-cli/blob/4a614b8f35ab541e43c84cce8e0eab8d36035898/internal/processor/reconciler_handler.go#L76

Describe the solution you'd like
When encountering an inactive reconciliation error, the cli should be able to automatically find the block with missing operations and log it (this may require additional functionality to be added to the reconciler package in rosetta-sdk-go).

Is your feature request related to a problem? Please describe.
currently with an existing data dir, and --bootstrap-balances flag, the cli return error:
2020/07/08 22:38:36 cannot bootstrap accounts, already started syncing: unable to bootstrap balances

Describe the solution you'd like
Would be nice, if rosetta-cli ignore --bootstrap-balances flag when resuming from an existing check, so user can simply use one command to start a check and resume from a check.

Describe the bug

The create_account and request_funds workflows are not needed if using prefunded_accounts.

To Reproduce

Use prefunded_accounts and create_limit: 0 and omit workflows for create_account and request_funds. The tool will fail, unless empty workflows are created with those names.

Expected behavior

Require no workflows, unless the checking is done dynamically based on what is required by the other scenarios.

Is your feature request related to a problem? Please describe.
In the rosetta-cli, we use : to access subcommands instead of Cobra's native handling for this.

Describe the solution you'd like
Use subcommands properly (rosetta-cli view:block -> rosetta-cli view block). This will allow users to run rosetta-cli view -h to print all view child commands.

Is your feature request related to a problem? Please describe.

When processing a mainnet blockchain from scratch it might be super interesting to have some progress feedback.

Describe the solution you'd like

• Understand the percentage it has already progrecessed (something like block 1000 / 4000 (25%) )
• See the blocks / second or transactions / second (3 bps | 10 tps )
• Some kind of ETA (ETA 23m:12sec)

[STATS] Blocks: 979521 (Orphaned: 0) Transactions: 569393 Operations: 2014444 Reconciliations: 6341132 (Inactive: 4585972) | Block 1000 / 4000 (25%) | 3 bps - 10 tps | ETA 23m:12sec

Even if this values might no be accurate (as @patrick-ogrady pointed out it might happen that some parts of the blockchain are heavier to process than others) it's also a good feedback for the user. Also we need to consider that sometimes, the people that will run this software knows what's the blockchain txs distribution and this values will actually might make a lot of sense.

Something like this will be enough, nothing fancier is required (IMO)

Ideally this values should be able to be turned on / off using cli params.

Describe alternatives you've considered

N/A

Additional context

We are processing a 4.5M blocks blockchain on a VPS so it would be ideal to undestand where are we.

Unreviewed commit(s) COMPARE was merged to master by patrick-ogrady.

Comment with appropriate number of 👍 to approve.

Is your feature request related to a problem? Please describe.
It is not easy to view the status of a deployed Rosetta implementation with the CLI.

Describe the solution you'd like
We should support viewing status like we support viewing a block or account (ex: view:status).

Describe alternatives you've considered
Write a simple script to get network status 👎

Is your feature request related to a problem? Please describe.
When using the rosetta-cli check as an integration test, running until a certain block height (using the --end flag) is often times not sufficient. When stopping with --end, we have no idea if we have synced to tip, how many reconciliations have been performed, or how many times an account has been reconciled?

Describe the solution you'd like
Allow for specifying custom completion points in a configuration file.

Replaced by #112

Problem

The recently added Construction API testing (#81) does not support multi-currency sends, on-chain address generation, testing non-transfer transactions, or testing account/utxo transfers on the same network. This makes it impossible to test the Construction API of many projects implementing the Rosetta API.

Proposed Solution

Expand scenarios to support the execution of generic Actions whose Results can be referenced while populating the operations that will be sent to the Construction API. Think of this abstraction as a scripting language of sorts.

There will be 2 required scenarios: create_account (called when sending to a new address) and request_funds (called when prompting for more funds). Other functionality can be tested by creating arbitrary scenarios.

Goal

Allow an integrator to automatically test any transaction they would ever create in production.

Possible Code

type ActionType string
const (
	// GenerateKey creates a new *types.PublicKey.
	GenerateKey ActionType = "generate_key"

	// SaveKey saves an address and *keys.KeyPair to KeyStorage.
	SaveKey ActionType = "save_key"

	// Derive calls `/construction/derive` with a *keys.PublicKey.
	Derive ActionType = "derive"

	// FindAccount finds some controlled addresses with a minimum
	// balance.
	FindAccount ActionType = "find_account"

	// FindCoin finds some coin with an amount above a provided value.
	FindCoin ActionType = "find_coin"

	// RandomString generates some string that matches a provided regex.
	RandomString ActionType = "random_string"

	// RandomAmount returns a random *types.Amount between a [min, max)
	// for a provided *types.Currency.
	RandomAmount ActionType = "random_amount"

	// RequestFunds prompts the user to deposit funds to some address.
	// It considers the funding complete once the address provided
	// has some minimum balance. Not, this is different than
	// the `request_funds` scenario (which may include other actions).
	RequestFunds ActionType = "request_funds"

	// AmountSum calculates the sum of a collection of *types.Amount.
	AmountSum ActionType = "amount_sum"

	// AmountDifference subtracts a collection of *types.Amount
	// from a single *types.Amount.
	AmountDifference ActionType = "amount_difference"

	// NegateAmount could be replaced by amount_difference with no minuend.
	NegateAmount ActionType = "negate_amount"

	// SetVariable allows for setting the value of any
	// variable (as opposed to calculating it using
	// other actions).
	SetVariable ActionType = "set_variable"
)

type SavedVariable string

type Action struct {
	// Save output variable in an action map (each function has clear output type and can only be used by certain functions)
	Variable string

	Type ActionType

	// used for "generate_key"
	Curve CurveType

	// used for "save_key"
	Address SavedVariable    // used in "save_key" and "request_funds"
	KeyPair SavedVariable

	// used for "derive"
	PublicKey SavedVariable

	// Used for "find_account", each used address (and all sub accounts) are locked
	NotAccountIdentifier []SavedVariable
	RequiredSubAccount *SubAccountIdentifier // select certain addresses with balance in SubAccount -> check all addresses + sub account

	// Used for "find_account", "find_coin", "request_funds"
	Amounts            []*Amount             // minimum amounts must have (allows for multiple currencies)

	// Used for "find_coin", each used coin is locked
	NotCoinIdentifier []SavedVariable // should be previously selected coins

	// Used for "random_string", generated randomly (account name)
	Regex string

	// Used for "random_amount", generate randomly of a certain amount [min, max)
	Min      string
	Max      string
	Currency *Currency

	// Used for "request_funds" ... funds request may need to populate arbitrary string for origination
	Message           string    // variables can be inserted (could include random string or pubkey)

	// Used for "amount_sum"
	SumAmounts []SavedVariable

	// Used for "amount_difference"
	Minuend     SavedVariable
	Subtrahends []SavedVariable

	// Used for "negate_amount"
	NegateAmount SavedVariable

	// Used for "set_variable"
	Value *Result // set the value of some SavedVariable to an arbitrary value (could be max fee)
}

// Results are the result of performing some action.
// fn(Action) = Result
// vars[action.OutputName] = Result
type Result struct {
	KeyPair  *keys.KeyPair
	AccountIdentifier *types.AccountIdentifier
	CoinIdentifier    *types.CoinIdentifier
	Amount  *types.Amount
	Value   string // arbitrary string
}

// If you want to get funds into a particular subaccount state,
// you must step through a scenario to get there (AKA stake).
type Scenario struct {
	Name string // must exist "create_account" and "request_funds"

	Frequency int // all scenario frequencies are summed to determine how often a particular scenario is run.

	Actions []*Action

	// If no operations, considered success with no broadcast (ex: request funds or create address with no on-chain origination)!
	Template []*OperationTemplate // after all actions are completed, template is populated with all set variables

	// To allow for long-running scenarios (AKA acruing staking rewards), use a very long confirmation depth.
	ConfirmationDepth int64
}

// Allows for specifying variables (otherwise not correct types)
type OperationTemplate struct {
	AccountIdentifier SavedVariable
	CoinIdentifier    SavedVariable
	Amount            SavedVariable

	Operation *types.Operation
}

// Output of this function sent to `/construction/preprocess`
func Process(scenario *Scenario) []*types.Operation {
	variables := map[string]interface{}{}

	for _, action := range scenario.Actions {
		variables[action.Variable] := ProcessAction(action, variables)
	}

	// If no operations in template, we don't try to broadcast anything.
	// This can occur when creating an address or requesting funds.
	return PopulateOperations(scenario.Template, variables)
}

Example

BTC

[
  {
    "name": "create_account",
    "actions": [
      {
        "variable": "keypair",
        "type": "generate_key",
        "curve_type": "secp256k1"
      },
      {
        "variable": "address",
        "type": "derive",
        "public_key": "keypair"
      },
      {
        "type": "save_key",
        "address": "address",
        "key_pair": "keypair"
      }
    ]
  },
  {
    "name": "request_funds",
    "actions": [
      {
        "variable": "keypair",
        "type": "generate_key",
        "curve_type": "secp256k1"
      },
      {
        "variable": "address",
        "type": "derive",
        "public_key": "keypair"
      },
      {
        "type": "save_key",
        "address": "address",
        "key_pair": "keypair"
      },
      {
        "type": "request_funds",
        "message": "waiting for funds on {{ address }}",
        "address": "address",
        "amounts": [
          {
            "value": "10000",
            "currency": {
              "symbol": "BTC",
              "decimals": 8
            }
          }
        ]
      }
    ]
  },
  {
    "name": "transfer",
    "actions": [
      {
        "variable": "fee",
        "type": "set_variable",
        "value": {
          "amount": {
            "value": "500",
            "currency": {
              "symbol": "BTC",
              "decimals": 8
            }
          }
        }
      },
      {
        "variable": "input_coin",
        "type": "find_coin",
        "amounts": [
          {
            "value": "10000",
            "currency": {
              "symbol": "BTC",
              "decimals": 8
            }
          }
        ]
      },
      {
        "variable": "output_address",
        "type": "find_account"
      },
      {
        "variable": "output_amount",
        "type": "amount_difference",
        "minuend": "input_coin",
        "subtrahends": [
          "fee"
        ]
      }
    ],
    "template": [
      {
        "amount": "input_coin",
        "coin_identifier": "input_coin",
        "account_identifier": "input_coin",
        "operation": {
          "operation_identifier": {
            "index": 0
          },
          "type": "Vin",
          "coin_change": {
            "coin_action": "coin_spent"
          }
        }
      },
      {
        "amount": "output_amount",
        "account_identifier": "output_address",
        "operation": {
          "operation_identifier": {
            "index": 1
          },
          "type": "Vout"
        }
      }
    ]
  }
]

Account-based, Origination-required, PoS Chain

[
  {
    "name": "create_account",
    "actions": [
      {
        "variable": "keypair",
        "type": "generate_key",
        "curve_type": "edwards25519"
      },
      {
        "variable": "creator",
        "type": "find_account",
        "amounts": [
          {
            "value": "100000",
            "currency": {
              "symbol": "COIN",
              "decimals": 8
            }
          }
        ]
      },
      {
        "variable": "address",
        "type": "random_string",
        "regex": "^[a-z]{0,10}"
      },
      {
        "type": "save_key",
        "address": "address",
        "key_pair": "keypair"
      }
    ],
    "template": [
      {
        "account_identifier": "creator",
        "operation": {
          "operation_identifier": {
            "index": 0
          },
          "type": "create_account"
        }
      },
      {
        "amount": "output_amount",
        "account_identifier": "output_address",
        "operation": {
          "operation_identifier": {
            "index": 1
          },
          "related_operations": [
            {
              "index": 0
            }
          ],
          "type": "create_account",
          "account_identifier": {
            "address": "{{ address }}"
          },
          "metadata": {
            "public_key": "{{ keypair }}"
          }
        }
      }
    ]
  },
  {
    "name": "request_funds",
    "actions": [
      {
        "variable": "keypair",
        "type": "generate_key",
        "curve_type": "edwards25519"
      },
      {
        "variable": "address",
        "type": "random_string",
        "regex": "^[a-z]{0,10}"
      },
      {
        "type": "save_key",
        "address": "address",
        "key_pair": "keypair"
      },
      {
        "type": "request_funds",
        "message": "waiting for funds on {{ address }} with public key {{ keypair }}",
        "address": "address",
        "amounts": [
          {
            "value": "100000",
            "currency": {
              "symbol": "COIN",
              "decimals": 8
            }
          }
        ]
      }
    ]
  },
  {
    "name": "transfer",
    "actions": [
      {
        "variable": "sender",
        "type": "find_account",
        "amounts": [
          {
            "value": "10000",
            "currency": {
              "symbol": "COIN",
              "decimals": 8
            }
          }
        ]
      },
      {
        "variable": "sender_amount",
        "type": "negate_amount",
        "negate_amount": "sender"
      },
      {
        "variable": "recipient",
        "type": "find_account",
        "not_account_identifier": [
          "sender"
        ]
      }
    ],
    "template": [
      {
        "amount": "sender_amount",
        "account": "sender",
        "operation": {
          "operation_identifier": {
            "index": 0
          },
          "type": "transfer"
        }
      },
      {
        "amount": "sender",
        "account": "recipient",
        "operation": {
          "operation_identifier": {
            "index": 1
          },
          "related_operations": [
            {
              "index": 0
            }
          ],
          "type": "transfer"
        }
      }
    ]
  },
  {
    "name": "stake",
    "confirmation_depth": 1000,
    "actions": [
      {
        "variable": "staker",
        "type": "find_account",
        "required_sub_account": "locked",
        "amounts": [
          {
            "value": "10000",
            "currency": {
              "symbol": "COIN",
              "decimals": 8
            }
          }
        ]
      }
    ],
    "template": [
      {
        "account_identifier": "staker",
        "operation": {
          "operation_identifier": {
            "index": 0
          },
          "type": "stake",
          "amount": {
            "value": "-100",
            "currency": {
              "symbol": "COIN",
              "decimals": 8
            }
          }
        }
      },
      {
        "operation": {
          "operation_identifier": {
            "index": 1
          },
          "related_operations": [
            {
              "index": 0
            }
          ],
          "type": "stake",
          "account": {
            "address": "{{ staker }}",
            "sub_account": {
              "address": "staking",
              "metadata": {
                "validator": "val a"
              }
            }
          },
          "amount": {
            "value": "100",
            "currency": {
              "symbol": "COIN",
              "decimals": 8
            }
          }
        }
      }
    ]
  }
]

Alternatives Considered

• Pre-fund addresses that require an on-chain creation
• Manual testing of complex transactions

Open Questions

• Is it better to define a DSL for these scenarios?
• Update configuration to read from YAML instead of JSON
  • complication here is that structs are not unmarshaled like JSON, so some custom unmarshaler is required.

Steps

Pending feedback on https://community.rosetta-api.org/t/feedback-request-automated-construction-api-testing-improvements/146

• Define clear input and output for each Action
  • 🚀 All variable data is JSON and is accessed using https://github.com/stretchr/objx notation (standard JSON dot and array notation) -> example: https://play.golang.org/p/Q-A0oYeff-f
• Update variable name access to be {{ VAR_NAME | OBJX Lookup into VAR }}
• Rewrite scenario to populate scenario by running Actions
  • support generate_key
  • support derive
  • support find_account
  • support find_coins
  • support request_funds (don't allow for writing message, just iterate through all balances requested)
  • support random_string
  • support random_amount
  • support amount_sum
  • support amount_difference
  • support set_variable
• Update configuration file fields to support specifying new []*Scenario type instead of just []*types.Operation
• Extract scenario construction logic from constructor and replace with updated scenario work
• Add ability to populate metadata in request to /construction/preprocess. https://community.rosetta-api.org/t/rosetta-cli-check-construction-dynamic-parameter/145
• Run scenarios on all networks or be able to specify specific networks: https://community.rosetta-api.org/t/feedback-request-automated-construction-api-testing-improvements/146/11
• Reuse Variables from a linked scenario (may want to link staking account, etc)...call this a flow
• Pull data from confirmed transaction in a flow

Is your feature request related to a problem? Please describe.
The check command doesn't automatically validate all sub-networks in a sharded blockchain (it validates the first sub-network it sees). I'd want to be able to manually run the validator against different sub-networks.

Describe the solution you'd like
The ability to pass a configuration file to the validator's check command. For example, the validator currently supports creating a configuration file using the create:configuration command. It would be useful to edit this configuration to target a specific sub-network and pass this to the check command.

Describe alternatives you've considered
I've been able to edit the rosetta /network/list endpoint logic to return a specific sub-network during testing. But this is not ideal when testing multiple sub-networks simultaneously.

Describe the bug
Getting an error when installing rosetta-cli.

To Reproduce
Steps to reproduce the behavior:
install go with homebrew
run go get github.com/coinbase/rosetta-cli
Error:
# github.com/dgraph-io/badger/y
go/src/github.com/dgraph-io/badger/y/metrics.go:70:24: cannot convert "badger_v2_num_allocs" (type string) to type expvar.Int
go/src/github.com/dgraph-io/badger/y/calloc.go:49:15: cannot use n (type int) as type int64 in argument to NumAllocs.Add
go/src/github.com/dgraph-io/badger/y/calloc.go:62:17: cannot use -sz (type int) as type int64 in argument to NumAllocs.Add

Expected behavior
rosetta-cli command should start working

Additional context

Describe the bug
We are getting closer to a beta release of our node written using the JS SDK. However, we are running into some issues when testing the Construction API.

First, the Rosetta CLI instance is trying to find the balance on a random address, which is expected behaviour.

rosetta_cli_1       | 2020/09/13 05:04:04 processing workflow "transfer"
rosetta_cli_1       | 2020/09/13 05:04:04 looking for balance {"value":"250000000","currency":{"symbol":"DGB","decimals":8}}
rosetta_cli_1       | 2020/09/13 05:04:04 processing workflow "create_account"
rosetta_cli_1       | 2020/09/13 05:04:05 processed workflow "create_account" for job "0"
rosetta_cli_1       | 2020/09/13 05:04:05 processing workflow "transfer"
rosetta_cli_1       | 2020/09/13 05:04:05 looking for balance {"value":"250000000","currency":{"symbol":"DGB","decimals":8}}
rosetta_cli_1       | 2020/09/13 05:04:05 processing workflow "request_funds"
rosetta_cli_1       | 2020/09/13 05:04:05 looking for balance {"value":"0","currency":{"symbol":"DGB","decimals":8}}
rosetta_cli_1       | 2020/09/13 05:04:05 processed workflow "request_funds" for job "1"
rosetta_cli_1       | 2020/09/13 05:04:05 processing workflow "request_funds" for job "1"
rosetta_cli_1       | 2020/09/13 05:04:05 looking for balance {"value":"80000","currency":{"symbol":"DGB","decimals":8}} on address sn6VmygQpFCQNaWabpwmNQcixL63qAykro
rosetta_cli_1       | 2020/09/13 05:04:05 processing workflow "transfer"
rosetta_cli_1       | 2020/09/13 05:04:05 looking for balance {"value":"250000000","currency":{"symbol":"DGB","decimals":8}}
rosetta_cli_1       | 2020/09/13 05:04:05 waiting for available jobs...
rosetta_cli_1       | [STATS] Transactions Confirmed: 0 (Created: 0, In Progress: 0, Stale: 0, Failed: 0) Addresses Created: 1

However, as soon as I have issued a transaction of 784119600500 units, it still tries to find the balance although the following output looks very good.

{
    "block_identifier": {
        "index": 104,
        "hash": "d41046d0727d50c094e7e3259317f7cde6c6ac471111c5c7bb8ed11fff7b0260"
    },
    "balances": [
        {
            "value": "784119600500",
            "currency": {
                "symbol": "DGB",
                "decimals": 8
            }
        }
    ]
}

To Reproduce
Steps to reproduce the behavior:

• Using the official example of automated construction API testing with an Ethereum implementation.

Expected behavior
A transaction should be crafted and sent to the mempool.

Question
Is the rosetta-cli searching for the exact balance on that specific account or would it be happy with a minimum balance?
I do not want you to reproduce the issue unless the latter is true. In that case I would be happy to provide further instructions to reproduce the scenario.

https://community.rosetta-api.org/t/rosetta-cli-check-data-memory-requirement/150

Questions

• What is the required RAM to run check:data?
• Can anything be optimized to reduced memory pressure?

TODO

• Make syncer continuous (so don't cache 1000 blocks in memory at once) coinbase/mesh-sdk-go#89
• Tune BadgerDB (#88)

Is your feature request related to a problem? Please describe.
It would be useful to see stats printed (maybe in cyan) periodically while running check. It is not clear how many:

• reconciliations have been performed
• transactions seen
• blocks processed a second
• balance changes found
• accounts seen
• operations seen

Describe the solution you'd like
Print stats in a different color every 30 seconds in this output...

Describe alternatives you've considered
Look at log files? No real alternatives available.

Describe the bug
The documentation for the duplicate hash check makes it sound like an error will be thrown if a block and a transaction have the same hash. This is incorrect. An error will only be thrown if two blocks or two transactions have the same hash.

Describe the bug
Starting today, I am getting waiting for implementation to reach tip before testing... for check:construction

020/10/07 16:17:15 0 Balances Updated
2020/10/07 16:17:15 check:construction status server running on port 9090
2020/10/07 16:17:15 waiting for implementation to reach tip before testing...
[MEMORY] Heap: 1101.724197MB Stack: 0.656250MB System: 1260.756973MB GCs: 2
[STATS] Transactions Confirmed: 0 (Created: 0, In Progress: 0, Stale: 0, Failed: 0) Addresses Created: 0
2020/10/07 16:17:25 waiting for implementation to reach tip before testing...
[MEMORY] Heap: 1101.786850MB Stack: 0.656250MB System: 1260.756973MB GCs: 2

To Reproduce
Steps to reproduce the behavior:
run rosetta-cli check:construction --configuration-file <config.json>

Expected behavior
It should start check:construction flow

Additional context

 "online_url": <URL>,
  "data_directory": "",
  "http_timeout": 300,
  "max_retries": 5,
  "retry_elapsed_time": 0,
  "max_online_connections": 0,
  "max_sync_concurrency": 0,
  "tip_delay": 60,
  "log_configuration": false,
  "construction": {
    "offline_url": <URL>,
    "max_offline_connections": 0,
    "stale_depth": 0,
    "broadcast_limit": 0,
    "ignore_broadcast_failures": false,
    "clear_broadcasts": false,
    "broadcast_behind_tip": false,
    "block_broadcast_limit": 0,
    "rebroadcast_all": false,
    "constructor_dsl_file": "stacks.ros",
    "end_conditions": {
      "create_account": 10,
      "transfer": 10
    }
  }

Describe the bug
Restarting the check command with a persistent store fails after > 200 accounts have been stored.

To Reproduce
Steps to reproduce the behavior:

1. Check a lot of blocks from a Rosetta server using a persistent data-dir: rosetta check --data-dir=./cli-data
2. Stop checking
3. Start the checking blocks again: rosetta check --data-dir=./cli-data

Expected behavior
check should always start without error.

Additional context

Describe the bug
I have a blockchain which contains a transaction in the genesis block. This transaction contains a single output going to an address. I have found that the reconciliation step fails, expecting
computed balance to be twice the node balance (which is simply just the only output from the genesis transaction). The failure log shows, for example:

Reconciliation failed for ss1q7q3h4chglps004u3yn79z0cp9ed24rfr9dn5nv at 0 computed: 4004420000HNS node: 2002210000HNS

To Reproduce

• Create a genesis block containing a transaction generating an output to an address.

• Run rosetta-cli check

Expected behavior

• Check fails with the reconciliation expecting a balance equal to double the node balance.

Additional context

rosetta-cli version v0.2.3, go version v0.2.3 on ubuntu linux, bionic

Is your feature request related to a problem? Please describe.

When creating workflows, all the inputs are stored as JSON inside strings, which means " must be constantly escaped. JSON is already difficult to read, and this makes it even worse.

Describe the solution you'd like

Ideally the JSON could be replaced with some simple language. For example:

set_variable would work like:

transfer.network = {"network": "mainnet", "blockchain": "libra"}

and math would work like:

sender_amount = 0 - {{recipient_amount}}

If JSON must be used, use it directly, and put only the templatized things in strings. For example:

"actions": [
...
{
  "type": "math",
  "input": {
    "operation": "subtraction",
    "left_value": 0,
    "right_value: "{{recipient_amount}}"
}
...
]

Is your feature request related to a problem? Please describe.
When doing rosetta-cli check, timeouts can happen especially when endpoints depends on slow function calls.

Describe the solution you'd like
Using a global flag like --timeout or as a part of a config file.

Describe alternatives you've considered
Also, it could be useful to know if a certain call takes longer than a certain amount of (configurable) time. Since at first we may wanna test that the endpoints actually respond some valid data, then optimize for time performance.

Is your feature request related to a problem? Please describe.
The --lookup-balance-by-block flag has caused a lot of confusion and should be inferred automatically from an implementation.

Describe the solution you'd like
There are 2 possible solutions:

1. [RECOMMENDED] Add some boolean to NetworkOptionsResponse: coinbase/mesh-specifications#19
2. Try to make a historical lookup and fallback if it causes an error (no specification change). The major drawback of this strategy is that an implementation meaning to implement historical lookup may do so incorrectly and the CLI would not raise any error (likely very undesirable).

Is your feature request related to a problem? Please describe.
Multiple teams have expressed confusion about the --end flag that can be provided when running check.

Specifically, a lack of clear documentation has given the impression that running check with a populated --end flag and getting success means that an implementation meets all testing expectations. This is not true.

Describe the solution you'd like
Update the docs to more clearly state that --end is should only be used to run small tests to help with debugging (ex: check a range of blocks for valid responses) but it is not a replacement for running check without the --end flag. Some important tests (ex: balance consistency checking) can b avoided when populating --end.

Is your feature request related to a problem? Please describe.
Our blockchain has 10 sub-networks (called "chains" in our nomenclature), but rosetta-cli check only validates the first sub-network it sees.

Describe the solution you'd like
Validate all sub-networks, if multiple exist. It seems that this is known issue:
https://github.com/coinbase/rosetta-cli/blob/master/cmd/check.go#L499

Describe alternatives you've considered
This is related to another issued I opened: #47
But, in general, the work around has been to manually change the node's rosetta implementation to return the specific sub-network I want to test.

Describe the bug
When using the --end option to rosetta-cli check, it always exits with status code 1, even when there are no errors in the output.
This makes it difficult to use as part of an automated continuous-integration workflow or testing suite.

To Reproduce
Steps to reproduce the behavior:

1. Run rosetta-cli check --end 42; echo $?.

Expected behavior
If there are no validation errors, an exit code of 0 should be returned. Exit code 1 should be returned only if validation errors have occurred.

While running on small chains a Decred simnet (local-only network used for development with a one-second target block time), check:data will sometimes hang indefinitely due to not consistently reaching an end condition.

Using just end_conditions.tip = true will sometimes either fail (due to very fast block times) or exit too early without having checked all accounts yet.

Using end_conditions.reconciliation_coverage = x will sometimes also cause the check not to terminate due to very fast processing of check:data on simnet (it will reconcile a large % of the total accounts before reaching tip).

I'd like to add a AbsoluteReconciliationCoverage end condition such that we can specify an absolute % of accounts that should have been reconciled before returning (as opposed to a relative % after it has reached tip).