Acceptance criteria

• Evalulate alternatives for OpenAPI generation
• Write annotation for KM endpoints
• Generate OpenAPI files
• Including pipeline docs publish and build

Acceptance criteria

• Support for soft-delete on hashicorp plugin
• Integrate soft-delete on KM

• Circle ci pipeline
• Linting
• Integration tests
• Unit tests

Implement ready and live endpoints in the key manager. Following the same implementation than in Orchestrate, on a dedicated port.

Acceptance criteria

• Implement a simple health check over the readiness of the API service
• Check the liveness on the e2e tests

To be implemented in core/store/keys/hashicorp

Define type Config struct

• Implement New(cfg *Config) *Store creating store from configuration
• Implement Create(...)
• Implement Get(...)
• Implement List(...)
• Implement Sign(...)
• Implement Sign....(...)

All other methods should return NotImplementedError
IMPORTANT: Compare impl with https://github.com/ConsenSys/quorum-signer-plugin-for-hashicorp-vault

Implement acceptance tests for secrets store:

• Hashicorp secret
• AKV secret using external AKV instance.

To be implemented in core/store/eth1/base

• Implement Create(...)
• Implement Import(...)
• Implement Get(...)
• Implement Update(...)
• Implement List(...)
• Implement Sign(...)
• Implement SignHomestead(...)
• Implement SignEIP155(...)
• Implement SignEEA(...)
• Implement SignPrivate(...)
• Implement ECRevocer(...)

Implement http.Handler exposing the base keys.Store functionalities.

Method and route naming are to be discussed and confirmed as part of this task

• POST /keys/{id}
• PATCH /keys/{id}/import
• GET /keys/{id}
• GET /keys
• PATCH /keys/{id}
• DELETE /keys/{id}
• GET /keys/deleted
• GET /keys/deleted/{id}
• DELETE /keys/{id}/destroy
• POST /keys/{id}/sign
• POST /keys/{id}/verify

Implement http.Server logic,

• should define the type Config struct for the main application to run
• should expose a non-TLS listener.
• use gorilla as webserver ???

Description

#32

When communicating with the keys endpoints. Return results in base64 instead of 0x.

Write acceptance test for the following:

• eea_sendTransaction
• eth_accounts
• eth_sendTransaction (with and without private_args)
• eth_sign
• eth_sign_transaction

Include Besu and go-quorum

IMPORTANT: Current Orchestrate client impl align with the one required here

To be implemented in core/store/keys/hashicorp

• Define type Config struct
• Implement New(cfg *Config) *Store creating store from configuration
• Implement Create(...)
• Implement Get(...)
• Implement Update(...)
• Implement List(...)
• Implement Sign(...)
• Implement Sign....(...)
• All other methods should return NotImplementedError

IMPORTANT: Compare impl with https://github.com/ConsenSys/quorum-signer-plugin-for-hashicorp-vault

• Implement function taking manifest specs, extracting it to create the corresponding store

• Implement a Manifest Loader that loads manifest files from a filesystem folder this is expected to be static (load only once)
  As part of this task, we should define the type of file format json, yaml, toml...? Initialize Hashicorp Vault secret store

The goal of this ticket is to evaluate different logger alternatives, such as go-kit or zap

Requirements are:

• Decouple from our application and easy to unplug
• Support Context logging
• Use interfaces instead of structs for logging

Acceptance criteria

• Run e2e
• Release KeyManager images.
• Review go-quorum docker tagging https://docs.google.com/document/d/1D81RASiixAA5E13LxvDDQG-5HtCEWU2UxDPCRxCu0LE/edit

Open Questions

• Where images should be published due to the usage of BSL License

Write acceptance test for the following:

• eea_sendTransaction
• eth_accounts
• eth_sendTransaction (with and without private_args)
• eth_sign
• eth_sign_transaction

Include Besu and go-quorum

We should get in touch with pliny and collaborate to release the first version of KM docs

To be implemented in core/store/keys/local

• Implement Create(...)
• Implement Get(...)
• Implement Update(...)
• Implement List(...)
• Implement Sign(...)
• All other methods should return NotImplementedError

Acceptance criteria

• Support in AWS
• Support in Azure
• Support Baseline signing flow

1. [HIGH] Node component appears to be one of our application core component which it belongs to our business layer, so that, two things about it:
   Node should not be aware of the outside layer of our application what it is at the moment because its interface it exposes httpClient, jsonrpc.Client, jsonrpc.Handler, all of them are part of what we could called INFRA layer. Alternatively I think Node should implement jsonrpc call methods required by the service layer (a proxy receiving a method and a payload req) and help methods such as getGasPrice, getNonce to be used on the business layer
   This component should be place at /src/core/node

2. [HIGH] IMO sessions should not be manage by the node components. Session will not necessarily depend on the node but on the users making the requests to the same node. For that the most standardized way to do is SESSION_ID which lives in the Request object and exchanged between the client and the server. To achieve that gorilla provides a library which I believe we could easily integrate https://github.com/gorilla/sessions

3. [MEDIUM] Everything related to jsonrpc server code, proxy logic and the preparer /pkg/http/request/preparer_proxy.go should be moved together under /src/infra. As far as I see all that logic appear to be out outside facing layer including the proxy preparer which IMO it is not a preparer but part of business logic of our proxy server.

4. [MEDIUM] If I am not mistaken we could skip the wrapper implementations of json-rpc request and response and it would be enough with a RequestMSG and ResponseMSG. As far as I understand a request will flow over different request preparers till it reaches the intended Besu/Quorum node and same for the response over the modifiers till it comes back to the user. Accounting on that I think it would be cleaner if we attach things directly over the http.request and http.response and make all the work of transforming to the right formats using static parsing methods fetching the context attached on those objects (alternatively on the context in case those object can not hold key-value fields)

5. [MEDIUM-LOW] Config structs should be a things initialize once and we never modify over the course of execution. The initialization of those config struct should only happen when the application inits, or when a new request it is coming into our application via our service layer. So that all config struct should remain under infra package (edited)

6. [LOW] We should avoid prefixing every modified and preparer by that name by adding them into a subfolder call with the same name

To be implemented in core/store/keys/azure-key-vault

• Define type Config struct
• Implement New(cfg *Config) *Store creating store from configuration
• Implement Create(...)
• Implement Get(...)
• Implement Update(...)
• Implement List(...)
• Implement Sign(...)
• All other methods should return NotImplementedError

The goal of this ticket is to investigate which alternatives we can use to connect to AKV and create tickets around them

To be implemented in core/store/keys/aws-kms

• Define type Config struct
• Implement New(cfg *Config) *Store creating store from configuration
• Implement Create(...)
• Implement Get(...)
• Implement Update(...)
• Implement List(...)
• Implement Sign(...)
• All other methods should return NotImplementedError

Acceptance criteria

• Remove refresh endpoints from store endpoints
• Remove version from HTTP interfaces for simplicity

Currently, the ETH account data comes from the underlying key store. This data is not enough to index the accounts because the address is not present in the keys. We need to store the address to be able to query accounts by address:

• DB server implementation (ex: PG)
• DB local implementation (SQLite, Redis)
• In-memory solution

Acceptance criteria

• Decide on a storage implementation for accounts, stores, audit logs and other persistent data.

Please read comments of the ticket

JSON-RPC use-case (e.g. eth_sendTransaction) needs some functional logic in order to synchronize various operation involving secure Stores, downstream node, etc.

What is the best approach for implementation?

Currently, the Hashicorp plugin keys domain uses Ethereum [R||S||V] signature scheme instead of the standard ECDSA [R||S] signatures. This should be modified to make the keys compatible with other stores (AKV, AWS).

Acceptance criteria:

• Stop using go-ethereum libraries for signature, use ecdsa standard package.
• Return base64 format
• integration tests
• Eth1.Store should work with the Hashicorp key store.

Discussion:
Change the key format to return x and y values?

Create a command to start the Key-Manager

Part of this ticket consist in proposing the library for command line (cobra?)

Add pagination to endpoints returning lists.

To be implemented in core/store/secrets/hashicorp

• Define type Config struct
• Implement New(cfg *Config) *Store creating store from configuration
• Implement Set(...)
• Implement Get(...)
• Implement Update(...)
• Implement List(...)
• All other methods should return NotImplementedError

In order to align with best practices standard over key management Hashicorp should provide support for soft-delete on keys.

Acceptance criteria

• Return custom error in stores
• Add logging to stores (with Info/Warn/Error, etc. levels)
• Mapping of Errors to correct HTTP codes in the service layer
• Acceptance test & unit test of error returned

Currently the development environment fails to start when running the command make dev.

• Fix the docker-compose network issues
• Start The Dependencies (hashicorp only now)
• Start key manager connected to hashicorp

Implement integration tests for Hashicorp key store.

Consist in implementing a JSON-RPC proxy able to extract JSON-RPC information.

Implementing interceptors is not part of this issue. In the end it should be a pass-through, possibly logging.

Waiting for research over if this feature would be useful to be implemented based on the web3 usage

Description

Implement the AWS secret store by following the specifications and by taking example from the other secret stores:

• Define implementation detail by following the AWS KMS: https://docs.aws.amazon.com/kms/index.html

Specification

https://docs.google.com/document/d/1GkGUW79iKAhzbFnwY28TVizZGxxMiO8f2XQvZyE2zl4/edit?usp=sharing

Acceptance criteria

• Complete the secret store implementation by:
• Implementation AWS KMS client in infra layer
• Implement the AWS KMS secret store in store layer by taking example from the hashicorp secret store (also AKV)
• Implement the integration tests by adding AWS localstack: https://github.com/localstack/localstack

Implement a Manifest Loader that

• loads manifest files from a filesystem folder
• this is expected to be static (load only once)

As part of this task, we should define the type of file format json, yaml, toml...?

Remove orchestrate dependency from Vault plugin:

• Remove usage of crypto package from Orchestrate
• Copy package content into pkg of plugin

https://docs.google.com/document/d/1GkGUW79iKAhzbFnwY28TVizZGxxMiO8f2XQvZyE2zl4/edit#heading=h.enhjo8a8lj0p

Currently the Hashicorp plugin deals with camel case, it should be changed to snake case to comply with Hashicorp. keys have already been updated.

Improve error handling of the Vault plugin:

• Remove dependency on Orchestrate errors and define custom errors in the vault plugin directly
• Map errors to a HTTP response (400, 404, 422, 500)

Methods to implement

• eth_sendTransaction
• eth_sign
• eth_signTypedData
• eth_accounts
• eth_signTransaction
• eea_sendTransaction
• personal_newAccount
• personal_listWallets (does it makes sense to rename personal_listStores?)
• personal_* (to be silence)
• default (pass through)

Should this issue broken is smaller ones?

Implement token renewal runnable service to renew token, following what has been implemented in Orchestrate (at least for this first version).

Decide between next options:

• Runnable stores
• Runnable client
• Manage by store manager

json-rpc protocol support multiple request at once by appending requests into body. The goal is to make the interceptor aware of this and handle it

Acceptance

• Verify endpoints for keys (ecdsa, eddsa)
• Verify endpoints for eth1 accounts