We need to figure out what API endpoints we can make RPC calls to get the following data for ETH2:

• Beacon block header data
• Light client update data
• Execution header data

Once we figure out what API endpoint to use, we can use this library to actually generate the JSON files with the requisite data: https://github.com/aurora-is-near/rainbow-bridge/tree/master/eth2near/eth2near-block-relay-rs.

Overview

Currently, the client connects to the default node locally running node on ws://127.0.0.1:9944. Instead, we should use the substrate URL from the config file to form a connection with the node.

let api : OnlineClient<PolkadotConfig> = OnlineClient::<PolkadotConfig>::from_url(config.substrate_url).await?;

Overview

We probably need to update lighthouse dependencies to match with the one required by Tangle inorder to run with it.

error: failed to select a version for `libsqlite3-sys`.
    ... required by package `rusqlite v0.28.0`
    ... which satisfies dependency `rusqlite = "^0.28.0"` of package `types v0.2.1 (https://github.com/webb-tools/lighthouse.git#dfcb3363)`
    ... which satisfies git dependency `types` of package `eth2-to-substrate-relay v0.1.0 (https://github.com/webb-tools/pallet-eth2-light-client?tag=v0.4.3#c089a76d)`
    ... which satisfies git dependency `eth2-to-substrate-relay` of package `pallet-eth2-light-client-relayer-gadget v0.0.1 (https://github.com/webb-tools/pallet-eth2-light-client?tag=v0.4.3#c089a76d)`
    ... which satisfies git dependency `pallet-eth2-light-client-relayer-gadget` of package `tangle-standalone v0.4.2 (/Users/salman01zz/Webb-Tools/tangle/standalone/node)`
versions that meet the requirements `^0.25.0` are: 0.25.2, 0.25.1, 0.25.0

the package `libsqlite3-sys` links to the native library `sqlite3`, but it conflicts with a previous package which links to `sqlite3` as well:
package `libsqlite3-sys v0.26.0`
    ... which satisfies dependency `libsqlite3-sys = "^0.26.0"` (locked to 0.26.0) of package `sqlx-sqlite v0.7.1`
    ... which satisfies dependency `sqlx-sqlite = "=0.7.1"` (locked to 0.7.1) of package `sqlx v0.7.1`
    ... which satisfies dependency `sqlx = "^0.7.1"` (locked to 0.7.1) of package `fc-db v2.0.0-dev (https://github.com/paritytech/frontier.git?branch=polkadot-v1.0.0#b520b749)`
    ... which satisfies git dependency `fc-db` (locked to 2.0.0-dev) of package `fc-cli v1.0.0-dev (https://github.com/paritytech/frontier.git?branch=polkadot-v1.0.0#b520b749)`
    ... which satisfies git dependency `fc-cli` (locked to 1.0.0-dev) of package `tangle-standalone v0.4.2 (/Users/salman01zz/Webb-Tools/tangle/standalone/node)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the links ='libsqlite3-sys' value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

Overview

Create a relayer context structure that stores relayer configuration, shutdown signals and implements substrate provider and other utility functions

/// LightClientRelayerContext contains Relayer's configuration and shutdown signal.
#[derive(Clone)]
pub struct LightClientRelayerContext {
	pub lc_relay_config: RelayConfig,
	pub lc_init_config: InitConfig,
	/// Broadcasts a shutdown signal to all active connections.
	///
	/// The initial `shutdown` trigger is provided by the `run` caller. The
	/// server is responsible for gracefully shutting down active connections.
	/// When a connection task is spawned, it is passed a broadcast receiver
	/// handle. When a graceful shutdown is initiated, a `()` value is sent via
	/// the broadcast::Sender. Each active connection receives it, reaches a
	/// safe terminal state, and completes the task.
	notify_shutdown: broadcast::Sender<()>,
}

Goal :

The goal of pallet-light-proposals is to verify the block_header and proofs submitted by the relayers and create proposals. The high level flow is :

• Relayer submits the eth_blockHeader and merkle proofs to the pallet-light-proposal
• pallet-light-proposals checks its a valid block header by :
  1. Ensure the block_hash generated from the blockHeader is recorded in eth-light-client pallet
  2. Verify the two merkle proofs uploaded : https://github.com/webb-tools/webb-rs/blob/main/bridge-proofs/src/lib.rs
• Make the anchor update proposals and submit to unsigned proposal queue

Pallet :

• Extrinsic calls for relayers to submit proposals

pub struct LightProposalInput<B,P,S> {
  pub block_header : B,
  pub root_merkle_proof : P,
  pub storage_merkle_proof : S
}

#[pallet::call_index(1)]
#[pallet::weight(Weight::from_ref_time(10_000) + T::DbWeight::get().reads_writes(1,1))]
pub fn submit_proposal(origin: OriginFor<T>, proposal : LightProposalInput<>) -> DispatchResultWithPostInfo {
  // generate the block hash
  let block_hash = Self::generate_block_hash(proposal.block_header);

  // validate the block hash against the eth-light-client storage
  ensure!(Self::is_block_hash_valid(block_hash), Error::<T>::InvalidBlock);

 // validate the merkle proofs
 ensure!(Self::validate_merkle_proof(proposal.proof), Error::<T>::InvalidProof);

 // prepare the proposal
Self::prepare_proposal(proposal);
}

References :
https://www.notion.so/hicommonwealth/Proposals-System-v2-311fe027f2e640c4a0170c0281e4a38a

Overview

Subxt errors are not parsed correctly and they return errors including all the metadata.

Error :

Runtime(Module(ModuleError { metadata: Metadata { 
inner: Metadata { types: PortableRegistry { types: [
PortableType { id: 0, ty: Type { path: Path { segments: ["sp_core", "crypto", "AccountId32"] }, type_params: [], type_def: 
.
.
raw: RawModuleError { pallet_index: 41, error: [0, 0, 0, 0] } }))

Expect Error : Pallet AlreadyInitialized

Overview

Add spec for Verification pallet design and construction of Merkle proof for tx receipt.

For reference ggxchain/transaction-receipt-relayer#26

SPEC Task Checklist

• Construction of Merkle proof for tx receipt.
• Pallet Design
• Break design into small tasks it required
• Initial flowchart

Develop and integrate pallet-light-proposals to tangle-runtime

Checklist :

• Prep initial flow chart and proposal verification spec
• [Pallet] Develop the extrinsic / pallet that verifies a storage proof and triggers an AUP (#24)
• [Relayer] Listen for event on source chain.
• [Relayer] Check / query Tangle to see when the block header has been relayed.
• [Relayer] Once relayed, generate storage proof.
• [Relayer] Submit storage proof + metadata necessary for extrinsic.
• [Tangle] Integrate pallet-light-proposals to tangle runtime

Tasks (WIP):

• Update Eth2 crates

• Update config files

• Update block data

• Update Eth2ClientTrait

• Update eth2subsrate_relay.rs

• Update last slot searcher (is deleted)

• Tests for the Pallet trait, use shell scripts (possibly) for starting/stopping blockchain

Overview

The main checklist for tracking what needs to get done to get this working.

Components

1. Pallet
2. Types libraries (pulled from lighthouse / rainbow-bridge
3. Relayer service (pulled from rainbow-bridge, using webb-relayer)
4. Substrate client lib to interact with pallet (subxt)

Tasks

• Pallet scaffold
• Pallet preliminary tests
• Pallet no_std/wasm compilation in a substrate node runtime.
• Implement EthClientPalletTrait for EthClientPallet (used to interface with pallet from relayer)
• Integrate Eth2SubstrateRelay into a LightClientRelayer service using our relayer software.
• #22
• Integration test relayer w/ local chain with pallet-eth2-light-client and subxt
• Stable subxt, pallet, relayer dependencies
• Pallet built and tested (continue updating against any changes to rainbow-bridge)
• Light client relayer crates building and tested (continue updating against any changes to eth2near-relay)

Mock testing

Testing pallet in Substrate usually starts with unit tests against a mocked runtime. You can find the mocked runtime for the light client inside the pallet directly, https://github.com/webb-tools/pallet-eth2-light-client/blob/drew/relay/pallet/src/mock.rs. Specifically, the mocked runtime has a user-defined struct such as Eth2Client that can be manipulated to interact with the pallet's API directly.

For mocked testing, we would likely import the eth2substrate-block-relay-rs library into the pallet as a dev-dependency and then run the relayer in the pallet testing environment.

Integration testing

Testing the pallet as an integration test will require integrating it into a full Substrate node. This testing setup will require running a live node adjacent to the relayer/tests of the relayer. Integration testing should only be done using subxt which is the library for making remote calls to a Substrate chain.

Overview

There are a few configuration files that are required to run the light client relayer and then once figured out, we want to ensure the system runs as expected within a live context.

Getting passed this step will help in crafting the proper README / docs for running the light client relayer.

Summary

• Fix build post substrate update v1.0.0 changes #35
• Fix webb-rs deps issue in dkg
• update light client with new dkg release

Overview

To test the relayer with unit tests, we need to finish implementing our mocked pallet client and hook the existing relayer tests into the pallet tests itself. This would allow each test to have a fresh environment when running the relayer as a library.

Checklist

• Implement mocked pallet client
• Migrate tests from the init-pallet to the pallet/tests
• Migrate tests from the eth2substrate-block-relay to the pallet/tests
• Add these tests to CI

https://github.com/webb-tools/pallet-eth2-light-client/blob/main/crates/eth2-pallet-init/src/substrate_pallet_client.rs#L358-L379

Overview

Since we don't intend to use webb-relayer here, let's remove it and also update the relevant places

• Remove webb-relayer gadget
• Remove configs
• Update Cli cmd
• Update readme

Overview

Let's use workspace inheritance for all the light client crates, This will help manage versions in dependencies and look clean

Task Checklist

• Create workspace inheritance for crates
• Remove unused dependencies

Overview

Simplyfy configuration, we can divide config into three parts

• LightClientRelayerConfig
• BeaconChainConfig
• TangleSubstrateConfig

and set default values for most of the values and just pass simple single configuration file

#[derive(Debug, Clone, Deserialize, Serialize, Default)]
#[serde(rename_all(serialize = "camelCase", deserialize = "kebab-case"))]
pub struct LightClientRelayerConfig {
    pub tangle_config : TangleSubstrateConfig,
    pub beacon_chain_config : BeaconChainConfig,
    // endpoint for the Ethereum full node, which supports Eth1 RPC API
    pub eth1_endpoint: String,
    pub get_light_client_update_by_epoch: Option<bool>,
    // the max number of headers submitted in one batch to eth client
    pub headers_batch_size: u32,
    
}

#[derive(Debug, Clone, Deserialize, Serialize, Default)]
#[serde(rename_all(serialize = "camelCase", deserialize = "kebab-case"))]
pub struct BeaconChainConfig {
    pub name: String,
    // Beacon http endpoint
    #[serde(skip_serializing)]
    pub http_endpoint: RpcUrl,
    // Beacon rpc version (V1_1, V1_2)
    pub beacon_rpc_version: BeaconRPCVersion,
    // sleep time on sync
    pub sleep_time_on_sync_secs: u64,

}

#[derive(Debug, Clone, Deserialize, Serialize, Default)]
#[serde(rename_all(serialize = "camelCase", deserialize = "kebab-case"))]
pub struct TangleSubstrateConfig {
    /// String that groups configuration for this chain on a human-readable name.
    pub name: String,
    /// Websocket Endpoint for long living connections
    #[serde(skip_serializing)]
    pub ws_endpoint: RpcUrl,
    #[serde(skip_serializing)]
    pub suri: Option<Suri>,
}

Overview

Create a verification pallet that will verify the TX proofs and storage proofs we can look at some work done by GGX chain here. We should try to make it generic and reusable.

ggxchain/transaction-receipt-relayer#26

SPEC: #80

Task List

• Add verification pallet
• Add tests for storage proof
• Add tests for tx proof

Checklist

• [BUG] Fix webb-proposals no_std issue.
• [TASK] Update the pallet-eth2-light-client eth2substrate-block-relay-rs.
• [TASK] Running it inside with Tangle (setting up configuration files)
• [TASK] Update README
• [TASK] (low-priority) Re-implement the eth2substrate-block-relay-rs w/ runtime-apis / RPCs / as a gadget.
• #67
• Implement retry client for BeacnRpc Client #78
• Implement retry client for EthRpc Client #82
• Handle substrate node connection drops (restart the lc-relayer) #76

Overview

Implement retry logic for submitting

• beacon_chain RPC calls,
• eth RPC calls
• light client update.

We also need to add retry logic to wait and reconnect if the connection to the tangle node is dropped

reproduce step

1. Set Infura API Key

export ETH1_INFURA_API_KEY=xxxx

2.Build light client

cargo build -p node-template

3. Run light client

//#terminal 1
./target/debug/node-template --tmp --chain local --alice
--rpc-external --unsafe-rpc-external --rpc-methods=unsafe
--port 30433

//#terminal 2
./target/debug/node-template --tmp --chain local --bob
--rpc-cors all --rpc-external --rpc-methods=unsafe
--port 30405
--relayer-config-dir=./gadget/config
--light-client-init-pallet-config-path=./crates/eth2-pallet-init/config.toml
--light-client-relay-config-path=./eth2substrate-block-relay-rs/config.toml
--rpc-port 9445 --bootnodes /ip4/127.0.0.1/tcp/30433/p2p/12D3KooWCzqQx1oEPJ94uDPXPa2VdHqDD5ftpCFmSd5KPHgxMivK

//#terminal 3
./target/debug/node-template --tmp --chain local --charlie
--rpc-cors all --rpc-external
--rpc-port 9448
--port 30408
--relayer-config-dir=./gadget/config
--light-client-init-pallet-config-path=./crates/eth2-pallet-init/config.toml
--light-client-relay-config-path=./eth2substrate-block-relay-rs/config.toml
--bootnodes /ip4/127.0.0.1/tcp/30433/p2p/12D3KooWCzqQx1oEPJ94uDPXPa2VdHqDD5ftpCFmSd5KPHgxMivK
--unsafe-rpc-external --rpc-methods=unsafe

the config file

./crates/eth2-pallet-init/config.toml

ethereum_network = "Goerli"
beacon_endpoint = "https://lodestar-goerli.chainsafe.io"
eth1_endpoint = "https://goerli.infura.io/v3/ETH1_INFURA_API_KEY"
eth_requests_timeout_seconds = 30
validate_updates = true
verify_bls_signature = true
hashes_gc_threshold = 51000
beacon_rpc_version = "V1_5"

substrate_network_name = "Tangle"
substrate_endpoint = "ws://127.0.0.1:9944"
path_to_signer_secret_key = "/tmp/empty/secret_key"

./eth2substrate-block-relay-rs/config.toml

beacon_endpoint = "https://lodestar-goerli.chainsafe.io"
eth1_endpoint = "https://goerli.infura.io/v3/ETH1_INFURA_API_KEY"
network_name = "Goerli"
headers_batch_size = 33
substrate_endpoint = "ws://127.0.0.1:9944"
path_to_signer_secret_key = "/tmp/empty/secret_key"
ethereum_network = "Goerli"
interval_between_light_client_updates_submission_in_epochs = 1
max_blocks_for_finalization = 5000
include_next_sync_committee_to_light_client = false
eth_requests_timeout_seconds = 30
state_requests_timeout_seconds = 1000
substrate_requests_timeout_seconds = 30
sleep_time_on_sync_secs = 0
sleep_time_after_submission_secs = 5
hashes_gc_threshold = 51000
beacon_rpc_version = "V1_5"

Relevant code

crates/bls/src/impls/milagro.rs

fn fast_aggregate_verify(
	&self,
	msg: Hash256,
	pubkeys: &[&GenericPublicKey<milagro::PublicKey>],
) -> bool {
	let pubkeys = pubkeys.iter().map(|pk| pk.point()).collect::<Vec<_>>();
	self.fast_aggregate_verify(msg.as_bytes(), &pubkeys)
}

milagro_bls-c6b4b448ce61a420/7d1776e/src/aggregates.rs

ate2_evaluation(&sig_point, &generator_g1_negative, &msg_hash, &key_point) return false

Overview

The relayer tests in the pallet-eth2-light-client repo currently won't work in Substrate due to the chain not being reset across tests that don't use test_externalities context. If we want to get the rainbow-bridge tests working for the relayer we need to refactor them inside this environment and run a light-client relayer from within that.

Overview

Add event watcher for relayer which will watch for the Vanchor deposit event and create lc-proposal and send them to pallet light-proposal-handler

/// Light Client Proposal.
///
/// On Vanchor deposit event the [`LightClientProposal`] sends a light storage proofs along with block headers to tangle
///  Tangle verifies this storage proofs and creates Anchor update proposals for each connected chains
/// The format of the proposal is:
/// ```text
/// ┌────────────────────┬─────────────────┬─────────────────────┐
/// │                    │                 │                     │
/// │    BlockHeader     │  Storage proof  │    Storage proof    │
/// │                    │     Root        │      Leaf Index      │
/// └────────────────────┴─────────────────┴─────────────────────┘
/// ```
#[derive(Proposal, Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct LightProposal {
    block_header: BlockHeader,
    merkle_root_proof: Vec<u8>,
    leaf_idx_proof: Vec<u8>,
}

Task Checklist

• Event watcher for deposit event (featured-flag)
• Query Tangle to see when the block header has been relayed.
• Fetch storage proofs for merkle-root and leaf-idx
• Construct BlockHeader
• Construct LightProposal and send to pallet light-proposal-handler

Overview

following are some pros for using thiserror instead of anyhow in light-client crate.

• We can define our own error type for the application LightClientError enum
• Easy to review and fix types
• convinient to use ? operator to unpack results
• reduces code.

Resources

https://github.com/dtolnay/thiserror#comparison-to-anyhow
https://www.shakacode.com/blog/thiserror-anyhow-or-how-i-handle-errors-in-rust-apps/

Overview

• Proper readme update
• What commit of rainbow-bridge we are using?
• How to run it?