Overview

In XAnchor, there is a lot of Edge cases in the gov. system, I will try to cover most of them here so we can keep track or address them by writing first a failing test and then later on fix it.

For all of the following tasks/checks we are going to refer to Chain A as the source chain and Chain B as the target chain.

Checklist

• Calling propose_to_link_anchor to create on-chain proposal on chain A could fail to link anchors if the proposal not passed.
• After the proposal passed on chain A, the send_link_anchor_message gets called we send two messages:
  • save_link_proposal as a sibling parachain, which just puts the link proposal in the pending link, what would happen if the target tree on the target chain is already linked to something else?
  • handle_link_anchor_message as a sovereign account, to actually kick the proposal on the other chain, what if it get called by normal Signed account?
  • handle_link_anchor_message as a sovereign account, to actually kick the proposal on the other chain, what if the parachain sovereign account does not have the value as balance?
• On chain B what happens if the proposal didn't pass? How the chain A could know about that? to remove that anchor from the PendingLinkedAnchors list?

Overview

We need to add logic for updating the chain ID to the new chain ID type into this repo.

• Add the chain ID type update to linkable-tree
• Update the zkp verify functions that utilise the chain ID

Overview

To get a testnet of dkg-standalone-nodes working, a user needs to complete the following actions. Example in a 3 node test network:

dkg-node-1:

1. startup the relayer: ./target/release/dkg-standalone-node --chain=chainspecs/arana_testnet_raw.json --port 30333 --ws-port 9944 --rpc-port 9933 --node-key <key> --validator --rpc-cors all

2. insert aura, grandpa, and dkg keys into the keystore:

   curl http://localhost:9933 -H "Content-Type:application/json" -d '{"jsonrpc":"2.0", "id":1, 
   "method":"author_insertKey", "params":["aura","<mnemonic>","<public key>"]}'
   
   curl http://localhost:9933 -H "Content-Type:application/json" -d '{"jsonrpc":"2.0", "id":1, 
   "method":"author_insertKey", "params":["gran","<mnemonic>","<public key>"]}'
   
   curl http://localhost:9933 -H "Content-Type:application/json" -d '{"jsonrpc":"2.0", "id":1, 
   "method":"author_insertKey", "params":["wdkg","<mnemonic>","<public key>"]}'
   
   curl http://localhost:9933 -H "Content-Type:application/json" -d '{"jsonrpc":"2.0", "id":1, 
   "method":"author_insertKey", "params":["acco","<mnemonic>","<public key>"]}'
   

3. Shutdown the node and purge the chain's db.

4. Repeat 1-3 for two more nodes.

5. Start up all 3 nodes with the same command in step 1.

Are there any ways to reduce these setup steps?

Overview

• Follow the xanchor, and regular anchor for demonstrated implementation

Chain Id should be fetched from the runtime (pallet storage or similar) and passed into the circuit

Overview

• Change references to darkwebb to webb
• Ensure README.md are updated

Description

The Ecosystem of ORML assets and native balances seems not to work as expected, asset with id 0 is considered the native asset
, however, the tokens pallet doesn't read the balance it sticks to zero.
maybe that's because the pallet balance doesn't read the balance? as in Polkadot API, they have an example to read the balances they use the system pallet

The mixer seems to consume the native asset if it was created with asset=0

Expected vs. Actual Behavior

• It's expected that the tokens pallet read native balance and it doesn't.
• if there is an asset with id 0 the assetRegistery>nextAssetId should be 1.

Steps to Reproduce

1. From a fresh main node, open substrate portal
2. Try out the native asset or ORML asset with id =0
   • Transfer some funds from the account dropdown
   • Go to chain state it's expected to see the native balance of that account under tokens>accounts and set the id to 0 which isn't the case.
   • Go to chain state balances>accounts it's an empty balance still, to view the balance try system>account
   • Chain state assetRegistery>nextAssetId, it still 0 event though the 0 is preserved to the native asset
3. Try out the native asset or ORML asset with id =1
   everything works as expected, the asset id jumps directly to 2

Environment

• Operating system: MacOS
• Template version/tag: 9d2408a
• Rust version (run rustup show):
  nightly-2021-11-20-x86_64-apple-darwin (overridden by '/Users/ahmed/projects/edgeware/webb/darkwebb-substrate/rust-toolchain.toml')
  rustc 1.58.0-nightly (a77da2d45 2021-11-19
  Logs, Errors or Screenshots

The keys for the updated arkworks gadgets need to be verified against the pallets.

In addition, we should make the tests match wasm-utils as closely as possible. Meaning:

1. We should use unchecked (uncompressed) proving keys
2. We should use setup_leaf_x5 and setup_proof_x5 for proof generation

Overview

Go through the following pallets and add / update documentation overview:

• - LinkableTree
• - Bridge
• - AnchorHandler
• - SignatureBridge
• - VAnchor
• - XAnchor

We want support for signed proposals to trigger updates on a pallet-signature-bridge.

• For testing, use a single ECDSA signature to govern things.
• For building, refer to Bridge.sol and SignatureBridge.sol.

Overview

The motivation here is to store a relayer configuration object on-chain and build it from the chain. This way a relayer can listen to this pallet and construct relayer configuration objects to execute on its own relayer.

Questions

• What fields of the config are intended to be on-chain?
  • Certainly the http-endpoint / ws-endpoint fields won't be there.
  • Just the contract addresses along with associated linked-anchors?
• What is the on-chain schema for a fixed bridge, variable bridge?
• How do we decide on adding a new chain to a bridge?

We should consider consolidating our utility functions into one place that can be used by both Typescript/Rust repos. One way we can do this is by adding functions to wasm-utils and re-exporting their TS versions from webb.js.

We would then have to import these tools from these libraries for each of the repos that are using them:

• relayer
• solidity / substrate
• etc.

Problem Description

Failed to get a deposit to work on native/non-native assets

Recreation Steps

• From a custom asset
  • Create a custom asset with a minimum balance of 5 units,and id 5
  • Mint tokens for an account (say 500 units)
  • Create a mixer for this asset with balances that is less and more than the minimum balance of the currency
  • Try to deposit on those sizes
  • There will be an error token.BalanceTooLow
• For native asset id 0
  • Transfer some balance to an account say (100K)
  • Create Mixer using Charlie with the amount of 5 and asset is native id 0
  • Try to deposit using an account with enough balance 100k units, or Charlie 8.9KK units
  • Error ⇒ balances.ExistentialDeposit

Triage

• Existential deposits or Minimum balance?

Overview

We have Config types that we want to move into the runtime storage to be configurable by a certain Origin type. In the future, the DKG will be responsible for updating these parameters and we'll use a handler pattern to do so. For now, we should move this into the storage and add tests for updating them.

• https://github.com/webb-tools/protocol-substrate/blob/main/pallets/vanchor/src/lib.rs#L137

		// Max deposit amount
		type MaxDepositAmount: Get<BalanceOf<Self, I>>;
		// Max withdraw amount
		type MinWithdrawAmount: Get<BalanceOf<Self, I>>;
		// Max external amount
		type MaxExtAmount: Get<BalanceOf<Self, I>>;
		// Max fee for relayer
		type MaxFee: Get<BalanceOf<Self, I>>;

Overview

We should set up a repo that will test interoperability between our ethereum and substrate Anchors.

Requirements:

• Launch a local ethereum network and deploy contracts, leave it running in the background
• Launch a local substrate network and leave it running in the background
• Run a relayer

Use https://github.com/openethereum/openethereum or equivalent to connect to the ethereum chain

Use https://github.com/scs/substrate-api-client or equivalent to connect to substrate chain

Use https://github.com/webb-tools/relayer for running a relayer

Checklist

Positive:

• Deposit on Ethereum, withdraw on Substrate chain
• Deposit on Substrate, withdraw on Ethereum chain

Negative:

• Deposit on Ethereum, specify Substrate chain_id, withdraw on Ethereum, should fail
• Deposit on Ethereum, specify Ethereum chain_id, withdraw on Substrate, should fail
• Do not connect the Anchors, then deposit on Ethereum, withdraw on Substrate chain, should fail

This spec is WIP

Overview

We want to "rescue tokens" by signature on the Substrate side which can come about using a handler or potentially not. This task should include investigate into the potential routes we can take given Substrate's capabilities.

Details

There is already a treasury account in protocol-substrate and the fees are being sent there:
https://github.com/webb-tools/protocol-substrate/blob/main/pallets/token-wrapper/src/lib.rs#L105
https://github.com/webb-tools/protocol-substrate/blob/main/pallets/token-wrapper/src/lib.rs#L291

So we don't need to implement a separate treasury pallet. What we can instead do is the following:

• Add a rescue_tokens function to the token-wrapper pallet. This function signature for rescue_tokens should look something like:

fn rescue_tokens(asset_id: T::asset_id, to: T::AccountId, amount_to_rescue: BalanceOf<T>, nonce: u32)

• Add an execute_rescue_tokens function to the token-wrapper-handler pallet.
• Test the execution of rescue_tokens via the signature-bridge.

Overview

Review resource encoding/decoding to ensure parity between protocol-solidity.

Overview

we should start using

#[transactional]

https://docs.substrate.io/rustdocs/latest/frame_support/attr.transactional.html

more in our pallets, especially the one that touches user balances.

Sometimes, when we take the amount from the user balances but later on along the xt it fails with an error, so we need to revert the whole transaction and the balance shouldn’t be touched, that’s what this macro does!

Also, for other pallets that other pallet gonna build on to of it, we should also use this one too, https://docs.substrate.io/rustdocs/latest/frame_support/attr.require_transactional.html to enforce the users of that pallet to mark their xt to be transactional

Overview

The refresh proposal logic asks on each block whether we should_refresh and if so proceeds to start a refresh proposal process. It does not however check whether we have started a refresh and have one in progress. This was overlooked by myself. We should add a way for the storage system to check whether we should_refresh AND are_refreshing. Then we won't have so many proposals being created locally for signing, which is creating many offline stages and likely causing performance issues.

fn on_initialize(n: BlockNumberFor<T>) -> frame_support::weights::Weight {
    // The bug is here since we will continue to create proposals every block once this is `true`.
    if Self::should_refresh(n) {
        if let Some(pub_key) = Self::next_dkg_public_key() {
            let uncompressed_pub_key = Self::decompress_public_key(pub_key.1).unwrap();

            let next_nonce = Self::refresh_nonce() + 1u32;
            let data = dkg_runtime_primitives::RefreshProposal {
                nonce: next_nonce,
                pub_key: uncompressed_pub_key,
            };

            match T::ProposalHandler::handle_unsigned_refresh_proposal(data) {
                Ok(()) => {
                    RefreshNonce::<T>::put(next_nonce);
                    frame_support::log::debug!("Handled refresh proposal");
                },
                Err(e) => {
                    frame_support::log::warn!("Failed to handle refresh proposal: {:?}", e);
                },
            }
        }
    }

    0
}

Types

• r_id (resource ID) - 32 byte string:
  • Last 6 bytes reserved for chain id type (2 bytes chain type + 4 bytes chain identifier)
  • Last adjacent 20 bytes are for ... (more on this in the next section).

More on r_id's

r_id's serve two purposes. Let's take the example of an anchor-update.

1. There is a r_id to method name mapping called Resources on the signature-bridge.
2. On the anchor-handler pallet, there is a mapping called AnchorList that maps r_id's to tree_id's. This is how the update method knows which anchor to update. Therefore, the r_id should contain some uniquely identifiable information about the tree_id (or the tree_id itself). Don't think putting just the tree_id will be enough since it just an integer...may not be unique enough. Might want to do something like the world 'ANCHOR" concatenated with the tree_id.

Overarching Schema of Proposal Data

Proposal Data:

• r_id: ResourceId (32 bytes)
• zeroes: [u8, 4] (4 bytes)
• nonce: T::ProposalNonce (4 bytes)
• call (the length of this is proposal specific)

Terminology: We will call the (r_id, zeroes, nonce) as the proposal header. The proposal header is a total of 40 bytes. We add 4 bytes of zeroes padding to match the length of the EVM/Solidity proposal header. On the EVM/Solidity side, these 4 bytes are taken up by the functionSig.

See how execute_proposal works (including pseudocode) in the comment below.

List of Proposals

1. Set Wrapping Fee Proposal
Pallet Flow: bridge/signature-bridge -> token-wrapper-handler -> token-wrapper
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure: Call::TokenWrapperHandler(Call::execute_wrapping_fee_proposal { r_id, wrapping_fee_percent, into_pool_share_id })

• r_id is the r_id
• wrapping_fee_percent is the new token wrapping fee
• into_pool_share_id is the id of the pool share we are setting the wrapping fee for (recall we have wrapping fees per pool share)

2. Add Token to Pool Share Proposal
Pallet Flow: bridge/signature-bridge -> token-wrapper-handler -> token-wrapper -> asset-registry
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure: Call::TokenWrapperHandler(Call::execute_add_token_to_pool_share { r_id, name, asset_id })

• r_id is the r_id
• name is the name of the pool share
• asset_id is the id of the asset being added to the pool share.

3. Remove Token from Pool Share Proposal
Pallet Flow: bridge/signature-bridge -> token-wrapper-handler -> token-wrapper -> asset-registry
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure: Call::TokenWrapperHandler(Call::execute_remove_token_from_pool_share { r_id, name, asset_id })

• r_id is the r_id
• name is the name of the pool share
• asset_id is the id of the asset being removed from the pool share.

4. Anchor Create Proposal
Pallet Flow: bridge/signature-bridge -> anchor-handler -> anchor
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure: Call::AnchorHandler(Call::execute_anchor_create_proposal { deposit_size, src_chain_id, r_id, max_edges, tree_depth, asset })

• deposit_size is the denomination size of the anchor
• src_chain_id is the id of the source chain
• r_id is r_id
• max_edges is the maximum number of anchors the created anchor can have an edge to
• tree_depth is the depth of the Merkle tree maintained by the anchor
• asset is the type of asset (CurrencyIdOf) that can be deposited into the created anchor

5. Anchor Update Proposal
Pallet Flow: bridge/signature-bridge -> anchor-handler -> anchor
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure: Call::AnchorHandler(Call::execute_anchor_update_proposal { r_id, anchor_metadata: EdgeMetadata { src_chain_id, root, latest_leaf_index } })

• r_id is the r_id
• src_chain_id is the id of the source chain
• root is the new root of the Merkle tree
• latest_leaf_index is the index of the latest inserted leaf (corresponds to the updated root above)

6. Register r_id Proposal
Pallet Flow: signature-bridge/bridge
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure:

7. Set fee recipient proposal
Pallet Flow: signature-bridge -> token-wrapper-handler -> token-wrapper
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure:

8. Rescue Tokens Proposal
Pallet Flow: signature-bridge -> token-wrapper-handler -> token-wrapper
Proposal Data: (r_id, zeroes, nonce, call)
Total Bytes: 40 + (size of call)
Call Structure:

Questions/Comments/Issues

1. How many bytes is a call?
2. How many bytes is a T::AssetId? It seems to just be an integer, so 4 bytes should be enough to cover the different asset types we will encounter.

Goal

We need to have a governance system for almost anything in our system, specifically speaking about pallets like xanchor and the tokens pool.

We need the system, from the start, to be abstract over the underlaying signing mechanism and by that I mean the system should not be directly dependent on say DKG or the democracy pallet, instead we should find a smart way where this system could work with both of them, or even in the future we can swap and change the underlaying gov. system without a lot of API or breaking changes.

Design

We would have to design a pallet that should expose the following functionality:

• a register_preimage method where it takes a Proposal and it creates a hash of that proposal
• a propose method where it takes a proposal_hash and adds this proposal to the backend proposal queue

That's it, it should be that simple!

Other pallets that depends on this pallet shall not know anything about the result of a proposal!

a quick example: Chain A wants to Link Anchor(id: 1) to Anchor(id: 42) on Chain B , when chain A sends that call say over XCM using xanchor pallet to chain B , chain B would create a proposal that when it passes it will link the both anchors together and post this proposal to it's proposal pallet. xanchors here does not care about the result of that proposal anymore, at least does not need to watch for the result, even it does not have to know how the vote process would go, like does my proposal pallet uses DKG for that? does it uses democracy pallet?

Use Cases and Scenarios

• Scenario #1

  We have two chains, chain A and chain B , they are both parachains, and chain A have an Anchor(id: 3) that operates on an asset USDT (webbUSD-1) and it wants to link that anchor on chain B where it knows that there is another anchor there with id 5 (Anchor(id: 5)) that operates on asset USDC so the flow goes as the following:

  • Notes

    Using Pallet: XAnchor

  1. Chain A creates a proposal with the following content:

  Proposal { targetChainId: PARA_B_ID, targetTreeId: TreeId(5), myTreeId: TreeId(3) }

  1. Then Chain A encode this proposal and send it to it's proposal pallet, where this proposal is going to be voted on somehow, Chain A does not care!
  2. when this proposal passes, a call that has been encoded into the proposal data will be dispatched on xanchor pallet to send a link anchors xt over XCM to Chain B.
  3. Chain B gets this message, and it creates another proposal with the following content:

  Proposal { targetChainId: PARA_A_ID, targetTreeId: TreeId(3), myTreeId: TreeId(5) }

  1. Then Chain B encode this proposal and send it to it's proposal pallet, where this proposal is going to be voted on, again Chain B does not care here about how the vote is going to be handled.
  2. when the proposal passes, another call that has been encoded in that proposal data will be dispatched on xanchor pallet to link both anchors cross chains, and send back to Chain A that the link is done on Chain B so that Chain A confirm that and link the anchors too!
  3. Chain A gets this message that Chain B accepted the link process and we should link now.

  Linking flow - Democracy proposal system

  1. Chain A creates a proposal with an encoded dispatchable call to link the anchor to Chain B.
     1. Chain A votes on this.
     2. If successful, chain A sends a message with its tree ID and a potentially empty target tree.
        1. If the target tree is empty, it signal that we want to create a new anchor.
        2. If the target tree is not-empty, then it signals we want to link an existing anchor.
  2. Chain B receives the proposal from chain A and votes on it.
     1. If successful, a new link is created and a message is now sent back to ACK this fact.
        1. A message containing the link metadata is always sent back and chain A stores it.

  Other metadata to add when linking

  Proposal {
  	targetChainId: PARA_A_ID,
  	targetTreeId: TreeId(3),
  	myTreeId: TreeId(5),
  	activeDKGPublicKey: [1,2,3,4]
  }

  Updating Flow - No proposal system once linked

  1. Once anchors are linked, we just need either
     1. Relayers to pay the XCM fee to broadcast updates.
     2. An autonomous poller / cron job on-chain using on_finalize.
     3. Users to pay the XCM fee.

Updating a token pool - DKG

All xanchors & anchors are anchors over token pools, nothing else. These might be token pools that are empty underneath.

1. When the system decides to add a new token to a token pool, we require the DKG to sign a signature of a message authorizing it.
   1. Can we make the DKG the RegistryOrigin?
2. Some proposer prepares the message m.
3. The proposer submits it to dkg-proposals.
4. Proposers vote on the message.
5. On successful proposal, the message is sent to dkg-proposal-handler to be signed (if it's a sign message).
6. On successful signing, the signed message is posted back to dkg-proposal-handler for anyone to submit anywhere it targets.
7. A relayer submits this (sig, m) to the update of the asset registry.

Overarching Schema of Proposals

TBD

Example Proposals

1. Anchor Update Proposal
Size in Bytes: TBD
Pallet Flow: bridge/signature-bridge -> anchor-handler -> anchor

2. Update Wrapping Fee Proposal
Size in Bytes: TBD
Pallet Flow: bridge/signature-bridge -> token-wrapper-handler -> token-wrapper

3. Add Token to Pool Share Proposal
Size in Bytes: TBD
Pallet Flow: bridge/signature-bridge -> token-wrapper-handler -> token-wrapper

4. Remove Token From Pool Share Proposal
Size in Bytes: TBD
Pallet Flow: bridge/signature-bridge -> token-wrapper-handler -> token-wrapper

Overview

Currently all tokens share the same wrapping fee defined in the token wrapper pallet as a Config item: https://github.com/webb-tools/protocol-substrate/blob/main/pallets/token-wrapper/src/lib.rs#L120.

We want to make this configurable per wrapped token. This spec should outline the ideas and process for making these changes. There is a possibility these changes will need to be made in the asset-registry if they cannot be made directly on the token wrapper. This spec should include details on where and how these changes can be made.

Overview

The resource ID encoding and decoding functions in primitives/src/utils is incorrect and does not match the derive_resource_id function that we are using primarily in Solidity. Instead of encoding bytes from the back it encodes the the TreeId as 20 bytes in the front of the ResourceId. This should be fixed.

The incorrectness of this implementation can also be seen simply by the comments as it states the first 20 bytes are the tree_id. This in fact is not consistent with how things are meant to be.

Current implementations

/// The ResourceId type is a 32 bytes array represented as the following:
/// ```md
/// +---+---+---+---+---+---+---+---+---+
/// | * |   |   |  ...  | * | * | * | * |
/// +-|-+---+---+---+---+-|-+---+---+---+
///   |                   +-> The last 4 bytes are the chain_id
///   +-> The first 20 bytes are the tree_id
/// ```
/// This takes the tree_id and the chain_id and combines them into a single 32
/// bytes array. the process is simple as convert the `tree_id` to 4 bytes array
/// (little-endian), pad the remaining `(20 - 4)` butes with zeros, next convert
/// the chain_id to 4 bytes array (little-endian) and append it to the last 4
/// bytes of the result array.
pub fn encode_resource_id<TreeId, ChainId>(tree_id: TreeId, chain_id: ChainId) -> ResourceId
where
	TreeId: Encode + Decode + AtLeast32Bit + Default + Copy,
	ChainId: Encode + Decode + AtLeast32Bit + Default + Copy,
{
	let mut result = [0u8; 32];
	let mut tree_id_bytes = tree_id.encode();
	tree_id_bytes.resize(20, 0); // fill the remaining 20 bytes with zeros
	let mut chain_id_bytes = chain_id.encode();
	chain_id_bytes.resize(4, 0); // fill the remaining 4 bytes with zeros

	debug_assert!(tree_id_bytes.len() == 20);
	debug_assert!(chain_id_bytes.len() == 4);
	result[0..20].copy_from_slice(&tree_id_bytes);
	result[28..].copy_from_slice(&chain_id_bytes);
	result
}

Correct implementation

pub fn derive_resource_id(chain: u32, id: &[u8]) -> ResourceId {
	let mut r_id: ResourceId = [0; 32];
	let chain = chain.to_le_bytes();
	// last 4 bytes of chain id,
	r_id[28] = chain[0];
	r_id[29] = chain[1];
	r_id[30] = chain[2];
	r_id[31] = chain[3];
	let range = if id.len() > 28 { 28 } else { id.len() }; // Use at most 28 bytes
	for i in 0..range {
		r_id[27 - i] = id[range - 1 - i]; // Ensure left padding for eth compatibility
	}
	r_id
}

Overview

The chain ID update has landed in the anchor/vanchor but there were complications getting it to work immediately in the xanchor. The reason for this is mainly due to the following:

• Parachain identifiers (para_ids) are u32 on the relay chain. It does not support (at least I don't think) updating this type to a u64.
• We want to use the updated chain ID type when proposing connections to other parachains, so even though we will preserve the para_id as a u32, we want to send messages that have u64 based chain identifiers.

Tasks

• Remove encode resource ID and use derive_resource_id instead.
• Ensure decode_resource_id works against derive_resource_id.
• Fix the tests

Import wasm-utils into the node to test it against Mixer and Anchor pallets

Overview

We have changed chain ID encoding to be 6 bytes of data instead of 4. These changes have landed on @protocol-solidity. We should also implement those changes here.

TBA