saitotech / saito-rust Goto Github PK

Hey folks,

There has been some internal discussion about serialization and encoding, so I thought it would make sense to document some of the ideas in these discussions more publicly.

Rust gives quite a few methods to implement serialization, and currently there's a short list of the ones that have been discussed, each containing their own sets of pros and cons.

Serde/bincode

Pros

• Compact, fast byte encoding native to Rust and its type system
• Supported by other projects, including google/tarpc a Rust implementation of RPC that requires no additional tooling or languages to create a message schema.
• Easy to implement, plug and play with Serde using the bincode crate.

Cons

• Rust specific and tightly coupled, meaning almost no other language support
• Less control in encoding, potentially meaning less performant

Protocol buffers

Pros

• Platform and language neutral
• Easier to implement than building a custom serializer

Cons

• Additional complexity to manage a separate language
• Less efficient than custom serializer

Serde/Custom Serializer

Pros

• Complete control over encoding our structures, easy to optimize where we need it

Cons

• Cost of development time, maintenance, and bug fixing
• Would require recreating it for each new implementation in another language

There's a few other open-ended questions to factor into this conversation that will influence the decision. One of them in my mind is will the Rust version be the only version that is allowed to run consensus? If so, being as tightly coupled to Rust isn't as big of an issue.

Personally, at time of writing I'm leaning most towards Protocol buffers. It's a good compromise of dev complexity, language agnosticism, and efficiency that gives us decent flexibility moving forward.

If there are other standards that should be considered, please mention them! Now is the time to lay it out before anything gets implemented.

To extend adding blocks to the longest chain, we'll need to implement saving Blocks to disk. We'll want a Storage module to easily read and write Blocks.

As we're using tokio, the best way to do this would be to implement this asynchronously. We'll also want to extend some traits within Block to make it easy to swap out serialization, as we've yet to decide what serialization method we'll be implementing in the long term. For now, using bincode to serialize should be fine, as long as it's implemented in a trait and not hardcoded.

The functions we want to add are:

• async read_block_from_disk(block_hash: [u8: 32]) -> Result<Block, Err>
• async write_block_to_disk(block: Block) -> Result<(), Err>

A synchronous, rough implementation of Storage can be found in the master branch.

Running cargo test on stable results in many tests not finishing within 60 seconds, and seem to go for many minutes.

...
test network::tests::test_blockchain_causes_sndblkhd has been running for over 60 seconds
test network::tests::test_network_message_sending has been running for over 60 seconds
test network::tests::test_sndblkhd has been running for over 60 seconds
test network::tests::test_sndchain has been running for over 60 seconds
test networking::api_message::tests::test_message_serialize has been running for over 60 seconds
test networking::message_types::handshake_challenge::tests::test_challenge_serialize has been running for over 60 seconds
test networking::message_types::request_block_message::tests::test_request_block_message_serialize has been running for over 60 seconds
test networking::message_types::request_blockchain_message::tests::test_request_blockchain_message_serialize has been running for over 60 seconds
test networking::message_types::send_block_head_message::tests::test_send_block_head_message_serialize has been running for over 60 seconds
test networking::message_types::send_blockchain_message::tests::test_send_blockchain_message_serialize has been running for over 60 seconds
test slip::tests::slip_addition_and_removal_from_utxoset has been running for over 60 seconds
test staking::tests::blockchain_roll_forward_staking_table_test_with_test_manager has been running for over 60 seconds
test staking::tests::blockchain_staking_deposits_test has been running for over 60 seconds
test staking::tests::staking_create_blockchain_with_many_staking_deposits_many_staker_payouts_per_block has been running for over 60 seconds
test staking::tests::staking_create_blockchain_with_two_staking_deposits_one_staker_payout_per_block has been running for over 60 seconds
test storage::tests::write_read_block_to_file_test has been running for over 60 seconds

We could use a real logging library, it would be nice to be able to have log levels and be able to pipe log message easily.

Routing Hops are currently 130 bytes on average. They each contain two publickeys (from, to) and a signature. The signature is generated using the to address and a piece of data associated with the transaction.

Optimizing the size of routing paths is important over the long-term given the number of transactions that will be included in blocks. One possible modification is removing the "from" publickey because it can be sourced from the "to" field of the previous routing hop. The first routing "from" address can be found as the sender of the first INPUT slip who would have needed to sign the transaction.

We can explore using a commutive encryption algorithm as well -- one where only a single signature is needed and that signature is generated and swapped-out as additional paths.

Possible savings are 33 bytes per routing hop, possibly the final "to" publickey as well (as the block creator will contain the publickey). And all but one signature, saving 64 bytes per block. Average data sizes per hop can be reduced considerably. This is well worth looking into once the software is up and operating reliably.

Once there are golden tickets #36 being produced in consensus, the system will start to have slips that will govern network functions.

Each node needs a way to collect and organize their slips, as well as create transactions with the proper slip balances to pay fees to the network. We'll want a Wallet struct that will be capable of doing this functionality.

The Wallet will need to contain:

• shared state that can safely be shared and accessed across multiple threads (Arc and Rwlock)
• A body that can be serialized
• Said body will need to contain important fields such as
  • Slip Array
  • Keypair
  • Balance
  • Pending slips in candidate block
  • Wallet Version
  • Default Fee when creating Transactions

The basic API we should support at first is

• add_slip(slip: Slip)
• remove_slip(slip: Slip)
• create_transaction(fee: u64, amount: u64)
• create_signature(data: [u8; 32])

A rough implementation can be found in the master branch of this repo.

I want to create some tooling so that we can actually simulate creating a blockchain and then hitting it with incoming blocks and transactions and observe the behavior. If Cargo bench can do all this, I think we shoudl use that. Otherwise, maybe this should a separate crate under a tools lib that interacts with the node runtime via the system(e.g. through some socket)

Acceptance Criteria:

• Passed UAT Test in Testnet
• Saito-rust Go-live
• Release of Communication on new Network
• Highly Critical Issues found in the next 16 hours are fixed

Subtasks:

• Deployment to Testnet
• UAT Test in Testnet
• Switchover to Production
• Communicate to community
• 16-hour support for critical bugs (Dev and Testers Standby)

I run the node I get listening on http://127.0.0.1:3030

how to connect to other nodes?

Acceptance Criteria

• All Peer-to-peer variables used by the network are in the network class
• Passed all test cases.

Subtasks

• PEERS_DB_GLOBAL & its usage
• PEERS_REQUEST_RESPONSES_GLOBAL & its usage
• PEERS_REQUEST_WAKERS_GLOBAL & its usage
• INBOUND_PEER_CONNECTIONS_GLOBAL & its usage
• OUTBOUND_PEER_CONNECTIONS_GLOBAL & its usage
• Retest all test cases.
• Review and signoff of task completion

https://trello.com/b/gWPiQFZ1/saito-rust-project-management

We are using the serializers branch as a place to benchmark and look at code for serialization.

We are using bincode for now because it's easist, but the thinking is that we need to either use protobufs or do custom serialization.

I think the benchmarks for this should be done in a decent way so that we can actually use them as benchmarks and devtools going forward. It would be nice to perhaps also add benchmarking to a CI pipeline in the future.

We have a keypair class that enables us to create keypairs, but the UX needs to be done.

Our vision is that users should be prompted to create a password encrypted keypair the first time they start a node which creates an encryted keypair file at a standard location.

If the keypair file already exists, the user should be prompted for the password.

The user should also be able to pass a command line argument with the path to another encryted keypair file(which should use the same password encrypting scheme and prompt them for a password also).

Another option might be to allow users to pass an unencrypted password file as a command line parameter, but this is I think just "nice to have". Could be done if it's very easy, but might also complicate UX significantly.

We're open to suggestions on changes to this design.

https://github.com/SaitoTech/saito-rust/runs/3468793166#step:3:1076

error: this boolean expression contains a logic bug
   --> src/blockchain.rs:315:16
    |
315 |             if save_to_disk || true {
    |                ^^^^^^^^^^^^^^^^^^^^ help: it would look like the following: `true`
    |
    = note: `#[deny(clippy::logic_bug)]` on by default
help: this expression can be optimized out by applying boolean operations to the outer expression
   --> src/blockchain.rs:315:16
    |
315 |             if save_to_disk || true {
    |                ^^^^^^^^^^^^
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#logic_bug

There's been some discussion about how we name things in the codebase, so I wanted to open up a discussion so that we could stick to some common conventions.

By default, we match the Rust styling: https://doc.rust-lang.org/1.0.0/style/README.html

In general, we should fully type all names unless the name runs significantly long. We don't cut corners by shortening variables or attributes, so block over blk and transaction over tx. In the future, well defined acronyms can describe processes like automatice transaction rebroadcasting (ATR for short).

Getter functions just reference the piece of data that we want access to -> id() returns id. This also applies to a function like latest_block in Blockchain.

Setter functions use the convention of set_${attribute}.

Additional functions that perform a more complex transform, or some other complex functionality will start with a verb, then describe what the function does. If there's not a guarantee that the operation will be successful, we use the verb try. Only if we're sure of the success of the operation would this be negated.

Here's an example with Blockchain. When we want to add a block to the blockchain, the function we call is try_add_block, and not add_block. The function add_block is only called when we are certain that the block should be added to the blockchain. In a similar vein, when we attempt to bundle a block in Mempool we call try_bundle_block, and only call bundle_block when the proper requirements are met.

For our primitive structs, we want to distinguish between Primitive and PrimitiveCore. Core version of the struct will be what's sent over the wire between nodes, and represents the simplest version of the struct. The full-bodied version is available to allow for each node to enrich its data.

Hi team, I've just noticed that we're using the outdated jemallocator.
Check this PR from rust-core for more details.

Updated jemalloc lib
This updated one is from tikv

TL;DR: this upgrade could help for sized deallocation's usage correctly, particularly to rustdoc.
So somehow it can deliver a sizable performance increase. We can do some benchmarks after that.

Lets consider my suggestion. Thank you.

For now, to generate slips and get Saito circulating in the system, we auto generate GoldenTicket Transactions and feed them into the Mempool where they're bundled and added to the chain.

We want a Miner module to do some mining work to generate these golden tickets. We want to port over the existing Miner from the master branch, remove the Lottery and LotteryGame cruft around it and add tests.

I think the implementation of add_block_to_blockchain will much simpler if we get the datastructures right.

I believe a having a map of block_id -> block_hash which represents the longest chain will make things must easier.

This should be in it's own class with only a handful of methods:

rollforward(newblock)
rollbackward() -> hashOfRemovedBlock
getBlockById(u64) -> hashOfSomeBlockInTheLC
getLatestBlock() -> hashOfLatestBlock
bottomOfEpochBlock() -> hashOfBlockThatsAboutToFallOffTheChain

At the top of the longest chain, we would need stack-like behavior. I.E. the latest block is pushed and popped in a LIFO manner.

At the bottom of the epoch, we'll also need to push and pop as blocks fall off the epoch in a FIFO(queue-like) manner and also keep a cache of recently-fallen-off blocks which can then be pushed back into the bottom of the epoch when we do rollback().

The simplest way I can think to handle all this is to have two stacks, one which has the latest block at the top, and another which holds the 2nd-oldest epoch. Since the length of the epoch is fixed, it's straightforward to use something like a ring-buffer in a fixed-length array to acheive this. When we rollforward, the block which is 2 epochs down the longest chain is simply lost. This supports a rollback the length of the entire epoch. We can worry about optimizing this later.

Currently running saito requires that there be a configuration file present under the current directory/configuration however this may not match realistic expectations when running saito in production. When executing the saito binary from a different directory it panics with thread 'main' panicked at 'Failed to read configuration.: configuration file "/Users/zyler/github/saito-live/configuration/base" not found', /Users/zyler/github/saito-live/saito-rust/src/consensus.rs:136:44

Options for handling this gracefully would be:

1. Sane defaults without a configuration file

2. Allow path to configuration file to be specified via command line ie saito --config=/path/to/config.yml or via environment variable ie $SAITO_CONFIG_PATH

3. If file not found search for configuration in a user-standard directory such as $HOME/.saito

4. If user config directory is not found search in os-specific config path for instance /etc/saito on linux

This error I encountered when using homebrew to install saito on my mac, and realized the node operator onboarding experience could be improved with better configuration options.

(https://github.com/saito-live/homebrew-saito to try installation via homebrew on mac.)

Acceptance Criteria:

• Onchain reorganization in Rust is identical with Saito-lite-rust implementation
• Passed the Tests after changes.

Subtasks

• move out logic relating to update_genesis_period() & fork_id to blockchain on_chain_reorganization() itself.
• add_block_success() update.
• passed the tests.

After @clayrab pointed out in #42, we'll want a generic way to create randomness in golden ticket creation. Traditionally, we've used a SHA256 miner to achieve this, but we'll want some sort of interface for multiple lottery games to conform to outside of just mining.

The way to go about this is creating a trait called LotteryGame that exposes an interface that all Saito lottery games need to conform to. A higher level struct called Lottery will control running the game, and will be used to initialize the LotteryGame when a node starts running.