Revamp the Sharding Client Entry Points / Architecture about prysm HOT 6 CLOSED

Yes. I agree this is the next logical issue/PR to tackle. Once this is done, we can begin service implementations for notary and proposer. It's gonna be fun! Let me know if you need help with the PR

from prysm.

Yeah agreed this is a good first step in order to tackle the issues raised in #122. What are the advantages of using a flag like --nodetype="proposer" instead of sharding-proposer for the entrypoint for sharding ?

from prysm.

I agree with refactoring to support a modular node framework. Our existing code relies on a running geth node to connect to. These changes will make the sharding actors run as independent and self contained nodes.

from prysm.

Ok so here's how I've been approaching this:

In /cmd/geth/shardingcmd.go

func shardingNode(ctx *cli.Context) error {
	// configures a sharding-enabled node using the cli's context.
	shardingNode := sharding.NewNode(ctx)
	return shardingNode.Start()
}

Then, sharding.NewNode has the responsibility of registering all the config options of the cli, registering different sharding services, and managing their lifecycle.

Service Registration

Then, within this NewNode func, we register a few services depending on the cli flags. If utils.flags.ClientType is set to notary, then we register a NewNotary service. Otherwise, we register a NewProposer service. All of the services attached to this sharding node are managed and initialized within the node's .Start() function.

We define the Notary and Proposer services as protocols and, in their initialization, they setup a few different goroutines within their .Start() functions. Each of these sets up a ProtocolManager struct that satisfies an interface specifying access to p2p networking specific to the functionality of the client type, access to SMC bindings, and more. Then, three event loops started:

// Within the Notary/Proposer protocols' .Start() functions...
go protocolManager.StartGethRPC()
go protocolManager.StartP2P()
go protocolManager.StartMainLoop()

StartGethRPC() sets up a connection to a Geth node via an IPC connection and handles the logic of setting up the SMC on the Geth node as well as setting up the bindings to allow each protocol to call functions on the SMC.

StartP2P() will handle all of the shardp2p peer discovery, requests/responses to and from other nodes speaking the same protocol (either notary or proposer).

StartMainLoop() will handle the logic of being a notary or proposer depending on the protocol at hand. For notaries, this involves checking if the notary was selected as eligible in a period and more. For proposers, this involves listening to incoming transactions and processing them into collations that will then be submitted to the SMC via SMC bindings created and attached to the proposer's ProtocolManager.

What This Achieves

This architecture is similar to what geth currently does to setup full/light nodes. This achieves separation of concerns between the sharding node and its underlying services. That is, everything related to notaries is contained within the notary package, and the notary will be responsible for handling the lifecycle of all the goroutines specific to it. Same goes for proposers.

Additionally, this allows us to define a nice Service interface and a ProtocolManager interface that allows us to be extensible to further changes in the research or if we want to add something new to our sharding client down the line without major rafactoring.

My concern with this architecture is perhaps repeating myself a bit with the code between protocols, which is why I suggested having a single ProtocolManager interface that specifies common methods for both, with logic that can be overwritten. However, both will need access to SMC bindings, which the go protocolManager.StartGethRPC() will handle. Could I instead abstract this one level higher at the sharding node level? Then, when registering a Notary protocol as a service to the sharding client, we could pass in a handler for SMC-related logic as an argument like:

shardingNode.Register(func() {
  return notary.NewNotary(shardingNode.ctx, shardingNode.SMCHandler)
})

This could keep things nice and abstract as we currently have them, but still trying to understand the best way to do this.

Thoughts on this? @prestonvanloon @terenc3t @nisdas @Magicking @enriquefynn?

from prysm.

I'm a fan of attaching services as protocols to each actor (proposer/notary). What should a client do if he just wants to be an observer? In that case he wouldn't register NewNotary or NewProposer service, but still needs to StartP2P protocolManager.StartP2P(). The observer will also wanna goto a specific shard, we can get the shard number from cli.

from prysm.

Yeah this model allows for us to easily do this via cli flags. If client type is not set then the node just becomes an observer in this case.

from prysm.