Testing framework for p2p networking tests about trin HOT 9 CLOSED

@victor-wei126 -- just some thoughts to get things started, following up on our chat earlier today. We can brainstorm and analyze on here, in case anyone else wants to join in the fun.

from trin.

Geth's p2p testing framework: https://github.com/ethereum/go-ethereum/tree/master/cmd/devp2p
Might be helpful to refer to what they are doing.

from trin.

Hey, I have started thinking about the testing framework and i'm leaning more towards running a dummy peer to communicate with the target client. I'm trying to set a minimum requirements of what exactly a "dummy" peer should do, probably something like:

• supporting discv5 and overlay network messages and events
• set a communication channel to send specific messages to the target client from the command line.

Let me know if this seems viable, i'm continuing to work on this and would like to prototype something.

from trin.

Can someone clarify what we mean by "dummy peer" vs. "target client"? From my understanding, the target client is the node under test, and the dummy peer is what sends the msgs to the target client (ping, findcontent). If this is the case, how can we test request msgs like ping, and ensure a proper response? Because it then seems to me that we can only test response msgs with this method.

Also, I've been looking into how Geth does their discv5 test suite. It seems similar to what Ognyan is proposing, but runs all full suite of tests automatically against a full implementation. At leasts that's what I've gotten from it.

from trin.

Can someone clarify what we mean by "dummy peer" vs. "target client"? From my understanding, the target client is the node under test, and the dummy peer is what sends the msgs to the target client (ping, findcontent).

Yes, a dummy peer is a bare-bones node that supports portal network calls. Target node is the node under test.

If this is the case, how can we test request msgs like ping, and ensure a proper response? Because it then seems to me that we can only test response msgs with this method.

This is a good point, I guess with a dummy node we can only test a target client response to a given messages.
Geth is running similar tests here for dicsv5. It looks like they are also sending requests to the target node and testing the responses.

I'm trying to prototype something similar here. I guess we still need to figure out and explore a lot of things... 🤔.

from trin.

I just thought of this, but regarding testing the requests...if we run a dummy node, and assume it to be correct all the time, we could write some command line program, that sends the requests, sorta like how geth does it with: devp2p ping <ENR>. Or we could just set up the nodes (similar to how you have it setup now in testing-framework-experiment branch) but instead of sending the requests from dummy, we send from test client??

I have also looked at what you're prototyping, and it seems like a good approach so far to me.

from trin.

I think we can split p2p testing in two different directions:

1. Portal testing framework for cross-client tests, where we can test a target node behavior and running p2p simulations. For this, I guess we can use dummy nodes to send p2p network calls and check the expected behavior from the target node.

2. internal p2p portal tests related only to trin implementation. We can accomplish this for example by running a test suite, something like:

struct Default2NodeTest {
  node1: discv5_protocol
  node2: discv5_protocol
  proto1: portal_protocol
  proto2: portal_protocol
  }

which will establish a connection between two trin nodes and then run portal Req/Resp tests on this connection. This will require some refactoring on the current trin-core crate to encapsulate the current logic for the base discv5 protocol, the overlay protocol and jsonrpc.

@carver, @SamWilsn any thoughts on this?

from trin.

I think we can split p2p testing in two different directions:

1. Portal testing framework for cross-client tests, where we can test a target node behavior and running p2p simulations. For this, I guess we can use dummy nodes to send p2p network calls and check the expected behavior from the target node.

2. internal p2p portal tests related only to `trin` implementation. We can accomplish this for example by running a test suite, something like:

struct Default2NodeTest {
  node1: discv5_protocol
  node2: discv5_protocol
  proto1: portal_protocol
  proto2: portal_protocol
  }

which will establish a connection between two trin nodes and then run portal Req/Resp tests on this connection. This will require some refactoring on the current trin-core crate to encapsulate the current logic for the base discv5 protocol, the overlay protocol and jsonrpc.

@carver, @SamWilsn any thoughts on this?

Yeah, given the general difficulty of testing network peers, I think coming at it from multiple angles at once is very reasonable. The tradeoffs in the different approaches will hopefully complement each other. +1 for exploring it.

from trin.

For now, I think we are in a "good enough" place, using ethportal-peertest and portal-hive. There's some discussion about how we could avoid writing tests twice (once for each), but that deserves its own issue IMO.

from trin.