chainsafe / js-libp2p-gossipsub Goto Github PK
View Code? Open in Web Editor NEWTypeScript implementation of Gossipsub
License: Apache License 2.0
TypeScript implementation of Gossipsub
License: Apache License 2.0
Before deploying gossipsub
into the wild in the libp2p
network, it would be great to have a suite of stress tests
Good work!!
Is this implementation of the updated spec? v1.1?
I am not aware of the differences between the two spec versions, but wanted to ask here.
Thanks
Let's consider a simple network with 3 peers {A, B, C}, with 2 connections established A<->B and B<->C
In this scenario, if A subscribes to a given topic and C publishes messages to that topic, A should be able to receive the messages.
It seems to me that https://github.com/ChainSafe/js-libp2p-gossipsub/blob/v0.8.0/ts/index.ts#L1034 this should not break here, otherwise we will not grab fanout peers. However, this does not seem to fix the issue, as I tried to do a small hacking in the code but messages were not sent to the peer.
Other than that, should we make canRelayMessage
options to be true by default? I would expect the default behaviour to relay messages, but users might turn that off if they like.
Test scenario to test: libp2p/js-libp2p-interfaces#83 (use this branch as a dep)
@wemeetagain any ideas on what is wrong here?
cc @Gozala
Browser tests for go-gossipsub-copied tests are disabled because they create many independent peers, and it is slow and leads to timeouts in a browser environment.
Figure out how to appropriately run these tests.
I don't have access to the chainsafe travis account so I am creating this issue. The libp2p team suggests that we use aegir.
cc'ing @GregTheGreek and @priom.
Gossipsub1.1 spec was extended to include: Validation queue protection with Random Early Drop
References:
Unfortunately, didn't test out releasing w/ Aegir in #77
There are a few issues with the current setup when trying to release with Aegir
.eslint.js
getting picked up by aegir for its linting step. Can be mitigated by either running with --lint=false
or renaming .eslint.js
to eg: .eslint.ts.js
tsc
never gets run during the release cycle, so aegir fails on the test and other steps. Can be mitigated by either checking in built js in src/
or adding a tsc step in aegirCurrently, we have a @ts-ignore
hiding the fact that time-cache
has no types.
This causes type errors for downstream users that don't have the time-cache
types accessible.
After asking about tagging peers on twitter, I was redirected to 2 issues related to the connection manager functionality in go-libp2p (1 and 2). These issues brought to my attention the js equivalent of the go connection manager, js-libp2p-connection-manager. This module manages connections to peers. However, it is lacking some of the functionality that the Go connection manager has.
Currently, I have copied over the peer class from the js-libp2p-floodsub as a temporary measure. However, I would have to manually add a lot of the functionality and I don't think it is worth the extra effort. My current thoughts are either to use js-libp2p-connection-manager as a super class and add the needed functionality (tagging,etc) and later on make a PR to the js-libp2p-connection-manager. Or make a PR to the js-libp2p-connection-manager to include some of this functionality before using it in our codebase.
There are more complex options in gossipsub like scoreParams, scoreThresholds and directPeers. We should have examples for their usage, otherwise people will need to look into the code.
Debugging for an unrelated problem (ChainSafe/lodestar#3110) I noticed the cache keeps items for too little.
@wemeetagain noticed it keeps items for 30 seconds.
See below the message block 2046119 (new) FdqanLvU71cfZa2GBXOd7TOPJ8c
, it's (new) and repeated
>>> Gossipssub - block 2046119 (new) 16.68s late FdqanLvU71cfZa2GBXOd7TOPJ8c Lighthouse/v1.5.0-90d5ab1/aarch64-linux 16Uiu2HAmBi4CiFxB9zQbEx4mkbGiE9bwSaaZ9nGZX8RhRGPXL2fh
Eph 63941/8 4.700 [CHAIN] info: Received block: 2046119
Eph 63941/8 6.014 [] info: Synced - block: 2046120 … (empty) - head: 2046119 0xe042…490b - finalized: 2046048 0x7916…f846 (63939) - peers: 14
>>> Gossipssub - block 2046114 (already known) 80.33s late AhdyuZn7FP8Ml6pgd2o6IZPzPSs Lighthouse/v1.5.0-90d5ab1+/x86_64-linux 16Uiu2HAmCWxz9iXTrstyF1YDbvmpN6ngG2R3cTw6bTAXMLZEN2b8
>>> Gossipssub - block 2046114 (already known) 80.34s late AhdyuZn7FP8Ml6pgd2o6IZPzPSs Lighthouse/v1.5.0-90d5ab1/x86_64-linux 16Uiu2HAmJj6y2ces4gjhRzPFXBEqRdB6NoKWSh3B1Mgevr41c4WA
>>> Gossipssub - block 2046114 (already known) 80.34s late AhdyuZn7FP8Ml6pgd2o6IZPzPSs rust-libp2p/0.31.0 16Uiu2HAm3gcqtY6W5717jFQXSfcHkMzGYAag8XuYgcqgDCebeFbz
>>> Requesting peer beacon_block IWANT 16Uiu2HAm2KKqQj4PdHEZoRegyRzTegTqDBuzHcPwp4foesYtzkvE [ 'L9aYtlGZxVxHBuNjtVR/3gHfth4' ]
>>> Gossipssub - block 2046115 (new) 72.12s late AH+MpdqfVsFB/nogYJhkIEdrWZM Prysm/v1.4.2/b3650903ea23b040928732a2bed8b75d584ba903 16Uiu2HAkybvBXJbc9dfmU5spX9yV79Nv44hEQxFiHiEpCmFZR7jA
Eph 63941/9 0.364 [SYNC] info: INFURA HEAD 2046121, delay 0.3640000820159912s
>>> Requesting peer beacon_block IWANT 16Uiu2HAm2KKqQj4PdHEZoRegyRzTegTqDBuzHcPwp4foesYtzkvE [ 'DYodRUy6nfRhpLYdNLTTYqYqFCs' ]
>>> Gossipssub - block 2046121 (new) 0.88s late M1lw4SOUyZFV8rGS79oDyupTZCA Prysm/v1.4.1/a9ee3ee06afdffa2a9b0e759d4142642e7488416 16Uiu2HAm2KKqQj4PdHEZoRegyRzTegTqDBuzHcPwp4foesYtzkvE
Eph 63941/9 0.899 [CHAIN] info: Received block: 2046121
Eph 63941/9 0.901 [CHAIN] error: Block error slot=2046121, errCode=BLOCK_ERROR_PARENT_UNKNOWN
Eph 63941/9 1.265 [ETH1] error: Error updating eth1 chain cache message=The user aborted a request.
AbortError: The user aborted a request.
at abort (/home/lion/Code/eth2.0/lodestar/node_modules/node-fetch/lib/index.js:1418:16)
at AbortSignal.abortAndFinalize (/home/lion/Code/eth2.0/lodestar/node_modules/node-fetch/lib/index.js:1433:4)
at AbortSignal.dispatchEvent (/home/lion/Code/eth2.0/lodestar/node_modules/event-target-shim/src/event-target.mjs:337:35)
at abortSignal (/home/lion/Code/eth2.0/lodestar/node_modules/@chainsafe/abort-controller/src/abort-signal.ts:68:12)
at AbortController.abort (/home/lion/Code/eth2.0/lodestar/node_modules/@chainsafe/abort-controller/src/abort-controller.ts:26:9)
at Timeout._a (/home/lion/Code/eth2.0/lodestar/packages/lodestar/src/eth1/provider/jsonRpcHttpClient.ts:107:18)
at listOnTimeout (internal/timers.js:557:17)
at processTimers (internal/timers.js:500:7)
Eph 63941/9 1.934 [CHAIN] info: Received block: 2046120
>>> Gossipssub - block 2046115 (already known) 76.10s late AH+MpdqfVsFB/nogYJhkIEdrWZM Prysm/v1.4.4/29d48dfe7eb38959752048aec700f31c3abf7484 16Uiu2HAmH3MSFR4ijLotNJ6qML15uVRqGxkr5S19UUJrp3vmaTSQ
Eph 63941/9 6.000 [] info: Synced - block: 2046121 0xa78e…0393 - head: 2046121 0xa78e…0393 - finalized: 2046048 0x7916…f846 (63939) - peers: 13
>>> Gossipssub - block 2046115 (already known) 82.79s late AH+MpdqfVsFB/nogYJhkIEdrWZM Lighthouse/v1.5.0-90d5ab1/x86_64-linux 16Uiu2HAmJj6y2ces4gjhRzPFXBEqRdB6NoKWSh3B1Mgevr41c4WA
>>> Gossipssub - block 2046115 (already known) 82.79s late AH+MpdqfVsFB/nogYJhkIEdrWZM rust-libp2p/0.31.0 16Uiu2HAm3gcqtY6W5717jFQXSfcHkMzGYAag8XuYgcqgDCebeFbz
Eph 63941/10 0.424 [SYNC] info: INFURA HEAD 2046122, delay 0.4240000247955322s
>>> Gossipssub - block 2046116 (new) 74.16s late OLcXFZm0n6fyIJoIAFo538tsDMg Prysm/v1.4.2/b3650903ea23b040928732a2bed8b75d584ba903 16Uiu2HAkybvBXJbc9dfmU5spX9yV79Nv44hEQxFiHiEpCmFZR7jA
Eph 63941/10 6.001 [] info: Synced - block: 2046122 … (empty) - head: 2046121 0xa78e…0393 - finalized: 2046048 0x7916…f846 (63939) - peers: 14
>>> Gossipssub - block 2046116 (already known) 78.10s late OLcXFZm0n6fyIJoIAFo538tsDMg Prysm/v1.4.4/29d48dfe7eb38959752048aec700f31c3abf7484 16Uiu2HAmH3MSFR4ijLotNJ6qML15uVRqGxkr5S19UUJrp3vmaTSQ
>>> Gossipssub - block 2046120 (new) 31.11s late 1zuLxwl+Rng+EuPfSDwtZySj0D4 Lighthouse/v1.5.0-90d5ab1/aarch64-linux 16Uiu2HAmBi4CiFxB9zQbEx4mkbGiE9bwSaaZ9nGZX8RhRGPXL2fh
Eph 63941/11 1.066 [SYNC] info: INFURA HEAD 2046123, delay 1.065999984741211s
>>> Gossipssub - block 2046117 (new) 75.91s late NfWh1gqsh3z2rFOZBXpom78S1iQ Prysm/v1.4.2/b3650903ea23b040928732a2bed8b75d584ba903 16Uiu2HAkybvBXJbc9dfmU5spX9yV79Nv44hEQxFiHiEpCmFZR7jA
>>> Gossipssub - block 2046116 (already known) 88.62s late OLcXFZm0n6fyIJoIAFo538tsDMg Lighthouse/v1.5.0-90d5ab1/x86_64-linux 16Uiu2HAmJj6y2ces4gjhRzPFXBEqRdB6NoKWSh3B1Mgevr41c4WA
>>> Gossipssub - block 2046116 (already known) 88.75s late OLcXFZm0n6fyIJoIAFo538tsDMg rust-libp2p/0.31.0 16Uiu2HAm3gcqtY6W5717jFQXSfcHkMzGYAag8XuYgcqgDCebeFbz
Eph 63941/11 6.001 [] info: Synced - block: 2046123 … (empty) - head: 2046121 0xa78e…0393 - finalized: 2046048 0x7916…f846 (63939) - peers: 14
>>> Gossipssub - block 2046118 (new) 70.53s late mNPQQTZJ5WUwTRI/pl6ZE8c+8vA Prysm/v1.4.2/b3650903ea23b040928732a2bed8b75d584ba903 16Uiu2HAkybvBXJbc9dfmU5spX9yV79Nv44hEQxFiHiEpCmFZR7jA
Eph 63941/12 0.704 [SYNC] info: INFURA HEAD 2046124, delay 0.7039999961853027s
>>> Gossipssub - block 2046119 (new) 65.38s late FdqanLvU71cfZa2GBXOd7TOPJ8c Prysm/v1.4.2/b3650903ea23b040928732a2bed8b75d584ba903 16Uiu2HAkybvBXJbc9dfmU5spX9yV79Nv44hEQxFiHiEpCmFZR7jA
When the message cache implementation is stable and fully tests (e.g when #3 is complete), it makes sense to refactor it into its own npm module. Several of these data structures that were created for libp2p such as the time cache have their own modules and are widely used for different projects.
In index.js
, upon receiving a protobuf encoded RPC message, we process each kind of control message in handleIHave
, handleIWant
, handleGraft
and handlePrune
. We need to test these methods.
Test for handleIHave
Test for handleIWant
Test for handleGraft
Test for handlePrune
Some nodes are implemented in golang using (go-libp2p-pubsub), but they never arrive using this js library. Is there any known issue ?
see https://travis-ci.com/github/ChainSafe/js-libp2p-gossipsub/jobs/343776073
Occasionally CI tests fail on this test:
gossip
1) should send gossip to non-mesh peers in topic
0 passing (2s)
1 failing
1) gossip
should send gossip to non-mesh peers in topic:
AssertionError: expected 1 to be above 1
+ expected - actual
at Context.<anonymous> (test/gossip.js:49:39)
Restarting the tests usually "fixes" the problem.
Moving libp2p dependencies to peerDependency would allow to specify minimal required libp2p version on which gossipsub will work.
With the new spec of the gossipsub protocol already finished and implemented in go-libp2p
, other implementations are starting to update to v1.1
While it is backwards compatible with v1.0, there are important updates regarding security, as well as bootstrapping that makes it important to update sooner rather than later.
@wemeetagain what are your expectations for this update?
Gossipsub v1.1 checklist
Polishing
An initial implementation of the message cache data structure used in the gossip sub protocol has been implemented. Due to the ambiguity of the associated types, tests need to be written to make sure that this implementation is somewhat equivalent to the Go implementation. The tests need to follow the tests written in Go.
An important aspect of the gossipsub implementation is to be able to accommodate nodes running floodsub. We should have a test suite that proves that, as discussed on ChainSafe/gossipsub-js#15
There seems to be issues when I set the policy to StrictNoSign
.
I was able to reproduced in the 2-nodes.spec.js, Publish 10 msg to a topic
test fails then policy is set: D4nte#1
When setting the policy on the js-libp2p example: https://github.com/libp2p/js-libp2p/blob/master/examples/pubsub/1.js Only the first message gets through.
Am I using the API correctly?
Hi hi,
I noticed that Go cannot publish messages that JS receives when using elliptic keys. That's because Go inlines the public key into the peer ID. We already worked on this issue with just general connections, but it appears it's broken in PubSub additionally.
This PR: libp2p/js-peer-id#101 will allow the issue to be fixed and enable finishing up this TODO: https://github.com/libp2p/js-libp2p-pubsub/blob/master/src/message/sign.js#L76
After the other tests are completed, we need to make sure that this JS implementation of gossipsub is interoperable with the Go reference implementation.
Using a libp2p daemon, test whether any peer running either JS or Go implementations of gossipsub can receive and handle each others messages. Based these tests on the ipfs pubsub interop tests.
Get started on these interop tests provided by libp2p
Some fixes have gone into master that we need in order to release a new version of js-IPFS, could a new version of this module be published please?
Currently, all received messages in a subscribed topic are automatically gossiped.
We would like to add an option to not automatically gossip, rather, we would rely on manually re-publishing a message if gossiping is desired.
One known use-case is for validating incoming messages before gossiping.
This probably looks like adding a boolean option, passed in the constructor, similar to how #40 was exposed.
gossipsub v1.1 features direct peers and peer exchange require gossipsub to initiate dialing new peers. We're also going to want access to the PeerStore
to add the contents of PeerRecord
s.
Should gossipsub be passed a reference to the global Libp2p object so it can dialProtocol
? Is there a better / different way to approach this?
If we have a reference to the global object, we might pick the connectionManager
, registrar
, peerStore
, etc. from one place, rather than passing each submodule in explicitly.
As always, testing needs to be done in order to make sure this works as intended.
Here are some cases that need to be tested:
Make sure that the gossipsub protocol is properly mounted on top of both nodes, A and
Check that peer A can dial into peer B (vice-versa)
Have peer B subscribe to a topic in peer A
Have peer A publish to topic and make sure that peer B receives it
Have peer B unsubscribe from topic that peer A publishes to
Unmount gossipsub protocol from peers.
I brought up TypeScript support in the Libp2p Roadmap document. Raul supports having TypeScript declarations for js-libp2p. I created a repo for this purpose and also found this other repo as well. The end goal would be to merge it into js-libp2p, just like we want to do with this library.
In either case, these will need to be integrated into this repo.
Any thoughts?
Currently we can either disable gossipIncoming
option or gossipUnverified messages.
Proposed solution would be to add hooks (optional param) when subscribing to topics, where we can pass function that decodes message and verifies it. If that hook throws, we ignore and don't gossip that message.
Proposed alternative
gossip.subscribe("topic", (message) => transformedMessage)
gossip.on("topic", (transformedMessage) => {})
We recently moved the seenCache
to the gossipsub layer, this is a sanity check to see if this is good.
In reading thru the go-libp2p-pubsub code, I see that their seenCache
is actually at the pubsub layer!
Their processing goes in the following order (here):
We don't need to mimic the go-libp2p-pubsub implementation exactly but we should probably respect the behavior as closely as possible.
Gossipsubv1.1 has the following spam protection:
In flight IWANT requests, sent as a response to an IHAVE advertisement, are probabilistically tracked. For each IHAVE advertisement which elicits an IWANT request, the router tracks a random message ID within the advertised set. If the message is not received (from any peer) within a period of time, then a behavioural penalty is applied to the advertising peer through P₇. This measure helps protect against spam IHAVE floods by quickly flagging and graylisting peers who advertise bogus message IDs and/or do not follow up to the IWANT requests.
In go-libp2p-pubsub, they can attribute the invalid message to a specific reason. This lets them properly score the peer/message based on the reason: https://github.com/libp2p/go-libp2p-pubsub/blob/master/score.go#L563
Unfortunately, based on the current design of js-libp2p-pubsub, we are not able to determine why a message is invalid. Only that it is (it returns a boolean, valid/invalid). Earlier in the gossipsubv1.1 effort, we extended our extension of validate
to include topic validators and a function to process topic validator errors: here, but this is insufficient to track other causes of a message being invalid, eg: invalid or missing message signature.
A fix for this very likely requires modifying js-libp2p-pubsub in some way.
One possibility is to have validate
throw an Error
using js-errcode
.
We could then extend validate
to handle errors of different codes, and make our scoring code quite similar to the above go-libp2p-pubsub example.
Here is another relevant section of go-libp2p-pubsub: https://github.com/libp2p/go-libp2p-pubsub/blob/8945f91465bc3ba418688684a2c29c384cd79647/pubsub.go#L924
Open to other ideas of how to proceed.
Control message piggybacking is a way to reduce the amount of messages that get passed around in gossipsub. It is an optional feature but I think should be implemented in the future to give developers the option to use it in their applications.
Add to the already existing heartbeat tests, tests containing more than one node in the network and with publish
and subscribe
actions around, in order to confirm the content of the messages.
In the context of finding an issue with the heartbeat
on ChainSafe/gossipsub-js#21
0.9.1
bumped the libp2p-interfaces
dep from 0.10.x
to 0.11.x
. The current libp2p
release depends on 0.10.x
so the types are incompatible.
Errors manifest as:
src/runtime/libp2p-nodejs.js(40,7): error TS2419: Types of construct signatures are incompatible.
Type 'new (libp2p: Libp2p, options?: Partial<GossipInputOptions> | undefined) => Gossipsub' is not assignable to type 'new (...args: any[]) => PubsubBaseProtocol'.
Construct signature return types 'Gossipsub' and 'PubsubBaseProtocol' are incompatible.
The types of 'peers' are incompatible between these types.
Type 'Map<string, import("/Users/alex/Documents/Workspaces/ipfs/js-ipfs/node_modules/libp2p-gossipsub/node_modules/libp2p-interfaces/dist/src/pubsub/peer-streams")>' is not assignable to type 'Map<string, import("/Users/alex/Documents/Workspaces/ipfs/js-ipfs/node_modules/libp2p-interfaces/dist/src/pubsub/peer-streams")>'.
Type 'import("/Users/alex/Documents/Workspaces/ipfs/js-ipfs/node_modules/libp2p-gossipsub/node_modules/libp2p-interfaces/dist/src/pubsub/peer-streams")' is not assignable to type 'import("/Users/alex/Documents/Workspaces/ipfs/js-ipfs/node_modules/libp2p-interfaces/dist/src/pubsub/peer-streams")'.
Types have separate declarations of a private property '_rawOutboundStream'.
src/runtime/libp2p-browser.js(35,7): error TS2322: Type 'typeof Gossipsub' is not assignable to type 'new (...args: any[]) => PubsubBaseProtocol'.
Trying to update libp2p and running latest v0.6.2 version of this package I get the following typing issues:
node_modules/libp2p-gossipsub/src/index.d.ts:10:28 - error TS7016: Could not find a declaration file for module 'time-cache'. '/Users/adam/Projects/iovlabs/rif-communications-pubsub/node_modules/time-cache/src/index.js' implicitly has an 'any' type.
Try `npm install @types/time-cache` if it exists or add a new declaration (.d.ts) file containing `declare module 'time-cache';`
10 import TimeCache = require('time-cache');
~~~~~~~~~~~~
node_modules/libp2p-gossipsub/src/index.d.ts:12:25 - error TS7016: Could not find a declaration file for module 'libp2p-interfaces/src/pubsub'. '/Users/adam/Projects/iovlabs/rif-communications-pubsub/node_modules/libp2p-interfaces/src/pubsub/index.js' implicitly has an 'any' type.
Try `npm install @types/libp2p-interfaces` if it exists or add a new declaration (.d.ts) file containing `declare module 'libp2p-interfaces/src/pubsub';`
12 import Pubsub = require('libp2p-interfaces/src/pubsub');
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I posted an issue in the libp2p github to tell them that we will be adding some extra functionality to some of the JS libp2p libraries in order to make them work like the same Go libp2p libraries.
These are the libraries that we will need to work on (may be extended):
js-libp2p-connection-manager with go-libp2p-connmgr and corresponding interface
Peers in the JS implementation aren't tagged
JS implementation doesn't have a grace period
JS implementation has a different peer weighting mechanism than the Go implementation. Also the ranges of the weights differ.
Go implementation has a trimming heuristic for removing extra peer connections. JS implementation evicts connections on demand. There's an issue discussing these differences.
js-libp2p-peer-info. There isn't an associated Go module but there is the js-libp2p module uses this in addition to the peer store for managing information about peers. The Go equivalent is go-libp2p-host.
[ x ] The creation of a separate router module. The concept of a router doesn't really get introduced until gossipsub. In floodsub, we can get away without using a router. There is a go-libp2p-pubsub-router module that manages this for the Go implementations of pubsub protocols.
Add support for determining the protocol a peer supports to js-peer-info. This issue addresses this problem. We might have to add this functionality to the swarm (inconsistently, called switch in JS) module since it handles dialling into peers.
Add interoperability tests with go-libp2p-pubsub. The interoperability repository is still in development.
This repo is named gossipsub-js
rather than the more standard js-libp2p-gossipsub
.
It would be nice to conform to the naming of all of the other libp2p repos.
Perhaps we should id all known links that would need to be updated if we updated the repo name here.
See:
#87 (comment)
libp2p/js-libp2p#675
#83 (comment)
Once we can more easily rely on emitted events to trigger peer IP refreshes, use that. (Instead of currently running a synchronous peer-wide IP update, applied periodically)
Originally posted by @vasco-santos in #83
The go-libp2p-gossipsub implementation has a throttled
state that a message can go to if too many messages get validated at once. What would this look like in our implementation?
Add manually-created or auto-generated typescript definitions.
in ts/index.ts
, libp2p
is required as a direct dependency. but it is declared as a devDep.
move libp2p
to dep or peerDep
In order to make sure our implementation is the best it can be, we will need to benchmark each step of the gossipsub protocol.
Subscribing to peers
Publishing to peers
Since in Gossipsub we need to handle peers that are either using Floodsub or Gossipsub, we need to find a way to find which protocol a peer is running.
Here are several ways to go about doing this:
ls
on multistream
(see this) to determine the list of available protocols.More work will need to be done to determine the best approach.
In GossipSub, we need to store the latest publish message and periodically run the heartbeat operation. The first can be easily done with default JS functionality but I don't think this is the case for the second. An initial Google search brings up using Promises to get similar functionality to Go's time.sleep
but it's still not the same since it only pauses the function in which the promise is called.
We need to find a way to do this deterministically and make sure that we don't get unnecessary hangs in our code.
All released versions of libp2p-gossisub use logical OR in the guarding condition of the function _processRpcControlMessage
:
_processRpcControlMessage (peer: Peer, controlMsg: ControlMessage): void {
if (!controlMsg) {
return
}
const iwant = this._handleIHave(peer, controlMsg.ihave)
const ihave = this._handleIWant(peer, controlMsg.iwant)
const prune = this._handleGraft(peer, controlMsg.graft)
this._handlePrune(peer, controlMsg.prune)
if (!iwant || !ihave || !prune) {
return
}
const outRpc = createGossipRpc(ihave, { iwant: [iwant], prune })
this._sendRpc(peer, outRpc)
}
I assume logical AND should be used instead, as an outbound control message may include any non-empty combination of IWANT, IHAVE, and PRUNE:
_processRpcControlMessage (peer: Peer, controlMsg: ControlMessage): void {
if (!controlMsg) {
return
}
const iwant = this._handleIHave(peer, controlMsg.ihave)
const ihave = this._handleIWant(peer, controlMsg.iwant)
const prune = this._handleGraft(peer, controlMsg.graft)
this._handlePrune(peer, controlMsg.prune)
if (!iwant && !ihave && !prune) {
return
}
const outRpc = createGossipRpc(ihave, { iwant: iwant && [iwant], prune })
this._sendRpc(peer, outRpc)
}
Note that composing the iwant array in outRpc
message requires an additional check now.
The go-libp2p-gossipsub
reference implementation of the corresponding function also uses this logic:
func (gs *GossipSubRouter) HandleRPC(rpc *RPC) {
ctl := rpc.GetControl()
if ctl == nil {
return
}
iwant := gs.handleIHave(rpc.from, ctl)
ihave := gs.handleIWant(rpc.from, ctl)
prune := gs.handleGraft(rpc.from, ctl)
gs.handlePrune(rpc.from, ctl)
if len(iwant) == 0 && len(ihave) == 0 && len(prune) == 0 {
return
}
out := rpcWithControl(ihave, nil, iwant, nil, prune)
gs.sendRPC(rpc.from, out)
}
I came across this issue in a use case where IHAVE messages received by a subscribing peer were not answered by IWANT messages which, in consequence, led to lost publication messages.
Thanks a lot for fixing this issue in a patch release!
Hubertus
BTW, the pull request for [WIP] Gossipsubv1.1
is also affected.
Need to test the gossipsub protocol for multiple nodes (>2).
Mount gossipsub protocol on 3 nodes, A, B and C
Establish connections between all peers
Subscribe to different topics on different nodes
Publish to different topics on different nodes
API docs need to be generated and added to the README, as discussed on ChainSafe/gossipsub-js#15.
Currently, this test sends 10k messages, which overloads local / CI infrastructure.
It needs to be tweaked to be able to run without performance issues.
When I startup libp2p-gossipsub as part of libp2p I get unhandled promise rejection error as follow:
(node:77341) UnhandledPromiseRejectionWarning: Error: Heartbeat timer is already running
at Heartbeat.start (/Users/adam/Projects/iovlabs/rds-ipfs/node_modules/libp2p-gossipsub/src/heartbeat.js:41:27)
at Gossipsub.<anonymous> (/Users/adam/Projects/iovlabs/rds-ipfs/node_modules/libp2p-gossipsub/src/index.js:340:28)
at Generator.next (<anonymous>)
at fulfilled (/Users/adam/Projects/iovlabs/rds-ipfs/node_modules/libp2p-gossipsub/src/index.js:24:58)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Do you have some ideas why is it happening? Can it affect something?
Currently, it seems that everything works as expected even with this error, but it raises concern...
Since the eventual goal of this library is to be used within the Libp2p js project, this library should be licensed under the MIT License. All Libp2p repositories are licensed under this license.
Also, since there is a lot of intersection between the gossipsub protocols and floodsub protocols, it is going to be easier to use open-sourced components in order to build out this library.
Currently, only the default gossipsub multicodec ('/meshsub/1.0.0'
) is used to initialize the gossipsub protocol. We would like to initialize the gossipsub protocol with an application-supplied multicodec instead.
One known use-case is for applications that specify/require non-standard gossipsub multicodecs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.