Comments (6)
@greatest-ape so I see you are a super hero. Thank you on behalf of everyone who dont yet know they need webtorrents on rustoids!
from aquatic.
Hey @rektide,
I've been working quite a bit on this and have implemented a working WebTorrent tracker and merged it into master. There are two executables now, aquatic_udp
and aquatic_ws
, with the ws
suffix of the latter indicating the web socket based WebTorrent protocol. I've successfully used it as the tracker for a file transfer, so all essential parts are up and running, including TLS support for the wss
protocol!
The new tracker already supports pruning old connections, torrents and peers (webtorrent/bittorrent-tracker#164). It is likely already quite performant, but I haven't done any profiling. Getting a load test like the one I wrote for the udp protocol up and running would be nice and greatly aid both performance and correctness analysis, but I probably won't have time for that in a while.
One remaining issue is that of whether peers announcing over IPv6 should receive IPv4 peers. They probably shouldn't, and in that case, state should be split based on IPv4/IPv6 (which shouldn't be that difficult). Some other (mostly minor) issues are listed in the TODO file.
from aquatic.
Hello @rektide ,
This looks interesting. With some more clarity on the exact protocol (ideally an accepted bittorrent protocol extension), I would be interested in investigating it further.
At first look, my understanding is that a webtorrent tracker acts as a middleman for sending messages between peers, and then the peers commence some kind of file transfer connection without the involvement of the tracker? Is this correct?
Is there any point to intermingling peers using different protocols, ie udp and ws? Can they communicate with each other? This impacts whether it would make sense to attempt to integrate the various protocols or run the ws tracker essentially as an independent system from the udp tracker, possibly even with its own binary (especially if bringing in the whole tokio stack.)
Are there any deficiencies in the existing (NodeJS) implementation? I assume performance (or possibly correctness) would be the motivation for a Rust implementation?
Some info for future reference: there seems to be multiple websocket implementations in Rust. These look interesting:
- https://crates.io/crates/ws
- https://crates.io/crates/tungstenite / https://crates.io/crates/tokio-tungstenite
Some info on alternatives: https://github.com/flosse/rust-web-framework-comparison#websocket-libraries
WebTorrent FAQ: https://github.com/webtorrent/webtorrent/blob/master/docs/faq.md
from aquatic.
Hello! Thanks for the speedy & very well considered reply! What a delight to be so well met with such a broad ask. I'm going to try to provide answers as best I can, but I'd say they really are more like educated guesses based off a somewhat weak understanding.
a webtorrent tracker acts as a middleman for sending messages between peers, and then the peers commence some kind of file transfer connection without the involvement of the tracker?
Yup, right on. Just like regular BitTorrent, where a peer can announce content and scrape to find people offering content, in WebTorrent a peer talks to a tracker to announce and scrape, again registering itself or finding those registered as having a bit of content (identified by info_hash
).
We can also see in that websocket example, that the webtorrent announce/scrape commands are JSON encoded messages with a couple fields (action
, info_hash
, & some message dependent fields).
Is there any point to intermingling peers using different protocols, ie udp and ws?
This is an interesting question. What I'm seeing again from checking out WebTorrent's bittorrent-tracker is that it maintains a global list of peers, but will not intermix between ws and (udp or http).
I misread it at first & was assuming it might, & leaving clients to figure out which peers it can connect to however it might, but if there's only a handful of peers on the client's protocol, it might take quite a while to find another using the same protocol that it can connect to. But it would be neat, in cases where one was using something like webtorrent-hybrid, where the peer has multiple connectivity options built in, and some protocols might have connectivity advantage that would let them make the connection that the other protocol lacked (webrtc having stun/turn for example). I suppose in that case the hybrid peer/client could scrape the tracker via both protocols. :)
Are there any deficiencies in the existing (NodeJS) implementation? I assume performance (or possibly correctness) would be the motivation for a Rust implementation?
Performance, security, & correctness are my main desires, yes. There has been much discussion about moving to a more efficient C++ based websocket implementation, uws. Two interesting enhancements I'd mention are pruning old torrents/swarm and a discussion around enhancing client privacy, but I wouldn't call these deficiencies.
Cheers. Thanks for the great work already.
from aquatic.
Thanks @rektide for the info. I've started to look into it and will mention progress in this issue :)
from aquatic.
WebTorrent support is pretty much up and running now, so I'm closing this issue.
from aquatic.
Related Issues (20)
- Consider using HMAC for stateless connection id handling HOT 5
- Add Dockerfile for aquatic_ws HOT 10
- Tracker reply has no peers field HOT 4
- Websocket tracker ping support HOT 3
- Docs for benchmark generation HOT 2
- [Feature] Add statistics for aquatic-ws HOT 2
- Lots of Errors HOT 2
- Aquatic Crash with latest Commit HOT 9
- Error: create rustls config: No private keys in file with Let's Encrypt HOT 1
- [feature] Add client versions statistics to Prometheus exporters HOT 5
- Question/Feature Request: TLS certificates with aquatic-ws HOT 6
- Question about WS trackers HOT 9
- [bug] aquatic-ws - Prometheus metrics discrepancies HOT 3
- aquatic with reverse proxy (CDN) HOT 4
- [bug] aquatic-ws - Memory Leak HOT 9
- Allow serializing UDP responses to JSON HOT 9
- Error trying to compile `http_load_test`: unknown feature `stdsimd` HOT 5
- aquatic_http: peer addr should already have been extracted by now HOT 6
- How can I keep the program running in the background? HOT 1
- [Crash] Cannot allocate memory HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aquatic.