Transmission allow to choose on which IP will transit all the traffic.
Would be nice to allow such feature in your library.
I hacked it but I'd like it to be clean and usable by torrent-search
Thanks

I haven't found a way to access the retrieved peers from the dht module.
I think one should be able to access all found peers via dht.getPeers() or something.

Right now, we're naively storing any values the network asks us to. This is a potential source of DOS.

"In these networks, a node A only forwards a lookup on key k to another node B if |B−k| < |A−k|, where “−” represents XOR in Kademlia". This approach means we need to have a way to estimate the network size, or at least look at our own table to decide if there are better/closer nodes that could store the key/value instead.

Or, we could take Transmission's approach of storing anything we're asked to, but capping the number of hashes the client is willing to track.

/* The maximum number of peers we store for a given hash. */
#ifndef DHT_MAX_PEERS
#define DHT_MAX_PEERS 2048
#endif

/* The maximum number of hashes we're willing to track. */
#ifndef DHT_MAX_HASHES
#define DHT_MAX_HASHES 16384
#endif

I was just reading through the code and I notice that there were lots of error handling in the code. nice but Isn't it frown upon to throw errors in npm packages.. in stead you should past it as the first argument of the callback.

Have a validation function for each message type. Validate each message before processing it. This would help to move all the type checking out of the main codebase (where it's sort of an afterthought now) and make it more rigorous.

Should help to prevent future security issues like #87

cc @mafintosh

Any way of doing this? Or is it all done on the receiving side from the message originating address?

Currently, some of the important operations are O(n)

Using one instance of dht we can add torrent, than after receiving infodicionary stop it/ remove it. And than add it again. But torrent can't just get dht.peers[infoHash] since it is not array of strings but converted by string2compact array. So it's better to have array of strings besides to compactPeerInfo and have index to check it before adding of peer to this array.
Also index can be used to speed up of adding and removing of peers (replace following code)

var exists = peers.some(function (peer) {
    return bufferEqual(peer, compactPeerInfo)
  })

https://github.com/feross/bittorrent-dht/blob/master/client.js#L327

But on the documentation it's say it optional.

Example:

dht = new DHT({bootstrap: false})
dht.addNode('127.0.0.1:20000') // throws an error!

TypeError: First argument needs to be a number, array or string.
    at new Buffer (buffer.js:188:15)
    at idToBuffer (node_modules\bittorrent-dht\client.js:1196:12)
    at DHT.addNode (node_modules\bittorrent-dht\client.js:278:12)
    ...

If the DHT returns nodes or peers with local IP addresses, should we store them? That means that a remote attacker can make the DHT or bittorrent peer connect to local services. Is this unsafe?

Local IP ranges:
192.168.0.0 - 192.168.255.255 (65,536 IP addresses)
172.16.0.0 - 172.31.255.255 (1,048,576 IP addresses)
10.0.0.0 - 10.255.255.255 (16,777,216 IP addresses)

Localhost:
127.0.0.1

Does anyone know how utorrent implements the "vote" system over the DHT?

I looked online but there is next to nothing about it.

Some information:
http://lists.ibiblio.org/pipermail/bittorrent/2011-January/002413.html

http://arxiv.org/ftp/arxiv/papers/1305/1305.0711.pdf

You get a message something like this:

{ a:
   { id: '907050177fb05376db71ce066663819503eb0a9a',
     target: '546d6d0e5aacfae66299f26a300a428c497b23b5',
     token: '5b3f3de3d4f5ba42721dac14b2076dc64bc829da',
     vote: 0 },
  q: 'vote',
  t: '+=\u0000\u0000',
  v: 'UT??',
  y: 'q' }

The only thing I dont get it the target They say its sha1(info-hash + "rating") but i dont know. So the rating 1-5 and 0 been the query for votes. So if a infohash has 5 votes ranging from 1 to 5 would you receive 5 responses?

Has anyone played around with this?

Reporting this in case someone else has the problem but this might be specific to my uses.

Maybe related to joyent/libuv#1210

Node version v0.11.9, I did not check the fix for the above bug.

While performing successive lookup in the DHT as explained here: https://github.com/Ayms/torrent-live#findspies it appears that after some time the DNS lookup for the bootstrap nodes does resolve "less and less" until contacts are returned as an empty array.

Then the following error fires:

../deps/uv/src/unix/core.c:413: uv__close: Assertion `fd > -1' failed.

Deactivating the dns lookup does "resolve" the issue.

Not sure if it's a libuv or dns issue, maybe you can close it for now.

There's a BEP that describes how to add support: http://www.bittorrent.org/beps/bep_0032.html

should be pretty straightforward to implement

it might make

test('`node` event fires for each added node (10000x)', function (t) {

excruciatingly long

How do I use bittorrent-dht in a chrome packaged app. The DNS module does browserify properly.

tl;dr When we receive a get_peers for an infohash we have announced, should we include ourselves as a peer? Currently we don't, and I'm not sure whether it's a bug.

Long version:

I've noticed that when one peer receives a get_peers for an infohash it published itself, it returns the K closest nodes instead of itself as a peer. So the upshot is that the only way an announce works is if:

node A announces
node B hears the announce
node C calls get_peers, and node A is returned to node C by node B

Whereas if only two nodes were online, it'd be:

node A announces
node C starts up, calls get_peers, node A receives the get_peers but does not include itself as a peer, so the download can not proceed

Any idea if this is a bug, vs. me misunderstanding the protocol? If it's a bug, is the fix to call _addPeer() for ourselves when we announce?

Hi guys, I have a algorithm that check the peers on a torrent, after sometime this error happen:

TypeError: Bad argument
    at new Buffer (buffer.js:194:21)
    at idToBuffer (/home/dhseja/torrent_crawl/node_modules/bittorrent-dht/client.js:1302:12)
    at DHT._addNode (/home/dhseja/torrent_crawl/node_modules/bittorrent-dht/client.js:336:12)
    at DHT._onData (/home/dhseja/torrent_crawl/node_modules/bittorrent-dht/client.js:714:10)
    at Socket.emit (events.js:98:17)
    at UDP.onMessage (dgram.js:440:8)

buffer.js:194
      this.parent = new SlowBuffer(this.length);
                    ^
TypeError: Bad argument
    at new Buffer (buffer.js:194:21)
    at idToBuffer (/home/dhseja/torrent_crawl/node_modules/bittorrent-dht/client.js:1302:12)
    at DHT._addNode (/home/dhseja/torrent_crawl/node_modules/bittorrent-dht/client.js:336:12)
    at DHT._onData (/home/dhseja/torrent_crawl/node_modules/bittorrent-dht/client.js:714:10)
    at Socket.emit (events.js:98:17)
    at UDP.onMessage (dgram.js:440:8)
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory

Here is the algorithm:

// checker.js
var torrentStream = require('torrent-stream');
var Discovery = require('torrent-discovery');
var DHT = require('bittorrent-dht');
var parseTorrent = require('parse-torrent');
var hat = require('hat');
var Promise = require('bluebird');


module.exports = function(magnet) {
  return new Promise(function(done) {

    var torrent = parseTorrent(magnet);

    var peers = 0;
    var dht = new DHT();
    dht.lookup(torrent.infoHash);
    var discovery = new Discovery({
      peerId: hat(160),
      port: 6000,
      dht: dht
    });

    try {
      discovery.setTorrent(torrent);
      discovery.on('peer', function() {
        peers++;
      });
    }catch(error) {
      console.log(error);
    }




    setTimeout(function() {
      discovery.stop();
      dht.destroy();
      discovery = undefined;
      done(peers);
    }, 60 * 1000);
  });
}

An example of use:

var checker = require('./checker.js');

checker("magnet:?xt=urn:btih:").then(function(peers) {
 console.log("Peers on this torrent: "+peers);
});

Any idea?
Thanks in advance.

BEP44 is very exciting because it introduces mutable content signed with a public key. With mutable hashes, applications like twitter-style feeds can be built on the DHT which may prove very useful for projects like ssb. libtorrent and utorrent already have support for dht_store but I'm not sure if anybody is using this feature in the wild yet.

I started implementing this BEP on top of the bittorrent-client plugin system, but it doesn't make much sense to implement on top of bittorrent-client because couples strongly with the dht implementation.

• http://bittorrent.org/beps/bep_0044.html
• http://www.libtorrent.org/dht_store.html

I want to download 5 torrents by magnet links in same time. I suppose that I should use 1 bittorrent-dht instance instead of 5 (please fix me if this wrong).

So i will do following

var infoHashes = [..., ..., ...];
//infoHashes.length == 5

var table = dht();

table.on('peer', function(peer, infoHash){
    //do something
});
table.on('ready', function() {
    infoHashes.forEach(function(infoHash){
        table.lookup(infoHash);
    })
});

I want to completely stop downloading infoHashes[2]. What I need to do to destroy lookup for infoHashes[2]?

Could we maybe remove the console.log(...) stuff from this module and maybe use a debug module instead? I would be happy to contribute a PR.

No point to throwing them away, especially if they contain new nodes or peers.

Can we actually use the response, even without the original callback in the self.transactions object?

For better reliability (esp. with TravisCI), easier offline development, etc.

See more extensive discussion here: webtorrent/webtorrent#11

We have been looking at securing the DHT, making it harder to attack (especially with sybils). One thing we’re implementing to support this is requiring DHT nodes to calculate their node ID based on their external IP, with some flexibility to support NATs and such. More info on Node ID enforcement can be found here.

• http://www.bittorrent.org/beps/bep_0042.html
• http://libtorrent.org/dht_sec.html
• http://engineering.bittorrent.com/2013/12/19/dht-bootstrap-update/

Full API here:
https://github.com/DevelopmentIL/krpc

(Seen on @substack's dht-store branch, but filing here.)

  bittorrent-dht [2f01a3c] put from 98.100.36.218:6881 +1ms

/home/cjb/gittorrent/node_modules/bittorrent-dht/node_modules/k-bucket/index.js:225
    if ((id.length <= bytesDescribedByBitIndex)
           ^
TypeError: Cannot read property 'length' of undefined
    at KBucket.determineBucket (/home/cjb/gittorrent/node_modules/bittorrent-dht/node_modules/k-bucket/index.js:225:12)
    at KBucket.get (/home/cjb/gittorrent/node_modules/bittorrent-dht/node_modules/k-bucket/index.js:258:18)
    at DHT._onPut (/home/cjb/gittorrent/node_modules/bittorrent-dht/client.js:486:27)
    at DHT._onQuery (/home/cjb/gittorrent/node_modules/bittorrent-dht/client.js:1052:30)
    at DHT._onData (/home/cjb/gittorrent/node_modules/bittorrent-dht/client.js:1036:10)
    at Socket.EventEmitter.emit (events.js:98:17)
    at UDP.onMessage (dgram.js:440:8)

I don't control this node, so have no idea what they were doing :)

More nodes = healthier DHT table.

im currently having to handle dht.put like this:

    dht.put({v: new Buffer('hello world')}, function (errors, hash) {
      if (errors) {
        errors.forEach(function (e) {
          console.log(e)
        })
      }
      console.log(hash)
    })

however, I find the errors array not that useful for the following reasons:

• there will almost always be errors, since not all clients on the dht support bep44. for example:

{ [Error: 204 Unknown query type.] address: '95.211.211.152:6881' }
{ [Error: 202 Server Error] address: '121.40.163.12:6881' }
{ [Error: query timed out] address: '24.43.1.206:27935' }
{ [Error: query timed out] address: '105.107.66.81:8037' }
{ [Error: query timed out] address: '93.47.84.119:49670' }
{ [Error: query timed out] address: '145.167.133.38:32995' }
{ [Error: query timed out] address: '82.250.244.230:38754' }
{ [Error: query timed out] address: '5.79.212.119:62470' }
{ [Error: query timed out] address: '179.164.228.17:22828' }
{ [Error: query timed out] address: '27.208.33.164:9101' }
{ [Error: query timed out] address: '94.128.247.247:15718' }
{ [Error: query timed out] address: '81.246.190.143:3612' }
{ [Error: query timed out] address: '103.240.161.164:9097' }
{ [Error: query timed out] address: '172.101.33.37:18667' }
{ [Error: query timed out] address: '46.188.123.196:21985' }

• i have no way of knowing if any of the peers actually stored my key if all I get are the ones who didn't. IMO it should give you back all the successful ones too. there's no way to know how many peers it tried to store it on currently.

other API suggestions welcome!

The following code snippet yields no peers for the lookup query. no peer is printed to stdout. IF i use the same DHT object for announce and lookup, the correct IP is found. However when creating a new DHT object, the IP is not found.

Am i misusing the library or is this expected behavior and why?

Thanks!

var DHT = require("bittorrent-dht")

var ih = "46f9e9a430f839e7acd76733d649b885985cff4b"
var dht =  new DHT()

dht.listen(20000, function () {
  console.log('now listening')
})


dht.on('ready', function () {
  console.log("dht is ready")

  dht.announce(ih, 6666, function (r) {
    var otherDHT = new DHT()

    console.log("announce result is " + r)

    otherDHT.listen(20001, function () {
    })
    otherDHT.on('ready', function () {
      console.log("other dht is ready")

      otherDHT.lookup(ih, function (err, r) {
      })
      otherDHT.on('peer', function (addr, ih, from) {
        console.log("got peer " + addr)
      })
    })
  })
})

We just fixed a security issue in the bittorrent-dht module that would allow an attacker to send a specific series of messages to a listening peer to make it disclose internal memory of the node.js process.

All users of bittorrent-dht should upgrade to version 5.1.3 or later.

Edit: I guess for 2 it doesn't work because nodes don't know their own ips

I wanted to see if how many DHT instances with each other in their routing tables were enough to do announce + successful lookup. I hoped 2, but as a DHT node never returns itself as a peer, this appears to be impossible. Is this by design? 4+ are usually enough, but not always, which makes me think there's some race condition.

test('test announce+lookup with different quantities of DHTs', function (t) {
  var from = 2;
  var to = 10;

  t.plan(to - from + 1);

  for (var i = from; i <= to; i++) {
    (function(numInstances) {      
      findPeers(numInstances, t, function(err) {
        if (err) t.error(err)
        else t.pass('Success with ' + numInstances + ' instances')
      });
    })(i)
  }
})

/**
 *  Initialize [numInstances] dhts, have one announce an infoHash and another perform a lookup
 *  Times out after a while
 */
function findPeers(numInstances, t, cb) {
  var timeoutId = setTimeout(function() {
    cleanup(new Error('Timed out for ' + numInstances + ' instances'))
  }, 5000);

  var infoHash = common.randomId()
  var dhts = [];

  for (var i = 0; i < numInstances; i++) {
    var dht = new DHT({ bootstrap: false });
    dhts.push(dht);
    common.failOnWarningOrError(dht, t);
  }

  // wait till everyone's listening
  parallel(dhts.map(function(dht) {
    return function(cb) {
      dht.listen(function(port) {
        cb(null, port);
      })
    }
  }), function(err, ports) {
    // add each other to routing tables
    makeFriends(dhts);
    dhts[0].announce(infoHash, 9999, function() {
      dhts[1].lookup(infoHash)      
    })
  })


  dhts[1].on('peer', function(addr, hash) {
    if (hash === infoHash.toString('hex')) {
      cleanup()
    }
  })

  function cleanup(err) {
    parallel(dhts.map(function(dht) {
      return dht.destroy.bind(dht)
    }), function() {
      clearTimeout(timeoutId)
      cb(err)
    })
  }
}

/**
 *  Adds every dht in [dhts] to every other dht's routing table
 */
function makeFriends(dhts) {
  for (var i = 0; i < dhts.length; i++) {
    for (var j = i + 1; j < dhts.length; j++) {
      var d1 = dhts[i]
      var d2 = dhts[j]
      d1.addNode('127.0.0.1:' + d2.port, d2.nodeId)
      d2.addNode('127.0.0.1:' + d1.port, d1.nodeId)
    }
  }
}

hello,

after some time, i got this error :

buffer.js:98
const ui8 = new Uint8Array(size);
^

RangeError: Invalid typed array length
at new Uint8Array (native)
at allocate (buffer.js:98:17)
at new Buffer (buffer.js:49:12)
at idToBuffer (/root/scripts/tracker/node_modules/bittorrent-dht/client.js:1639:12)
at DHT._addNode (/root/scripts/tracker/node_modules/bittorrent-dht/client.js:630:12)
at DHT._onData (/root/scripts/tracker/node_modules/bittorrent-dht/client.js:1025:10)
at emitTwo (events.js:87:13)
at Socket.emit (events.js:172:7)
at UDP.onMessage (dgram.js:481:8)

any idea ?

Best regards,
tsukasagenesis

            if(port == 0) {
                debugf("Announce_peer with forbidden port %d.\n", port);
                send_error(from, fromlen, tid, tid_len,
                           203, "Announce_peer with forbidden port number");
                break;
            }

From https://github.com/jech/dht/blob/master/dht.c

See pull request #34 for the fix.

Use the k-bucket ping event,.

How I can start exchanging of peers with passed to bittorrent-dht peers from outside?

e.g. a:

1. I got some peers 2) I cached them 2) I destroyed bittorrent-dht instance 3) I start new bittorrent-dht instance after 30 minutes and want to pass it cached peers

2. app does not have metadata (infoDictionary), but has infoHash.
   App got some peers from trackers, but did not get any from DHT (even with BOOTSTRAP_NODES help).

Is ok?

If a boostrap timeout occurs, the boostrap process is retried but no other timeout is set in case the boostrap process does not succeed again.

Hi just a quick question. Is there anyway I can crawl the bittorrent dht using this package. I have seen various implementations but they do not use this package. I am just stuck on how i can acquire the infohash of the swarm of various bittorrent peers. I have 0.05 BTC available for an appropriate solution Any help would be much appreciated. Cheers.

Am I reading the code wrongly or it is not possible right now to create once a dht and persist it for different lookup, as well as updating the routing table continuously (and testing the nodes)? (as far as I see the routing table is supposed to be recreated for each lookup)

I saw #16 but apparently the fix is to bootstrap the dht at creation.

Is it on purpose, or in the TODO list?

Hi, I'm seeing a race that leads to announces only happening on private IPs instead of public ones:

  bittorrent-dht [2ab538e] announce 5fbfea8de70ddc686dafdd24b690893f98eb9475 30000 +3ms
debug: in self.localAddresses.forEach with 10.20.30.3
  bittorrent-dht [2ab538e] addPeer 10.20.30.3:30000 5fbfea8de70ddc686dafdd24b690893f98eb9475 +2ms
  bittorrent-dht [2ab538e] announce end 5fbfea8de70ddc686dafdd24b690893f98eb9475 30000 +0ms
debug: in publicAddress with 192.34.86.36
[publicAddress is added to self.localAddresses here, too late for the announce's forEach loop over self.localAddresses]

(I wonder why we're announcing on private IPs anyway?)

• Expose method
• Republish to the DHT at regular intervals

For testing purposes, I attempted to run the following code (CoffeeScript):

BTDHT = require "bittorrent-dht"

dht1 = new BTDHT()
dht1.on "ready", -> console.log("dht1 ready"); dht1.lookup("dht-test")
    .on "peer", (addr, hash) -> console.log("Found peer for #{hash} at #{addr} on dht1")
    .listen 20000, -> console.log("dht1 listening")

dht2 = new BTDHT()
dht2.on "ready", -> console.log("dht2 ready"); dht2.lookup("dht-test")
    .on "peer", (addr, hash) -> console.log("Found peer for #{hash} at #{addr} on dht2")
    .listen 20000, -> console.log("dht2 listening")

Compiled Javascript is as follows:

(function() {
  var BTDHT, dht1, dht2;

  BTDHT = require("bittorrent-dht");

  dht1 = new BTDHT();

  dht1.on("ready", function() {
    console.log("dht1 ready");
    return dht1.lookup("dht-test");
  }).on("peer", function(addr, hash) {
    return console.log("Found peer for " + hash + " at " + addr + " on dht1");
  }).listen(20000, function() {
    return console.log("dht1 listening");
  });

  dht2 = new BTDHT();

  dht2.on("ready", function() {
    console.log("dht2 ready");
    return dht2.lookup("dht-test");
  }).on("peer", function(addr, hash) {
    return console.log("Found peer for " + hash + " at " + addr + " on dht2");
  }).listen(20000, function() {
    return console.log("dht2 listening");
  });

}).call(this);

... and that resulted in this:

dht1 listening
dht2 listening
dht2 ready
dht1 ready
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace: 
  at DHT.EventEmitter.addListener (events.js:160:15)
  at /home/sven/projects/syncy/node_modules/bittorrent-dht/client.js:330:10
  at /home/sven/projects/syncy/node_modules/bittorrent-dht/client.js:395:5
  at done (/home/sven/projects/syncy/node_modules/bittorrent-dht/node_modules/run-parallel/index.js:15:13)
  at /home/sven/projects/syncy/node_modules/bittorrent-dht/client.js:388:9
  at asyncCallback (dns.js:68:16)
  at Object.onanswer [as oncomplete] (dns.js:119:9)

I cannot reproduce the issue when commenting out the second instance, it only appears to happen with 2 (or more?) instances. I've successfully reproduced the issue on another system as well.

Affected configurations:

• openSUSE 12.3 x64, Node.js v0.10.26, bittorrent-dht 1.1.1
• Debian 7 x64, Node.js v0.10.26, bittorrent-dht 1.1.1