nitmir / btdht Goto Github PK

View Code? Open in Web Editor NEW

83.0 83.0 28.0 186 KB

A python implementation of the Bittorrent DHT (Distributed Hash Table)

License: GNU General Public License v3.0

Python 99.04% Makefile 0.67% Batchfile 0.29%

btdht's People

Contributors

Stargazers

Watchers

btdht's Issues

utils.pyx AttributeError: 'generator' object has no attribute 'next'

I'm getting this error with Python 3.5.2 and btdht 0.3.1:

Traceback (most recent call last):
  ...
  File "...", line 6, in <module>
    dht.start()
  File "btdht/dht.pyx", line 379, in btdht.dht.DHT_BASE.start (btdht/dht.c:7806)
    self._scheduler.add_dht(self)
  File "btdht/utils.pyx", line 917, in btdht.utils.Scheduler.add_dht (btdht/utils.c:17508)
    self.add_thread(name, function, user=user)
  File "btdht/utils.pyx", line 846, in btdht.utils.Scheduler.add_thread (btdht/utils.c:15984)
    typ = iterator.next()
AttributeError: 'generator' object has no attribute 'next'

Process finished with exit code 1

Script is nearly empty:

#!/usr/bin/env python3

import btdht
import binascii
dht = btdht.DHT()
dht.start()

python 3.11 install issues

pip install btdht fails with

gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -Ivenv/include -I/usr/include/python3.11 -c btdht/dht.c -o build/temp.linux-x86_64-cpython-311/btdht/dht.o
      btdht/dht.c:209:12: fatal error: longintrepr.h: No such file or directory
        209 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1

at the same time i'm able to successfully build after installing build deps, either directly in a git checkout or by using pip install git+https://github.com/nitmir/btdht.git

enumerate_ids not working on Python3

Hi,

I have been trying to use enumerate_ids but it seems to not work with Python3.

I have been doing some executions on Python2 to see how it works and all goes fine since bytes in python2 are handled as strings.

The problem resides in nset function. I tried to convert bytes to strings to handle them there. Like this:

    if i == 1:
        return s[:n//8]  + chr(s[n//8] | _NSET_BIT1[n % 8]).encode() + s[n//8+1:]
    elif i == 0:
        return s[:n//8]  + chr(s[n//8] & _NSET_BIT0[n % 8]).encode() + s[n//8+1:]
    else:
        raise ValueError("i must be 0 or 1")

The problem with this is that the second element of the output of ``enumerate_ids``` is 21 long char. And some executions start with the first element as a 22 long so the second one is 23 long, third 22...

I have been trying to solve this but I have run out of ideas. Any hint would be appreciated.

Multiple get_peer requests causes 100% CPU usage

Hi,

Im playing around with this library and have tried asking it for a small bunch of magnet hashes, in the start it goes without any issues at all. But after a random number of get_peer requests for some random hashes for popular torrents, the CPU usage spikes to 100% and stays there until i kill it.

I have tried to run only one request per 10 second, running it both in blocking and non-blocking mode, increasing the debug level, moving the query limit between 10 and 1000. No matter what changes i have done the issue appears.

I tried to reproduce the issue by making a script that simply polled the ubuntu example hash (and some more hashes based on that hash), but that did not seem to fail.

It appears that the last thing that happens before the lockup is that the library receives a couple of errors:

2984 nodes, 1607 goods, 59 bads | in: 51, out: 60 en 14s

ERROR:202:b'Server Error' pour (None, 1520126208.2239816, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:=\x08,\x902M1:y1:qe')
obj of type b'e'
8 nodes added to routing table
5 nodes added to routing table
ERROR:202:b'Server Error' pour (None, 1520126208.227913, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:\xd6\x92N\xd8 B1:y1:qe')
obj of type b'e'
8 nodes added to routing table
0 nodes added to routing table
7 nodes added to routing table
4 nodes added to routing table
ERROR:202:b'Server Error' pour (None, 1520126208.266789, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:d/ X\xaf\x911:y1:qe')
obj of type b'e'
8 nodes added to routing table
3 nodes added to routing table
8 nodes added to routing table
ERROR:202:b'Server Error' pour (None, 1520126208.3035867, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:1yV\x97\xc7F1:y1:qe')
obj of type b'e'
ERROR:202:b'Server Error' pour (None, 1520126208.2280884, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:\x8a,\n\x1f\xac\xa21:y1:qe')
obj of type b'e'
ERROR:202:b'Server Error' pour (None, 1520126208.2672093, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:x\xb7\xffe\x1c\xc81:y1:qe')
obj of type b'e'
ERROR:202:b'Server Error' pour (None, 1520126208.2678056, b"d1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xea6:target20:\x87\xd0\x0f\xdc\xdd'\x01,~(\xe0\x9f\xd4\x1c\\jz\xc6\x12be1:q9:find_node1:t6:\x10\xc5\x9e\xd7\x97'1:y1:qe")
obj of type b'e'
0 nodes added to routing table

I tried some profiling with Yappi, it says that these two calls were the most used, not sure if helpful but..

name ncall tsub ttot tavg
...5/queue.py:90 PollableQueue.empty 187.. 21.10275 42.51144 0.000023
..dist-packages/six.py:580 iteritems 187.. 23.93679 37.79221 0.000020

Any calls for get_peers afterwards will simply yield "no peers or nodes found"

I have not seen any special IO-activity while this happens,

This has been tested on the latest release

What could this be, and how can it be avoided?

how to download torrents file using this lib?

how to download torrents file using this lib?
Thanks.

Error during boostraping

def search_dht(dht, info_hash):
    peers = []
    for i in range(0, 5):
        peers = dht.get_peers(binascii.a2b_hex(info_hash))
        time.sleep(1.5)

    return peers


if __name__ == '__main__':
    dht = btdht.DHT()
    dht.start()  # now wait at least 15s for the dht to boostrap
    print("Initializing DHT", flush=True)
    time.sleep(20)
    print(search_dht(dht, '756D9FA81E9328A97A03EE134EBCA8C2CCE3B670'))

The output I get is

init socket for b'0c66c99cafff75350028b3ec10845835c75921b5'
Initializing DHT
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
None

I can't really find what "0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s" mean in the documentation. Why would I be getting this "error" and not the expected (IP, PORT) outcome?

make install : btdht/krcp.pyx:1472:29: Cannot assign None to int

i gotten error, when doing make on the latest code, can you help?

 Error compiling Cython file:
 ------------------------------------------------------------
 ...
                 else:
                     self.addr_addr_2 = addr[0]
                 self.addr_port = addr[1]
         def __del__(self):
             self.addr_addr = None
             self.addr_port = None
                             ^
 ------------------------------------------------------------

 btdht/krcp.pyx:1472:29: Cannot assign None to int
 Compiling btdht/dht.pyx because it changed.
 Traceback (most recent call last):
   File "setup.py", line 28, in <module>
 Compiling btdht/krcp.pyx because it changed.
 Compiling btdht/utils.pyx because it changed.
 [1/3] Cythonizing btdht/dht.pyx
 [2/3] Cythonizing btdht/krcp.pyx
     ext_modules = cythonize("btdht/*.pyx") if has_cython else c_extensions,
   File "/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py", line 1026, in cythonize
     cythonize_one(*args)
   File "/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py", line 1146, in cythonize_one
     raise CompileError(None, pyx_file)
 Cython.Compiler.Errors.CompileError: btdht/krcp.pyx
 make: *** [build] Error 1

Add bep44 support

Add ability to use DHT as k-v strorage
https://www.bittorrent.org/beps/bep_0044.html

utils.pyx UnboundLocalError: local variable 'data' referenced before assignment

I'm getting this error frequently with Python 2.7.9 and btdht 0.3.1:

init socket for f289d6ad5d78a5dbbf4605ddafb6bfdbe38aa7ab
Bootstraping
Exception in thread scheduler:io_loop:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "C:\Python27\lib\threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "btdht/utils.pyx", line 1145, in btdht.utils.Scheduler._io_loop (btdht/utils.c:23139)
    dht._process_incoming_message()
  File "btdht/dht.pyx", line 979, in btdht.dht.DHT_BASE._process_incoming_message (btdht/dht.c:21040)
    self.debug(0, "send:%r : (%r, %r)" % (e, data, addr))
UnboundLocalError: local variable 'data' referenced before assignment

Using the example from the readme:

import binascii
import btdht
import time
dht = btdht.DHT()
dht.start()
time.sleep(30)
print(dht.get_peers(binascii.a2b_hex("0403fb4728bd788fbcb67e87d6feb241ef38c75a")))

nitmir / btdht Goto Github PK

btdht's People

Contributors

Stargazers

Watchers

Forkers

btdht's Issues

utils.pyx AttributeError: 'generator' object has no attribute 'next'

python 3.11 install issues

enumerate_ids not working on Python3

Multiple get_peer requests causes 100% CPU usage

how to download torrents file using this lib?

Error during boostraping

make install : btdht/krcp.pyx:1472:29: Cannot assign None to int

Add bep44 support

utils.pyx UnboundLocalError: local variable 'data' referenced before assignment

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent