storjold / dataserv-client Goto Github PK

View Code? Open in Web Editor NEW

56.0 56.0 24.0 11.43 MB

Client for storing and auditing data.

Home Page: http://storj.io

License: MIT License

Makefile 1.76% Python 97.70% Batchfile 0.02% Shell 0.52%

dataserv-client's People

Contributors

Stargazers

Watchers

dataserv-client's Issues

Optimize Build Seeds

   def pop_seed(self, height):
        """Deterministically build a seed."""
        return self.build_seeds(height-1).pop()

    def build_seed(self, height):
        """Deterministically build a seed."""
        seed = self.sha256(self.address)
        for i in range(height):
            seed = self.sha256(seed)
        return seed

    def build_seeds(self, height):
        return list(map(self.build_seed, range(height)))

Iffy code block. The way the code is structured one could accidentally rebuild the seed list every time we call a seed. We want to generate the seed list once in memory, and then call items from the list.

dataserv-client.exe stopped working but didn´t exit to command prompt

The dataserv-client.exe just suddenly stopped working for no apparent reason (on windows 7, on external HDD attached to PC). An error window appeared saying "The program stopped to function correctly because of a problem. Windows will close the program and notify if a solution exists." I had to manually click the "close program" button to return to the command prompt in the command window.
No further info appeared in the command window. It just went back to command prompt. I restarted polling manually and now polling again without problems.

Error handling not ideal: ConnectionResetError: [Errno 104] Connection reset by peer

Could be better handled. Was building a ton of shards, have to revalidate 10k+ shards now.

Query url: http://104.236.104.117/api/height/<address>/12690
Traceback (most recent call last):
  File "/usr/local/bin/dataserv-client", line 24, in <module>
    cli.main(sys.argv[1:])
  File "/usr/local/lib/python3.4/dist-packages/dataserv_client/cli.py", line 128, in main
    return getattr(client, command_name)(**arguments)
  File "/usr/local/lib/python3.4/dist-packages/dataserv_client/api.py", line 130, in build
    cleanup=cleanup, rebuild=rebuild)
  File "/usr/local/lib/python3.4/dist-packages/dataserv_client/builder.py", line 64, in build
    self.on_generate_shard(shard_num + 1, seed, file_hash)
  File "/usr/local/lib/python3.4/dist-packages/dataserv_client/api.py", line 126, in on_generate_shard
    self._url_query('/api/height/{0}/{1}'.format(self.address, height))
  File "/usr/local/lib/python3.4/dist-packages/dataserv_client/api.py", line 68, in _url_query
    response = urllib.request.urlopen(self.url + api_call)
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/usr/lib/python3.4/urllib/request.py", line 473, in _open
    '_open', req)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 1202, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.4/urllib/request.py", line 1177, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.4/http/client.py", line 1172, in getresponse
    response.begin()
  File "/usr/lib/python3.4/http/client.py", line 351, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.4/http/client.py", line 313, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.4/socket.py", line 371, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer
$ dataserv-client version
1.3.0

Confirm allocated disk space is correct.

so I started my build with the following bytes 3298534883328. I wake up this morning to it complaining about disk space and I now see 3.47TB used in my drive. Why did it go so far past 3TB?

brandon 4:26 PM (in 'dev')

dataserv-client 2.0.0 [WinError 10061]

ERROR UPDATING DATASERV-CLIENT: urlopen error [WinError 10061] Nenhuma ligação pôde ser feita porque o computador de destino as recusou ativamente

I'm getting this error. What is wrong ?

Thanks

Bug at build

python2.7 dataserv-client --address=15Rs7YLUP4NdZF37a4GEsiCxEwnAKHWeUd --max_size=1600149724 --store_path=/home/dataserv-client-master/dataserv_client/bin/data/ build
Failed. Only 11 files is created. replace = space and do max_size multiple of 128_1024_1024 work

Can you ceil the max_size or do a warning.
Or fix this. Thanks

Hangs on Test_Build

usage: utrunner.py [-h] [--url URL] [--max_size MAX_SIZE]
                   [--store_path STORE_PATH] [--config_path CONFIG_PATH]
                   [--debug]
                   <command> ...

Dataserve client command-line interface.

optional arguments:
  -h, --help            show this help message and exit
  --url URL             Url of the farmer (default:
                        http://status.driveshare.org).
  --max_size MAX_SIZE   Maximum data size in bytes. (default: 1073741824).
  --store_path STORE_PATH
                        Storage path. (default: C:\Users\super3.storj\store).
  --config_path CONFIG_PATH
                        Config path. (default:
                        C:\Users\super3.storj\config.json).
  --debug               Show debug information.

commands:
  <command>
    version             Show version number.
    register            Register your node on the network.
    ping                Ping master node.
    poll                Let the network know your are online.
    build               Fill the farmer with data up to their max.
    config              Edit and display config.

Error
Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "C:\Python34\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "C:\Python34\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "C:\Python34\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "C:\Python34\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "C:\Python34\lib\http\client.py", line 857, in send
    self.connect()
  File "C:\Python34\lib\http\client.py", line 834, in connect
    self.timeout, self.source_address)
  File "C:\Python34\lib\socket.py", line 512, in create_connection
    raise err
  File "C:\Python34\lib\socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 46, in _url_query
    response = urllib.request.urlopen(req)
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "C:\Python34\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\tests\test_client.py", line 118, in test_build
    client.register()
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\api.py", line 68, in register
    registered = self.messanger.register(self.cfg["payout_address"])
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 101, in register
    self.auth_address(), payout_addr
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 43, in _url_query
    headers = self._create_authentication_headers()
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 78, in _create_authentication_headers
    msg = self._get_node_address() + " " + header_date
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 71, in _get_node_address
    self._server_address = self.address()
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 92, in address
    data = self._url_query("/api/address", authenticate=False)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 86, in _handle_connection_error
    return self._url_query(api_path, retries + 1, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 64, in _url_query
    self._handle_connection_error(api_path, retries, authenticate)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\messaging.py", line 84, in _handle_connection_error
    raise exceptions.ConnectionError(self._server_url)
dataserv_client.exceptions.ConnectionError: Could not connect to server http://127.0.0.1:5000!

ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

My client on my laptop build stopped polling with the following traceback:

Traceback (most recent call last):
File "dataserv-client", line 9, in
File "E:\dataserv-client\dataserv_client\cli.py", line 124, in main
File "E:\dataserv-client\dataserv_client\api.py", line 114, in poll
File "E:\dataserv-client\dataserv_client\api.py", line 102, in ping
File "E:\dataserv-client\dataserv_client\api.py", line 66, in _querry
File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
File "C:\Python34\lib\urllib\request.py", line 463, in open
File "C:\Python34\lib\urllib\request.py", line 481, in _open
File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
File "C:\Python34\lib\urllib\request.py", line 1185, in do_open
File "C:\Python34\lib\http\client.py", line 1171, in getresponse
File "C:\Python34\lib\http\client.py", line 351, in begin
File "C:\Python34\lib\http\client.py", line 313, in _read_status
File "C:\Python34\lib\socket.py", line 374, in readinto
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

so remote host disconnected my client apparently and returned me to the command prompt. I was able to restart polling manually with no further incidents. Could not identify the reason for the remote disconnect.

No warning on read only volume

Hello,
python3.3 dataserv-client --address XXXXXXXXXXXXXXXXXX --max_size 979789414400 --store_path /path/dataserv_client/bin/data/ poll
No warning when the file is on read only for the current user (rw-r--r-- root, current user: driveshare)
http://104.236.104.117/api/online -> show Height: 0
Thanks to fix it.

Registering and building in new client on the same machine disturbs an existing polling client

I had a 2TB store on E:/Storj which was polling.

Registered a new address in another CMD prompt on windows.
Starting a build on the new address, specifying a different store directory (G:/storj).

This then stopped the first client which was polling from being picked up on the server at : http://104.236.104.117/api/online
Although it continued to ping.

To fix it I had to start the client again.

Using windows 10.

Perhaps every command should require a build path and an address rather than remembering them somehow (Not sure how that works, but I am convinced the issue is it there is some shared data between the two running clients somewhere).

Rewards

I see with satisfaction that the snapshot will be taken in the next 24 hours.

We may be building, or we will have to be in pool ?

Thanks.

Change sort order on http://104.236.104.117/api/online

It would be great if the sort order could be changed so that the addresses with the largest Height are at the top.

http://104.236.104.117/api/online

OSX instructions

For info, here is how I installed dataserv-client on OSX:

brew install python3
rehash
pip3 install dataserv-client

(this presumes that brew is already installed.) Could someone add this to the readme?

build failed at last time

Hello,

At last time the build have failed.
Can you do the server call before start the build? (prevent network failed broke the build). Address not registred bug all the build and it's say after the build.
If reserve if forgot before build do it, if build if forgot before poll do it.

Cheers,

Thread the crypto (cpu) part and HDD write part for build

Hello,

The program seam alternate cpu usage and hdd write part. Maybe loop { compute ; write }
Can you thread the compute to transform your serial code into parallel code? And be able to use all cpu core?

Cheers,

Master Secret Reset

Master secret resets on rerun.
{"version": "2.0.0", "master_secret": null, "payout_address": "1JdEaubcM36ufmT64drdVsGu5SN65A3Z1L"}

Continue build for test group where it left off.

E.g if someone builds 200gb and wants to extend to 500gb, they should only have to build another 300gb.

Increase build command speed.

Currently it sets height everytime it generates a shard (takes longer then generating the shard). Add an option to set height every x shards (don't forget to always set the last time).

Possible do a binary search for resume point it case its continuing a build.

User setting fake height.

@fabe: someone is setting fake height again apparently: 1CounterpartyXXXXXXXXXXXXXXXUWLpVr | Last Seen: 2 second(s) | Height: 999999999999999999

heunland 2:35 PM

Used By Another Process Error

When running testing on Windows:

Resuming from height 1
Error
Traceback (most recent call last):
  File "C:\Users\super3\Code\dataserv-client\tests\test_builder.py", line 60, in test_builder_build
    bucket.build(self.store_path, cleanup=True)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\builder.py", line 89, in build
    rebuild=rebuild)
  File "C:\Users\super3\Code\dataserv-client\dataserv_client\builder.py", line 62, in generate_shard
    os.remove(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\super3\\AppData\\Local\\Temp\\tmpp11mbzg0\\7460b234c7de598b1e98224b7a3afb48d00c82f6bc5c0c8e0cc52efa9dd1dc64'

Wallet address validation

Validation of the wallet address when building, registering and polling commands are used.

upgrade does not work

sudo pip3 install dataserv-client --upgrade
Downloading/unpacking dataserv-client from https://pypi.python.org/packages/source/d/dataserv-client/dataserv-client-2.0.0.tar.gz#md5=b75fc5880377d255de7de8462f7b1693
Downloading dataserv-client-2.0.0.tar.gz
Running setup.py (path:/tmp/pip-build-bpaqoypm/dataserv-client/setup.py) egg_info for package dataserv-client
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip-build-bpaqoypm/dataserv-client/setup.py", line 35, in
install_requires=open("requirements.txt").readlines(),
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 17, in

File "/tmp/pip-build-bpaqoypm/dataserv-client/setup.py", line 35, in

install_requires=open("requirements.txt").readlines(),

FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-bpaqoypm/dataserv-client
Storing debug log for failure in /home/beulemann/.pip/pip.log

readme - version in setup part

$ sudo pip3 install dataserv-client==1.1.0

is this the right version?
or should it be 1.3.0?

Client doesn't complete expanded build with over 11TB max size

On Windows 10. I had a 11TB build which I tried to expand using first:

dataserv-client.exe --address <BITCOIN_ADDRESS> build --rebuild

That failed because of a server issue which wasn't responding a couple of days ago. So then I tried this instead:

dataserv-client.exe --store_path <PATH_TO_FOLDER> --max_size <MAX_DATA_SIZE_IN_BYTES> --address <BITCOIN_ADDRESS> build

This failed again when it reached about 67K Height. Below is the traceback from the app.

Traceback (most recent call last):
File "dataserv-client", line 24, in
File "E:\dataserv-client\dataserv_client\cli.py", line 128, in main
File "E:\dataserv-client\dataserv_client\api.py", line 130, in build
File "E:\dataserv-client\dataserv_client\builder.py", line 64, in build
File "E:\dataserv-client\dataserv_client\api.py", line 126, in on_generate_shard
File "E:\dataserv-client\dataserv_client\api.py", line 68, in _url_query
File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
return opener.open(url, data, timeout)
File "C:\Python34\lib\urllib\request.py", line 463, in open
response = self._open(req, data)
File "C:\Python34\lib\urllib\request.py", line 481, in _open
'_open', req)
File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
result = func(*args)
File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Python34\lib\urllib\request.py", line 1185, in do_open
r = h.getresponse()
File "C:\Python34\lib\http\client.py", line 1171, in getresponse
response.begin()
File "C:\Python34\lib\http\client.py", line 351, in begin
version, status, reason = self._read_status()
File "C:\Python34\lib\http\client.py", line 321, in _read_status
raise BadStatusLine(line)
http.client.BadStatusLine: ''

Expanding should be easier and faster in my opinion, if we need to scale to 1PB we need to make sure that users can Build and Expand faster than it is currently possible. I have been trying for 3 days to add more storage to my build unsuccessfully. The amount of time it takes to check existing shards is also too long, we should skip that process and get the client to only create new shards for the required expanded storage. Also having to fall back to 0 Height when expanding or rebuilding doesn't sound right as usually I'll be still polling when doing that, so starting from the current build size would be ideal then see the increases as the expansion progresses.

Sign auto updates.

From the chat:

nutterzuk [4:00 PM]
Just a quick question - how secure is the auto update?

nutterzuk [4:00 PM]
There is absolutely no risk someone manages to get everyones clients downloading a malicious update?

nutterzuk [4:03 PM]
Using some kinda certificate or signing perhaps?

shawn [4:04 PM]
yeah a multisig that we control needs to be added

nutterzuk [4:04 PM]
As in that's done already and needs to be used to sign an update?

nutterzuk [4:04 PM]
Or that needs to be added as in yeah, it's not implemented yet

shawn [4:04 PM]
make an issue for it

shawn [4:04 PM]
i don't think thats implemented

stack overflow when passing relative path to --store_path

For example:
dataserv-client --address=$BTCADDR --max_size="$SIZE" --store_path=data build

Traceback (most recent call last):
  File "/home/storj/venv/bin/dataserv-client", line 9, in <module>
    cli.main(sys.argv[1:])  # pragma: no cover
  File "/home/storj/venv/lib/python3.4/site-packages/dataserv_client/cli.py", line 121, in main
    store_path=arguments.pop("store_path"),
  File "/home/storj/venv/lib/python3.4/site-packages/dataserv_client/api.py", line 30, in __init__
    self._mkdir_recursive(store_path)
  File "/home/storj/venv/lib/python3.4/site-packages/dataserv_client/api.py", line 35, in _mkdir_recursive
    self._mkdir_recursive(sub_path)
.........
  File "/home/storj/venv/lib/python3.4/site-packages/dataserv_client/api.py", line 33, in _mkdir_recursive
    sub_path = os.path.dirname(path)
  File "/home/storj/venv/lib/python3.4/posixpath.py", line 147, in dirname
    sep = _get_sep(p)
  File "/home/storj/venv/lib/python3.4/posixpath.py", line 39, in _get_sep
    if isinstance(path, bytes):

Speed Up Unit Tests

Tests on travis are taking up to 17min to complete. Faster test would improve development feedback and its likely this can be sped up dramatically with just minimal effort.

Ubuntu installation failed

The setup instructions for Ubuntu yielded the following error:

$ sudo pip3 install dataserv-client
Downloading/unpacking dataserv-client
  Downloading dataserv-client-1.2.1.tar.gz
  Running setup.py (path:/tmp/pip-build-2p4_wfs8/dataserv-client/setup.py) egg_info for package dataserv-client
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip-build-2p4_wfs8/dataserv-client/setup.py", line 4, in <module>
        from esky.bdist_esky import Executable
    ImportError: No module named 'esky'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip-build-2p4_wfs8/dataserv-client/setup.py", line 4, in <module>

    from esky.bdist_esky import Executable

ImportError: No module named 'esky'

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-2p4_wfs8/dataserv-client
Storing debug log for failure in /home/chrisvdb/.pip/pip.log

dataserv-client does not automatically switch to poll mode after building has finished

The client does not automatically switch to polling and does not continue to poll after building has finished. The client stops after building has finished and appears offline.
The client should automatically switch to polling mode and continute to poll after building has finished.

When the client updates the payout address, it should also be updated server side.

We have no server api call to update the payout address!

Cannot install

Error during install:

sudo pip3 install dataserv-client
Downloading/unpacking dataserv-client
Downloading dataserv-client-1.2.1.tar.gz
Running setup.py (path:/tmp/pip_build_root/dataserv-client/setup.py) egg_info for package dataserv-client
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/dataserv-client/setup.py", line 4, in
from esky.bdist_esky import Executable
ImportError: No module named 'esky'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 17, in

UBUNTU 14.10

Replace IP with URL

Replace the IP of the status server to http://status.driveshare.org

Fix documentation to include authentication.

tl;dr use --wif=WALLET_PRIVATE_KEY instead of --address (this is now derived from the given wif)

Improve query performance by using urllib3

URLLib is slow. It will open and close a port to the web server for every http transaction. URLLib3 has the ability to leave a port open. If the server can handle the load, leaving the port open for new http transactions speeds things up considerably.

Reassessing seeds of files that have already been built is painfully slow and I think it is because of all of the overhead of opening and closing the connection to the server. URLLib3 could possibly speed that up considerably at the cost of server load.

Client should not require access to SJCX private key

The server needs a means of authenticating miners to ensure that:

A remote attacker cannot impersonate other miners to reduce the victim miner's height
A attacker with access to the victim miner's network traffic (MiTM) cannot modify the miner's messages to the server to tamper with their contracts / reduce their height

Recent potential solutions have been based on authenticating with the user's SJCX private key. This poses several problems:

If the miner must sign every message, it requires the miner to have access to the SJCX private key, which means that if an attacker compromises a miner, they can control the user's SJCX wallet
If the miner signs a single message, an attacker can perform replay attacks on a victim miner's messages if they have access to their network traffic (unless the server stores complicated state to try to prevent this)
Solutions that authenticate based on SJCX public key mean that a single user with multiple miners must use a separate SJCX address for each miner
Solutions that depend on copying private keys or signing a message with SJCX key are undesirable for usability as they require the user to go through a series of manual steps involving their SJCX private keys to start mining

I propose the following solution:

On first run, miners automatically generate a new keypair ("mining private key, mining public key") to authenticate to the server / peers. This is invisible to the user.
- If the user needs to migrate their contracts to a separate server, we provide a documented process of how to move their mining keypair along with their storage data
Users specify a SJCX payout address for their miner, which they do not need to prove ownership of (mining on behalf of other users is a permitted scenario)
- (depending on implementation difficulty) There is not a fixed mapping between mining keys and SJCX payout address. Users can dynamically change their SJCX payout address (e.g. "Every Tuesday, I point my miners at SJCX address 1N7a.... to donate a share of my profits to them.")
Miners authenticate each message to the server by signing the message with a timestamp and their mining private key.
Multiple miners can specify the same SJCX payout address

1.1.0 client did not restart after scheduled maintenance for upgrade. External drive dismounted

Maintenance for upgrade was announced, and clients on v1.1.0 should have automatically restarted dataserv-client polling. I have 3 builds of heights 4000, 8000 and 2800 polling on an external 2 TB HDD (ADATA HD710) attached to a Windows 7 desktop (HP Z620 workstation) Version of client running is 1.1.0.

Polling stopped during the upgrade, and then did not restart automatically as expected. Instead, the machine showed an error window stating:

"dataserv-client.exe -There is no hard drive.
There is no disk in the drive. Insert a disk in the drive \Device\Harddisk1\DR1."

When checking the list of drives in windows explorer, the external drive I was using in fact did not appear in the list of mounted drives, even though its indicator light was still on so the disk was working.

The command window of my first build gave the following traceback:

This build had gone back to the command prompt awaiting the next command.

The other 2 builds showed this traceback:

and the cursor just kept blinking but did not go back to command prompt until I forced the process to abort.

I was able to refresh the explorer so that the external drive was remounted, after which I restarted polling on the 3 builds without problem.

Power settings on this machine are so that the harddrives never go to sleep or shut down, nor that the monitor shuts off.

http.client.BadStatusLine

jawed 8:04 AM Had an exception last night with 1.3.0:
Traceback (most recent call last):
File "dataserv-client", line 24, in
File "E:\dataserv-client\dataserv_client\cli.py", line 128, in main
File "E:\dataserv-client\dataserv_client\api.py", line 116, in poll
File "E:\dataserv-client\dataserv_client\api.py", line 104, in ping
File "E:\dataserv-client\dataserv_client\api.py", line 68, in _url_query
File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
File "C:\Python34\lib\urllib\request.py", line 463, in open
File "C:\Python34\lib\urllib\request.py", line 481, in _open
File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
File "C:\Python34\lib\urllib\request.py", line 1210, in http_open
File "C:\Python34\lib\urllib\request.py", line 1185, in do_open
File "C:\Python34\lib\http\client.py", line 1171, in getresponse
File "C:\Python34\lib\http\client.py", line 351, in begin
File "C:\Python34\lib\http\client.py", line 321, in _read_status
http.client.BadStatusLine: ''
Windows client

Add Support for Config Encryption

If --password=SOMETHING_SECURE is pass the config should be encrypted if it is not already.
If it is done on the first command an unencrypted config should never hit the drive.
An additional command change_password should be added so the user can change it.
It may be worth have encryption required and printing the generated password to be used (would also ensure secure passwords).

The main code needed is already implemented in dataserv_client/encryptedio

Need last verification test for winners after making result shapshot to verify its height

There is strange winner 1Hh6A1v4ZBKs8vLywm4FiGRPpvPfRcbZqS, it appears before snapshot at the right time with more than 10000 height (6-10 hours after dissapearing from active list)!
It will be fair to check and confirm winners result with some verification tool, that the winner have the exact and valid files on his storage.

Crash/Bug

Traceback (most recent call last):
File "C:\Python34\lib\urllib\request.py", line 1182, in do_open
tunnel_headers[proxy_auth_hdr] = headers[proxy_auth_hdr]
File "C:\Python34\lib\http\client.py", line 1088, in request
def request(self, method, url, body=None, headers={}):
File "C:\Python34\lib\http\client.py", line 1126, in _send_request
# default charset of iso-8859-1.
File "C:\Python34\lib\http\client.py", line 1084, in endheaders
else:
File "C:\Python34\lib\http\client.py", line 922, in _send_output
msg += message_body
File "C:\Python34\lib\http\client.py", line 857, in send
if self.sock is None:
File "C:\Python34\lib\http\client.py", line 834, in connect
"""Connect to the host and port specified in init."""
File "C:\Python34\lib\socket.py", line 512, in create_connection

File "C:\Python34\lib\socket.py", line 503, in create_connection
except error as _:
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a
period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Z:\share_space\dataserv-client\dataserv_client\api.py", line 50, in _querry
File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
"""
File "C:\Python34\lib\urllib\request.py", line 463, in open
return response
File "C:\Python34\lib\urllib\request.py", line 481, in _open
if proto in ('http', 'https'):
File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
else:
File "C:\Python34\lib\urllib\request.py", line 1210, in http_open

File "C:\Python34\lib\urllib\request.py", line 1184, in do_open
# server.
urllib.error.URLError: <urlopen error [WinError 10060] A connection attempt failed because the connected party did not p
roperly respond after a period of time, or established connection failed because connected host has failed to respond>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "dataserv-client.py", line 9, in
File "Z:\share_space\dataserv-client\dataserv_client\cli.py", line 123, in main
File "Z:\share_space\dataserv-client\dataserv_client\api.py", line 95, in poll
File "Z:\share_space\dataserv-client\dataserv_client\api.py", line 83, in ping
File "Z:\share_space\dataserv-client\dataserv_client\api.py", line 68, in _querry
dataserv_client.exceptions.ConnectionError: Could not connect to server http://IPHERE!

25 TB limit not working

2 Addr with height 2626000 ~ 320TB are online. The 25 TB limit is not working.

Input of max_size - support for other measurements

Eg an argument of 2TB rather than however many bytes, perhaps support for b,kb,mb,gb,tb.

Probably low priority as this will be fixed with a GUI, but it would be good in case anyone else is creating third party GUIs of any sort.

Missing MSVCR100.dll on Windows 10

Extracted dataserv-client into C:\Users\jooize\Desktop\dataserv-client and ran dataserv-client <walletaddress> register in cmd.exe.

dataserv-client throws an error saying:

The program can't start because MSVCR100.dll is missing from your computer. Try reinstalling the program to fix this problem.

Running Windows 10 Pro 64-bit (upgraded from 8.1 Pro). Tried installing Microsoft Visual C++ 2010 64-bit with no result.

height stuck at 4, no error

Hi,
I built 750gb share and dataserv-client is running fine on my linux server, pinging and appearing in http://104.236.104.117/api/online for a week.
But the "height" reported is not increasing and keeps stuck at "4". Is it normal or does this mean I'm not storing anything?
Thanks

Traceback (most recent call last):
 File "/usr/local/bin/dataserv-client", line 24, in <module>
   cli.main(sys.argv[1:])
 File "/usr/local/lib/python3.4/site-packages/dataserv_client/cli.py", line 128, in main
   return getattr(client, command_name)(**arguments)
 File "/usr/local/lib/python3.4/site-packages/dataserv_client/api.py", line 116, in poll
   self.ping()
 File "/usr/local/lib/python3.4/site-packages/dataserv_client/api.py", line 104, in ping
   return self._url_query("/api/ping/{0}".format(self.address))
 File "/usr/local/lib/python3.4/site-packages/dataserv_client/api.py", line 68, in _url_query
   response = urllib.request.urlopen(self.url + api_call)
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161, in urlopen
   return opener.open(url, data, timeout)
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463, in open
   response = self._open(req, data)
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481, in _open
   '_open', req)
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441, in _call_chain
   result = func(*args)
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210, in http_open
   return self.do_open(http.client.HTTPConnection, req)
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185, in do_open
   r = h.getresponse()
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171, in getresponse
   response.begin()
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351, in begin
   version, status, reason = self._read_status()
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313, in _read_status
   line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
 File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374, in readinto
   return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer

storjold / dataserv-client Goto Github PK

dataserv-client's People

Contributors

Stargazers

Watchers

Forkers

dataserv-client's Issues

Recommend Projects

Recommend Topics

Recommend Org