Errors when country VPN servers are unavailable that mess up console text display about vpngate-with-proxy HOT 11 CLOSED

Hi @phd21

For 1.), do you have a screen shot of the terminal?

For 2.) 3.) 4.) , As python2 is going to be obsoleted, I'm working on toward the python3 version of the program. It now can save favourite servers to the local and even allow you add your custom .ovpn file too. Automation mode is also implemented, so it can do the boring stuff in the background. The UI is now separated from the main script which results in simpler and thus faster to build any GUI front-end on top of the main script. But as the GUI is not my main concern right now, it will not be done anytime sooner.

A good program should automatically pick the right python version. In case it doesn't, just simply invoke the right python version for it, such as python3 [some script or cmd]. There are python2 and python3 symlink in almost Unix/Linux, when you type python --version it is in fact symlink to python2. To set the default to python3, just make a symlink to it. To know where all these symlink and python executable files are, whereis python

Best regards

from vpngate-with-proxy.

Hi Dragon2fly,

1.) Screenshot is included in this. FYI: I currently "dual boot" Linux Mint KDE 17.3 (based on Ubuntu 14.04) and Linux Mint KDE 18.1 (based on Ubuntu 16.04). Linux Mint 17.x is incredible and super stable, everything just works perfectly (usually), but a lot of software developers are producing new updates and products that only work with Ubuntu 16.04+. I have been testing and using Linux Mint 18.1 since it came out, and with the new Plasma 5.8 or higher, it now works well and is stable too; But there are a lot of applications that have not been updated for Ubuntu 16.04+, and some others are "quirky".

I get no errors with "vpngate-with-proxy" in Linux Mint 17.x even when the server is not found, it just says "nothing to do". I do get errors in Linux Mint 18.x, see screenshot. I do not know why? I have both Python 2 and Python 3 installed in both systems.

Re: #2, #3, & #4: Your updates and improvements look and sound really great. I am looking forward to them.

I will be happy to test any updates, just let me know.

I have been thinking about learning Python (and "QT") development in Linux for awhile now, can you please recommend your favorite IDE's, editors, and or learning tutorials for me? Are there any particular Python add-ons and or programs that you recommend as well, like "wxWidgets", wxGlade, Glade, etc...? My email is [email protected]. Thank you ...

5.) Since I believe it is in everyone's best interest to use a VPN (free or not), I think it is important to have the Local and Public IP addresses easily available in the system tray panel, and or on the desktop. There are not that many options for this. It is pretty easy to find out your local IP address, not so with your Public IP address. That is another nice thing to have in your application, maybe included as part of the "mouse over" pop-up, or in the right click menu?

After researching this for people on the Linux Mint forum and for myself, one really nice "indicator" application I found is called "indicator-ip". But, for some reason it does not display the current public or local ip address in the system tray panel, nor does its icon even show up, yet there is a "space" where the icon should be, and it can be right clicked to display the various current Local and Public IP addresses. I have contacted the developer to see what may be done for Linux Mint users.

If you have any comments and or recommendations on this, please let me know...

Link to Linux Mint forum post.
https://forums.linuxmint.com/viewtopic.php?f=53&t=251802&sid=317555751851a97f71d08ea08a1229b9

Link to the "bovender/indicator-ip" Post (forked from sentientwaffle/unity-ip-indicator)
https://github.com/bovender/indicator-ip/issues

Original article - How to have your machine’s IP displayed in Ubuntu’s system tray
https://www.maketecheasier.com/display-ip-address-system-tray-ubuntu/

Best regards to you and yours,
Phil
phd21

from vpngate-with-proxy.

Hi @Dragon2fly
i second the error reported by @phd21 on Ubuntu 16.04 (and variants). my system information is 4.10.0-32-generic #36~16.04.1-Ubuntu SMP Wed Aug 9 09:19:02 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux and Python vesrions are 2.7.12 and 3.5.2 . basically, an Exception in thread is encountered when executing ./run tui command. i've updated config.ini with latest mirrors as well. stack trace turns out to be as following:

Exception in thread Thread-4: 
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner                          
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "vpnproxy_tui.py", line 231, in get_csv
    vpn_data = requests.get(gate, proxies=proxy, timeout=3).text.replace('\r', '')
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py", line 426, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))

looks like it all starts with a ConnectionError. HTTP request error 104 aka 'Connection reset by peer' results from a firewall filtering all traffic to a server. using ./run cli gives:

fetching data
using gate:  http://www.vpngate.net
('Connection aborted.', error(104, 'Connection reset by peer'))
Connection to gate http://www.vpngate.net failed

using gate:  http://218-40-237-91.ppp.bbiq.jp:44624
HTTPConnectionPool(host='218-40-237-91.ppp.bbiq.jp', port=44624): Read timed out.
Connection to gate http://218-40-237-91.ppp.bbiq.jp:44624 failed

using gate:  http://39.116.238.42:58443
HTTPConnectionPool(host='39.116.238.42', port=58443): Read timed out.
Connection to gate http://39.116.238.42:58443 failed

using gate:  http://117.16.138.29:11361
HTTPConnectionPool(host='117.16.138.29', port=11361): Read timed out.
Connection to gate http://117.16.138.29:11361 failed

using gate:  http://147.46.138.56:21084
HTTPConnectionPool(host='147.46.138.56', port=21084): Read timed out.
Connection to gate http://147.46.138.56:21084 failed

using gate:  http://h86-62-74-246.ln.rinet.ru:17396
HTTPConnectionPool(host='h86-62-74-246.ln.rinet.ru', port=17396): Read timed out.
Connection to gate http://h86-62-74-246.ln.rinet.ru:17396 failed

summing up, vpngate.net is not accessible and connection times out for the mirror servers.

from vpngate-with-proxy.

Hi @phd21 and @cod3g3nki,

I can't replicate the errors with my ubuntu 16.04.1. There was no Traceback printed out.
Also since the tui and the cli have the same code that handle the exception, if there was a problem, the tui and the cli should both print out the Traceback.

Anyway, I changed the tui so that it catches ConnectionError and ConnectionTimeout separately. Could you clone it and test again?

from vpngate-with-proxy.

I just cloned the repo and tried again with following configuration:

[proxy]
use_proxy = no
address = 
port = 
ip = 

[sort]
key = score

[country_filter]
country = all
port = all
score = all

[DNS_leak]
fix_dns = yes
dns = 8.8.8.8, 84.200.69.80, 208.67.222.222

[openvpn]
verbose = yes

[mirror]
url = http://p76ed4cd5.tokynt01.ap.so-net.ne.jp:16169, http://103.1.249.67:29858, http://211.217.242.42:3230, http://zp018093.ppp.dion.ne.jp:36205, http://218-40-237-91.ppp.bbiq.jp:44624, http://39.116.238.42:58443, http://117.16.138.29:11361, http://147.46.138.56:21084, http://h86-62-74-246.ln.rinet.ru:17396

added five working mirror servers to the list:

 Current VPNGate's mirrors:
  0 http://p76ed4cd5.tokynt01.ap.so-net.ne.jp:16169
  1 http://103.1.249.67:29858
  2 http://211.217.242.42:3230
  3 http://zp018093.ppp.dion.ne.jp:36205
  4 http://218-40-237-91.ppp.bbiq.jp:44624
  5 http://39.116.238.42:58443
  6 http://117.16.138.29:11361
  7 http://147.46.138.56:21084
  8 http://h86-62-74-246.ln.rinet.ru:17396

Using tui still raises ConnectionError.

Exception in thread Thread-4: 
Traceback (most recent call last):                                                                
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner                          
    self.run()                                                                                    
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "vpnproxy_tui.py", line 231, in get_csv
    vpn_data = requests.get(gate, proxies=proxy, timeout=3).text.replace('\r', '')
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py", line 426, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))

screen shot of the error is as follows:

trying to refresh at this stage gives Invalid: please wait for last refresh to be finished.

using cli has not changed either. here it's worth mentioning that mirrors listed in default config.iniare not reachable, while mirrors listed on vpngate.net are reachable but time out (even in browser) if you try to access csv list at /api/iphone/:

fetching data
using gate:  http://www.vpngate.net
('Connection aborted.', error(104, 'Connection reset by peer'))
Connection to gate http://www.vpngate.net failed

using gate:  http://p76ed4cd5.tokynt01.ap.so-net.ne.jp:16169
HTTPConnectionPool(host='p76ed4cd5.tokynt01.ap.so-net.ne.jp', port=16169): Max retries exceeded with url: /api/iphone/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f68dc854dd0>: Failed to establish a new connection: [Errno 113] No route to host',))
Connection to gate http://p76ed4cd5.tokynt01.ap.so-net.ne.jp:16169 failed

using gate:  http://103.1.249.67:29858
HTTPConnectionPool(host='103.1.249.67', port=29858): Max retries exceeded with url: /api/iphone/ (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7f68dc854e50>, 'Connection to 103.1.249.67 timed out. (connect timeout=3)'))
Connection to gate http://103.1.249.67:29858 failed

using gate:  http://211.217.242.42:3230
HTTPConnectionPool(host='211.217.242.42', port=3230): Max retries exceeded with url: /api/iphone/ (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7f68dc854d50>, 'Connection to 211.217.242.42 timed out. (connect timeout=3)'))
Connection to gate http://211.217.242.42:3230 failed

using gate:  http://zp018093.ppp.dion.ne.jp:36205
HTTPConnectionPool(host='zp018093.ppp.dion.ne.jp', port=36205): Max retries exceeded with url: /api/iphone/ (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7f68dc854e10>, 'Connection to zp018093.ppp.dion.ne.jp timed out. (connect timeout=3)'))
Connection to gate http://zp018093.ppp.dion.ne.jp:36205 failed

using gate:  http://218-40-237-91.ppp.bbiq.jp:44624
HTTPConnectionPool(host='218-40-237-91.ppp.bbiq.jp', port=44624): Read timed out.
Connection to gate http://218-40-237-91.ppp.bbiq.jp:44624 failed

using gate:  http://39.116.238.42:58443
HTTPConnectionPool(host='39.116.238.42', port=58443): Read timed out.
Connection to gate http://39.116.238.42:58443 failed

using gate:  http://117.16.138.29:11361
HTTPConnectionPool(host='117.16.138.29', port=11361): Read timed out.
Connection to gate http://117.16.138.29:11361 failed

using gate:  http://147.46.138.56:21084
HTTPConnectionPool(host='147.46.138.56', port=21084): Read timed out.
Connection to gate http://147.46.138.56:21084 failed

using gate:  http://h86-62-74-246.ln.rinet.ru:17396
HTTPConnectionPool(host='h86-62-74-246.ln.rinet.ru', port=17396): Read timed out.
Connection to gate http://h86-62-74-246.ln.rinet.ru:17396 failed

Failed to get VPN servers data
Check your network setting and proxy

for the sake of clarity, i must tell you that i'm in China and vpngate website is blocked by government firewall.

from vpngate-with-proxy.

Hi @cod3g3nki,

for the sake of clarity, i must tell you that i'm in China and vpngate website is blocked by government firewall.
I heard that they have blocked all kind of VPN too. Is OpenVPN protocol still working? Is SoftetherVPN protocol still working?

Thank for your detailed report. The traceback is still not shown up in my test so I think I need to debug on your computer. Could you open vpnproxy_tui.py and replace the function get_csv in line 266 with the following:

if an exception is not caught, the thread will not return so we catch all exceptions and see what it is

    def get_csv(self, url, queue, proxy={}):
        self.messages['debug'].appendleft(' using gate: ' + url)
        # self.messages['debug'].appendleft(str(proxy))
        self.messages['debug'].appendleft('Get inside get_csv')
        try:
            gate = url + '/api/iphone/'
            self.messages['debug'].appendleft('Before fetching data')
            vpn_data = requests.get(gate, proxies=proxy, timeout=3).text.replace('\r', '')
            self.messages['debug'].appendleft('After fetching data')
            servers = [line.split(',') for line in vpn_data.split('\n')]
            if servers[0][0] == '*vpn_servers':
                vpndict = {s[0]: Server(s) for s in servers[2:] if len(s) > 1}
                self.messages['debug'].appendleft(' gate ' + url + ': success')
                queue.put((1, vpndict))
            else:
                self.messages['debug'].appendleft(' Received WRONG data file')
                self.messages['debug'].appendleft(' Connection to gate ' + url + ' failed')
                self.messages['debug'].appendleft(vpn_data)
                queue.put((0, {}))
        except Exception as e:
            self.messages['debug'].appendleft(str(e))
            self.messages['debug'].appendleft(' Connection to gate ' + url + ' failed')
            queue.put((0, {}))

from vpngate-with-proxy.

Hi Dragon2fly,

I did not get a chance to try this out until now, but I replaced the code you suggested to "cod3g3nki" in your current revision, and it seems to be working fine now, nice... I do not get a message saying "no servers", but thankfully I don't get a screen full of code, it just goes back to start screen...

Can you please add to the "tui" interface the ip addresses of the available servers in the list? Also, when it says connected to server (VPN connection established) and lists the IP address, is it also possible to list the first 2 DNS addresses? I like that the "DNS Fix" defaults to yes too.

Also, is there a list of the 2 digit country codes in use, are they standard codes like in the link below?
http://www.nationsonline.org/oneworld/country_code_list.htm

Is it better to hit "q" for quit or "Ctrl+C", or does it not matter?

Thank you.

Best regards to you and yours,
Phil
phd21

from vpngate-with-proxy.

Hi @phd21

I'm glad that it helps. But I still need to investigate further why it behaves differently on different flavor.

I don't have much free time right now and tweaking the "tui" is quite time consuming so would not do that anytime sooner. As for DNS, one must know what he want and their setting! Beside, other process can change it, turn on another internet adapter also reset it. Thus it is not necessary and 100% reliable that you see is the current DNS. You better check it by cat /etc/resolv.conf or for absolute proof https://www.dnsleaktest.com/

2 digit country code is standard.

It does not matter! Even shutdown the computer without turning it off is not matter. It will perform cleaning up when SIGINT or SIGTERM is received.

Your sincerely

from vpngate-with-proxy.

Hi @phd21 and @cod3g3nki

The problem in newer os like Mint18 is caused by python-pip, a tool that is used to install other dependencies. The recent version of pip has it own version of requests module, which is different from the one that our program using. When checking for pip (by import pip), it inserts it own modules' path to the head of sys.path. Subsequently, import requests looks at pip path first, found and use Pip’s module instead of our right one. -> the exception class is different and never been caught, leaving a mess in terminal of unhandled exception!

It is very tricky and surprising to find out that pip was the problem. Solution is not to import pip. Now the program uses shell to check for pip indirectly and the problem has gone away.

from vpngate-with-proxy.

Hi Dragon2fly,

Thanks for posting this. Glad you are still checking and updating this wonderful application.

I am not sure what this means? Does this mean there is an updated version that we can download and use, or that the instructions for installing have changed, and if so how exactly, or both?

Best regards to you and yours,
Phil
phd21

from vpngate-with-proxy.

Hi @phd21,

The developers of pip and requests are trying to deal with this problem too. But their solution is only temporary and they said that it will break when the next version comes out. So we don't use their patch, stick to the way we are used to and find a way around the problem.

So, the instruction for installing is not changed. This program it self has taken care of the change to the library it uses. Just clone it and run, thing should be done automatically as usual.

Have a good day.

from vpngate-with-proxy.