Giter VIP home page Giter VIP logo

fpnd's Introduction

fpnd - FreePN Network Daemon and Tools

image

GitHub tag (latest SemVer, including pre-release)

GitHub CI Build Status

Codecov

image

Announcements

7/31/21

FreePN's infrastructure servers are currently offline. If you wish to use FreePN, you will need to configure your own infrastructure, as detailed in this repo's documentation.

What FreePN is (and is not)

FreePN is a set of open source (see FLOSS definition) privacy tools for an improved online user experience (and yes, there's even an ISO standard for that).

The FreePN network daemon (fpnd) is a P2P implementation of a distributed virtual private network (dVPN) that creates an anonymous "cloud" of peers where each peer is both a client node and an exit node. Peers are randomly connected on startup and reconnected to new (random) peers as needed.

The FreePN desktop user interface (freepn-gtk3-tray) currently supports GTK3-based XDG-compliant desktop environments, eg, Gnome, Unity, XFCE, and derivatives.

FreePN is not a full VPN solution (eg, openvpn or vpnc) and does not require setup of any pre-shared keys or certs. Traffic over FreePN network links is always encrypted, however, since each network link is independent, the traffic must be decrypted as it passes out of each peer host. When running in "peer" mode, each peer is assumed to be an untrusted host; when running in "adhoc" mode, the hosts can be assumed to be trusted hosts (as they belong to the user).

Prototype design limitations

  • we only route www (http and https) and dns (optional) traffic
  • traffic routing supports IPv4 only (IPv6 is dropped unless configured not to)
  • DNS privacy depends entirely on your DNS config
  • the most common local-only DNS config is not routable out-of-the-box
  • it takes you to make the changes to stop DNS privacy leaks

Docs for FreePN tools

Important

If you already installed one of the early 0.9.x or older releases, please upgrade to the latest release (as a bonus you will get an stunnel upgrade plus several lemony-fresh bug fixes). See issue #88 for more details.

Quick Start

We assume you're reading this because you want to take back some control over your online privacy, so the first thing you should do is read the short DNS Privacy doc and then evaluate your own DNS Setup and decide which secure DNS providers suit your needs. If you've already done that (or you're just super impatient ;) then proceed with the following to install the software.

Note

You will need one of the supported Linux distributions below with at least Python 3.6 in order to run this software. On Gentoo you should also have a 5.x kernel.

Tested Linux distributions

We test mainly on Gentoo, Ubuntu LTS (Bionic, Focal, Groovy) and Kali Linux. However, the default Python on Xenial lacks sufficient asyncio support, so will not work out-of-the-box.

Pre-install

Before you can install any FreePN packages, you'll need to add the required package repository or overlay.

For all Ubuntu series, make sure you have the gpg and add-apt-repository commands installed and then add the PPA:

$ sudo apt-get install -y software-properties-common
$ sudo add-apt-repository -y -s ppa:nerdboy/embedded

Note that on kali you will need to edit the file created under /etc/apt/sources.list.d for the PPA and change the series name to focal, then run sudo apt-get update again.

For Gentoo or derivatives based on Portage, first install the portage overlay.

Create a repos.conf file for the overlay and place the file in the /etc/portage/repos.conf directory. Run:

$ sudo nano /etc/portage/repos.conf/freepn-overlay.conf

and add the following content to the new file:

[freepn-overlay]

# Various python ebuilds for FreePN
# Maintainer: nerdboy <[email protected]>

location = /var/db/repos/freepn-overlay
sync-type = git
sync-uri = https://github.com/freepn/freepn-overlay.git
priority = 50
auto-sync = yes

Adjust the path in the location field as needed, then save and exit nano.

Run the following command to sync the repo:

$ sudo emaint sync --repo freepn-overlay

Install

After following the pre-install setup, use the appropriate package manager to install the package for your distro:

  • Gentoo - sudo emerge freepn-gtk3-tray
  • Ubuntu - sudo apt-get install freepn-gtk3-indicator

Then add your local <username> to the fpnd group:

  • sudo usermod -aG fpnd <username>

Note

Replace <username> with your actual login ID when you run the above command, then log out and log back in again.

Check your group memberships with the id command:

$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),115(netdev),118(lxd),995(fpnd)

If you really don't want/need a desktop, you can still install the network component:

  • Gentoo - sudo emerge net-misc/fpnd
  • Ubuntu - sudo apt-get install python3-fpnd

Post-Install Updates

Once installed, the above packages should update normally along with any other updated system packages (both @world and dependencies). It's very important to keep all your system packages up to date, and especially important when running an alpha release (eg, fpnd-0.9.x) since (software) interfaces may (and probably will) break between releases. Both Gentoo and Ubuntu have ways to notify you when you have updates waiting (or at least when your package tree is getting stale).

  • Gentoo
    • Use your preferred tools to keep the overlay synced and check for updates frequently; using the "live" ebuild is left as an exercise for the reader...
  • Ubuntu
    • Use the Software Updater GUI -- or --
    • Use apt|apt-get from a terminal window.

When packages update on Ubuntu, you might see something like "The following packages have been kept back:" followed by one or more package names. This might happen with FreePN packages if new dependencies are added between releases. If you see python3-fpnd in the list of packages kept back, you can try one of the following commands (instead of the typical sudo apt-get upgrade) to resolve dependencies. Start with the first one:

  • sudo apt-get --with-new-pkgs upgrade
  • sudo apt upgrade
  • sudo apt-get install python3-fpnd

Config Settings

Although there is currently no user config file for the GUI component, the fpnd package installs the system settings file /etc/fpnd.ini. Most of the entries are there to provide a consistent set of defaults for different runtime environments, but some are intended for the end-user to adjust when appropriate.

DNS settings:

  • route_dns: Default is False; only set to True if you've configured your global (plain-text) DNS settings to use an external/public DNS server, eg Cloudflare (or you already have secure DNS in place)
  • private_dns_only: Default is False; only set to True when you're already running a secure local DNS resolver, eg stubby

Misc settings:

  • drop_ip6: Default is True; you can disable this if you need local IPv6 working, but you should have your own ipv6 firewall rules in place
  • default_iface: Default is None; you should only need this if you have multiple network interfaces with active routes, then you should set this to the interface name you want FreePN to use
  • debug: Default is True; set this to False for slightly smaller log files

GUI Usage

Select FreePN Tray Control from the Applications View or the Internet menu in your desktop of choice, eg, Gnome, Unity, XFCE, etc. You can also run it from an X terminal to get some debug output.

$ freepn-gtk3-indicator

Some screenshots

Gentoo Gnome profile (X1 Carbon)

Gentoo Gnome profile (X1 Carbon)

Ubuntu Bionic Desktop (Pinebook OG)

Ubuntu Bionic Desktop (Pinebook OG)

Mate Bionic Desktop (pi-top[1] Rpi 3B Rev 1.2)

Mate Bionic Desktop (pi-top[1] Rpi 3B Rev 1.2)

fpnd's People

Contributors

sarnold avatar xyzib avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fpnd's Issues

script is in "failed" state on reboot / startup in bionic

When you start up a box with fpnd installed (on bionic ubuntu 18.04 LTS), the output of systemctl status fpnd.service is 'failed'. In order for the service run properly, it needs to be stopped and then started again. The service should not be in 'failed' state on startup; it should require manually being started, not start on boot.

doesn't connect even after restarting

Installed FreePN, restarted my laptop, stopped the application/service and started again and it still doesn't seem to work, there's no difference in the response and it still doesn't connect.
FreePn_issue
then the WAITING status is what is repeated, there's no difference in the output
sys_specs

stale processes and message errors with early 0.9.x releases

Mainly seen on Bionic, along with issues related to stunnel and openssl versions. This leaves one or more instances of fpnd.py still running after stopping and starting resulting in a small flood of announce/config messages.

Fixed in https://github.com/freepn/fpnd/releases/tag/0.9.5

So the first thing to do is make sure you have the latest packages; this should include upgrades to both python3-fpnd and stunnel4. So, open a terminal and do:

$ sudo apt-get update
$ sudo apt-get upgrade

As of today you should end up with these new versions:

$ dpkg -l python3-fpnd
ii  python3-fpnd   0.9.5-0   <= important part
$ dpkg -l stunnel4
ii  stunnel4       3:5.55-2   <= important part

Now stop the stunnel service; it will start again when you start the fpnd service:

$ sudo systemctl stop stunnel4.service

Then stop the fpnd service and check for any stale processes; after opening a terminal window:

$ sudo systemctl stop fpnd.service
$ ps ax | grep py

If you see fpnd.py in the output, you'll need to manually kill those processes before starting the service again:

$ ps ax | grep py
  747 ?        Ssl    0:00 /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers
 1726 ?        Sl     0:02 /usr/bin/python3 /usr/share/system-config-printer/applet.py
17701 ?        Sl     0:02 /usr/bin/python3 /usr/share/system-config-printer/applet.py
26810 ?        Sl     0:00 /usr/bin/python3 /usr/lib/fpnd/fpnd.py start
26847 pts/2    S+     0:00 grep --color=auto py

Note the process ID 26810 and kill it:

$ sudo kill 26810

It might be stubborn, so check again and use -s 9 if it ignores the first kill:

$ sudo kill -s 9 26810

Now you should at least get a clean startup, but there may still be a lot of bad nodes in use until more people upgrade...

Pop_OS 20.04 Stuck on Config

OS: Pop_OS 20.04
CPU: i5 4570
GPU: RX 480

I started the service, it has only connected once, but every other time I tried to enable it, it has been stuck on Config for hours on end. (I have left it overnight, and it has never connected once.

Ubuntu 20.04 Stuck On "Waiting"

Hi I'm newcomers, when I try to run freepn from terminal it says WAITING for more than 2 minutes. Is there anything I can provide to solve this issue so it could be more clear?.

I tried all on the FAQ, restart, start - stop the service but nothing works (the GUI app also doesn't work).
a photo

  • OS: Ubuntu 20.04
  • Kernel: 5.4.0-51-generic

datrie build error in python nightly (on travis-ci)

check out what the upstream devs did, try the "fix":

  creating build/temp.linux-x86_64-3.10
  creating build/temp.linux-x86_64-3.10/src
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -I/usr/include/datrie -I/home/travis/build/freepn/fpnd/.tox/py310/include -I/opt/python/3.10-dev/include/python3.10 -c src/cdatrie.c -o build/temp.linux-x86_64-3.10/src/cdatrie.o
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -I/usr/include/datrie -I/home/travis/build/freepn/fpnd/.tox/py310/include -I/opt/python/3.10-dev/include/python3.10 -c src/datrie.c -o build/temp.linux-x86_64-3.10/src/datrie.o
  src/datrie.c: In function โ€˜__Pyx_PyList_Appendโ€™:
  src/datrie.c:1789:23: error: lvalue required as left operand of assignment
           Py_SIZE(list) = len+1;
                         ^
  src/datrie.c: In function โ€˜__Pyx_ListComp_Appendโ€™:
  src/datrie.c:1816:23: error: lvalue required as left operand of assignment
           Py_SIZE(list) = len+1;
                         ^

Site feedback

I think this is issue and for that is github, i was linked to https://github.com/freepn from https://www.freepn.org/ and was unable to find better repo than this one to ask.

The web page https://www.freepn.org and this repo README.md does not explain how the software is working/principle of working so people can understand what it do. Instead visitor is supplied a Download link. Seems weird. Visitor should see 1) what it is, 2) how it works 3) download link
I am unable to easily see the point 2 on that page. For the beginning text explanation and if hard to explain in words, maybe image illustration. Thank you

Blocking on the Waiting value

I don't know if I'm the one using the FreePN tool in the wrong way. That said, when I start fpnd.service, at first everything is fine. After the third iteration of waiting, there is an error, then the iterations of waitings loop. Is this normal ?

~$ systemctl start fpnd.service
~$ tail -f /run/fpnd/fpnd.state
WAITING
WAITING
ERROR
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
WAITING
...

Need Debian Support

Can you please make a Debian-based project ??? Or is there any way to use it ubuntu package in Debian??

systemd kill causes undefined behavior in cleanup handler

No problems using openrc with cgroups, but in spite of an almost identical man page description, letting systemd kill the daemon process on stop results in undefined behavior. The "fix" is setting

KillType=none

in the unit file and letting ExecStop kill it manually.

  • does NOT shut down correctly:
    • systemd 237-3ubuntu10.41, fpnd 0.8.5, bionic
    • systemd 245.5-3, fpnd 0.8.5, Kali/rolling (focal ppa pkgs)

Correct log snippet:

2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +123: INFO [5526] CLEANUP: shutting down fpn0
2020-06-02 22:00:18 UTC node_tools.network_funcs.do_net_cmd +302: INFO [5526] net cmd fpn0-down.sh result: Success
2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +126: INFO [5526] CLEANUP: leaving network ID: edf70dc89ab3fb28
2020-06-02 22:00:18 UTC node_tools.node_funcs.run_ztcli_cmd +273: DEBUG [5526] got data: 200 leave OK
2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +128: DEBUG [5526] CLEANUP: action leave returned: 200 leave OK
2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +123: INFO [5526] CLEANUP: shutting down fpn1
2020-06-02 22:00:18 UTC node_tools.network_funcs.do_net_cmd +302: INFO [5526] net cmd fpn1-down.sh result: Success
2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +126: INFO [5526] CLEANUP: leaving network ID: edf70dc89ac82488
2020-06-02 22:00:18 UTC node_tools.node_funcs.run_ztcli_cmd +273: DEBUG [5526] got data: 200 leave OK
2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +128: DEBUG [5526] CLEANUP: action leave returned: 200 leave OK
2020-06-02 22:00:18 UTC node_tools.node_funcs.run_moon_cmd +345: DEBUG [5526] deorbit on 9790eaaea1 result: 200 deorbit OK
2020-06-02 22:00:18 UTC root.call +289: DEBUG [5526] * Client will send payload: [{'ses': None, 'tok': None, 'ver': 1, 'met': 'offline', 'arg': ('ddfd7368e6',), 'ref': '5633c6b4-2fa5-4c27-b14d-56b39f9a18b4'}]
2020-06-02 22:00:18 UTC node_tools.node_funcs.do_cleanup +133: DEBUG [5526] CLEANUP: offline reply: [{'ref': '5633c6b4-2fa5-4c27-b14d-56b39f9a18b4', 'result': 'ddfd7368e6'}]

Broken log snippet with an extra log command to verify the script:

2020-06-02 21:48:00 UTC daemon.stop +231: INFO [12914] Stopping...
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:00 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +123: INFO [12428] CLEANUP: shutting down fpn0
2020-06-02 21:48:01 UTC node_tools.node_funcs.do_cleanup +125: INFO [12428] CLEANUP: got net cmd: ['/usr/lib/fpnd/fpn0-down.sh']

non-legacy iptables backend change breaks net scripts

At some point after bionic the default iptables (command) backend changed from legacy to nf_tables that results in the current net scripts failing to setup the correct masq ports.

Log errors:

2020-06-02 19:13:00 UTC node_tools.network_funcs.do_net_cmd +308: ERROR [2912] net cmd fpn0-setup.sh msg: iptables v1.8.4 (nf_tables): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.8.4 (nf_tables): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.8.4 (nf_tables): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.8.4 (nf_tables): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
4 warnings/errors

Current manual workaround:

As documented here (https://github.com/kubernetes/website/commit/9cdaf4ead1f4c514a6a2582ab00e01efad331a0d/) the affected user node can be "fixed" by setting the default back to legacy using the following command:

update-alternatives --set iptables /usr/sbin/iptables-legacy

followed by a restart of the fpnd service.

Found in:

# cat /etc/os-release 
PRETTY_NAME="Kali GNU/Linux Rolling"
NAME="Kali GNU/Linux"
ID=kali
VERSION="2020.2"
VERSION_ID="2020.2"
VERSION_CODENAME="kali-rolling"
ID_LIKE=debian
ANSI_COLOR="1;31"
HOME_URL="https://www.kali.org/"
SUPPORT_URL="https://forums.kali.org/"
BUG_REPORT_URL="https://bugs.kali.org/"

msg_responder needs knowledge of the IP address associated with each msg

This is required for both (internal) node ID/addr matching and f2b regex matching in the logs. Currently the msg_responder is a separate (and very simple) daemon that only does the following:

  • validates msg size and hex format

  • logs the msg (node ID) or an error

    • if valid msg, return response str (msg), else log error msg

The msg_responder uses the responder service interface so the (source) IP addr is not currently visible for logging. From the fpnd side we can see both the IP addr and node ID in the peer data, BUT we currently allow "late" node msgs with no extra validation.

LEAF peer with no address causes exception in state runner

We've seen these peers before, but apparently did not remain in this state long enough to cause trouble:

# zerotier-cli peers
200 peers
<ztaddr>   <ver>  <role> <lat> <link> <lastTX> <lastRX> <path>
34e0a5e174 -      PLANET   137 DIRECT 7192     2047     147.75.92.2/9993
3a46f1bf30 -      PLANET    26 DIRECT 3389     2156     185.180.13.82/9993
778cde7190 -      PLANET   481 DIRECT 12198    12121    103.195.103.66/9993
8ca1de52b4 -      LEAF      -1 RELAY
992fcf1db7 -      PLANET   180 DIRECT 7192     7019     195.181.173.159/9993
ddfd7368e6 1.4.6  LEAF       3 DIRECT 3389     3389     192.168.0.67/9993

"cold" exit node startup and missing cfg ref

Where "cold" startup means the node has not (recently) orbited the root; this issue has not been seen on a node restart after missing the initial cfg ref. If it happens, the root never gets the cfg msg from the ctlr node, because the ctlr node never sees the node msg, so the staging queue is never filled, so there's no bootstrap (ie, no networks are created or configured).

  • mbr node: gets the first msg ref, but fails to get the cfg msg (so keeps trying)
  • root node: sees the cfg request but never responds
  • ctlr node: never sees the pub msg

Erratic service file behavior with systemd >= 246

The systemd service file log shows repeated complaints about KillMode=none (with nothing useful in the 246 man page) and the service "stop" fails miserably. Tweaking the service file does change the behavior, ie, without changes, there is no more signal to tell the daemon to shutdown and run cleanup handlers, and tweaking has so far only resulted in even more erratic shutdown behavior (eg, one cleanup func gets run 5 times in a row but others never run). This means that fpnd can no longer send the offline msg or run the down scripts to cleanup routes and iptables rules.

avahi-daemon.conf requires interface whitelisting to ignore vpn interfaces

The default (as installed) avahi-daemon config allows all interfaces and denies none so it will attempt to bind to any interface it sees. The "fix" is editing the avahi-daemon.conf file to explicitly whitelist the interfaces avahi should listen on. This fix should also resolve the problem described in issue #39.

If you're not using avahi (or don't know what it is) and you're concerned about your privacy/security, then you can either a) remove the packages or b) disable and mask the service. To remove the primary avahi packages run:

sudo apt-get remove avahi-autoipd avahi-daemon avahi-utils

To keep the packages but stop any avahi services from running if you're using systemd (either Ubuntu or Gentoo) you'll need to run the following commands:

sudo systemctl disable avahi-daemon.socket avahi-daemon.service
sudo systemctl mask avahi-daemon.socket avahi-daemon.service

otherwise, for openrc run the following:

sudo /etc/init.d/avahi-daemon stop
sudo rc-update del avahi-daemon

If you are using avahi and have not modified the default config, you should uncomment the allow-interfaces line in the above config file and add your ethernet and/or wifi devices after the = (if more than one device, use a comma to separate the interface names).

Open the file in your favorite editor and change this:

use-ipv6=yes
#allow-interfaces=eth0
#deny-interfaces=eth1

to something like:

use-ipv6=yes
allow-interfaces=eth0,wlan0
#deny-interfaces=eth1

Do substitute the interface names for your system, since it may or may not be using predictable interface names. Check the names with ifconfig:

$ ifconfig 
enx000ec68929d2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet XXX.XXX.XXX.XXX  netmask 255.255.255.0  broadcast XXX.XXX.XXX.XXX
        inet6 fe80::20e:dead:beef:29d2  prefixlen 64  scopeid 0x20<link>
        ether 00:0e:c6:89:29:d2  txqueuelen 1000  (Ethernet)
        RX packets 14688  bytes 748393 (748.3 KB)
        RX errors 0  dropped 2572  overruns 0  frame 0
        TX packets 10444  bytes 2048942 (2.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

If ifconfig is not installed, use the ip command:

$ sudo ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether b8:27:eb:04:24:0e brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.XXX.XXX/24 brd XXX.XXX.XXX.XXX scope global dynamic eth0
       valid_lft 2019sec preferred_lft 2019sec
    inet6 fe80::dead:beef:fe04:240e/64 scope link 
       valid_lft forever preferred_lft forever

and use the displayed device name instead of eth0.

error in adding the PPA from Launchpad

Here is the process in ubuntu 20.10

sudo add-apt-repository -y -s ppa:nerdboy/embedded
Note: PPA publishes dbgsym
You need to add 'main/debug' component to install the ddebs,
but apt update will print warning if the PPA has no ddebs
Repository: 'deb http://ppa.launchpad.net/nerdboy/embedded/ubuntu/ groovy main'
Description:
Packages for beaglebone, raspberrypi, allwinner, ubuntu touch, etc (even amd64 and x86 too)
More info: https://launchpad.net/~nerdboy/+archive/ubuntu/embedded
Adding repository.
Found existing deb entry in /etc/apt/sources.list.d/nerdboy-ubuntu-embedded-groovy.list
Adding deb entry to /etc/apt/sources.list.d/nerdboy-ubuntu-embedded-groovy.list
Found existing deb-src entry in /etc/apt/sources.list.d/nerdboy-ubuntu-embedded-groovy.list
Adding deb-src entry to /etc/apt/sources.list.d/nerdboy-ubuntu-embedded-groovy.list
Adding key to /etc/apt/trusted.gpg.d/nerdboy-ubuntu-embedded.gpg with fingerprint 88EF78CEDAA422E7BFF547A041113ED57774ED19
Hit:1 http://security.ubuntu.com/ubuntu groovy-security InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu groovy InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu groovy-updates InRelease
Hit:4 http://us.archive.ubuntu.com/ubuntu groovy-backports InRelease
Ign:5 http://ppa.launchpad.net/nerdboy/embedded/ubuntu groovy InRelease
Err:6 http://ppa.launchpad.net/nerdboy/embedded/ubuntu groovy Release
404 Not Found [IP: 91.189.95.83 80]
Reading package lists... Done
E: The repository 'http://ppa.launchpad.net/nerdboy/embedded/ubuntu groovy Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

(something usable in the github CI chain) .deb packaging automation

So what we more or less want to accomplish here to start is this:

Adding the debian/* (ubuntu/, fedora/, etc.) folders and the docker setup to each (dependency) pkg's respective git repo (so nanoservice, python-nanomsg, etc.).

Probably makes sense to pick off one or two of these dependencies to start, then fork the dep repo and make a branch to work on.


Things for us are currently in the following state:

Ubuntu PPA pkgs:

  1. per-package updates to debian/* for version/rev-bumps are manual and local-only (once tested locally, pkg builds for a single series are automated after uploading signed source pkg to ppa)
  2. PPA base support is mainly for Ubuntu series

Debian pkgs:

  1. local/manual builds only

All .deb pkgs:

  1. multiple build profiles required for dbg symbols and ZT ctlr build

Examples of existing tools found on github so far that we could use:

  1. scripts to process pkgs prior to uploading to build host: https://github.com/64studio/debuild

  2. (self-hosted) replacement for launchpad ppa: https://github.com/debomatic/debomatic

also plenty of one-pkg-only docker build setups.

inconsistent cachedir on bionic

2020-10-20 21:48:06 UTC node_tools.data_funcs.state_check +130: INFO [1833] Node online with id: deadbeef00
2020-10-20 21:48:06 UTC node_tools.helper_funcs.get_cachedir +171: WARNING [1833] Falling back to temp dir: /tmp

zt network data can be incomplete until routes appear

When networks are created on a ZT node there is a finite amount of time (on the order of seconds) between when the client "sees" the new network and when the network routing data is fully populated. The end result is a possible exception when nodestate.py collects the netStatus data from the cache; note the size of the route list is '0' in the log snippet below:

2020-01-03 01:59:28 UTC node_tools.cache_funcs.get_endpoint_data +52: DEBUG [29823] Entering get_endpoint_data with key_str: net
2020-01-03 01:59:28 UTC node_tools.cache_funcs.get_endpoint_data +60: DEBUG [29823] Appending data for key: net-500000000000000
2020-01-03 01:59:28 UTC node_tools.cache_funcs.get_endpoint_data +63: DEBUG [29823] Leaving get_endpoint_data
2020-01-03 01:59:28 UTC node_tools.cache_funcs.get_net_status +76: DEBUG [29823] net-500000000000000 route list size: 0
2020-01-03 01:59:28 UTC node_tools.helper_funcs.update_state +115: ERROR [29823] update_state exception: list index out of range
2020-01-03 01:59:28 UTC node_tools.data_funcs.do_logstats +32: DEBUG [29823] 13 items currently in cache.
2020-01-03 01:59:29 UTC node_tools.data_funcs.do_logstats +33: DEBUG [29823] Cache items: ['node-500000000000000', 'peer-500000000000000', 'peer-500000000000001', 'peer-500000000000002', 'peer-500000000000003', 'peer-5
00000000000004', 'peer-500000000000005', 'peer-500000000000006', 'moon-500000000000000', 'nstate-500000000000000', 'mstate-500000000000000', 'utc-time', 'net-500000000000000']
2020-01-03 01:59:29 UTC node_tools.data_funcs.do_logstats +31: DEBUG [29823] Leaving update_runner
2020-01-03 01:59:29 UTC node_tools.data_funcs.do_logstats +32: DEBUG [29823] 13 items currently in cache.
2020-01-03 01:59:29 UTC node_tools.data_funcs.do_logstats +33: DEBUG [29823] Cache items: ['node-500000000000000', 'peer-500000000000000', 'peer-500000000000001', 'peer-500000000000002', 'peer-500000000000003', 'peer-5
00000000000004', 'peer-500000000000005', 'peer-500000000000006', 'moon-500000000000000', 'nstate-500000000000000', 'mstate-500000000000000', 'utc-time', 'net-500000000000000']
2020-01-03 01:59:29 UTC node_tools.data_funcs.wrapper +64: INFO [29823] Get data result: ENODATA

avahi-autoipd link-local conflicts with zerotier interfaces

Yet another cause of netscript/routing failures, if avahi-daemon is running with the default bionic config it will break routing, although you can't see this until the interfaces are brought up:

root@arm:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.0.254   0.0.0.0         UG    203    0        0 eth0
link-local      0.0.0.0         255.255.0.0     U     210    0        0 ztevwxwlya
link-local      0.0.0.0         255.255.0.0     U     211    0        0 ztevw4x5ik
172.16.1.4      0.0.0.0         255.255.255.252 U     0      0        0 ztevw4x5ik
172.16.1.16     0.0.0.0         255.255.255.252 U     0      0        0 ztevwxwlya
192.168.0.0     0.0.0.0         255.255.255.0   U     203    0        0 eth0

And in the fpnd.log you can see this with VERBOSE enabled:

2020-06-11 02:16:42 UTC node_tools.data_funcs.wrapper +99: INFO [25480] Get data result: OK
2020-06-11 02:16:42 UTC node_tools.data_funcs.wrapper +106: DEBUG [25480] New cache time is: 2020-06-11 02:16:42 UTC
2020-06-11 02:16:42 UTC node_tools.helper_funcs.log_fpn_state +210: INFO [25480] fpn0 is UP
2020-06-11 02:16:42 UTC node_tools.helper_funcs.log_fpn_state +210: INFO [25480] fpn1 is UP
2020-06-11 02:16:42 UTC node_tools.helper_funcs.run_event_handlers +294: DEBUG [25480] running net_change_handler for iface fpn0 and state True
2020-06-11 02:16:42 UTC node_tools.helper_funcs.net_change_handler +233: DEBUG [25480] run_net_cmd using cmd: ['/usr/lib/fpnd/fpn0-setup.sh']
2020-06-11 02:16:42 UTC node_tools.helper_funcs.run_event_handlers +294: DEBUG [25480] running net_change_handler for iface fpn1 and state True
2020-06-11 02:16:42 UTC node_tools.helper_funcs.net_change_handler +233: DEBUG [25480] run_net_cmd using cmd: ['/usr/lib/fpnd/fpn1-setup.sh']
2020-06-11 02:16:42 UTC node_tools.helper_funcs.run_event_handlers +297: DEBUG [25480] running net_id_handler for iface fpn_id0 and net id edf70dc89ac63d90
2020-06-11 02:16:42 UTC node_tools.helper_funcs.net_id_handler +256: DEBUG [25480] Added network id edf70dc89ac63d90 to net_q
2020-06-11 02:16:42 UTC node_tools.helper_funcs.run_event_handlers +297: DEBUG [25480] running net_id_handler for iface fpn_id1 and net id edf70dc89a76e79a
2020-06-11 02:16:43 UTC node_tools.helper_funcs.net_id_handler +256: DEBUG [25480] Added network id edf70dc89a76e79a to net_q
2020-06-11 02:16:44 UTC schedule.Job.run +537: INFO [25480] Running job Job(interval=1, unit=seconds, do=run_net_cmd, args=(['/usr/lib/fpnd/fpn0-setup.sh'],), kwargs={})
2020-06-11 02:16:46 UTC node_tools.network_funcs.do_net_cmd +302: INFO [25480] net cmd fpn0-setup.sh result: Checking iptables binary...
Checking kernel rp_filter setting...
  RP needs garlic filter...
Checking network...
  Found edf70dc89ac63d90
Using FPN0 ID: edf70dc89ac63d90
Checking for FPN routing table...
  Inserting routing table...
Checking FPN network settings...
 "allowGlobal": true,

Found these devices and parameters:
  FPN interface: ztevw4x5ik
  FPN address: 172.16.1.6
  FPN gateway: 172.16.1.5
  FPN network id: edf70dc89ac63d90

  INET interface: eth0
  INET address: 192.168.0.158

Success
2020-06-11 02:16:46 UTC node_tools.network_funcs.run_net_cmd +268: DEBUG [25480] run net cmd ['/usr/lib/fpnd/fpn0-setup.sh'] returned tuple: (True, b'Checking iptables binary...\nChecking kernel rp_filter setting...\n 
 RP needs garlic filter...\nChecking network...\n  Found edf70dc89ac63d90\nUsing FPN0 ID: edf70dc89ac63d90\nChecking for FPN routing table...\n  Inserting routing table...\nChecking FPN network settings...\n "allowGlob
al": true,\n\nFound these devices and parameters:\n  FPN interface: ztevw4x5ik\n  FPN address: 172.16.1.6\n  FPN gateway: 172.16.1.5\n  FPN network id: edf70dc89ac63d90\n\n  INET interface: eth0\n  INET address: 192.16
8.0.158\n\nSuccess\n', 0)
2020-06-11 02:16:46 UTC node_tools.sched_funcs.wrapper +97: DEBUG [25480] JOB: Job(interval=1, unit=seconds, do=run_net_cmd, args=(['/usr/lib/fpnd/fpn0-setup.sh'],), kwargs={}) claims success: (True, b'Checking iptable
s binary...\nChecking kernel rp_filter setting...\n  RP needs garlic filter...\nChecking network...\n  Found edf70dc89ac63d90\nUsing FPN0 ID: edf70dc89ac63d90\nChecking for FPN routing table...\n  Inserting routing tab
le...\nChecking FPN network settings...\n "allowGlobal": true,\n\nFound these devices and parameters:\n  FPN interface: ztevw4x5ik\n  FPN address: 172.16.1.6\n  FPN gateway: 172.16.1.5\n  FPN network id: edf70dc89ac63d
90\n\n  INET interface: eth0\n  INET address: 192.168.0.158\n\nSuccess\n', 0)
2020-06-11 02:16:46 UTC schedule.Job.run +537: INFO [25480] Running job Job(interval=1, unit=seconds, do=run_net_cmd, args=(['/usr/lib/fpnd/fpn1-setup.sh'],), kwargs={})
2020-06-11 02:16:46 UTC node_tools.network_funcs.do_net_cmd +308: ERROR [25480] net cmd fpn1-setup.sh msg: Checking iptables binary...
FPN networks found, parsing network IDs...
Checking network...
  No gateway found
Checking network...
  Found edf70dc89a76e79a
Using FPN1 ID: edf70dc89a76e79a
Found these devices and parameters:
  FPN SRC interface: ztevwxwlya
  FPN SRC address: 172.16.1.17
  FPN SRC network: 169.254.0.0/16
172.16.1.16/30
  FPN SRC network id: edf70dc89a76e79a

  INET interface: eth0
  INET address: 192.168.0.158
  INET gateway: 192.168.0.254
Allow forwarding for FPN source traffic
net.ipv4.ip_forward = 1
iptables v1.6.1: host/network `169.254.0.0/16
172.16.1.16' not found
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.6.1: host/network `169.254.0.0/16
172.16.1.16' not found
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.6.1: host/network `169.254.0.0/16
172.16.1.16' not found
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.6.1: host/network `169.254.0.0/16
172.16.1.16' not found
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.6.1: host/network `169.254.0.0/16
172.16.1.16' not found
Try `iptables -h' or 'iptables --help' for more information.

5 warnings/errors
2020-06-11 02:16:46 UTC node_tools.network_funcs.run_net_cmd +268: DEBUG [25480] run net cmd ['/usr/lib/fpnd/fpn1-setup.sh'] returned tuple: (False, b'', 1)

missing kernel module(s) cause net script failures

If the user is missing any of the required kernel modules for iptables or packet marking, the fpn_setup (net) scripts will fail. We can only check for kernel config (ebuild) or set a pkg dependency (.deb) which still needs to be implemented. This is a (user node) show-stopper BUT we can't do anything to "fix" it so the label is Watch Item instead.

New use-case for "adhoc" mode feature

Mainly a road-warrior (reverse) freepn tunnel for using your own exit node (pre-configured in your own "trusted" location, ie, home or office, etc). Proposed for "alpha-1" release while full p2p mode is not yet complete.

automatic redirect of port 80 to 443 (external sites) not compatible with "tight" routing

First off, best practices say do not do automatic redirects like that on your web site. Second, fpnd network links are purposely "tight" so we do not take over the default route on a user host. Instead, we route only a few protocols (ie, http/https and dns/dot) between specific ports, which does not take into account working around upstream sites doing such redirects. This does not affect https requests, only http requests to a site doing automatic redirects of 80=>443. Using http with a site that only offers http should still work (and is still horribly insecure).

Feature/Enhancement: Raspi

Good day

Can I assist in helping add Raspi CLi support?
use case: headless Raspi as a VPN routing node to add more nodes cheaply

Is there anything specific that would need to be done? Python should be fine on Armv7, but I haven't looked closer to see if there are any platform specific network calls.

node msg errors and openssl upgrades

This is actually two things happening at once in the node cfg messaging over stunnel:

  • stunnel messages using IPv6 instead of IPv4 (depends on local configuration)
  • stunnel too old for current openssl version (mainly on bionic)

The "infrastructure" node running stunnel needs both stunnel tuning in the ssl layer and better filtering for allowing only IPv4 addressing. Bionic nodes also need a newer stunnel package to work with its (already) upgraded openssl libs.

add-apt-repository command fails on some bionic server installs

For some as-yet unknown reason, the above command fails on one bionic RPi server instance:

# apt-add-repository ppa:nerdboy/embedded
Traceback (most recent call last):
  File "/usr/bin/apt-add-repository", line 107, in <module>
    sp = SoftwareProperties(options=options)
  File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 117, in __init__
    self.reload_sourceslist()
  File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 612, in reload_sourceslist
    self.distro.get_sources(self.sourceslist)
  File "/usr/lib/python3/dist-packages/aptsources/distro.py", line 146, in get_sources
    self.get_mirrors()
  File "/usr/lib/python3/dist-packages/aptsources/distro.py", line 461, in get_mirrors
    self, mirror_template="http://%s.archive.ubuntu.com/ubuntu/")
  File "/usr/lib/python3/dist-packages/aptsources/distro.py", line 170, in get_mirrors
    et = ElementTree(file=fname)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 557, in __init__
    self.parse(file)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 597, in parse
    self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 600, column 36

apt-get update / Domain resolution issue on Ubuntu 20.04 in Chromium / Firefox browsers

When you type in a domain to the URL bar in the format: xxx.com, it doesn't resolve / times out while trying to load, however https://xxx.com (clicking a link to https://xxx.com also resolves immediately.

Connected to IP address: 172.104.16.201 in the USA.

Additionally, the apt-get update command seems to not be working while connected to the VPN.

Could possibly have to do with this piece of networking functionality on the browser side:
https://thegeekpage.com/disable-web-browser-automatically-adding-www-url/

apt-get could be using different ports / a different kind of traffic that's unsupported.

de-wedging fails for nodes who never route

Occurs on startup if upstream neighbor node cannot route at all; if the wedge message is sent while the network configuration is still in progress, then the message is ignored by the network controller.

fpnd and freepn-gtk3-tray should be bundled in .deb (PPA) packages

There should be 2 installation options for fpnd:

  • fpnd-only (this is just the 'base' fpnd configuration option - useful for things like raspi's, servers, etc.)

  • fpnd-and-freepn-gtk3-tray (this would be a 'bundled' installation option, such that both the UI and it's underlying packages are installed at the same time - useful (and expected behavior) for most desktop-environments / desktop Linux users)

ensure net script cleanup - filter, nat, route

Repeated testing with the same devices shows plenty of network cruft from previous fpn networks; update scripts for better cleanup, possibly add custom chain names. Note this caused one of the bootstrap nodes to stop routing properly after a "fresh" start of fpnd (all test nodes had been running for several days without restarting the devices).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.