adulau / forban Goto Github PK

I opened ports 12555 and forban still shows nothing. My router connects up to the provider.
What hosts might it find through this port? Does it use the global DHT and is supposed to find others globally? Or, otherwise, what are the other hosts? How can they

I think, you should explain this in details here http://www.foo.be/forban/ to prevent such questions from being asked.

Also, my router doesn't have such choices source any, destination any. In such case, you should explain that users should set 0.0.0.0.

It would be nice to have a ./bin/forbanctl --foreground so as to watch what is going on.

Following a discussion with Krunch at OHM2013 about the attacks against the opportunistic mode in Forban, he came with a simple script using the Linux sparse file to always present files bigger than any nodes.

#!/bin/bash
set -eu

# flibustier - overwrite all the files shared over Forban
#
# This reads the Forban index files and create/extend
# all the files locally such that they are a bit larger than
# what is being shared by other nodes.
# Since Forban only looks at the file size to decide whether
# to download a file (bigger is better), this should overwrite
# everything that is being shared (at least for the instances
# running in opportunistic mode).
#
# Copyright © 2013 Krunch <[email protected]>
# This work is free. You can redistribute it and/or modify it under the
# terms of the Do What The Fuck You Want To Public License, Version 2,
# as published by Sam Hocevar. See http://www.wtfpl.net/ for more details.

# where Forban is running
forbandir='/usr/src/Forban'

sharedir="$forbandir/var/share/"
lootdir="$forbandir/var/loot/"

function debug {
    echo "$@"
}

# looking for $lootdir/$uuid/cache/forban/index
find "$lootdir" -mindepth 4 -maxdepth 4 -name index | while read index
do
    # reading stuff like this:
    #   file name,42
    rev $index | sed 's/,/ /' | while read size filename
    do
        size=$(echo $size | rev)
        filename="$(echo "$filename" | rev)"

        # i don't really want to think about what happens
        # if we also mess with the forban metadata
        if [[ "$filename" =~ ^forban/ ]]
        then
            continue
        fi

        debug "Advertised with size $size: $filename"

        localsize=$(stat -c '%s' "$sharedir/$filename" || true)
        if [ -n "$localsize" ] && [ "$localsize" -ge "$size" ]
        then
            continue
        fi

        # get the size in MiB, rounded down
        sizemb=$(((size / (1024*1024))))
        # bump to the next MiB
        sizemb=$(((sizemb + 1)))

        debug "Advertising with size ${sizemb}M"

        filedir="$(dirname "$filename")"
        if [ ! -d "$sharedir/$filedir" ]
        then
            mkdir -p "$sharedir/$filedir"
        fi
        dd "of=$sharedir/$filename" bs=1M seek=$sizemb count=0
    done

Hi,
the current implementation downloads work like normal browser downloads. Is there a resume functionality- I assume not.
Even the detection if the remote file is bigger, than my local one and then start to download is not very optimal.

Under heavy network-load or in disconnect situation the "download one file completly" is not very useful.
What do you think of a DHT-torrent implementation for optimistic mode (and serving it)? (optional feature)

Matthias

Somehow Forban does not find its ipv4 interface anymore.
You can access Forban with your browser as usual. It only shows up v6 , no v4

It only sends ipv6 anouncements, too. But no ipv4 announcements.

00:20:30.103977 IP6 fe80::fad1:11ff:febd:575c.60618 > ff02::1.12555: UDP, length 113
00:21:30.367215 IP6 fe80::fad1:11ff:febd:575c.34404 > ff02::1.12555: UDP, length 113

root@piratebox:~# ifconfig
br-lan Link encap:Ethernet HWaddr F8:D1:11:BD:57:5C
inet addr:192.168.56.10 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::fad1:11ff:febd:575c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2517 errors:0 dropped:2 overruns:0 frame:0
TX packets:1985 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:166242 (162.3 KiB) TX bytes:271136 (264.7 KiB)

eth0 Link encap:Ethernet HWaddr F8:D1:11:BD:57:5C
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2560 errors:0 dropped:41 overruns:0 frame:0
TX packets:1985 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:204624 (199.8 KiB) TX bytes:271136 (264.7 KiB)
Interrupt:4

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:291 errors:0 dropped:0 overruns:0 frame:0
TX packets:291 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:24854 (24.2 KiB) TX bytes:24854 (24.2 KiB)

mon.wlan0 Link encap:UNSPEC HWaddr F8-D1-11-BD-57-5C-00-48-00-00-00-00-00-00-00-00
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:52609 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:10311911 (9.8 MiB) TX bytes:0 (0.0 B)

wlan0 Link encap:Ethernet HWaddr F8:D1:11:BD:57:5C
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:201 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:0 (0.0 B) TX bytes:24061 (23.4 KiB)

Hi Alexandre,
I'm currently packing the forban package for Pirateboxes on OpenWRT.
On my prototype I'm running Forban located at the piratebox-folder
/opt/piratebox/forban

Python is, piratebox typically, installed under
/usr/local/

So the libs are located at /usr/local/lib

Fine. At first I had to slowdown your startUp because these multiple python threads pushes the MR3020 into a reboot :(
That wasn't this big.

Now I'm encountering a problem, which I can't solve (I don't have python skills :( ). At first, I'running Forban in "shared" mode.

On startup I'm getting the following Messages:

forban_announce is starting with pid:
(pid)
Traceback (most recent call last):
File "/opt/piratebox/forban/bin/forban_announce.py", line 66, in
import announce
File "/opt/piratebox/forban/lib/announce.py", line 29., in
import hmac
EOFError: EOF read where object expected
...
forban_share has the same issues with hmac

I think, it can't load the libary hmac. hmac exists in /usr/local/lib/python2.6
Copieng it stupidly over (for testing purpose) doesn't fix it either.

Another python script is running fine without modifications (droopy and shoutbox-service).

Any ideas?

Thank you
kind regards
Matthias

Hi,
I need an option for placing the pid files into another place on folder.
As I mentioned on the other thread, I wanted to place the /var folder to /tmp , that it gets removed after reboot. This seems to cause other problems.

Please, can u please create an option, where the pid files should be located?

Thank you.

Matthias,
btw: remove the blank space on "indexrebuild" option (I ran into an error because I only removed the # ;) ) it's an noob-failure point ;)

Following a feedback from a user, should we add empty directories into the index? or should we provide a way to add a hidden file for allowing sharing empty directories.

Hi,
is it possible to configure a interface Forban only work with?
Currently, It is working on all given interfaces- but running i.e. on a router requires to work only on specific interfaces.

Interface should be given with command line and/or config-file.

Best regards
Matthias

Apparently this software uses wget. How about you add --continue to the wget command spo that partially downlaoded files can resume from where they stopped if for some reason the have stopped.

Or it won't display the css and image file that you put in /forban.

»› Traceback (most recent call last):
File "/Users/thomas/bin/forban/bin/forban_announce.py", line 127, in
forbanindex.build()
File "/Users/thomas/bin/forban/lib/index.py", line 57, in build
f = open (workingloc,"w")
IOError: [Errno 2] No such file or directory: '/Users/thomas/Volatile/share/forban/index.79109'
Traceback (most recent call last):
File "/Users/thomas/bin/forban/bin/forban_opportunistic.py", line 128, in
allindex.build()
File "/Users/thomas/bin/forban/lib/index.py", line 57, in build
f = open (workingloc,"w")
IOError: [Errno 2] No such file or directory: '/Users/thomas/Volatile/share/forban/index.79111'

»› git diff
diff --git a/bin/forban_discover.py b/bin/forban_discover.py
index 58050ef..3f87838 100644
--- a/bin/forban_discover.py
+++ b/bin/forban_discover.py
@@ -34,6 +34,8 @@ try:
disable_ipv6 = config.get('global','disabled_ipv6')
except ConfigParser.NoOptionError:
disable_ipv6 = 0
+except ConfigParser.NoSectionError:

• disable_ipv6 = 0

try:
forbanpath = config.get('global','path')

Hi,
sorry, me again :)

I've done a basic mesh-network setup with a Piratebox running forban and my eePC joining the mesh and running forban.
I'm using B.A.T.M.A.N. , so I have a mesh0 interface (which stands for the wlan interface used) and a bat0 Interface, which gets an IP-Adress and is the valid interface for reaching other boxes.

I assigned Box1 192.168.78.1
I assigned Box2 192.168.78.2

Ping works, Traffic on bat0 showes up.

Remote access on :12555 works fine.

My eePC shows up only "access v6" (no link)
My Box only v4 (link working)

I can't see any broadcasts on packet dumps. Bash-Forban on eePC only monitors nothing. :(
Seems it can't see itself, too :(

Maybe it is a "test-setup problem"? Maybe forban can't figure out where to broadcast & listen?

Thank you so much.
Matthias

edit: Sidenote: Wired access server <--> OpenWRT-Piratebox worked VERY fine

Single line install copy and paste into terminal add to READme

git clone git://github.com/adulau/Forban.git && cd Forban && ./bin/forbanctl start && cd ./var/share && touch sharedfile.txt && echo "dafdasF" >> sharedfile.txt && sleep 4 && open http://127.0.0.1:12555/

failed:
... Git Clone with git clone [email protected]:adulau/Forban.git

Hi,
the browse function should please provide an ordered file list.

Thank you
Matthias

3h hours after started Forban, It doesn't discover itself anymore.
There are no announcement sent through set network either.

Logfile still shows up a working forban_announce thread:

1970-01-01 03:58:53,567 - forban_announce - DEBUG - Index rebuilt
1970-01-01 03:58:54,648 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;a2690654-6187-4ab9-9995-bf24c59b6e4e;hmac;880521dc5ca02b3beb9266168d19ab2bf336560d
1970-01-01 03:59:54,990 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;a2690654-6187-4ab9-9995-bf24c59b6e4e;hmac;880521dc5ca02b3beb9266168d19ab2bf336560d

The following threads are running (shared mode):
2078 root 8288 S python /opt/forban/bin/forban_discover.py
2079 root 9584 S python /opt/forban/bin/forban_announce.py
2080 root 38812 S python /opt/forban/bin/forban_share.py

Interfaces:

root@piratebox:/opt/forban/var/log# ifconfig
br-lan Link encap:Ethernet HWaddr F8:D1:11:BD:57:5C
inet addr:192.168.56.10 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::fad1:11ff:febd:575c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5610 errors:0 dropped:46 overruns:0 frame:0
TX packets:3354 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:584295 (570.6 KiB) TX bytes:608845 (594.5 KiB)

eth0 Link encap:Ethernet HWaddr F8:D1:11:BD:57:5C
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6053 errors:0 dropped:443 overruns:0 frame:0
TX packets:3350 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:689415 (673.2 KiB) TX bytes:608437 (594.1 KiB)
Interrupt:4

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2006 errors:0 dropped:0 overruns:0 frame:0
TX packets:2006 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:148644 (145.1 KiB) TX bytes:148644 (145.1 KiB)

Restart didn't work.

Currently, every box sends out regular broadcasts about itself.

What do you think about turning this around?
In a network with mostly "shared"-mode-Forban, you don't need a high frequent broadcast. It may suffice when a user can initiate a "probe" and every box answers with a announce package (as a callback style) - the initiating can be done automatically after x hours since last visit, or something like that.
Other boxes can "hear" this answers too and save the result in their cache.

Opportunistic-Forbans will initiate the broadcast itself every x intervall (maybe first try to get everything your found, then re-broadcast).

What do you think about that? Let this discuss :)

When a proxy is enabled, Forban is trying to fetch the local indexes of Forban using the defined proxy. Proxy should be disabled by default for the urlfetch functions as the Forban are locally reachable.

With usage of cleanloot the index-file is not existent:
Message on Web-UI:

Missing index from this loot [browse]

forban_announce DEBUG-output:

1970-01-01 00:21:30,351 - forban_announce - DEBUG - Index rebuilt
1970-01-01 00:21:30,360 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:22:30,426 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:23:30,506 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:24:30,586 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:25:30,787 - forban_announce - DEBUG - Index rebuilt
1970-01-01 00:25:30,795 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:26:30,866 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:27:30,936 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119
1970-01-01 00:28:31,006 - forban_announce - DEBUG - forban;name;piratebox.lan;uuid;6e6dc51b-6255-48b6-a215-b8a84ac1fd2f;hmac;75ba4f7f6643712cdc6b5db902ae377ed22b7119

Clicking on "browse" returns an internal Server Error:

[01/Jan/1970:00:33:42] HTTP Traceback (most recent call last):
File "/opt/forban/lib/ext/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/opt/forban/lib/ext/cherrypy/lib/encoding.py", line 188, in call
self.body = self.oldhandler(_args, *_kwargs)
File "/opt/forban/lib/ext/cherrypy/_cpdispatch.py", line 34, in call
return self.callable(_self.args, _self.kwargs)
File "/opt/forban/bin/forban_share.py", line 283, in l
tempindex = mindex.search("^((?!forban).)$", uuid)
File "/opt/forban/lib/index.py", line 119, in search
f = open(location, "r")
IOError: [Errno 2] No such file or directory: '/opt/forban/var/loot/6e6dc51b-6255-48b6-a215-b8a84ac1fd2f/cache/forban/index'

[01/Jan/1970:00:33:42] HTTP
Request Headers:
REFERER: http://192.168.56.10:12555/
HOST: 192.168.56.10:12555
CONNECTION: keep-alive
Remote-Addr: ::ffff:192.168.56.5
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.3
USER-AGENT: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.152 Safari/535.19
ACCEPT-LANGUAGE: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
ACCEPT-ENCODING: gzip,deflate,sdch

Hi,
on a remote system, I have a file with a "ö".
This causes to break the search on every connected forban.

Browsing works.

Error in forbarn_share_error.log:

--- will included soon ---

Any idea?

Matthias

Running cleanloot without loot-folder causes an Error:

Traceback (most recent call last):
File "/opt/forban/bin/forbanctl", line 189, in
deleteloots()
File "/opt/forban/bin/forbanctl", line 105, in deleteloots
for dirs in os.listdir(lootpath):
OSError: [Errno 2] No such file or directory: '/opt/forban/var/loot'

A message: "No loot found" - would be better (like cleaning PIDs)

Hi @adulau
Thank you for this great program. I heard about it at EMF and ran it, but it hasn't been updated for a while, and there could be loads of features that could be added.
I am no good at coding, but what can I do to help?
Thanks,

Hi,
I have a PirateBox special, again :)
The most devices doesn't keep their date-time with power-off. So the Boxes start at a minimum Time and counts up.
If you restart the Box with Forban, you always get a list with negative values... and these Boxes are already offline:

Access  Name    Last seen   Size    How many files are missing from yourself?   
v4      grml    -7787.0 secs ago    11.97KB Missing no files from this loot [missing?] [browse] 
v4      anoia   -1076.0 secs ago    11.97KB Missing no files from this loot [missing?] [browse] 
v4      piratebox.lan   -7740.0 secs ago    564.46MB    Missing 1 files from this loot [browse] yourself

1. How can I do a houskeeping on these entries?
2. Is it possible to "ignore" Boxes with negative values?

Thank you :)

Matthias