freifunk-gluon / packages Goto Github PK

It is currently unclear how well the DFS stack currently used in OpenWrt and Linux in general performs, especially with different radio drivers, that each have to reimplement the pulse pattern detection, so the general idea is to collect data to compare different radios and locations, to eventually see the area in which a DFS event took place and check if the devices back each others claims..

Testing could be done with an SDR and https://github.com/TobleMiner/dfs-pulse-tester, if @TobleMiner can confirm it works for that purpose.

cfg80211 can be queried via nl80211 and offers the following state:

• NL80211_FREQUENCY_ATTR_DFS_STATE is one of usable, available, unavailable, unknown
• NL80211_FREQUENCY_ATTR_DFS_TIME holds the time, that a channel is in the above state

So the proposed format would be a list of channels containing their respective state and the time they've been in that state.

Basically:

• loop over all channels
  • drop NL80211_FREQUENCY_ATTR_DISABLED
  • drop NL80211_FREQUENCY_ATTR_INDOOR_ONLY
  • report state and time

{
  "wireless": {
    "dfs": {
      "100": {"state": "available", "for": "35580"},
      "104": {"state": "usable", "for": "35769"},
      ...
    }
  }
}

I really dislike the fact that writing a data for respondd now has to be done in C. This makes the whole process of monitoring a lot more static. It should be a lot easier and faster to add new data to respondd.

The common data sources in providers are:

• uci
• ubus call
• file
• constant

The idea is now to cover this data in a config file and add some post processing. Maybe we can do this with kind of a domain specific language (DSL) in lua:

The main functionality would be given by this functions:

provide(source, postprocessing_handler, cache_time)
emit(json_path, value)
as(json_path)(value, emit) -- higher order function, simply "emits" its value

Constant:

provide(const('1'), as('/node/software/status-page/apiverison'), 60000)

File:

provide(file('/sys/module/batman_adv/version'),
        as('/node/software/batman-adv/version'), infinity)

UCI value:

provide(uci_option('system.@system[0].hostname'), as('/node/hostname'), 60000)

Custom post processing handler:

provide(uci_option('autoupdater.settings'), function (result, emit)
  emit('/node/software/autoupdater/enabled', result['enabled'])
  emit('/node/software/autoupdater/branch', result['branch'])
end, 60000)

UBus call:

provide(ubus_call('system', 'info'), function (result, emit)
  emit('/statistics/loadavg', result['load'][0])
  emit('/statistics/uptime', result['uptime'])
  emit('/statistics/memory', result['memory'])
end, 60000)

I think the fact that the previous respondd implementation with lua providers caused a lot of memory consumption, was caused by the fact, that a lot of lua modules (uci, ubus, ...) were loaded by the provider modules. Maybe this could be solved by implementing the above mentioned functions in C using the native bindings.

What do you think about this idea?

When i do:
uci set autoupdater.testing.mirror=http://MirrorURL

then cat /etc/config/autoupdater
is see:
config branch 'testing'
........
option name 'testing'
----> option mirror 'http://MirrorURL'

instead of:
list mirror 'http://MirrorURL'

so autoupdater is failing:
/usr/bin/lua: /usr/sbin/autoupdater:301: bad argument #1 to 'remove' (table expected, got string)
stack traceback:
[C]: in function 'remove'
/usr/sbin/autoupdater:301: in main chunk
[C]: ?

when i switch it manually everything works fine. is uci using an old option here?

is this a bug or a feature?

The stripped-down version of ebtables-tiny is unfortunately missing the redirect target (for the broute table). Is there an easy switch (config option) to build gluon with the original ebtables version instead of tiny? Or could you please include the redirect extension (ebt_redirect.c) in the tiny package as well? Thanks!

Unfortunately respondd seems to be not vlan aware. This produces incorrect data about the connection type for maps such as HopGlass. See also:
https://forum.freifunk.net/t/hopglass-development-thread/10984/116

root@ma42-cpe:~# autoupdater 
Retrieving manifest from http://autoupdater-broken.mkg20001.io/broken/experimental.manifest ...
^Cautoupdater: warning: manifest http://autoupdater-broken.mkg20001.io/broken/experimental.manifest only carried 0 valid signatures, 1 are required
Retrieving manifest from http://1.1.1.1/404/experimental.manifest ...
Segmentation fault

reproduce:

config branch 'experimental'
	list mirror 'http://autoupdater-broken.mkg20001.io/broken'
	list mirror 'http://1.1.1.1/404'
	option good_signatures '1'
	option name 'experimental'
	list pubkey '5b755c0064558fe7a6c809a086d57ede9b89b07b25d95e1efe3cfe048b176621'

autoupdater-broken is simply 8.8.8.8 behind cloudflare (really any ip that blackholes port 80)

when autoupdater is fetching this hanging request and the fetching is interrupted with ctrl+c, it just goes on fetching the next url, but apparently without cleaning up properly

(note that it must fetch autoupdater-broken first, then 1.1.1.1 for the crash to occur. also the request must go as far as establishing a connection and sending the GET there)

also I don't think it should even do anything except cleanup after ctrl+c

If a node has a value for branch in the config that is not in the list of autoupdater mirrors from site.conf, then the selectbox is mislieadingly showing the first option as selected.

In that case, it should either show an empty selection or better, show the stored selection as new added option to the selectbox.

There are three readme files, that are copied onto the nodes, which are not needed in each subfolder in

https://github.com/freifunk-gluon/packages/tree/master/admin/autoupdater/files/usr/lib/autoupdater/

Those should be excluded in the Makefile

this issue was detected during freifunk-gluon/gluon#1496
currently, the new autoupdater doesn't handle the exit code of the sysupgrade script it calls.
therefore, it doesn't put the node back into an accessible or working state after sysupgrade fails - a manual power cycle is necessary in this case.

The Tunneldigger OpenWrt Makefile currently depends on on the kmod packages kmod-l2tp, kmod-l2tp-eth and kmod-l2tp-ip. kmod-l2tp-ip shouldn't be needed, as Tunneldigger uses the UDP L2TP transport and not raw IP.

Am I overlooking something, or can this dependency be removed? I have not tried removing it myself, as I don't have a Tunneldigger setup to test the change.

cc @RalfJung @kaechele

The README does not match the current implementation;vpn is called wg_mesh now, I think.
@lemoer said something about a documentation overhaul a few days back; maybe this can become part of it.

(Probably related to #67)

We created a package for Freifunk Frankfurt to provide:

• Configured channels/TX Power for wifi interfaces
• The batman nexthop
• Wifi airtime

Module: https://github.com/freifunk-ffm/ffffmlib-playground

The module is implemented as a shared library providing the functionality, a lua module using it , three alfred/old-respondd modules and two respondd modules.

We would like to have it integrated into gluon packages proper. For that I have some questions:

• Is this acceptable for gluon packages?
• Is this layout acceptable?

Of course I would remove all the ffffm strings.

Cc @oszilloskop, @eberhab

When executed in a VM for example, I see the following in the logs a lot:

respondd[2552]: netlink.c: error on line 28

The netlink.c actually is from respondd-module-airtime, and it's probably caused by the lack of wifi interfaces and thus the lack of netlink answers.

I'm currently using autoupdater to update the firmware on an Aerohive HiveAP 330. Which takes ages.
So I've used that time to find out about what's holding it up.
The HiveAP 330 uses a 9600bps console output, so any output takes a long time.

The while loop in recv_image_cb outputs the progress in every iteration:

Maybe it would be beneficial if the downloading progress would only get shown, like, every 128th iteration or so?

On a sidenote, it also looks to me like the SHA256 checksum is calculated for every iteration as well? Is this necessary? Wouldn't it be sufficient to calculate it once the download is completed?

Okay, it finally finished downloading, took about 20 minutes for the 44MB image...

EDIT:
using # autoupdater >/dev/null it took merely a couple seconds to download the image. So it definitely is related to terminal output.

the main batman-adv-mac "primary0" is not listed/marked explicitly.
Since this primary0 is the one you need to now to use features like "batctl tp", it's a guessing-game to identify the right mac from the "mixed list".
(propably it's the one ending on "b" or "3", but that's just a guess, it would be better to have it clearly identified on the datagrams provided by respondd.)

Next to the checks missing as noted in #157 it seems like there are a few more allocation checks missing.

In the C standard, a NULL pointer dereference is undefined. While on Linux it usually just kills the program with a SIGSEGV, it seems that a NULL pointer dereference together with compiler optimizations can cause more serious issues. Read Tim Armstrong's reply here for instance (which references/summarizes an LLVM blog post):

https://www.quora.com/What-actually-happens-when-dereferencing-a-NULL-pointer

Currently unchecked allocation examples:

• json_object_new_*()
• malloc()
• (and all nla_get_*()s and nla_parse() )

as agreed on @IRC, there needs to be a v2018.2.x packages branch which does not contain everything the master branch now has.

The tunneldigger client in gluon is broken and doesn't allow unique sessions on the server. This causes to server to drop every tunneling attempt (except the first one). Patches to fix that were added to the client in wlanslovenija/tunneldigger#57

The DIR-860L with mediatek wifi does not offer rx/tx time:

$ iw dev wlan0 survey dump
Survey data from wlan0
  frequency:      5180 MHz [in use]
  channel active time:    4320701 ms
  channel busy time:    1316279 ms

The airtime module expects these fields an crashes:

root@device:/etc/init.d# dmesg | grep SEG                                                                                              
[  152.200000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                            
[  212.200000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                            
[  272.200000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                            
[  332.200000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                            
[  392.200000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                            
[  452.790000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                            
[230475.420000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004                                           
root@device:/etc/init.d# logread | grep "kern.info"                                                                                    
Mon Jan 30 20:53:13 2017 kern.info kernel: [230475.420000]                                                                                   
Mon Jan 30 20:53:13 2017 kern.info kernel: [230475.420000] do_page_fault(): sending SIGSEGV to respondd for invalid read access from 00000004
Mon Jan 30 20:53:13 2017 kern.info kernel: [230475.440000] epc = 76f1ebf5 in airtime.so[76f1e000+11000]                                      
Mon Jan 30 20:53:13 2017 kern.info kernel: [230475.450000] ra  = 76f1ebeb in airtime.so[76f1e000+11000]                                      

It used to be the case that (following Debian's handling of version numbers) firmware version 1.7.0 was considered newer than 1.7.0~rc1. However, with the latest gluon 2018.1.3, that does not seem to work any more: We have rolled out 1.7.0~rc1 on our beta branch, and now we built 1.7.0, but even when doing a force update check the autoupdater just says "No new firmware available."

As @NeoRaider explained me in irc, this is a temporary trick to prevent amplifications attacks.

is it intentional that there is no @reboot trigger?

I had a quick review on the new implementation of the autoupdater, I am not sure if this behavior is present in the older lua implementation. When a negative priority is presented in the manifest [1] the probability mechanism implement in [2] is basically deactivated, except for the case when diff itself is negative.

[1] https://github.com/freifunk-gluon/packages/blob/master/admin/autoupdater/src/manifest.c#L113
[2] https://github.com/freifunk-gluon/packages/blob/master/admin/autoupdater/src/autoupdater.c#L155

It would be cool, if respondd publish the "main" domain of reverse ip address.

On this way it is possible to detect the isp.

I am trying to troubleshoot my Gluon node not appearing on the Freifunk map. This is a rather common problem, especially after some recent changes to Gluon.

In the process, I wanted to check that respondd is working properly. So I wanted to test it from the command line. I got as far as this:

printf "GET nodeinfo" | socat STDIO UDP6-DATAGRAM:[fe80::5054:ff:fe12:3456]:1001

The trouble is, the data returned is compressed. How can I decompress it with command-line tools? "gunzip -d" does not work. Some people suggest using "openssl zlib -d" or some other such tool, but I could not find anything readily available on Gluon.

I would like to improve respondd's documentation with the information collected here.

As discussed in the gluon meetup today, we want to allow using wgpeerselector in combination with wgkex.

The only (big) missing piece is an HTTPS request to the broker as done here.

I would suggest to add the request here (after the NTP check):

However, the wgkex broker currently expects requests that include {"domain": "'"$SEGMENT"'", "public_key": "'"$PUBLICKEY"'"} in the http body. From the context of the wgpeerselector this information is not tangible. As of now, the wgpeerselector does not know about "domains", as it does not contain gluon code. So this problem needs to be addressed in some way.

@goligo wants to take a look at this. I am open to discuss this here.