Add Unbound support to pihole container

Just tried to build a new stack now, and unfortunately, deCONZ is not added to the stack, despite the fact that the config menu is shown. For some reason, the selected service.yml (e.g. .templates/deconz/service_conbee_II.yml) is not copied to the docker-compose.yml.

This is a new issue, since I used the latest version of SensorsIot/IOTstack to successfully install deCONZ 5 days ago.

whenever the container is updated, it does not auto start on system restart and also pushes with a warning to remove the foreign container, can u suggest me a fix. Thanks in advance.

Hello,
i'm trying to follow instructions from README.md (or youtube video) but aftter docker-compose up -d (or via menu.sh) i get following error

pi@amalthea: ~/IOTstack $ docker-compose up -d
ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
pi@amalthea: ~/IOTstack $ ./menu.sh
~/IOTstack ~/IOTstack
checking for project update
From https://github.com/SensorsIot/IOTstack

• branch master -> FETCH_HEAD
  An update is available for the project
  checking docker version
  Docker version >= 18.2.0. You are good to go.
  ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.
  For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

I am running Raspbian on Raspberry pi 3 B+
pi@amalthea:~/IOTstack $ uname -a
Linux amalthea 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

pi@amalthea:~/IOTstack $ docker -v
Docker version 19.03.12, build 48a6621

my docker-compose.yml (created by menu.sh)
version: '3.6'
services:

portainer:
container_name: portainer
image: portainer/portainer
restart: unless-stopped
ports:
- 9000:9000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./volumes/portainer/data:/data

nodered:
container_name: nodered
build: ./services/nodered/.
restart: unless-stopped
user: "0"
privileged: true
env_file: ./services/nodered/nodered.env
ports:
- 1880:1880
volumes:
- ./volumes/nodered/data:/data

influxdb:
container_name: influxdb
image: "influxdb:latest"
restart: unless-stopped
ports:
- 8086:8086
- 8083:8083
- 2003:2003
env_file:
- ./services/influxdb/influxdb.env
volumes:
- ./volumes/influxdb/data:/var/lib/influxdb
- ./backups/influxdb/db:/var/lib/influxdb/backup

grafana:
container_name: grafana
image: grafana/grafana
restart: unless-stopped
user: "0"
ports:
- 3000:3000
env_file:
- ./services/grafana/grafana.env
volumes:
- ./volumes/grafana/data:/var/lib/grafana
- ./volumes/grafana/log:/var/log/grafana

mosquitto:
container_name: mosquitto
image: eclipse-mosquitto
restart: unless-stopped
user: "1883"
ports:
- "1883:1883"
volumes:
- ./volumes/mosquitto/data:/mosquitto/data
- ./volumes/mosquitto/log:/mosquitto/log
- ./volumes/mosquitto/pwfile:/mosquitto/pwfile
- ./services/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
- ./services/mosquitto/filter.acl:/mosquitto/config/filter.acl

I am not familiar with docker.
Please advise. Thank you.

The ~/IOTStack/.template/influxdb/influxdb.env contains the following:

#INFLUXDB_DB=mydb
INFLUXDB_DATA_ENGINE=tsm1
INFLUXDB_REPORTING_DISABLED=false
#INFLUXDB_HTTP_AUTH_ENABLED=true
INFLUXDB_ADMIN_ENABLED=true
#INFLUXDB_ADMIN_USER=myadminuser
#INFLUXDB_ADMIN_PASSWORD=myadminpassword
INFLUXDB_USER=nodered
INFLUXDB_USER_PASSWORD=nodered
#INFLUXDB_READ_USER=myreaduser
#INFLUXDB_READ_USER_PASSWORD=myreadpassword
#INFLUXDB_WRITE_USER=mywriteuser
#INFLUXDB_WRITE_USER_PASSWORD=mywritepassword

and, because a question about the following key has come up before, I have added it to the list:

#INFLUXDB_HTTP_FLUX_ENABLED=true

Test 1 – search the documentation page

With the Configure InfluxDB OSS page open, use the browser's built-in search facility to look for the exact keys:

Test 2 - use Google to search the site

Of the keys that are not documented, use Google to perform a restricted search using this syntax:

"key" site:.docs.influxdata.com

The hit for "INFLUXDB_USER" is at a page about Creating InfluxDB and Kapacitor connections and the context is that of a substitution key, not an environment key.

Test 3 - use the site's search capability

Sticking to the keys that were not documented in test 1, use the "Search the docs…" field at the top of the Configure InfluxDB OSS page:

• [a] appears to be matching on "InfluxDB" and "DB" but finds nothing relevant
• [b] appears to be matching on "InfluxDB" and "user" but finds nothing relevant

Test 4 - look for partial matches

Using the same approach as test 1, look for partial matches as a way of (hopefully) picking up typos and/or renamed keys:

The bottom line for environment keys

I think test 1 was fairly conclusive all by itself. Tests 2 and 3 were aimed at discovering whether environment keys might be documented in places other than Configure InfluxDB OSS. The answer to that question seems to be "no". Test 4 was aimed at eliminating typos or renames. That was fruitless as well.

That leaves us with these keys:

INFLUXDB_HTTP_AUTH_ENABLED
INFLUXDB_HTTP_FLUX_ENABLED
INFLUXDB_REPORTING_DISABLED

The convention in the documentation is to show the default value of the key, so the IOTstack .template file should probably become:

INFLUXDB_HTTP_AUTH_ENABLED=false
INFLUXDB_HTTP_FLUX_ENABLED=false
INFLUXDB_REPORTING_DISABLED=false

There seems little point in commenting-out keys and setting them false. All that achieves is to require two edits to activate a key.

The alternative representation would be:

#INFLUXDB_HTTP_AUTH_ENABLED=true
#INFLUXDB_HTTP_FLUX_ENABLED=true
#INFLUXDB_REPORTING_DISABLED=true

On balance, I'm more inclined to go with the first approach and adopt the standard established in the documentation.

IOTstack documentation for InfluxDB

The IOTstack InfluxDB documentation says:

The credentials and default database name for influxdb are stored in the file called influxdb/influx.env. The default username and password is set to "nodered" for both. It is HIGHLY recommended that you change them. The environment file contains several commented out options allowing you to set several access options such as default admin user credentials as well as the default database name. Any change to the environment file will require a restart of the service.

That's somewhere between misleading and wrong. The relevant documentation page is Authentication and authorization in InfluxDB and there is mention of these two environment keys:

INFLUX_USERNAME
INFLUX_PASSWORD

but the context is by way of providing defaults for the CLI rather than setting a username and password combination that will then be enforced by InfluxDB for all comers.

For the record, there was no INFLUX_DB (or similar) environment key described at Authentication and authorization in InfluxDB. I tried adding it to see if it had any effect: it did not. The influx - InfluxDB command line interface documentation is silent on this question.

All handling of credentials is internal to the database with commands like:

> CREATE USER testadmin WITH PASSWORD 'easyToGuess' WITH ALL PRIVILEGES
> CREATE USER testuser WITH PASSWORD 'easierToGuess'

> SHOW USERS
user      admin
----      -----
testadmin true
testuser  false

> CREATE DATABASE test
> USE test
> GRANT ALL ON test TO testadmin
> GRANT READ ON test TO testuser

> SHOW GRANTS FOR testadmin
database privilege
-------- ---------
test     ALL PRIVILEGES

> SHOW GRANTS FOR testuser
database privilege
-------- ---------
test     READ

> DROP USER testadmin
> DROP USER testuser
> DROP DATABASE test

My view is that the whole field of database access control is beyond the scope of IOTstack documentation. Giving the impression username and password combinations can ever be "set and forget" is quite misleading. Covering the field properly would involve duplicating Authentication and authorization in InfluxDB. The IOTstack documentation should point to that page and leave the rest to the user.

EOF

I'll wait a day or two for comments, and then prepare a pull request to:

1. Change the template influxdb.env to:

   INFLUXDB_HTTP_AUTH_ENABLED=false
   INFLUXDB_HTTP_FLUX_ENABLED=false
   INFLUXDB_REPORTING_DISABLED=false
   

2. Change the documentation to:

   • a discussion of the environment keys provided in the template
   • a pointer to the InfluxDB documentation as the source for more environment keys
   • how to activate new or changed environment key
   • a pointer to the Influx documentation on security.

I noticed that my backups had stopped syncing to Gdrive by chance. It turns out the rclone sync command fills up the trash folder and then fails silently in the backup script.

I added --drive-use-trash=false to the rclone sync line in docker-backup.sh as per the documentation, that seems to fix the problem. Would it make sense to add that change to the repo?

Also, does anyone have suggestions on some kind of alarm if the backup is not successful? It's a bit disconcerting that this was a completely silent error.

https://github.com/joohoi/acme-dns

https://github.com/exentriquesolutions/nip.io or https://github.com/resmo/nip.io

These should work and auto-configure with nginx and PiHole once the new menu is completed.

add let's encrypt for so pihole dashboard can be accessed via https

Update to Documentation structure to make it easier to find what you are looking for by putting containers into category's like:

• Communication
• Home automation
• Logging
• Dashboards
• Data storage

This allows a user to easily find the documentation that he is looking for

Hello

Has something happened with the Home Assistant Installation via the IoTStack? I've now have everything running but HA. I just got a RPI4 to use a SSD and wanted to add HA (as my SD burned out, surprise :)). And when i try to install HA it reports "Not supported" and doesnt continue.
This happens with both RPI4 32 and 64 bit.

Thank you in advance

Hi guys! Does anybody here have an idea why I can't update the Node-RED to the newest version? I still have 1.0.6 and there is already 1.1.2 out. Any suggestion will be highly appreciated. Thanks in advance.

Rewrite some or all of menu.sh to allow for more advanced configurations, and automated deployments.

Consider using python or node.
Consider having some of the deployment done inside docker containers so that node and python does not need to be installed on the host system.

Freshly downloaded Buster image.
Installed git, no issues.
Installed IOTstack, no issues.
Followed instructions, saw docker needed an update:

pi@old-trout:~ $ cd ~/IOTstack && bash ./menu.sh
~/IOTstack ~/IOTstack
checking for project update
From https://github.com/SensorsIot/IOTstack
 * branch            master     -> FETCH_HEAD
Project is up to date
checking docker version
./menu.sh: line 197: docker: command not found
./menu.sh: line 202: [: : integer expression expected

Docker version less than 18.02.0 consider upgrading or you may experience issues
Upgrade by typing: 'sudo apt upgrade docker docker-compose'
~/IOTstack

Updated docker:
pi@old-trout:~ $ sudo apt upgrade docker docker-compose
Finished without error.
Restarted IOTstack, same issues:

pi@old-trout:~ $ cd ~/IOTstack && bash ./menu.sh
~/IOTstack ~/IOTstack
checking for project update
From https://github.com/SensorsIot/IOTstack
 * branch            master     -> FETCH_HEAD
Project is up to date
checking docker version
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.39/version: dial unix /var/run/docker.sock: connect: permission denied
./menu.sh: line 202: [: : integer expression expected

Docker version less than 18.02.0 consider upgrading or you may experience issues
Upgrade by typing: 'sudo apt upgrade docker docker-compose'
~/IOTstack

Basically, no matter what I tried, this was the result:

pi@old-trout:~/IOTstack $ docker-compose up -d
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
pi@old-trout:~/IOTstack $ ps -aux | grep docker
root     12250  0.9  6.1 974360 54672 ?        Ssl  21:19   0:04 /usr/sbin/dockerd -H fd://
root     12259  0.5  2.3 943644 21328 ?        Ssl  21:19   0:02 docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level info
pi       19816  0.0  0.0   7348   472 pts/0    S+   21:26   0:00 grep --color=auto docker
pi@old-trout:~/IOTstack $ service docker restart
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to restart 'docker.service'.
Authenticating as: root
Password: 
polkit-agent-helper-1: pam_authenticate failed: Authentication failure
==== AUTHENTICATION FAILED ===
Failed to restart docker.service: Access denied
See system logs and 'systemctl status docker.service' for details.
pi@old-trout:~/IOTstack $ docker-compose up -d
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

Suggestions?

Hello,
in my Node-RED flow I have a javascript statement that puts out a datetime string to the dashboard with "var ts = d.toLocaleString('de-DE');".
So I expect to get the time like "13:27:14". But in the dashboard I see "1:27:14 PM" which is the locale "en-US".
Probably that happens because the underlying npm was built with "small-icu" and not with "full-icu"?

The whole day today I'm trying to install NextCloud. Tried already several times on clean and "refurbished" systems, with setting up passwords before running docker-compose.

First I thought that maybe some ports from nextcloud_db are not ok as I had also separate MariaDB instance, but after clean start without installing MariaDB situation is the same.

When I try to activate NextCloud with SQLite everything is ok, but when doing the same with MariaDB/nextcloud_db no visible error occurs but page is loading last two hours and nothing happens like script is stuck somewhere...

Is port 9001 needed for Mosquitto?

Summary:

• Out-of-the box, the eclipse-mosquitto image does not listen to port 9001.
• The evidence suggests that the docker-compose port mapping for 9001 is redundant.
• Does anyone know any different?
• If there are no objections, I'm proposing removal.

Chain of evidence

What eclipse-mosquitto image is on my RPi4?

$ docker images eclipse-mosquitto
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
eclipse-mosquitto   latest              a2b55301913b        11 days ago         5.86MB

What does Docker know about this image?

$ docker inspect eclipse-mosquitto
[
    {
        … [snip] …
        "RepoTags": [
            "eclipse-mosquitto:latest"
        ],
        … [snip] …
        "Created": "2020-06-02T21:51:19.217586203Z",
        … [snip] …
        "ContainerConfig": {
            … [snip] …
            "ExposedPorts": {
                "1883/tcp": {}
            },
            … [snip] …
            "Env": [
                … [snip] …
                "VERSION=1.6.10",
                … [snip] …
            ],
        },
    }
]

Observation: as far as docker inspect is concerned, only port 1883 is exposed.

Assume IOTstack is not running.

What does netstat have to say about the baseline situation on the RPi4:

netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*                               

Observation: no process is listening to either port 1883 or 9001.

Run-up mosquitto all by itself, explicitly specifying ports 1883 & 9001 (ie how docker-compose.yml does it at the moment):

$ docker run -d --name mosquitto -p 1883:1883 -p 9001:9001 eclipse-mosquitto
38589d808156e385aa58ebce2e25121b500c009c731995e610dd29c0e41fdb56

What does netstat think outside the container?

$ netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::9001                 :::*                    LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*                               

Observation: a process (Docker) is listening to ports 1883 & 9001.

What does netstat think inside the container?

pi@sec-dev:~/IOTstack $ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN      

Observation: a process (mosquitto) is listening to port 1883 but nothing is listening to port 9001.

Any complaints in the log?

$ docker logs mosquitto
1591924206: mosquitto version 1.6.10 starting
1591924206: Config loaded from /mosquitto/config/mosquitto.conf.
1591924206: Opening ipv4 listen socket on port 1883.
1591924206: Opening ipv6 listen socket on port 1883.

Observation: no complaints.

OK. Tear that down.

$ docker stop mosquitto
$ docker rm mosquitto

Now, repeat the test but this time without port 9001:

$ docker run -d --name mosquitto -p 1883:1883 eclipse-mosquitto
248a2763bf72f4d9c704a590083ce2170130a3cb82768d381c6e0e509fdb9ba7

The netstat view from outside:

$ netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*                               

Observation: no listener for port 9001 (as expected).

The netstat view from inside:

$ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN      

Observation: no change – still only port 1883 (also as expected).

Any complaints in the log?

$ docker logs mosquitto
1591924553: mosquitto version 1.6.10 starting
1591924553: Config loaded from /mosquitto/config/mosquitto.conf.
1591924553: Opening ipv4 listen socket on port 1883.
1591924553: Opening ipv6 listen socket on port 1883.

Observation: mosquitto is still happy.

OK. Tear that down.

Now for the acid test. Comment-out the line in docker-compose.yml:

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    restart: unless-stopped
    user: "1883"
    ports:
      - 1883:1883
#      - 9001:9001
    volumes:
      - ./volumes/mosquitto/data:/mosquitto/data
      - ./volumes/mosquitto/log:/mosquitto/log
      - ./services/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
      - ./services/mosquitto/filter.acl:/mosquitto/config/filter.acl

Bring up the stack:

$ cd ~/IOTstack
$ docker-compose up -d

Look at netstat both inside and outside:

$ netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::9000                 :::*                    LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::2222                 :::*                    LISTEN     
tcp6       0      0 :::7920                 :::*                    LISTEN     
tcp6       0      0 :::2003                 :::*                    LISTEN     
tcp6       0      0 :::8083                 :::*                    LISTEN     
tcp6       0      0 :::53                   :::*                    LISTEN     
tcp6       0      0 :::8086                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::1880                 :::*                    LISTEN     
tcp6       0      0 :::5432                 :::*                    LISTEN     
tcp6       0      0 :::3000                 :::*                    LISTEN     
tcp6       0      0 :::8089                 :::*                    LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::53                   :::*                               
udp6       0      0 :::67                   :::*                               
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*                               

$ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 127.0.0.11:38641        0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN      
udp        0      0 127.0.0.11:37575        0.0.0.0:*                           

Observation: no sign of port 9001.

Anything in the log?

$ tail volumes/mosquitto/log/mosquitto.log 
2020-06-12T00:59:27: mosquitto version 1.6.10 terminating
2020-06-12T00:59:27: Saving in-memory database to /mosquitto/data/mosquitto.db.
2020-06-12T01:19:18: mosquitto version 1.6.10 starting
2020-06-12T01:19:18: Config loaded from /mosquitto/config/mosquitto.conf.
2020-06-12T01:19:18: Opening ipv4 listen socket on port 1883.
2020-06-12T01:19:18: Opening ipv6 listen socket on port 1883.
2020-06-12T01:19:27: New connection from 172.27.0.6 on port 1883.

Observation: No complaints.

Conclusion: Port 9001 is not used and is probably not needed.

Googling

There are some references to port 9001:

• fiware-tutorials

  Port 9001 is the standard port for HTTP/Websocket communications

• steves-internet-guide

  MQTT over Websockets usually uses port 9001 but it isn’t fixed.

• A restricted Google search:

   "9001" site:.mosquitto.org
  

  returns:

   Your search - "9001" site:.mosquitto.org - did not match any documents
  

  which suggests that, if there ever was a case for Mosquitto needing port 9001, it's disappeared into the bit-bucket of history.

Off-hand, I can't think of a use-case for Mosquitto to be involved in HTTP/Websocket comms. Can anyone else?

I do have a couple of use-cases for IoT devices communicating over HTTP but they do it with Node-Red, not Mosquitto:

• TrigBoards searching for firmware updates (a quick check over HTTP is better for battery life than firing up OTA); and
• A PurpleAir air-quality monitor (which doesn't support user-logging via MQTT).

Some brute force

Assuming there was a use-case for Mosquitto to chat over port 9001, it needs a bit more than just the port mapping in docker-compose.yml to get it going. In particular, these lines also need to be added to mosquitto.conf:

listener 1883
listener 9001

Both lines are needed - if you only add 9001 then 1883 stops working.

That definitely causes Mosquitto to listen on the port:

$ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN      
tcp        0      0 127.0.0.11:40117        0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN      
tcp        0      0 :::9001                 :::*                    LISTEN      
udp        0      0 127.0.0.11:55876        0.0.0.0:*                           

but it's an open question as to what you can actually do with it.....

Bottom line

My view is that port 9001 should be removed from the Mosquitto template, with the two-step how-to of enabling port 9001 put in the documentation for anyone who subsequently discovers a use-case.

If nobody objects to that plan, I'll prepare a pull request.

Can you please add Folding@Home to the stack for contributing towards the global pandemic?

https://hub.docker.com/r/linuxserver/foldingathome

Jellyfin is an excellent alternative to Plex and puts users in control of their data and is 100% free and open source. https://hub.docker.com/r/jellyfin/jellyfin/

Hi all,
I have a strange problem which I did not find anyone else having. So, I'm sorry if it's a very newbie issue...
I went through the instructions from the video and successfully installed everything. I have a sensor connected to COM port and when I try to store the value from 'Serial In' node to InfluxDB I get the error: "Error: connect ECONNREFUSED 127.0.0.1:8086" and "no host available". I have a user "pi" in influx which I inputted it's name and password and created a database there to receive the values. Just can't figure out why it can't connect to it.
Thanks in advance!

IOTstack – Experiments with Bluetooth

This issue documents my approach to getting node-red-contrib-generic-ble working under IOTstack. I'm hoping that people with a lot more experience than me will spot any flaws in my approach.

I'm not going to keep typing node-red-contrib-generic-ble because it's boring. I'm going to use the abbreviation BLE.

I don't have a use-case for Bluetooth so I can't go any further than I've documented here. It would be very helpful to get confirmation that the result is fit for purpose and that useful communications can be achieved between Node-Red and Bluetooth devices.

If there's enough interest, I will volunteer to turn this into a documentation page for the IOTstack project. If someone else wants to tackle the documentation using this issue as a starting point, please go right ahead.

Installation

Give the current user access to the Bluetooth hardware

$ sudo usermod -G bluetooth -a $(whoami)
$ sudo reboot

Update to the latest and greatest of everything

Make sure Raspbian (ahem, "Raspberry Pi OS") is up-to-date:

$ cd ~/IOTstack
$ docker-compose down
$ cd
$ sudo apt update
$ sudo apt upgrade
$ sudo reboot

Make sure IOTstack is up-to-date:

$ cd ~/IOTstack
$ git pull origin master

Make sure container images are up-to-date:

$ docker rmi "iotstack_nodered" "nodered/node-red"
$ docker-compose pull
$ docker-compose up --build -d

Demonstrate that nodered is in non-host mode (the default)

In the following, 192.168.132.1 is my local router.

$ docker exec nodered traceroute 192.168.132.1 
traceroute to 192.168.132.1 (192.168.132.1), 30 hops max, 38 byte packets
 1  172.18.0.1 (172.18.0.1)  0.011 ms  0.015 ms  0.012 ms
 2  myrouter.mydomain.com (192.168.132.1)  0.288 ms  0.263 ms  0.143 ms
$ 

Two hops = non-host mode.

Edit the nodered configuration

Edit ~/IOTstack/docker-compose.yml and add these lines:

      - /var/run/docker.sock:/var/run/docker.sock
      - /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket
    devices:
      - "/dev/ttyAMA0:/dev/ttyAMA0"
      - "/dev/vcio:/dev/vcio"
      - "/dev/gpiomem:/dev/gpiomem"
    network_mode: "host"

The final result should be:

  nodered:
    container_name: nodered
    build: ./services/nodered/.
    restart: unless-stopped
    user: "0"
    privileged: true
    env_file: ./services/nodered/nodered.env
    ports:
      - 1880:1880
    volumes:
      - ./volumes/nodered/data:/data
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket
    devices:
      - "/dev/ttyAMA0:/dev/ttyAMA0"
      - "/dev/vcio:/dev/vcio"
      - "/dev/gpiomem:/dev/gpiomem"
    network_mode: "host"

Remember, you can't use tabs in docker-compose.yml. You must use spaces.

Apply the changes:

$ docker-compose up -d
…
Recreating nodered ... done
$

Confirm that nodered is in host mode

$ docker exec nodered traceroute 192.168.132.1
traceroute to 192.168.132.1 (192.168.132.1), 30 hops max, 38 byte packets
 1  myrouter.mydomain.com (192.168.132.1)  0.252 ms  1.610 ms  0.200 ms
$

One hop = host mode.

Confirm that Node-Red (the service) is reachable

You can either open a browser and point it to port 1880 on your Raspberry Pi's IP address or domain name, or you can do it from the command line:

$ curl -I localhost:1880
HTTP/1.1 200 OK
…
$

As long as you get the "200 OK" you should be fine.

Install node-red-contrib-generic-ble

Open a shell into the nodered container:

$ docker exec -it nodered bash
#

Confirm that BLE is not already installed:

# ls -ld node_modules/node-red-contrib-generic-ble
ls: node_modules/node-red-contrib-generic-ble: No such file or directory
#

BLE has a dependency that the package does not take care of:

# apk add eudev-dev
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/armv7/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/armv7/APKINDEX.tar.gz
(1/2) Installing pkgconf (1.6.3-r0)
(2/2) Installing eudev-dev (3.2.9-r1)
Executing busybox-1.31.1-r9.trigger
OK: 240 MiB in 67 packages
#

Install BLE:

# npm install node-red-contrib-generic-ble

The compile chucks out a lot of notes and warnings but succeeds.

Confirm that BLE has been installed:

# ls -ld node_modules/node-red-contrib-generic-ble
drwxr-xr-x    5 root     root          4096 Jun  3 21:00 node_modules/node-red-contrib-generic-ble
# 

Exit the container shell:

# exit
$

Restart nodered. This picks up the new BLE node:

$ docker-compose restart nodered

Check for errors:

$ docker logs nodered

The output I get is:

> [email protected] start /usr/src/node-red
> node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"

3 Jun 21:05:05 - [info] 

Welcome to Node-RED
===================

3 Jun 21:05:05 - [info] Node-RED version: v1.0.6
3 Jun 21:05:05 - [info] Node.js  version: v10.20.1
3 Jun 21:05:05 - [info] Linux 4.19.118-v7l+ arm LE
3 Jun 21:05:06 - [info] Loading palette nodes
3 Jun 21:05:08 - [info] Dashboard version 2.22.1 started at /ui
3 Jun 21:05:09 - [info] Settings file  : /data/settings.js
3 Jun 21:05:09 - [info] Context store  : 'default' [module=memory]
3 Jun 21:05:09 - [info] User directory : /data
3 Jun 21:05:09 - [warn] Projects disabled : editorTheme.projects.enabled=false
3 Jun 21:05:09 - [info] Flows file     : /data/flows.json
3 Jun 21:05:09 - [info] Server now running at http://127.0.0.1:1880/
3 Jun 21:05:09 - [warn] 

---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.

You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------

3 Jun 21:05:09 - [info] Starting flows
3 Jun 21:05:09 - [info] Started flows
3 Jun 21:05:09 - [info] [sqlitedb:eb5d5c76.848308] opened /data/Databases/Mailbox/mailbox.db ok
3 Jun 21:05:09 - [info] [sqlitedb:f51495af.860dd8] opened /data/Databases/TrigBoard/trigboard.db ok
3 Jun 21:05:09 - [info] [mqtt-broker:Merle MQTT] Connected to broker: mqtt://mosquitto:1883
3 Jun 21:05:09 - [info] [mqtt-broker:Merle MQTT] Connected to broker: mqtt://mosquitto:1883
3 Jun 21:05:09 - [info] [mqtt-broker:Merle MQTT] Connected to broker: mqtt://mosquitto:1883
3 Jun 21:05:09 - [info] [mqtt-broker:Merle MQTT] Connected to broker: mqtt://mosquitto:1883
3 Jun 21:05:09 - [info] [mqtt-broker:Docker MQTT] Connected to broker: mqtt://mosquitto:1883
$

No errors!

Testing

1. Use your browser to connect to Node-Red.
2. Created a new empty flow.
3. Scroll down in the Node Palette until you see "Generic BLE in" and drag it onto the canvas.
4. Double-click the BLE-in node to open its properties.
5. Click the pencil icon next to "Add new Generic BLE...".
6. Turn on the "BLE Scanning" checkbox.
7. Wait. If all goes well, the "Scan Result" popup menu will populate and you will be able to open that popup menu and see all your Bluetooth MACs.

Discussion

First attempt at getting BLE working (failure):

• Starting docker-compose.yml configuration:

     nodered:
       container_name: nodered
       build: ./services/nodered/.
       restart: unless-stopped
       user: "0"
       privileged: true
       env_file: ./services/nodered/nodered.env
       ports:
         - 1880:1880
       volumes:
         - ./volumes/nodered/data:/data
  

• I forced a complete rebuild of iotstack_nodered. The nodes installed at build time were the defaults plus SQLite.

• I edited docker-compose.yml, to add:

       network_mode: "host"
  

  Then restarted nodered and checked that it was in host mode.

• At HilscherAutomation I came across the requirement for:

       devices:
         - "/dev/ttyAMA0:/dev/ttyAMA0"
         - "/dev/vcio:/dev/vcio"
         - "/dev/gpiomem:/dev/gpiomem"
  

  There were more lines in the HilscherAutomation list but the above three were the only ones that mapped to anything on my RPi4. They seemed like a good idea but I do not know whether they are required.

• My first attempt at compiling BLE failed because of a missing dependency. I worked out how to install eudev-dev, after which BLE compiled.

• Restarting the nodered container to activate BLE produced an error about /var/run/docker.sock so I added a mapping:

       volumes:
         - ./volumes/nodered/data:/data
         - /var/run/docker.sock:/var/run/docker.sock
  

• That cured the docker.sock problem but nodered would still not come up. The new error was:

   Error: connect ENOENT /var/run/dbus/system_bus_socket at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1107:14)
  

• I tried to solve that by adding another mapping:

       volumes:
         - ./volumes/nodered/data:/data
         - /var/run/docker.sock:/var/run/docker.sock
         - /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket
  

  but nodered kept restarting with the same error.

I checked a few more things (eg possible permission conflicts) but that's where I gave up.

Second attempt at getting BLE working (success):

• Starting docker-compose.yml configuration:

     nodered:
       container_name: nodered
       build: ./services/nodered/.
       restart: unless-stopped
       user: "0"
       privileged: true
       env_file: ./services/nodered/nodered.env
       ports:
         - 1880:1880
       volumes:
         - ./volumes/nodered/data:/data
         - /var/run/docker.sock:/var/run/docker.sock
         - /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket
       devices:
         - "/dev/ttyAMA0:/dev/ttyAMA0"
         - "/dev/vcio:/dev/vcio"
         - "/dev/gpiomem:/dev/gpiomem"
  

• I forced a complete rebuild of iotstack_nodered. The nodes installed at build time were the defaults plus SQLite.

• I edited docker-compose.yml, to add:

       network_mode: "host"
  

  Then restarted nodered and checked that it was in host mode.

• I installed the eudev-dev dependency and compiled BLE.

• I restart the nodered container.

And it just worked.

To say that I was astonished would be a severe understatement.

My guess (and it is no more than a guess) is that some or all of the volumes and/or devices additions to docker-compose.yml needed to be in place before BLE was compiled.

More work

I tried to see if I could add node-red-contrib-generic-ble to the menu but that, of course, came with the eudev-dev pre-requisite. I tried adding:

RUN apk add eudev-dev

as the second line of the Dockerfile but docker-compose hated it.

I will leave it to others to solve this problem.

Just be aware that, in the meantime, a rebuild of nodered will require a manual install of BLE.

I want to install Teslamate on my Pi with IOTstack already running lots of containers.

However, Teslamate installed with Docker, has its own set of Grafana, Mosquitto and Postgres DB. How can I install Teslamate without interferring with IOTstack containers?

Teslamate docs: https://docs.teslamate.org/docs/installation/docker

When I try to to add "RTL_433 to mqtt" by using container selection I get following message after docker-compose:
Step 7/8 : RUN apt-get update && apt-get install -y git libtool libusb-1.0.0-dev librtlsdr-dev rtl-sdr cmake automake && git clone https://github.com/merbanan/rtl_433.git /tmp/rtl_433 && cd /tmp/rtl_433/ && mkdir build && cd build && cmake ../ && make && make install
---> Running in 33366bfc61e4
Err:1 http://security.debian.org/debian-security buster/updates InRelease
Temporary failure resolving 'security.debian.org'
Err:2 http://deb.debian.org/debian buster InRelease
Temporary failure resolving 'deb.debian.org'
Err:3 http://deb.debian.org/debian buster-updates InRelease
Temporary failure resolving 'deb.debian.org'
Reading package lists...
W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease Temporary failure resolving 'deb.debian.org'
W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease Temporary failure resolving 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease Temporary failure resolving 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package git
E: Unable to locate package libtool
E: Unable to locate package libusb-1.0.0-dev
E: Couldn't find any package by glob 'libusb-1.0.0-dev'
E: Couldn't find any package by regex 'libusb-1.0.0-dev'
E: Unable to locate package librtlsdr-dev
E: Unable to locate package rtl-sdr
E: Unable to locate package cmake
E: Unable to locate package automake
ERROR: Service 'rtl_433' failed to build: The command '/bin/sh -c apt-get update && apt-get install -y git libtool libusb-1.0.0-dev librtlsdr-dev rtl-sdr cmake automake && git clone https://github.com/merbanan/rtl_433.git /tmp/rtl_433 && cd /tmp/rtl_433/ && mkdir build && cd build && cmake ../ && make && make install' returned a non-zero code: 100

Hi, I'm missing Chronograph in your stack. Could you add it?

Step 3/5 : COPY requirements.txt ./
ERROR: Service 'python' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder007370358/requirements.txt: no such file or directory

i thought that maybe this error was due to me using an external hdd to run everything from so i did a clean install on a sd without the ssd and still had this error. not sure if others are also getting it...

Tested Services on experimental:

• ☑ adminer
• ☑ blynk_server - Redirects to blynk.cc
• ☑ deconz
• ☑ diyhue
• ☑ dozzle
• ☑ espruinohub
• ☑ gitea
• ☑ grafana
• ☐ home_assistant (Container)
• ☐ homebridge
• ☑ influxdb
• ☑ mariadb
• ☑ mosquitto
• ☐ motioneye
• ☑ nextcloud
• ☑ nodered
• ☑ openhab
• ☑ pihole
• ☐ plex
• ☑ portainer
• ☑ postgres
• ☑ python
• ☐ rtl_433
• ☑ tasmoadmin
• ☑ telegraf
• ☑ timescaledb
• ☑ transmission
• ☑ webthings_gateway
• ☐ zigbee2mqtt
• ☐ zigbee2mqtt_assistant

Issues:

• gitea - Need specific builds for arm and x86 - go-gitea/gitea#12323
• nextcloud - WSL specific, umask is 0000, causes permissions error. - nextcloud/docker#1188

Native Installs:

• ☑ Home Assistant (Supervisor). See: https://github.com/SensorsIot/IOTstack/blob/experimental/docs/Home-Assistant.md
• ☐ RTL_433
• ☐ RPIEasy

Issues:

Other features

• ☐ Install script (mentioned in readme).
• ☐ Backup and restore

Issues:

Menus

• ☑ Docker Commands
• ☑ Miscellaneous Commands
• ☐ Backup and restore

Issues:

Add Netdata to install stack for performance and monitoring.

https://hub.docker.com/r/netdata/netdata/tags

The current main page https://sensorsiot.github.io/IOTstack/ is outdated and much more complicated than needed. The README.md text or similar would be better for the main page. Perhaps a youtube thumbnail would be nice as well, but that is a minor thing.

Keep up the good work! :)

Have you struck this with PiHole?

I have tracked down this problem to an issue that is (probably) unique to IOTstack. It occurs under a very specific set of circumstances:

• gcgarner/IOTstack installed on or before 3 December 2019 (doesn't matter if it has since been upgraded to SensorsIot/IOTstack); and
• menu.sh run between 28 November 2019 and 3 December 2019.

It is also possible that the issue could cause problems with containers other than PiHole.

A quick test to see if this applies to you

Try the following:

$ cd ~/IOTstack
$ ls -ald backups volumes
drwxr-xr-x+  3 pi   pi   4096 Aug 19 11:39 backups
drwxr-xr-x+ 10 root root 4096 Aug 19 11:16 volumes

The "+" at the end of the flags indicates that an Access Control List (ACL) is present. If you see this on both backups and volumes then this issue almost certainly applies to your IOTstack.

More generally, if you see a "+" anywhere within the backup or volumes hierarchy then this issue might apply to your IOTstack.

The fix

$ cd ~/IOTstack
$ docker-compose down
$ sudo setfacl -bR ./backups ./volumes
$ docker-compose up -d

The gory details

At the time of writing (19 Aug 2020), lines 95..105 of menu.sh contain:

# this function creates the volumes, services and backup directories. It then assisgns the current user to the ACL to give full read write access
docker_setfacl() {
	[ -d ./services ] || mkdir ./services
	[ -d ./volumes ] || mkdir ./volumes
	[ -d ./backups ] || mkdir ./backups
	[ -d ./tmp ] || mkdir ./tmp

	#give current user rwx on the volumes and backups
	[ $(getfacl ./volumes | grep -c "default:user:$USER") -eq 0 ] && sudo setfacl -Rdm u:$USER:rwx ./volumes
	[ $(getfacl ./backups | grep -c "default:user:$USER") -eq 0 ] && sudo setfacl -Rdm u:$USER:rwx ./backups
}

All of the above lines, except one, were added by Graham Garner on 28 November 2019. The exception is the line ending in mkdir ./tmp added by Slyke on 28 May 2020 as part of PR#40 but, as we will see in a moment, that line doesn't do anything.

Lines 428..429 of menu.sh contain:

		#set the ACL for the stack
		#docker_setfacl

Graham Garner did the commenting-out on 3 December 2019 (which is why the mkdir ./tmp added on 28 May 2020 doesn't do anything).

See also gcgarner/IOTstack/issues/91 - an issue about Mosquitto which chucked up a problem with setfacl not being installed.

Distilling all that down to its essence, there was a period between 28 November 2019 and 3 December 2019 when running menu.sh would have resulted in these two commands being executed:

sudo setfacl -Rdm u:$USER:rwx ./volumes
sudo setfacl -Rdm u:$USER:rwx ./backups

As the comment for docker_setfacl() explains, the intention was to give the current user full access to everything in backups and volumes. On an RPi, the current user is pi (user ID 1000).

Each time PiHole launches, it runs some checks and fixes things it doesn't like. Those fixes include setting owner and group IDs equal to 999 for:

• ~/IOTstack/volumes/pihole/etc-pihole
• ~/IOTstack/volumes/pihole/etc-pihole/gravity.db
• ~/IOTstack/volumes/pihole/etc-pihole/pihole-FTL.conf

Those paths are outside the container. Inside the container, etc-pihole maps to /etc/pihole/.

Inside the container, user ID 999 is username "pihole", the same with group ID 999 being group "pihole". Outside the container, user ID 999 does not exist while group ID 999 is group "spi".

The combination of the ACL outside the container and the change of owner and group inside the container results in:

$ getfacl ~/IOTstack/volumes/pihole/etc-pihole/gravity.db 
getfacl: Removing leading '/' from absolute path names
# file: home/pi/IOTstack/volumes/pihole/etc-pihole/gravity.db
# owner: 999
# group: spi
user::rw-
user:pi:rwx			#effective:rw-
group::rwx			#effective:rw-
mask::rw-
other::r--

I don't pretend to understand precisely what is going on but this combination somehow causes PiHole (the running container) to be denied write access to gravity.db.

PiHole is running as root both inside and outside the container:

$ docker exec pihole ps -C pihole-FTL -o euser,ruser,suser,fuser,comm
EUSER    RUSER    SUSER    FUSER    COMMAND
root     root     root     root     pihole-FTL

$ ps -C pihole-FTL -o euser,ruser,suser,fuser,comm
EUSER    RUSER    SUSER    FUSER    COMMAND
root     root     root     root     pihole-FTL

and, from the perspective of pihole-FTL running inside the container, gravity.db has these permissions:

$ docker exec pihole ls -l /etc/pihole/gravity.db
-rw-rw-r--+ 1 pihole pihole 5255168 Aug 19 13:52 /etc/pihole/gravity.db

so I can't quite see why this results in write access being refused.

Still, any IOTstack where menu.sh was not run between 28 November 2019 and 3 December 2019 is unlikely to have any ACLs present so they obviously aren't needed. The fix above definitely cured my PiHole problem so my advice is, "if you see ACLs anywhere in your IOTstack, run the fix".

Add Wireguard installation template

Because the repo https://github.com/gcgarner/IOTstack is no longer active and has been migrated over to here, I post my problem as a new issue:

First I have installed the NRCHKB-Node in Node-Red (docker-container from the github-Repo "gcgarner/IOTstack") and I could see the device in Homekit but if I try to add that device it just runs into a timeout.

Then I have installed the NRCHKB-Node in Node-Red (docker-container from the forked github-Repo "SensorsIot/IOTstack") and now I can't see any new Homekit-device anymore !!!

The problems above don't occur with Node-Red standard installation. So it must have something to do with Docker or the network configuration in Docker. All Firewall-Rules between my VLAN's are de-activated...

On the github repo to the NRCHKB docker file (https://github.com/RaymondMouthaan/node-red-homekit-docker) is mentioned that you have to add network_mode: "host" to your config otherwise it will not work. When I do this, however, it has no effect on the network settings: in the Portainer Web-UI, the nodered-container still remains assigned to the network "iotstack_default" (network mode bridged). When I try to connect the nodered container to the "host" network in the Portainer Web UI, an error message appears.

Maybe I would have more information if I could start node-red in debug mode as described by Shaquu (NRCHKB/node-red-contrib-homekit-bridged#252) but I don't know how to use the extended debug command "DEBUG = NRCHKB, Accessory, HAPServer, EventedHTTPServer node-red" on a docker container.

Mosquitto security does not persist

Problem

After working through the documentation on how to set up a username and password for Mosquitto, I have found that it "works" but is not persistent.

Rather than just prepare a pull request, I wanted to draw attention to the security issue in case my proposed solution raises red flags for anyone.

The basic problem is that the internal container path for the password file is not mapped to any external path in the template service.yml and, therefore, not in the docker-compose.yml and not anywhere in the scope of ~/IOTstack/volumes/mosquitto.

As soon as the container is rebuilt, the internal password file disappears and the container constantly restarts until you disable the line in mosquitto.conf that enables password checking.

I do not understand why this problem hasn't been reported before. The only mention I can find is:

• gcgarner/IOTstack issue 32

but that is about a path inconsistency.

Discussion

At the moment, the mosquitto template service.yml looks like this:

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    restart: unless-stopped
    user: "1883"
    ports:
      - 1883:1883
      - 9001:9001
    volumes:
      - ./volumes/mosquitto/data:/mosquitto/data
      - ./volumes/mosquitto/log:/mosquitto/log
      - ./services/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
      - ./services/mosquitto/filter.acl:/mosquitto/config/filter.acl

Given the documented command for creating a password:

mosquitto_passwd -c /mosquitto/config/pwfile MYUSER

The obvious solution would be another volumes mapping like:

      - ./volumes/mosquitto/pwfile:/mosquitto/config/pwfile

I am not a great fan of volume-mapping files. They can be pernickety and a bit restrictive. I prefer mapping directories.

Proposed solution

1. Add this volume mapping to the template service.yml:

   - ./volumes/mosquitto/pwfile:/mosquitto/pwfile
   

2. Add this line to the template directoryfix.sh:

   sudo mkdir -p ./volumes/mosquitto/pwfile/
   

3. Change the relevant line in the template mosquitto.conf to read:

   #password_file /mosquitto/pwfile/pwfile
   

4. Change the documentation so that the command reads:

   mosquitto_passwd -c /mosquitto/pwfile/pwfile «MYUSER»
   

Hi & thank you for putting IOTstack together!

I noticed that right after installing a small set of IOTstack containers (i.e. portainer, node-red, influxdb, grafana, eclipse-mosquitto, pi-hole, rtl_433 to mqtt) on a fresh raspbian OS there are quite frequent writes to the disk on an otherwise completely idle pi. Over 24 hours these writes amounted to some 5 GB, which go towards shortening the life of the SD card or SSD. I was wondering if you could offer any hints to limit these writes? Are they really necessary considering the system is completely idle otherwise?

I've tried to identify the "hot file(s)" but I got nowhere so far (I could not get sysdig to work due to some compilation errors when building dkms and I didn't have time to troubleshoot it further; sysdig docker container also throws some errors so no luck with that either).

Many thanks!
Dan

Copying this from: gcgarner#128

Found some example code that will achieve this: https://stackoverflow.com/questions/47424865/merge-two-yaml-files-in-python

my-custom.yml example:

services:
  pihole:
    restart: always
    extra_hosts:
      - router: 192.168.1.1
      - iotstack 192.168.1.5

The current service.yml for pihole:

pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "8089:80/tcp"
      #- "443:443/tcp"
    env_file:
      - ./services/pihole/pihole.env
    volumes:
       - ./volumes/pihole/etc-pihole/:/etc/pihole/
       - ./volumes/pihole/etc-dnsmasq.d/:/etc/dnsmasq.d/
    dns:
      - 127.0.0.1
      - 1.1.1.1
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
    restart: unless-stopped

The output to docker-compose.yml:

  pihole:
    extra_hosts:
      - router: 192.168.1.1
      - iotstack 192.168.1.5
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "8089:80/tcp"
      #- "443:443/tcp"
    env_file:
      - ./services/pihole/pihole.env
    volumes:
       - ./volumes/pihole/etc-pihole/:/etc/pihole/
       - ./volumes/pihole/etc-dnsmasq.d/:/etc/dnsmasq.d/
    dns:
      - 127.0.0.1
      - 1.1.1.1
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
    restart: always

You can see that restart is overridden by the value in my-custom.yml and that extra_hosts with it's properties is added in.

Will likely add this in as a Python script. That means Python will have to be installed on the host system. Could also just use a docker container with this script in it to generate. Will do some research.

Hi,

i get an error with the backup script.
The result is an empty backup on my dropbox :(

pi@intranet:~/IOTstack $ ~/IOTstack/scripts/docker_backup.sh
~/IOTstack ~/IOTstack
/home/pi/IOTstack/scripts/docker_backup.sh: line 8: list.txt: Permission denied
/home/pi/IOTstack/scripts/docker_backup.sh: line 9: list.txt: Permission denied
/home/pi/IOTstack/scripts/docker_backup.sh: line 10: list.txt: Permission denied
Moving old influxdb backups if they exist
backing up influxdb database
influxdb backup complete
/home/pi/IOTstack/scripts/docker_backup.sh: line 15: list.txt: Permission denied
compressing stack folders

.templates/motioneye/service.yml has an issue where the last 2 lines are commented out, causing usb cameras to not work. uncommenting, or adding an option in ./menu.sh to toggle comments, or warning the user that this happened for security would be nice.

Any reason this isn't included as an option?
If not, I believe I have it working and can do a PR.

After running menu.sh setup, I immediately ran docker-compose up -d as suggested in the instructions. I received the following error message:

pi@raspberrypi:~/IOTstack $ docker-compose up -d
ERROR: Named volume "transm_config_volume:/config:rw" is used in service "transmission" but no declaration was found in the volumes section.

I'm too much of a Docker noob to know how to get in and tinker with this, so I'm afraid I can't provide much insight as to why this is happening. I couldn't find any documentation to fall back to; it looks like the Transmission container is a recent experimental addition to the stack.

Installed new Raspbian with new IOTstack with more services. But Homebridge and Dozzle don't have assigned ports.

pi@raspberrypi:~ $ docker ps --format "table {{.Names}}\t{{.Ports}}"
NAMES               PORTS
portainer           0.0.0.0:9000->9000/tcp
homebridge
nodered             0.0.0.0:1880->1880/tcp
mosquitto           0.0.0.0:1883->1883/tcp
dozzle
tasmoadmin          443/tcp, 0.0.0.0:8088->80/tcp
grafana             0.0.0.0:3000->3000/tcp
influxdb            0.0.0.0:2003->2003/tcp, 0.0.0.0:8083->8083/tcp, 0.0.0.0:8086->8086/tcp
deconz              0.0.0.0:443->443/tcp, 0.0.0.0:8090->80/tcp, 0.0.0.0:5901->5900/tcp

When building stack with Node RED and the Home Assistant websocket plugin, the HA plugin is not installed in Node RED. I also tried to execute the npm command directly in the Node RED Docker terminal, and it returns the following error:

bash-5.0# npm install node-red-contrib-home-assistant-websocket
npm ERR! code E404
npm ERR! 404 Not Found - GET http://registry.npmjs.org/@hapi%2fjoi - Not found
npm ERR! 404 
npm ERR! 404  '@hapi/[email protected]' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of 'node-red-contrib-home-assistant-websocket'
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-08-06T09_32_26_807Z-debug.log

The end of the debug log:

59 verbose stack Error: 404 Not Found - GET http://registry.npmjs.org/@hapi%2fjoi - Not found
59 verbose stack     at res.buffer.catch.then.body (/usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:104:15)
59 verbose stack     at process._tickCallback (internal/process/next_tick.js:68:7)
60 verbose statusCode 404
61 verbose pkgid @hapi/[email protected]
62 verbose cwd /usr/src/node-red
63 verbose Linux 5.4.51-v7+
64 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "node-red-contrib-home-assistant-websocket"
65 verbose node v10.21.0
66 verbose npm  v6.14.4
67 error code E404
68 error 404 Not Found - GET http://registry.npmjs.org/@hapi%2fjoi - Not found
69 error 404
70 error 404 '@hapi/[email protected]' is not in the npm registry.
71 error 404 You should bug the author to publish it (or use the name yourself!)
72 error 404 It was specified as a dependency of 'node-red-contrib-home-assistant-websocket'
73 error 404 Note that you can also install from a
74 error 404 tarball, folder, http url, or git url.
75 verbose exit [ 1, true ]

Just deleted the Node-RED container (and the volumes/nodered folder) and tried installing it again via IOTStack.

I tried pre-installing some nodes from the menu, but the selected notes were not installed.

Regardless of my selection, only pigpiod, Home Assistant and influxdb nodes are pre-installed. Even if I deselect them when installing Node-RED.

Add Ubiquiti UniFi controller support

Taken from: https://stackoverflow.com/questions/38839444/in-ssh-shell-script-check-for-docker-version

SERVER_VERSION=$(docker version -f "{{.Server.Version}}")
SERVER_VERSION_MAJOR=$(echo "$SERVER_VERSION"| cut -d'.' -f 1)
SERVER_VERSION_MINOR=$(echo "$SERVER_VERSION"| cut -d'.' -f 2)
SERVER_VERSION_BUILD=$(echo "$SERVER_VERSION"| cut -d'.' -f 3)

if [ "${SERVER_VERSION_MAJOR}" -ge 19 ] && \
   [ "${SERVER_VERSION_MINOR}" -ge 0 ]  && \
   [ "${SERVER_VERSION_BUILD}" -ge 3 ]; then
    echo "Docker version >= 19.0.3 it's ok"
else
    echo "Docker version less than 19.0.3 can't continue"
    exit 1
fi

When the script is starting, warn the user if they do not have the minimum docker version (as specified: https://docs.docker.com/compose/compose-file/compose-versioning/ ) of 18.02.0+

Hi, do you have any idea why I am getting an error when installing hassocks.io?
Thank you, Daniel

ERROR: No NetworkManager support on host.

As mentioned in the broader issue on rewriting the menu #33 and as discussed more specifically in PR #24 a configuration option / screen for each container would be very useful and a much-appreciated feature. (This is my way of including the discussion surrounding this feature in an issue rather than in a pull request)

Until the compose-override feature #38 is implemented, I'm not sure how to merge configurations (amendments to the service.yml files in the services folder) into docker-compose without running through the build menu a second time, which is how I do it now. Is there a command that will simply rerun the docker-compose merging that happens when using the menu (without having to run through the menu again)? Or is there a way to add a configuration menu (like Node-RED) in the current version of the menu.sh?

Some users have been reporting wrong version in Docker compose errors. This is probably due to #7 Issue #28 should be able to mitigate this error.

Hi guys,

I've just found out there is a wrong path in mosquitto.conf file for enabling passwords.
There stands:

password_file /mosquitto/pwfile/pwfile

As this didn't work for me I looked inside into the container and saw the pwfile sitting in the config folder not pwfile folder. I don't know why is the pwfile folder there and mapped to local pwfile folder. I suppose the pwfile should sit in this folder.

If I changed the path to:

password_file /mosquitto/config/pwfile

everything runs smoothly again.

I'm a big fan of this stack, great work!
I preferred running this on an Ubuntu server rather than a Raspberry Pi and thought I would share some of the changes I had to make. Maybe this is something for the documentation? The chapter on running on a Virtual Machine in the wiki is not the most extensive 😄

I had an issue with docker versions, manually changing the docker-compose file to version 3.3 fixed this. Not sure what the issue here is.

for some containers, the specification in the docker-compose file is not suitable for ubuntu. For example I had to change the homebridge entry to: oznu/homebridge:latest

There were also some issues with starting the stack, this was also reported on the original repo

run:

export DOCKER_HOST=127.0.0.1:2375
sudo service docker start
sudo docker-compose up -d

after rebooting, run:

sudo service docker start
sudo docker-compose up -d

https://www.home-assistant.io/blog/2020/01/29/changing-the-home-assistant-brand/

After the name change users might otherwise think they're are getting an old version.

We re-cloned from this forked project and went through the setup again. Got the same error as before.

root@wanda:/IOTstack# docker-compose up -d
Building python
Step 1/5 : FROM python:3
---> ac7e76fc5dfc
Step 2/5 : WORKDIR /usr/src/app
---> Using cache
---> 3482ade3362a
Step 3/5 : COPY requirements.txt ./
ERROR: Service 'python' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder249099125/requirements.txt: no such file or directory
root@wanda:/IOTstack#

Hi, any idea why I am not able to install Node-RED? I get the following erro:
ERROR: Service 'nodered' failed to build: error creating aufs mount to /var/lib/docker/aufs/mnt/6fc4c2becaeb615527e0c4b88bfb259114b5f71a14f26395110d13e6fcc6c49d-init: mount target=/var/lib/docker/aufs/mnt/6fc4c2becaeb615527e0c4b88bfb259114b5f71a14f26395110d13e6fcc6c49d-init data=br:/var/lib/docker/aufs/diff/6fc4c2becaeb615527e0c4b88bfb259114b5f71a14f26395110d13e6fcc6c49d-init=rw:/var/lib/docker/aufs/diff/5da31066f8873d60b8d3d1bce5e0e2550b166b8efccda28fab9adaa7ca4a4969=ro+wh:/var/lib/docker/aufs/diff/2820ff712f3d99dabe38e3d3d727aa646af5dc4bedc530085d16e482405372f4=ro+wh:/var/lib/docker/aufs/diff/d3ac95d081d6cc3182673e6572e5b0452232560425ce0a1a5fe247d4d2647d27=ro+wh:/var/lib/docker/aufs/diff/d42c6013397cae57cc7e71419101e8b10234abbbbd8eccaa75968493b60bb62c=ro+wh:/var/lib/docker/aufs/diff/0772d6721a4a5b57a56447a079ea4c1bc4e38279eb972571c8379d00dd90ee6d=ro+wh:/var/lib/docker/aufs/diff/05ea7484c8147dadf5a5b97f25a89270f2eb7a065220ba139a841400586b29f9=ro+wh:/var/lib/docker/aufs/diff/81ae9e728993651e6a53cea7e36aa87f3cf4648fee5e318d19c6aa0394369c0b=ro+wh:/var/lib/docker/aufs/diff/2eb07fff6b64e783713e8241d6f991c7830d9f2e449beecab23755aafde48248=ro+wh:/var/lib/docker/aufs/diff/bc4dff8d2bff05f109928047588af6e94795711927259eff0943ebc10da57173=ro+wh:/var/lib/docker/aufs/diff/a7c3b9932c3885b2e4694fdd2c5bed640a32dd480f846b136e9ad9cec9d1d357=ro+wh:/var/lib/docker/aufs/diff/263c59e30d0324cb90475d48ba953a84e2db3662881ab045043ca48d6e6c4c3b=ro+wh,dio,xino=/dev/shm/aufs.xino: invalid argument