owntracks / quicksetup Goto Github PK

View Code? Open in Web Editor NEW

12.0 4.0 3.0 2.73 MB

A (mostly) automated installer for OwnTracks Recorder, Frontend with MQTT and Let's Encrypt

Home Page: https://owntracks.org/booklet/guide/quicksetup/

Shell 10.42% Python 32.24% Jinja 57.34%

acme ansible configuration frontend mqtt owntracks recorder

quicksetup's Issues

Show TLS cert subject during enrollment / deployment

How Quicksetup logs Let's Encrypt certificate renewals

This process is logged in the file /tmp/lego.log; note that /tmp/ was a deliberate choice to have it cleared out on reboot.

The content of this file increases over time, until it is removed, and can be used to determine whether Let's Encrypt renewals are working.

As an example, this following was logged on one of our systems:

2024-03-18T13:24:53
2024/03/18 14:24:53 No key found for account jane@example. Generating a P256 key.
2024/03/18 14:24:53 Saved key to /usr/local/owntracks/tls/.lego/accounts/acme-v02.api.letsencrypt.org/jane@example/keys/[email protected]
2024/03/18 14:24:54 [INFO] acme: Registering account for jane@example
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/usr/local/owntracks/tls/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2024/03/18 14:24:54 [INFO] [example.org] acme: Obtaining bundled SAN certificate
2024/03/18 14:24:54 [INFO] [example.org] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/327752274377
2024/03/18 14:24:54 [INFO] [example.org] acme: Could not find solver for: tls-alpn-01
2024/03/18 14:24:54 [INFO] [example.org] acme: use http-01 solver
2024/03/18 14:24:54 [INFO] [example.org] acme: Trying to solve HTTP-01
2024/03/18 14:24:55 [INFO] [example.org] Served key authentication
2024/03/18 14:24:55 [INFO] [example.org] Served key authentication
2024/03/18 14:24:55 [INFO] [example.org] Served key authentication
2024/03/18 14:25:00 [INFO] [example.org] The server validated our request
2024/03/18 14:25:00 [INFO] [example.org] acme: Validations succeeded; requesting certificates
2024/03/18 14:25:01 [INFO] [example.org] Server responded with a certificate.
2024-03-19T03:23:01
2024/03/19 04:23:02 [example.org] The certificate expires in 89 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-19T09:29:13
2024/03/19 10:29:14 [example.org] The certificate expires in 89 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-20T03:23:01
2024/03/20 04:23:01 [example.org] The certificate expires in 88 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-21T03:23:01
2024/03/21 04:23:02 [example.org] The certificate expires in 87 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-22T03:23:01
2024/03/22 04:23:01 [example.org] The certificate expires in 86 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-23T03:23:01
2024/03/23 04:23:01 [example.org] The certificate expires in 85 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-24T03:23:01
2024/03/24 04:23:02 [example.org] The certificate expires in 84 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-25T03:23:01
2024/03/25 04:23:02 [example.org] The certificate expires in 83 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-26T03:23:01
2024/03/26 04:23:02 [example.org] The certificate expires in 82 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-27T03:23:01
2024/03/27 04:23:01 [example.org] The certificate expires in 81 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-28T03:23:01
2024/03/28 04:23:01 [example.org] The certificate expires in 80 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-29T03:23:01
2024/03/29 04:23:01 [example.org] The certificate expires in 79 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-30T03:23:01
2024/03/30 04:23:02 [example.org] The certificate expires in 78 days, the number of days defined to perform the renewal is 30: no renewal.
2024-03-31T02:23:01
2024/03/31 04:23:01 [example.org] The certificate expires in 77 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-01T02:23:01
2024/04/01 04:23:02 [example.org] The certificate expires in 76 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-02T02:23:01
2024/04/02 04:23:02 [example.org] The certificate expires in 75 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-03T02:23:01
2024/04/03 04:23:01 [example.org] The certificate expires in 74 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-04T02:23:01
2024/04/04 04:23:02 [example.org] The certificate expires in 73 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-05T02:23:01
2024/04/05 04:23:02 [example.org] The certificate expires in 72 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-06T02:23:01
2024/04/06 04:23:02 [example.org] The certificate expires in 71 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-07T02:23:01
2024/04/07 04:23:01 [example.org] The certificate expires in 70 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-08T02:23:01
2024/04/08 04:23:02 [example.org] The certificate expires in 69 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-09T02:23:01
2024/04/09 04:23:02 [example.org] The certificate expires in 68 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-10T02:23:01
2024/04/10 04:23:01 [example.org] The certificate expires in 67 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-11T02:23:01
2024/04/11 04:23:01 [example.org] The certificate expires in 66 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-12T02:23:01
2024/04/12 04:23:02 [example.org] The certificate expires in 65 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-13T02:23:01
2024/04/13 04:23:02 [example.org] The certificate expires in 64 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-14T02:23:01
2024/04/14 04:23:02 [example.org] The certificate expires in 63 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-15T02:23:01
2024/04/15 04:23:07 [example.org] The certificate expires in 62 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-16T02:23:01
2024/04/16 04:23:02 [example.org] The certificate expires in 61 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-16T09:43:53
2024/04/16 11:43:54 [example.org] The certificate expires in 61 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-17T02:23:02
2024/04/17 04:23:02 [example.org] The certificate expires in 60 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-18T02:23:01
2024/04/18 04:23:01 [example.org] The certificate expires in 59 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-19T02:23:01
2024/04/19 04:23:01 [example.org] The certificate expires in 58 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-20T02:23:01
2024/04/20 04:23:02 [example.org] The certificate expires in 57 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-21T02:23:01
2024/04/21 04:23:01 [example.org] The certificate expires in 56 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-22T02:23:01
2024/04/22 04:23:01 [example.org] The certificate expires in 55 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-23T02:23:01
2024/04/23 04:23:02 [example.org] The certificate expires in 54 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-24T02:23:01
2024/04/24 04:23:02 [example.org] The certificate expires in 53 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-25T02:23:01
2024/04/25 04:23:02 [example.org] The certificate expires in 52 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-26T02:23:01
2024/04/26 04:23:01 [example.org] The certificate expires in 51 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-27T02:23:01
2024/04/27 04:23:02 [example.org] The certificate expires in 50 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-28T02:23:01
2024/04/28 04:23:02 [example.org] The certificate expires in 49 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-29T02:23:01
2024/04/29 04:23:01 [example.org] The certificate expires in 48 days, the number of days defined to perform the renewal is 30: no renewal.
2024-04-30T02:23:01
2024/04/30 04:23:01 [example.org] The certificate expires in 47 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-01T02:23:01
2024/05/01 04:23:02 [example.org] The certificate expires in 46 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-02T02:23:01
2024/05/02 04:23:01 [example.org] The certificate expires in 45 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-03T02:23:01
2024/05/03 04:23:02 [example.org] The certificate expires in 44 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-04T02:23:01
2024/05/04 04:23:01 [example.org] The certificate expires in 43 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-05T02:23:01
2024/05/05 04:23:02 [example.org] The certificate expires in 42 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-06T02:23:01
2024/05/06 04:23:01 [example.org] The certificate expires in 41 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-07T02:23:01
2024/05/07 04:23:01 [example.org] The certificate expires in 40 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-08T02:23:01
2024/05/08 04:23:02 [example.org] The certificate expires in 39 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-09T02:23:01
2024/05/09 04:23:02 [example.org] The certificate expires in 38 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-10T02:23:01
2024/05/10 04:23:02 [example.org] The certificate expires in 37 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-11T02:23:01
2024/05/11 04:23:02 [example.org] The certificate expires in 36 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-12T02:23:01
2024/05/12 04:23:02 [example.org] The certificate expires in 35 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-13T02:23:01
2024/05/13 04:23:01 [example.org] The certificate expires in 34 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-14T02:23:01
2024/05/14 04:23:01 [example.org] The certificate expires in 33 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-15T02:23:01
2024/05/15 04:23:02 [example.org] The certificate expires in 32 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-16T02:23:01
2024/05/16 04:23:02 [example.org] The certificate expires in 31 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-17T02:23:01
2024/05/17 04:23:02 [INFO] [example.org] acme: Trying renewal with 730 hours remaining
2024/05/17 04:23:02 [INFO] renewal: random delay of 5m14.287798892s
2024/05/17 04:28:16 [INFO] [example.org] acme: Obtaining bundled SAN certificate
2024/05/17 04:28:17 [INFO] [example.org] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/351713246817
2024/05/17 04:28:17 [INFO] [example.org] acme: Could not find solver for: tls-alpn-01
2024/05/17 04:28:17 [INFO] [example.org] acme: use http-01 solver
2024/05/17 04:28:17 [INFO] [example.org] acme: Trying to solve HTTP-01
2024/05/17 04:28:17 [INFO] [example.org] Served key authentication
2024/05/17 04:28:18 [INFO] [example.org] Served key authentication
2024/05/17 04:28:18 [INFO] [example.org] Served key authentication
2024/05/17 04:28:18 [INFO] [example.org] Served key authentication
2024/05/17 04:28:18 [INFO] [example.org] Served key authentication
2024/05/17 04:28:23 [INFO] [example.org] The server validated our request
2024/05/17 04:28:23 [INFO] [example.org] acme: Validations succeeded; requesting certificates
2024/05/17 04:28:24 [INFO] [example.org] Server responded with a certificate.
2024-05-18T02:23:01
2024/05/18 04:23:02 [example.org] The certificate expires in 88 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-19T02:23:01
2024/05/19 04:23:01 [example.org] The certificate expires in 87 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-20T02:23:01
2024/05/20 04:23:01 [example.org] The certificate expires in 86 days, the number of days defined to perform the renewal is 30: no renewal.
2024-05-21T02:23:01
...

bootstrap.sh should detect if it's being run as priviledged user

.. and bail otherwise.

Test http mode

We've done almost all testing on quicksetup with MQTT, which is obvious as we prefer that protocol.

I've tested HTTP POST, but we need to test whether an OwnTracks device (Android, iOS) correctly works on HTTP.

Reminder: we do not support WS over MQTT.

Mosquitto doesn't start up if LE isn't configured

Mosquitto doesn't launch if certificates / key aren't available when it starts up which, in our case can occur when LE isn't configured. (Reported by Ton.)

resource consumption on a 512 MB Digital Ocean droplet after 6 days of deployments

Not bad, I think.

$ free
               total        used        free      shared  buff/cache   available
Mem:          468740      125608       55036        4040      304200      343132
Swap:        2097148       52204     2044944

$ mosquitto_sub -u _lr -P "$(cat /usr/local/owntracks/userdata/.lr.pw)" -v -t '$SYS/broker/load/messages/#'

$SYS/broker/load/messages/received/1min 24.08
$SYS/broker/load/messages/sent/1min 105.69
$SYS/broker/load/messages/received/5min 11.80
$SYS/broker/load/messages/sent/5min 48.34
$SYS/broker/load/messages/received/15min 4.61
$SYS/broker/load/messages/sent/15min 18.64

Feature: A dedicated log-in page

Hi, thanks a lot for this web interface, it is great! You included everything I could wish for.

I was wondering if it would be possible to implement a separate log in page. At this time, the webpage requests a username and password in a pop-up window. This makes it more difficult to use a password manager to log in. If it would not be too much of a hassle, could you consider adding a /login page that would be shown to logged out users?

Installing new Frontend version leaves old files in place

When installing a new Frontend version we don't remove old files, i.e. no longer used code. This manifests itself, eg. when upgrading from 2.12 to 2.13.0 where the structure of the directories has changed.

Install ansible-core + required collections via pip

Instead of installing via packages, we should install via pip (to user). Installing ansible-core and the two required collections is likely faster and will consume less space.

The advantage for us is pinning the release we want and thus having same versions on Debian and Ubuntu.

Must verify we then have

passlib
requests
paho.mqtt

Friend relationships in MQTT / HTTP modes

Upon configuring Quicksetup we automagically set up friends and add them to mosquitto.acl, give them passwords in mosquitto.pw and htpasswd.

In Mosquitto's ACL we configure one friend can see all which at least for the moment is okay'ish.

But when configuring httpmode for a friend, there's no automatic addition of anybody to the friends in HTTP mode Recorder database.

This is both an enhancement and a bug. ;-)

User-specific waypoints for inline/otrc configuration

Similarly to how we create friend-specific MQTT ACLs, we could add user-specific waypoints to the inline/otrc configuration during bootstrapping.

A directory waypoints/ would contain <username>.json with an array of waypoints which is merged into .otrc.

This permits, say, a family to pre-configure "Home", "Sports", etc. for certain or all members.

Test on a Raspi

There's no real reason why it shouldn't work on Raspbian but we should test

Verify fresh Android install and OSM map

"mapLayerStyle" : "OpenStreetMapNormal"

that setting is now in .otrc config / inline config

configuration.yaml: do we need to differenciate ios/android devices?

OTRC config can contain keys which either OS doesn't support, but there are some keys with values that need to differ:

mqttProtocolLevel :
- ios: 5
- android: 3
waypoints :
- ios: +follow
- android: none

Add forwardme to waypoints in otrc

With randon rid

Trouble installing server

OwnTracks looks really interesting and I want to dabble with it, though my skills are obviously limited...

This is my first stab at setting up an OwnTracks server. I'm using an Ubuntu 22.04 VM on a locally hosted Proxmox server. It is definitely a dip of the toe in the water for me and I've tried to simplify it as much as possible, but I've still not done very well.

I followed the Quicksetup steps in the OwnTracks Booklet

downloaded quicksetup
edited configuration.yaml
Set dns_domain to owntracks.lan (it can be pinged)
Removed email address for Lets Encrypt config
Added OpenCage API
Added users
Ran bootstrap.sh

I got an error about installing ot-recorder but it seems to be installed (using 'which ot-recorder') but the service isn't running.
The mosquitto and nginx services are running

TESTING
I copied the otrc file and ran it on my Android device but it comes up with a status error of "Connection Refused"
I went to http://owntracks.lan and it came up with the "Welcome to nginx!" landing page, with no OwnTracks information.
I ran "mosquitto_sub -v -t 'owntracks/#'" but there was no output

SYS.INFO
Last bootstrap: 2024-08-04T16:58:46Z
Ansible version: 2.17.2
OS distro: Ubuntu / 22
OS distribution: jammy

I'm obviously doing something wrong but I don't know where to start looking. Any pointers you can give we would be greatly appreciated.

Front end URL could be improved

Steps to reproduce

Configure and run a new quicksetup installation with the hostname owntracks.example.com
Browse to https://owntracks.example.com.
Observe that there's nothing there but a default nginx page. Further observe that in order to access the expected content, you have to browse to https://owntracks.example.com/owntracks.

Preferred behavior

Since quicksetup assumes (and reasonably so) that it controls the server, then either:

The default URL should be https://owntracks.example.com; or
Browsing to / should redirect to /owntracks.

I would submit a pull request, but I don't know enough about ansible to even find where this is configured.

Do we support MQTT over Websockets?

Add users to mosquitto.pw idempotently

Would be nice to have: an Ansible module which does that, somewhat like community.general.htpasswd

mosquitto restart issue during install

I'm using IONOS VPS and have tried this a couple of times now. It fails at the same stage. Even reimagining my server and starting fresh it appears. Anyone else encountered this?

Using Ubuntu 22.4

Some further info.

root@ubuntu:~# journalctl -xeu mosquitto.service
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit mosquitto.service has entered the 'failed' state with result 'exit-code'.
May 25 00:53:01 ubuntu systemd[1]: Failed to start Mosquitto MQTT Broker.
░░ Subject: A start job for unit mosquitto.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit mosquitto.service has finished with a failure.
░░
░░ The job identifier is 3958 and the job result is failed.
May 25 00:53:01 ubuntu systemd[1]: mosquitto.service: Scheduled restart job, restart counter is at 5.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ Automatic restarting of the unit mosquitto.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
May 25 00:53:01 ubuntu systemd[1]: Stopped Mosquitto MQTT Broker.
░░ Subject: A stop job for unit mosquitto.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A stop job for unit mosquitto.service has finished.
░░
░░ The job identifier is 4043 and the job result is done.
May 25 00:53:01 ubuntu systemd[1]: mosquitto.service: Start request repeated too quickly.
May 25 00:53:01 ubuntu systemd[1]: mosquitto.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit mosquitto.service has entered the 'failed' state with result 'exit-code'.
May 25 00:53:01 ubuntu systemd[1]: Failed to start Mosquitto MQTT Broker.
░░ Subject: A start job for unit mosquitto.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit mosquitto.service has finished with a failure.
░░
░░ The job identifier is 4043 and the job result is failed.

Any help or advice greatly appreciated.

Add optional configuration for payload encryption

~~Key would have to be identical for all friends, as Recorder supports a single key only.~~

Recorder has support for per/user encryption keys.

Quicksetup acme "cant find solver" "firewall problem?"

I'm running a ubuntu jammy 24.04 vps on oracle cloud and a seperately hosted domain.
I followed all the steps in the "new" quicksetup in the booklet and everything worked fine running bootstrap.sh until task lego:enroll at letsencrypt.

I think its best if I just paste the error message here. I replaced what I thought was sensitive with IplaceholdersI

fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/usr/local/owntracks/lego/enroll.sh"], "delta": "0:00:18.573103", "end": "2024-05-04 22:08:31.223056", "msg": "non-zero return code", "rc": 1, "start": "2024-05-04 22:08:12.649953", "stderr": "", "stderr_lines": [], "stdout": "2024-05-04T22:08:12\n2024/05/04 22:08:13

[INFO] [IdomainI] acme: Obtaining bundled SAN certificate\n2024/05/04 22:08:14 
[INFO] [IdomainI] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/IauthidI\n2024/05/04 22:08:14 
[INFO] [IdomainI] acme: Could not find solver for: tls-alpn-01\n2024/05/04 22:08:14
[INFO] [IdomainI] acme: use http-01 solver\n2024/05/04 22:08:14 
[INFO] [IdomainI] acme: Trying to solve HTTP-01\n2024/05/04 22:08:31 
[INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/IauthidI\n2024/05/04 22:08:31 Could not obtain certificates:\n\terror: one or more domains had a problem:\n[IdomainI] acme: error: 400 :: urn:ietf:params:acme:error:connection
:: IIP-AddressI: Fetching http://IdomainI/.well-known/acme-challenge/Irandom lettersI: Timeout during connect (likely firewall problem), url: ", "stdout_lines": ["2024-05-04T22:08:12", "2024/05/04 22:08:13
 
[INFO] [IdomainI] acme: Obtaining bundled SAN certificate", "2024/05/04 22:08:14 
[INFO] [IdomainI] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/IauthidI", "2024/05/04 22:08:14 
[INFO] [IdomainI] acme: Could not find solver for: tls-alpn-01", "2024/05/04 22:08:14 
[INFO] [IdomainI] acme: use http-01 solver", "2024/05/04 22:08:14 
[INFO] [IdomainI] acme: Trying to solve HTTP-01", "2024/05/04 22:08:31 
[INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/IauthidI", "2024/05/04 22:08:31 Could not obtain certificates:", "\terror: one or more domains had a problem:", "[IdomainI] acme: error: 400 :: urn:ietf:params:acme:error:connection :: IIP-AddressI: Fetching http://IdomainI/.well-known/acme-challenge/Irandom lettersI: Timeout during connect (likely firewall problem), url: "]}

My question is mainly where the problem exactly lies. Is it a problem with the domain? With the settings of the VPS on Oracle Cloud? Or is it just a directory that doesnt have proper chmod. Im struggling with this error message for days now. I already configured the iptables to allow everything but still it just says "firewall problem"

This is the first Issue I ever post on Github, so Im sorry if im labeling it wrong 😅
Thanks in advance

-Gus

Change "mqttProtocolLevel"

Currently this is set to 3 (MQTTV31), but should be 4 (MQTTV311). We cannot use MQTTV5 because Android doesn't support it yet.

But we have not experience with MQTTV31, which was replaced by MQTTV311 in 2014!!!

Multiuser and MQTT passwords

wenn es mehrere devices per user gibt und dort verschiedene passwörter angegeben werden klappt der MQTT connect nicht, weil im otrc nicht das erste sondern die verschiedenen Passwörter stehen

Permission issue -- testing Quicksetup

I'm running MQTT and owntracks on the same raspberry pi. I have everything working pretty well except for a couple oddities with having a few older phones in my setup (running older versions of owntracks). It still works, but can't do certain things like enable them to receive commands to force a location update. (Seperate issue but not a show stopper)

On the server side, I want to be able to have more control over manipulating data. The data store is located in the default location which works, but I started seeing issues when I tried to delete a user. During testing I created a user with a typo in the name. Re-created a user on that device, but the old user is still there.

I tried the "Kill" command but got the "No comprendo" message. Doing a version check, however, shows that "WITH_KILL = yes" so ?
deleting a file is combersome as I can't use cyberduck or GUI due to permissions. I have to telnet in and use sudo commands to delete stuff in the var/spool/owntracks/recorder/store/rec/(user_directory)/(YYYY-MM.rec).

When I finally deleted the user file and folder under rec directory, the user is still visible on devices and on the recorder page.

I also tried getting around the permissions thing by moving the store to home/me/Documents/owntracks...../store (I made a full copy at that location).... then I get an error with owntracks... something about ghash Permissions Denied.

So I know I messed up something somewhere but I'm not sure what direction to go in now.

How-to change login password at https://owntracks.example/owntracks/ ?

Been following this: https://owntracks.org/booklet/guide/quicksetup/
I am using an Ubuntu VPS. It is all set up now and I was able to login to the page at https://owntracks.example/owntracks/

However, I wanted to change the password that was given to me automatically to something simple. So, in my VPS, I navigated to /usr/local/owntracks/userdata/ and then did nano username.pass. I changed the password here, pressed CTRL+X to exit, then chose "Y" to save the file.

Well, when I go to login to https://owntracks.example/owntracks/, it is still taking the old password and not the new one I set, even after a reboot of the VPS, and even though the username.pass file is showing my new password that I wanted. Help?

Battery shows 0% no matter what at https://owntracks.domain.com/owntracks/table/

So, when I locally hosted OwnTracks in a Docker container on Windows, my battery percentage showed properly on the server localhost table URL. I am now using a Linux VPS and I noticed at the table URL (https://owntracks.domain.com/owntracks/table/), the battery is showing 0%, even though I am at 100%. Why is this?

Image showing 0%: https://i.imgur.com/21K430O.png

Not sure how it's work fine on Docker container but not now that I'm using a proper VPS. If it matters, was using HTTP mode on Docker, but MQTT on VPS.

Safari keeps prompting for basic authentication (apache / nginx)

Cannot reproduce on either Firefox or Chrome (on Mac). Safari periodically asks for basic auth credentials in spite of saving them.

Other Configuration items

Is there documentation for quicksetup's configuration.yaml to specify the following:
--> Connect to and use an already existing MQTT broker within the network
--> Specifying an alternative storage directory

Assertion failing during OwnTracks installation

Hi, I'm trying to install the latest version. However, I'm facing the following issues during the bootstrap phase:

TASK [verify some requirements] ************************************************
fatal: [localhost]: FAILED! => {
"assertion": "ansible_distribution_release in [ 'bookworm', 'jammy' ]",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}

I'm running Ubuntu 23.10 and carefully followed all installation steps you provided.

tks!

DNS Challenge

Hi, would it be possible to use the Let's Encrypt DNS Challenge when running your quickstart script rather than the HTTP challenge? Port 80 is not allowed to be open on our firewall.

Nginx web issues

I have successfully installed OwnTracks using the quicksetup on a rpi3.
Everything works, except when i visit my dns domain it shows:

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

Quicksetup: Where does the data live?

I installed Owntracks using quicksetup. Both for the purposes of debugging an issue I'm facing and for planning a backup strategy, I'm wondering where the data lives. I haven't seen anything relevant in the documentation, nor have I found it by poking around my system.

Installing behind nginx proxy

Hello.

I tried to install owntracks behind an existing nginx reverse proxy.
TASK lego: enroll at letsencrypt failed.
What can i do to fix this error?

greets

User-specific MQTT ACLs?

When Quicksetup configures a system, we create a mosquitto.conf which can, if necessary, be augmented by user-specific files dropped into conf.d/; these will be picked up during a Mosquitto restart.

Assuming a user needs specific ACLs, however, we have no provision for specifying these. Bootstrapping creates a mosquitto.acl which, if necessary, is overwritten at each run.

It's probably worth our while, long-term, to look at the dynamic security plugin which is auto-installed with the Debian Mosquitto package.

check for minimum Ansible version in bootstrap

MQTT configuration changes

I'm working on two things that are both MQTT related with the recent change.

Short Question:
1) Where do I put the extra configuration lines for the own tracks-cards (listener 9001)

2) Can the 'listener 1883 127.0.0.1' be chaged to just 'listener 1883' for local (non ssl) access to the MQTT broker?

Long version of question 1)
Cards: I used the online tool to create a card https://avanc.github.io/owntracks-cards/. Since I'm really new to MQTT, I tried publishing the card via MQTT Explorer https://apps.apple.com/us/app/mqtt-explorer/id1455214828?mt=12

In that application, I used:
topic owntracks/indigo/myphone/info
copy/paste the json from the saved file created with the online tool. Click "Publish"

On the owntracks web interface... "Device Table"... it worked. The icon is there, and name is now "Indigo Server" instead of "IS". However, on the other devices, the card didn't populate? From my device "bill/myphone" I still see "IS" and just the circle with an "IS" in the middle. So it kida worked.

Next I decided to add the card page thing to my raspberry pi. https://github.com/avanc/owntracks-cards

Where I'm stuck.... where to put the extra configuration lines? Since mosquitto now uses owntracks.conf, I considered adding it to the end of that file (spoiler: don't do that) crashed mosquitto, removed the extra lines, restart rPi a couple times, now I'm back to square 1.

# -- 1883 ----- Plain (loopback only)
listener 1883 127.0.0.1

# -- 8883 ----- TLS
listener 8883
# cafile /etc/mosquitto/certs/.lego/certificates/isl.mynetgear.com.issuer.crt
cafile /usr/local/owntracks/tls/cert.crt
certfile /usr/local/owntracks/tls/cert.crt
keyfile /usr/local/owntracks/tls/cert.key
# -- ends

# -- 9001 ----- Cards
listener 9001
protocol websockets
http_dir /home/williammoore/owntracks-cards

Long version of question 2)
I'm having a booger of a time connecting my home automation system (on the same LAN network) to the MQTT broker. IPaddress/User/password is right. Tried DNS instead of IPaddress. Selected SSL/TLS. Tried adding the .cert file to my indigo server from /usr/local/owntracks/tls/cert.crt Every thing I have tried so far has yielded zero. But it is a local connection, so I'm not too concerned about the security of it and I closed port forwarding on 1883 when the MQTT changed to 8883. Can the 1883 config line be changed to:

# -- 1883 ----- Plain 
listener 1883
# -- 1883 ------ loopback
listener 1883 127.0.0.1

or something similar?

Add template validation for mosquitto. when 2.1 is released

Mosquitto 2.1 will add a --test-config switch:

Avoid clobbering passwords / otrc files when multiple devices in configuration.yaml have the same username / device

OwnTracks supports username/devicename, but during creation of quicksetup we assumed a user would have one device only.

Test whether this is a problem. After reviewing the mkpasswords filter, i don't actually think so ...

Should we configure unattended upgrades?

I've had good experience with unattended upgrades on Debian/Ubuntu, and I know from several people who say the same.

Should we clean up (rec files, retained messages, etc.) when users are deprovisioned from config.yaml?

Maybe a bit overkill at the moment, but doable.

Do we take care of all growing logfiles?

Where do ot-recorder logs end up (systemd in debian)? I think this is handled by systemd
mosquitto and nginx use logrotate
There is another log /var/log/php8.2-fpm.log ...

Check task duration: why does otrc generation take so long?

system: generate user .otrc files in userdata ------------------------------------- 12.14s
system: create password files in userdata ------------------------------------------ 8.43s
ot-recorder: add/delete keys for users which have them ----------------------------- 6.75s
nginx: add users to htpasswd ------------------------------------------------------- 6.25s
system: add ufw open ports --------------------------------------------------------- 4.71s
system: install required packages -------------------------------------------------- 4.11s
lego: enroll at letsencrypt -------------------------------------------------------- 1.87s
frontend: unpack dist -------------------------------------------------------------- 1.85s
ot-recorder: install package ------------------------------------------------------- 1.60s
system: create directories --------------------------------------------------------- 1.58s
ot-recorder: launch service -------------------------------------------------------- 1.54s
Gathering Facts -------------------------------------------------------------------- 1.46s
restart_mosquitto ------------------------------------------------------------------ 1.20s
lego: get certificate information -------------------------------------------------- 1.13s
system: enable ufw ----------------------------------------------------------------- 1.10s
system: template out sys.info ------------------------------------------------------ 1.09s
nginx: install OwnTrack's index.php ------------------------------------------------ 1.06s
lego: template out enroller -------------------------------------------------------- 1.06s
system: install OwnTracks repository key ------------------------------------------- 1.05s
lego: template out certificate/key installer --------------------------------------- 1.01s

Removing a user's secret from configuration doesn't clear Recorder's key for the user

If a friend is configured and deployed with secret key, and the key is later removed from configuration.yaml, the user-device entry Recorder's keys database remains in place.

In terms of idempotency we should fix this.

Track Frontend dist/ releases

We'll have to add automation for creating Frontend dist/ releases. There's an open request for doing so.

Ubuntu has no ansible_distribution_minor_version

ii  ansible                         2.10.7+merged+base+2.10.8+dfsg-1        all

$ ansible --version
ansible 2.10.8
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]

        "ansible_distribution": "Ubuntu",
        "ansible_distribution_file_parsed": true,
        "ansible_distribution_file_path": "/etc/os-release",
        "ansible_distribution_file_variety": "Debian",
        "ansible_distribution_major_version": "22",
        "ansible_distribution_release": "jammy",
        "ansible_distribution_version": "22.04",

Post quicksetup connection issues

Hey 👋

I've gone through and followed the quicksetup booklet and run the .bootstrap, which resulted in no errors. All of the expected services are running (e.g. ot-recorder, mosquitto, nginx), the expected ports are open, and the server shows no obvious errors.

I've used the resulting front end to configure my android device (via either/both the URLconfig and OTRC file options), however it has been unable to connect. And I see no payloads when running mosquitto_sub -v -t '#'. While tailing the mosquitto logs shows only pings being sent and received to/from otrec.

Any suggestions/help would be much appreciated!

The app provides the following log output:

2024-05-23 12:49:42.710 D MessageProcessor: Initializing MessageProcessor
2024-05-23 12:49:42.711 D MessageProcessor: message:null, 
2024-05-23 12:49:42.712 D MessageProcessor: Reloading outgoing message processor. ThreadID: Thread[main,5,main]
2024-05-23 12:49:42.715 D MessageProcessor: Starting outbound message loop. ThreadID: Thread[backgroundDequeueThread,5,main]
2024-05-23 12:49:42.715 D Scheduler: WorkManager queue task PERIODIC_TASK_MQTT_RECONNECT as b82ec93f-788d-40d9-8195-3b1dcc6d9ba1
2024-05-23 12:49:42.720 D MessageProcessorEndpointMqtt: Sending message Thread: Thread[networkHandlerThread,5,main]
2024-05-23 12:49:42.722 D MessageProcessorEndpointMqtt: Connecting to broker. ThreadId: Thread[networkHandlerThread,5,main]
2024-05-23 12:49:42.722 D MessageProcessorEndpointMqtt: Connecting on non-ui worker thread: Thread[networkHandlerThread,5,main]
2024-05-23 12:49:42.723 D MessageProcessorEndpointMqtt: Acquired connecting lock
2024-05-23 12:49:42.724 D MessageProcessor: message:null, 
2024-05-23 12:49:42.725 D MessageProcessorEndpointMqtt: Initializing new mqttClient
2024-05-23 12:49:42.726 D MessageProcessorEndpointMqtt: 
2024-05-23 12:49:42.801 I MQTTReconnectWorker: MQTTReconnectWorker started on threadID: Thread[androidx.work-1,5,main]
2024-05-23 12:49:58.579 D MessageProcessorEndpointMqtt: Releasing connectinglock
2024-05-23 12:49:58.581 D MessageProcessor: message:MQTT Error: , 
2024-05-23 12:49:58.583 W MessageProcessorEndpointMqtt: failed connection attempts: 1
2024-05-23 12:49:58.585 E MessageProcessor: Message delivery failed. queueLength: 106, messageId: 1716464982095-550a5c
2024-05-23 12:49:58.587 D MessageProcessorEndpointMqtt: MQTT not current connecting
2024-05-23 12:49:58.588 W MessageProcessor: Error sending message. Re-queueing
2024-05-23 12:49:58.588 D MessageProcessorEndpointMqtt: Connecting to broker. ThreadId: Thread[networkHandlerThread,5,main]
2024-05-23 12:49:58.590 D MessageProcessorEndpointMqtt: Connecting on non-ui worker thread: Thread[networkHandlerThread,5,main]
2024-05-23 12:49:58.590 D MessageProcessorEndpointMqtt: Acquired connecting lock
2024-05-23 12:49:58.592 D MessageProcessor: message:null, 
2024-05-23 12:49:58.593 D MessageProcessorEndpointMqtt: Disconnecting mqtt Client
2024-05-23 12:49:58.596 I MessageProcessor: Waiting for 1 s before retrying
2024-05-23 12:49:58.596 D MessageProcessorEndpointMqtt: Error disconnecting from mqtt client.
 (32101)
	at okio.Okio__OkioKt.createMqttException(Unknown Source:9)
	at org.eclipse.paho.client.mqttv3.internal.ClientComms.disconnect(SourceFile:113)
	at org.eclipse.paho.client.mqttv3.MqttAsyncClient.disconnect(SourceFile:51)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.connectToBroker(SourceFile:135)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.reconnect(SourceFile:46)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.lambda$reconnect$1(Unknown Source:0)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.$r8$lambda$3UR4gQL2b_UbF_ekJLIMhafolpU(Unknown Source:0)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt$$ExternalSyntheticLambda0.run(Unknown Source:18)
	at android.os.Handler.handleCallback(Handler.java:942)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loopOnce(Looper.java:201)
	at android.os.Looper.loop(Looper.java:288)
	at android.os.HandlerThread.run(HandlerThread.java:67)

2024-05-23 12:49:58.597 D MessageProcessorEndpointMqtt: Initializing new mqttClient
2024-05-23 12:49:58.598 D MessageProcessorEndpointMqtt: 
2024-05-23 12:50:14.474 D MessageProcessorEndpointMqtt: Releasing connectinglock
2024-05-23 12:50:14.476 D MessageProcessor: message:MQTT Error: , 
2024-05-23 12:50:14.479 E MessageProcessorEndpointMqtt: Failed to reconnect to MQTT broker
org.owntracks.android.services.MqttConnectionException:  (0) - java.net.SocketException: Connection reset
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.connectToBroker(SourceFile:243)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.reconnect(SourceFile:46)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.lambda$reconnect$1(Unknown Source:0)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.$r8$lambda$3UR4gQL2b_UbF_ekJLIMhafolpU(Unknown Source:0)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt$$ExternalSyntheticLambda0.run(Unknown Source:18)
	at android.os.Handler.handleCallback(Handler.java:942)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loopOnce(Looper.java:201)
	at android.os.Looper.loop(Looper.java:288)
	at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by:  (0) - java.net.SocketException: Connection reset
	at androidx.work.impl.utils.WorkForegroundUpdater$1.run(SourceFile:433)
	at java.lang.Thread.run(Thread.java:1012)
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:191)
	at java.net.SocketInputStream.read(SocketInputStream.java:143)
	at org.conscrypt.ConscryptEngineSocket$SSLInputStream.readFromSocket(Unknown Source:24)
	at org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(SourceFile:172)
	at org.conscrypt.ConscryptEngineSocket$SSLInputStream.access$100(Unknown Source:0)
	at org.conscrypt.ConscryptEngineSocket.doHandshake(Unknown Source:90)
	at org.conscrypt.ConscryptEngineSocket.startHandshake(Unknown Source:36)
	at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(Unknown Source:83)
	at androidx.work.impl.utils.WorkForegroundUpdater$1.run(SourceFile:174)
	... 1 more

Rate-Limited By OpenCage?

EDIT: Found the issue. Needed to re-import config file, even though I manually made the changes in the app on my end.

Hi, I just wanted to ask: Is the following log due to being rate-limited by OpenCage? If so, is there a better free alternative that allows many more, or even unlimited?

2024-03-21 15:49:54.385 W OpenCageGeocoder: Rate-limited, not querying until 2024-03-22T00:00:00Z
2024-03-21 15:49:59.299 W OpenCageGeocoder: Rate-limited, not querying until 2024-03-22T00:00:00Z
2024-03-21 15:50:01.054 W MessageProcessorEndpointMqtt: failed connection attempts: 13

Only noticed it because location data stopped updating in OT server.
I have tried removing the OpenCage API key from configuration & re-using sudo ./bootstrap.sh, as well as switching from OpenCage to Google in the app itself, but it seems I am still not getting location updates?

And now I am getting:

2024-03-21 16:10:08.254 W MessageProcessorEndpointMqtt: failed connection attempts: 34
2024-03-21 16:10:08.254 E MessageProcessor: Message delivery failed. queueLength: 4813, messageId: 1711050466922-87debc
2024-03-21 16:10:08.258 W MessageProcessor: Error sending message. Re-queueing
2024-03-21 16:10:08.262 I MessageProcessor: Waiting for 120 s before retrying
2024-03-21 16:10:18.613 I ServiceStarter$Impl: starting service
2024-03-21 16:10:40.221 I MQTTReconnectWorker: MQTTReconnectWorker started on threadID: Thread[androidx.work-2,5,main]
2024-03-21 16:10:40.753 E MessageProcessorEndpointMqtt: Failed to reconnect to MQTT broker
org.owntracks.android.services.MqttConnectionException:  (5)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.connectToBroker(SourceFile:243)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.reconnect(SourceFile:46)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.lambda$reconnect$1(Unknown Source:0)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt.$r8$lambda$3UR4gQL2b_UbF_ekJLIMhafolpU(Unknown Source:0)
	at org.owntracks.android.services.MessageProcessorEndpointMqtt$$ExternalSyntheticLambda0.run(Unknown Source:18)
	at android.os.Handler.handleCallback(Handler.java:938)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loop(Looper.java:246)
	at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by:  (5)
	at okio.Okio__OkioKt.createMqttException(Unknown Source:15)
	at org.eclipse.paho.client.mqttv3.internal.ClientState.notifyReceivedAck(SourceFile:259)
	at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(Unknown Source:91)
	at java.lang.Thread.run(Thread.java:923)

Please Delete This

Don't install a firewall

I just ran the script to set up Owntracks, and ended up losing SSH access to my server. Eventually I figured out this was due to ufw blocking the ports, and I was able to fix it. It was not mentioned in the installation guide anywhere that a firewall was going to be installed, so I was very confused about what had happened before I finally figured it out.

I don't think it's Owntracks's job to install a firewall, but if it really has to this has to be mentioned in the installation guide.

owntracks / quicksetup Goto Github PK

quicksetup's Issues

Steps to reproduce

Preferred behavior

Recommend Projects

Recommend Topics

Recommend Org