openwisp / ansible-openwisp2 Goto Github PK

#19 introduced the openwisp-network-topology module developed by @rohithasrk but we forgot to add the cronjob to run ./manage.py update_topology which is needed to automatically update the topologies using the FETCH strategy every x minutes.

When openwisp2_network_topology is set to true we should add a cron that runs the update_topology command, eg:

3 * * * * /root/.virtualenvs/ninux_graph/bin/python /var/www/ninux_graph/manage.py update_topology

In order to control the cron frequency, we could copy the syntax used by the role finn.letsencrypt (which we suggest in our README to handle the SSL cert).

┆Attachments: openwisp-config #38 Automatic naming of specific sections

A small optimization that's easy to add: https://docs.djangoproject.com/en/1.10/ref/templates/api/#django.template.loaders.cached.Loader

But we need to activate the django cache with django redis with these two:

• http://niwinz.github.io/django-redis/latest/#_configure_as_cache_backend
• http://niwinz.github.io/django-redis/latest/#_configure_as_session_backend

We need these options to be added to nginx:

gzip on;
gzip_comp_level 6;
gzip_proxied any;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/html image/svg+xml application/json application/javascript text/xml text/css application/xml application/x-font-ttf font/opentype;

They need to be configurable (so it can be also turned off and details may be changed.

I think we can use this chance to add a generic mechanism to add configuration variable to nginx (which will contain the default gzip values too), eg:

openwisp2_nginx_ssl_config:
  gzip: on
  gzip_comp_level 6
  # etc

The variables of openwisp2_nginx_ssl_config will be added only to the :443 section of the nginx configuration.

Ubuntu 16 has python3.5 preinstalled, the role doesn't work out of the box.

i think the nginx spdy module is replaced by httpd2. But HTTP/2 needs OpenSSL 1.0.2+ and most distributions deliver older OpenSSL Versions.

https://www.nginx.com/blog/supporting-http2-google-chrome-users/

I realized only now that with 3151bd0 we run the risk of installing more versions of dependencies that are not compatible with openwisp-controller unintentionally.

Suppose we release a new major version of one of the dependencies which is out of the range specified in the requirements of openwisp-controller, what would happen? It would be installed and OpenWISP 2 would break. That would affect negatively every user trying to install/upgrade their instance.

So the solution is to reintroduce the variables with the _dev suffix, and install the modules specified in the variables with the _pip suffix only if the corresponding _dev is enabled, and each _pip suffix variable should point by default to each repo's github tarball of master.

Hi, thanks for this new interface, the ruby one gave me headaches :)

I'm new to ansible and am having a hard time to understand how it works.

If I understood correctly, I have to deploy the server every time it restarts, is that correct?

If so, is there a way to deploy a server permanently, so it resists restarts?

What's the correct way to have a production server running with your software?

Regards!

Cassio

Hey,

I set up my local PC and the device as described in the Readme in section "Install OpenWISP2 locally (laptop, desktop pc)".
However, binding the ansible via the playbook to the localhost makes it accessible from your own PC but does not expose the ansible instance to other devices.

I fixed this by making the following changes to the playbook:

- hosts: <IP-ADDRESS OF PC>
  roles: [openwisp.openwisp2]
  vars:
    postfix_myhostname: "<HOSTNAME>"

Maybe change the Readme accordingly?

Installing locally fails on TASK [openwisp.openwisp2 : migrate], outputting the following error:

fatal: [0.0.0.0]: FAILED! => {"changed": false, "cmd": "./manage.py migrate --noinput", "msg": "\n:stderr: Traceback (most recent call last):\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 60, in get_new_connection\n    cur.execute(\"SELECT load_extension(%s)\", (self.spatialite_lib,))\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py\", line 328, in execute\n    return Database.Cursor.execute(self, query, params)\nsqlite3.OperationalError: /usr/lib/x86_64-linux-gnu/libspatialite.so.7: undefined symbol: sqlite3_spatialite_init\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"./manage.py\", line 10, in <module>\n    execute_from_command_line(sys.argv)\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line\n    utility.execute()\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/core/management/__init__.py\", line 356, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/core/management/base.py\", line 283, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/core/management/base.py\", line 330, in execute\n    output = self.handle(*args, **options)\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/core/management/commands/migrate.py\", line 81, in handle\n    connection.prepare_database()\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 72, in prepare_database\n    with self.cursor() as cursor:\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/db/backends/base/base.py\", line 254, in cursor\n    return self._cursor()\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/db/backends/base/base.py\", line 229, in _cursor\n    self.ensure_connection()\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/db/backends/base/base.py\", line 213, in ensure_connection\n    self.connect()\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/db/backends/base/base.py\", line 189, in connect\n    self.connection = self.get_new_connection(conn_params)\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 65, in get_new_connection\n    six.reraise(ImproperlyConfigured, ImproperlyConfigured(new_msg), sys.exc_info()[2])\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/utils/six.py\", line 685, in reraise\n    raise value.with_traceback(tb)\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 60, in get_new_connection\n    cur.execute(\"SELECT load_extension(%s)\", (self.spatialite_lib,))\n  File \"/opt/openwisp2/env/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py\", line 328, in execute\n    return Database.Cursor.execute(self, query, params)\ndjango.core.exceptions.ImproperlyConfigured: Unable to load the SpatiaLite library extension \"libspatialite.so.7\" because: /usr/lib/x86_64-linux-gnu/libspatialite.so.7: undefined symbol: sqlite3_spatialite_init\n", "path": "/opt/openwisp2/env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin", "state": "absent", "syspath": ["/tmp/ansible_7an2FN", "/tmp/ansible_7an2FN/ansible_modlib.zip", "/tmp/ansible_7an2FN/ansible_modlib.zip", "/usr/lib/python2.7", "/usr/lib/python2.7/plat-x86_64-linux-gnu", "/usr/lib/python2.7/lib-tk", "/usr/lib/python2.7/lib-old", "/usr/lib/python2.7/lib-dynload", "/usr/local/lib/python2.7/dist-packages", "/usr/lib/python2.7/dist-packages"]}

It seems to be a problem with spatialite.

Fedora is a popular linux distribution that is suitable to be used for hosting an openwisp2 instance.

The parts to change to add support to Fedora are the following ones:

• management of system packages: use yum module instead of apt and the names of some packages are different
• nginx configuration, path differ slightly
• supervisord configuration, a few things differ here too

The current release fails on older systems (eg: ubuntu 14 LTS, debian 8), while it seems the latest commit fixes the issues but breaks installation on newer systems (ubuntu 16 LTS).

We need to solve this asap.

When the first release of openwisp-network-topology is ready, add a way to install, configure and upgrade the module.

@rohithasrk will help me implementing this feature. @leonardomaccari and @mikysal78 are also interested in using this feature.

It would be great to figure out a general way to extend ansible-openwisp2 with additional roles, that way we could allow adding extra features to ansible-openwisp2 without complicating it too much: additional features like the network-topology and the freeradius modules would then become additional ansible roles that depend on ansible-openwisp2, but this is just an early-stage idea.

I think it would be better to replace the section named "Install OpenWISP2 locally (laptop, desktop pc)" with a section that explains how to install OpenWISP 2 in a virtualbox VM, the same task we have asked GCI students to perform and for which they encountered all the same issues.

We can use their experience to prepare a step by step tutorial for this case that would probably work better than the kind of suggestion we give now.

1. Create a database of error's (maybe in wiki?)
   For example I (and HizkiFW) had this problem:

ERROR! the role 'openwisp.openwisp2' was not found in /home/artem/openwisp2-ansible-playbook/roles:/home/artem/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/artem/openwisp2-ansible-playbook
The error appears to have been in '/home/artem/openwisp2-ansible-playbook/playbook.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
roles:

• openwisp.openwisp2
  ^ here

2. Is working ansible-playbook -i hosts playbook.yml -u <user> -k --become -K command? (bcs I used root all the time)
3. Swap "Install OpenWISP2 locally" and steps (like "Install ansible") place
4. Add subtitles for video's of installing/using openwisp
5. And maybe record new videos about openwisp

Tried to install controller on SBC with armbian based on Debian Strech and got veeery long error.

The major clue in it was:
" The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source."
After installing libjpeg-dev, instalation went smoothly

https://github.com/openwisp/openwisp-controller/tree/gis

┆Attachments: gis

I think the following passages are a bit ambiguous/hard to understand, and could be improved.

• for "production server" we mean a server (not a laptop or a desktop computer!) with public ipv4 / ipv6 which is used to host openwisp2
• for "local machine" we mean the host from which you launch ansible, eg: your own laptop

Ansible is a configuration management tool that works by entering production servers via SSH, so you need to install it and configure it on the machine where you launch the deployment and this machine must be able to SSH into the production server.

This section does not explain that ansible is going to install openwisp2 on the production server. It also needs to make it clear that the local machine is going to be installing openwisp2 on the production server by controlling it via ssh.

sudo ansible-galaxy install openwisp.openwisp2

Using sudo to install the openwisp2 role messed with some of the permissions, preventing ansible from reading the role. It could be fixed by chowning the role directory, but could easily be prevnted by not using sudo during this step.

Create a new file hosts on your local machine with the following contents:

Could be reworded to mention the working directory that was made in the previous step.

Create a new playbook file playbook.yml on your local machine with the following contents:

Also add instructions to replace the hosts part in the yml file to the production server's hostname

Substitute openwisp2.mydomain.com with your hostname - DO NOT REPLACE openwisp2.mydomain.com WITH AN IP ADDRESS, otherwise email sending through postfix will break, causing 500 internal server errors on some operations.

This is the part that confused me when I first installed it. It says to replace the hostname with your hostname, which is very ambiguous, since your could refer to either the local machine or the production server. Since the user is currently working on the local machine, it is assumed that the your here is the local machine, when in fact it should be the production server.

The line become: "{{ become | default('yes') }}" means ansible will use the sudo program to run each command. You may remove this line if you don't need it (eg: if you are using the root user).

This line could be improved by stating that the root user here is for the production server, not for the local machine.

Run the playbook from your local machine with:

ansible-playbook -i hosts playbook.yml -u <user> -k --become -K

Substitute <user> with your user.

'with your user' is ambiguous; it could mean either the local machine's username or the production server.

Tip: if you have an error like Authentication or permission failure then try to use root user ansible-playbook -i hosts playbook.yml -u root -k

Also add another tip, if there is an error about adding the host's fingerprint to the known_hosts file, it could be easily done by manually connecting to the host via SSH before doing the install process.

When the playbook is done running, if you got no errors you can login at:

https://openwisp2.mydomain.com/admin
username: admin
password: admin

Substitute openwisp2.mydomain.com with your hostname.

Again, 'your' here is ambiguous. Should be replaced with 'production server'.

I will make have made a PR fixing these issues.

#25

┆Attachments: ansible-openwisp2 #25 [pip] Add a way to specify a different source for each openwisp2 module

The message added in #35 now is always shown, also on subsequent upgrades.

It would be better to show this message only when the superuser is created, we have a variable we can use for this.

so it would be just a matter of adding something like this to the debug msg:

when: '"created" in load_initial_data_result.stdout'

The host only-adapter is causing issues with internet connectivity on my VM and hence ansible can't download packages.

This issue is regarding the VM installation instructions.

Django 2.0 officially suggests to use python 3.5 or higher.

OpenWISP will soon be python 3 only.

Many users have troubles installing ansible via pip because they miss several dependencies:

• cryptography (python) and its system dependencies
• git

It would be useful to add a way to specify a different branch or repository for each module in order to test new features that are being developed in separate branches or in other repositories.

The postfix role fails if the apt cache on the production server isn't up to date. We can fix this by doing the APT cache update task before it.

Avoid email being sent as "webmaster@localhost".

The setting should have a good default value but also be customizable.

Installation in Debian Buster (aka testing) fails on the migrate step. Not entirely sure why.
Running in x86_64.

TASK [openwisp.openwisp2 : migrate] **********************************************************************************************************************
fatal: [openwisp.home.lan]: FAILED! => {"changed": false, "cmd": "./manage.py migrate --noinput", "msg": "\n:stderr: Traceback (most recent call last):\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 51, in get_new_connection\n    conn.load_extension(self.spatialite_lib)\nsqlite3.OperationalError: mod_spatialite: cannot open shared object file: No such file or directory\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"./manage.py\", line 10, in <module>\n    execute_from_command_line(sys.argv)\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/core/management/__init__.py\", line 371, in execute_from_command_line\n    utility.execute()\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/core/management/__init__.py\", line 365, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/core/management/base.py\", line 288, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/core/management/base.py\", line 335, in execute\n    output = self.handle(*args, **options)\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/core/management/commands/migrate.py\", line 77, in handle\n    connection.prepare_database()\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 61, in prepare_database\n    with self.cursor() as cursor:\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/db/backends/base/base.py\", line 255, in cursor\n    return self._cursor()\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/db/backends/base/base.py\", line 232, in _cursor\n    self.ensure_connection()\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/db/backends/base/base.py\", line 216, in ensure_connection\n    self.connect()\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/db/backends/base/base.py\", line 194, in connect\n    self.connection = self.get_new_connection(conn_params)\n  File \"/opt/openwisp2/env/lib/python3.6/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 55, in get_new_connection\n    ) from exc\ndjango.core.exceptions.ImproperlyConfigured: Unable to load the SpatiaLite library extension \"mod_spatialite\"\n", "path": "/opt/openwisp2/env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "state": "absent", "syspath": ["/tmp/ansible_5L_TsX", "/tmp/ansible_5L_TsX/ansible_modlib.zip", "/tmp/ansible_5L_TsX/ansible_modlib.zip", "/usr/lib/python2.7", "/usr/lib/python2.7/plat-x86_64-linux-gnu", "/usr/lib/python2.7/lib-tk", "/usr/lib/python2.7/lib-old", "/usr/lib/python2.7/lib-dynload", "/usr/local/lib/python2.7/dist-packages", "/usr/lib/python2.7/dist-packages"]}

Ansible fails in Ubuntu 14.04 on the migrate task.
See https://travis-ci.org/hispanico/ansible-openwisp2/jobs/368290716

We should automatically add ./manage.py clearsessions in a daily cronjob.

See this thread on the mailing list.

• Create a dockerfile

┆Attachments: #21 [idea] Device context

I think it's appropiate to add a note in the install instructions which warn about this issue with postfix:

https://groups.google.com/d/msg/openwisp/SeBTNkZbxDs/2peMEzzNCAAJ

So users can avoid it.

┆Attachments: ansible-openwisp2 #24 [django] Automatically add the clearsessions command to crontab

Invalid

By automating the creation and update of a letsencrypt SSL certificate we can offer a secure default installation.

Hi everyone :)

TLDR: GitMate.io acted on closed issues (and PRs) unneededly tonight, likely on your repo. The issue is fixed now and we're improving QA. More info at https://gitlab.com/gitmate/open-source/gitmate-2/issues/275.

You're getting this email because you have been affected by a bug in GitMate.io this night and you're using the feature that was buggy.

In short: GitMate's stale rules acted on closed issues and PRs. That might have been comments like "this issue will be closed now because it's inactive" or the addition of stale labels on already closed issues.

We are sorry that this happened and fixed the problem as soon as we were aware. We are also introducing new measures to improve our stability with a new QA stage in addition to our existing ones so those kinds of issues don't happen again.

Please let us know if you would like to get a script to revert those actions; it seems though as most communities would rather keep it than sending out additional notifications.

We've summarized the incident at https://gitlab.com/gitmate/open-source/gitmate-2/issues/275 .

Please reach out to us if you have any further questions about this.

Sincerely,

Lasse Schuirmann

[email protected]
Founder of http://coala.io/, http://viperdev.io/ and http://gitmate.io/

Add uid=www-data, ensuring dirs on which uwsgi has to write are writable.

Sometimes admins don't need or prefer not to have access to the management console of a controller from an untrusted network, although they will still want clients to connect to the controller and get configurations from these same untrusted netwoks. I suggest a role variable which would enable admins to restrict access to the management console allowing only computers from trusted networks to do so while Openwrt/LEDE clients from trusted and untrusted networks will still be able to work with Openwisp. This would improve security reducing the surface of attacks such as weak passwords and possibly other types of attacks.

We need to make it possible to enable CORS because other applications may need to access some APIs to get data.

We can use this module: django-cors-headers - which is configurable via django settings, we have to make sure we can configure every available setting of this module, possibly using a single ansible variable containing a dictionary.

This feature should be optional and turned off by default now.

These lines seem duplicated:

• https://github.com/openwisp/ansible-openwisp2/blob/master/.travis.yml#L31
• https://github.com/openwisp/ansible-openwisp2/blob/master/.gitlab-ci.yml#L16-L19

Maybe we can extract those lines in a script that gets called by both CI systems

While installing ansible-openwisp2 on Debian Stretch RC1 I face an uwsgi crash as ansible fetched uwsgi 2.0.14 from the PyPi repo. The crash is related to an OpenSSL 1.1 incompatibility.

However while digging and experimenting, Debian successfully installed uwsgi from its own repo.
In order to have a functionnal installation I modified the file :

/etc/ansible/roles/openwisp.openwisp2/tasks/pip.yml

with the following informations :

- name: Install uwsgi
  pip:
    name: http://http.debian.net/debian/pool/main/u/uwsgi/uwsgi_2.0.14+20170111.orig.tar.gz
    state: latest
    virtualenv: "{{ virtualenv_path }}"
    virtualenv_python: "{{ openwisp2_python }}"
    virtualenv_site_packages: yes
  with_items:
    - uwsgi
  notify: reload supervisor

That way pip fetch, build and doesn't complain.

Thanks

One patch still needs to be sent: nemesifier/Stouts.postfix@3c42624

Change the dependencies in meta/main.yml to use the latest Stouts.postfix role (instead of my fork, my patches have been merged months ago), remove my fork, install the main Stouts.postfix role in the local environment and test the installation again to ensure it works.

Try accessing the controller via HTTP, it should redirect to HTTPS but on Chrome it downloads a file instead.

Add support for django-netjsonconfig 0.3 and release new version.

I've started evaluating this project once again after a long break. I'm getting the following error when I try to deploy using ansible

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/etc/ansible/roles/openwisp.openwisp2/tasks/main.yml': line 3, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

## tasks
- import_tasks: variables.yml
  ^ here


The error appears to have been in '/etc/ansible/roles/openwisp.openwisp2/tasks/main.yml': line 3, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

## tasks
- import_tasks: variables.yml
  ^ here

Things work when I replace import_tasks with include.

ansible version - 2.3.1.0
Ubuntu 16.04.2 LTS

A cron to run save_snapshot command. This issue can be referenced after merging openwisp/openwisp-network-topology#17.