Giter VIP home page Giter VIP logo

django-loci's Introduction

django-loci

CI build status Dependency monitoring https://img.shields.io/gitter/room/nwjs/nw.js.svg?style=flat-square downloads code style: black

Reusable django-app for storing GIS and indoor coordinates of objects.

Indoor coordinates Map coordinates Mobile coordinates

Dependencies

Compatibility Table

django-loci Python version
0.2 2.7 or >=3.4
0.3 - 0.4 >=3.6
1.0 >=3.7
dev >=3.8

Install stable version from pypi

Install from pypi:

pip install django-loci

Install development version

First of all, install the dependencies of GeoDjango:

Install tarball:

pip install https://github.com/openwisp/django-loci/tarball/master

Alternatively you can install via pip using git:

pip install -e git+git://github.com/openwisp/django-loci#egg=django_loci

If you want to contribute, install your cloned fork:

git clone [email protected]:<your_fork>/django-loci.git
cd django_loci
python setup.py develop

Setup (integrate in an existing django project)

First of all, set up your database engine to one of the spatial databases suppported by GeoDjango.

Add django_loci and its dependencies to INSTALLED_APPS in the following order:

INSTALLED_APPS = [
    # ...
    'django.contrib.gis',
    'django_loci',
    'django.contrib.admin',
    'leaflet',
    'channels'
    # ...
]

Configure CHANNEL_LAYERS according to your needs, a sample configuration can be:

ASGI_APPLICATION = "django_loci.channels.asgi.channel_routing"
CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels.layers.InMemoryChannelLayer",
    },
}

Now run migrations:

./manage.py migrate

Troubleshooting

Common issues and solutions when installing GeoDjango.

Unable to load the SpatiaLite library extension

If you get the following exception:

django.core.exceptions.ImproperlyConfigured: Unable to load the SpatiaLite library extension

You need to specify the SPATIALITE_LIBRARY_PATH in your settings.py as explained in the django documentation regarding how to install and configure spatialte.

Issues with other geospatial libraries

Please refer to the geodjango documentation on troubleshooting issues related to geospatial libraries.

Settings

LOCI_FLOORPLAN_STORAGE

type: str
default: django_loci.storage.OverwriteStorage

The django file storage class used for uploading floorplan images.

The filestorage can be changed to a different one as long as it has an upload_to class method which will be passed to FloorPlan.image.upload_to.

To understand the details of this statement, take a look at the code of django_loci.storage.OverwriteStorage.

DJANGO_LOCI_GEOCODER

type: str
default: ArcGIS

Service used for geocoding and reverse geocoding.

Supported geolocation services:

  • ArcGIS
  • Nominatim
  • GoogleV3 (Google Maps v3)

DJANGO_LOCI_GEOCODE_FAILURE_DELAY

type: int
default: 1

Amount of seconds between geocoding retry API calls when geocoding requests fail.

DJANGO_LOCI_GEOCODE_RETRIES

type: int
default: 3

Amount of retry API calls when geocoding requests fail.

DJANGO_LOCI_GEOCODE_API_KEY

type: str
default: None

API key if required (eg: Google Maps).

System Checks

geocoding

Use to check if geocoding is working as expected or not.

Run this checks with:

./manage.py check --deploy --tag geocoding

Extending django-loci

django-loci provides a set of models and admin classes which can be imported, extended and reused by third party apps.

To extend django-loci, you MUST NOT add it to settings.INSTALLED_APPS, but you must create your own app (which goes into settings.INSTALLED_APPS), import the base classes of django-loci and add your customizations.

Extending models

This example provides an example of how to extend the base models of django-loci by adding a relation to another django model named Organization.

# models.py of your app
from django.db import models
from django_loci.base.models import (AbstractFloorPlan,
                                     AbstractLocation,
                                     AbstractObjectLocation)

# the model ``organizations.Organization`` is omitted for brevity
# if you are curious to see a real implementation, check out django-organizations


class OrganizationMixin(models.Model):
    organization = models.ForeignKey('organizations.Organization')

    class Meta:
        abstract = True


class Location(OrganizationMixin, AbstractLocation):
    class Meta(AbstractLocation.Meta):
        abstract = False

    def clean(self):
        # your own validation logic here...
        pass


class FloorPlan(OrganizationMixin, AbstractFloorPlan):
    location = models.ForeignKey(Location)

    class Meta(AbstractFloorPlan.Meta):
        abstract = False

    def clean(self):
        # your own validation logic here...
        pass


class ObjectLocation(OrganizationMixin, AbstractObjectLocation):
    location = models.ForeignKey(Location, models.PROTECT,
                                 blank=True, null=True)
    floorplan = models.ForeignKey(FloorPlan, models.PROTECT,
                                  blank=True, null=True)

    class Meta(AbstractObjectLocation.Meta):
        abstract = False

    def clean(self):
        # your own validation logic here...
        pass

Extending the admin

Following the previous Organization example, you can avoid duplicating the admin code by importing the base admin classes and registering your models with them.

But first you have to change a few settings in your settings.py, these are needed in order to load the admin templates and static files of django-loci even if it's not listed in settings.INSTALLED_APPS.

Add django.forms to INSTALLED_APPS, now it should look like the following:

INSTALLED_APPS = [
    # ...
    'django.contrib.gis',
    'django_loci',
    'django.contrib.admin',
    #      ↓
    'django.forms', # <-- add this
    #      ↑
    'leaflet',
    'channels'
    # ...
]

Now add EXTENDED_APPS after INSTALLED_APPS:

INSTALLED_APPS = [
    # ...
]

EXTENDED_APPS = ('django_loci',)

Add openwisp_utils.staticfiles.DependencyFinder to STATICFILES_FINDERS:

STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'openwisp_utils.staticfiles.DependencyFinder',
]

Add openwisp_utils.loaders.DependencyLoader to TEMPLATES:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'OPTIONS': {
            'loaders': [
                'django.template.loaders.filesystem.Loader',
                'django.template.loaders.app_directories.Loader',
                # add the following line
                'openwisp_utils.loaders.DependencyLoader'
            ],
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    }
]

Last step, add FORM_RENDERER:

FORM_RENDERER = 'django.forms.renderers.TemplatesSetting'

Then you can go ahead and create your admin.py file following the example below:

# admin.py of your app
from django.contrib import admin

from django_loci.base.admin import (AbstractFloorPlanAdmin, AbstractFloorPlanForm,
                                    AbstractFloorPlanInline, AbstractLocationAdmin,
                                    AbstractLocationForm, AbstractObjectLocationForm,
                                    AbstractObjectLocationInline)
from django_loci.models import FloorPlan, Location, ObjectLocation


class FloorPlanForm(AbstractFloorPlanForm):
    class Meta(AbstractFloorPlanForm.Meta):
        model = FloorPlan


class FloorPlanAdmin(AbstractFloorPlanAdmin):
    form = FloorPlanForm


class LocationForm(AbstractLocationForm):
    class Meta(AbstractLocationForm.Meta):
        model = Location


class FloorPlanInline(AbstractFloorPlanInline):
    form = FloorPlanForm
    model = FloorPlan


class LocationAdmin(AbstractLocationAdmin):
    form = LocationForm
    inlines = [FloorPlanInline]


class ObjectLocationForm(AbstractObjectLocationForm):
    class Meta(AbstractObjectLocationForm.Meta):
        model = ObjectLocation


class ObjectLocationInline(AbstractObjectLocationInline):
    model = ObjectLocation
    form = ObjectLocationForm


admin.site.register(FloorPlan, FloorPlanAdmin)
admin.site.register(Location, LocationAdmin)

Extending channel consumers

Extend the channel consumer of django-loci in this way:

from django_loci.channels.base import BaseLocationBroadcast
from ..models import Location  # your own location model


class LocationBroadcast(BaseLocationBroadcast):
    model = Location

Extending AppConfig

You may want to reuse the AppConfig class of django-loci too:

from django_loci.apps import LociConfig


class MyConfig(LociConfig):
    name = 'myapp'
    verbose_name = _('My custom app')

    def __setmodels__(self):
        from .models import Location
        self.location_model = Location

Installing for development

Install sqlite:

sudo apt-get install sqlite3 libsqlite3-dev libsqlite3-mod-spatialite gdal-bin

Install your forked repo:

git clone git://github.com/<your_fork>/django-loci
cd django-loci/
python setup.py develop

Install test requirements:

pip install -r requirements-test.txt

Create database:

cd tests/
./manage.py migrate
./manage.py createsuperuser

Launch development server and SMTP debugging server:

./manage.py runserver

You can access the admin interface at http://127.0.0.1:8000/admin/.

Run tests with:

# pytests is used to test django-channels
./runtests.py && pytest

Contributing

  1. Announce your intentions in the OpenWISP Mailing List
  2. Fork this repo and install it
  3. Follow PEP8, Style Guide for Python Code
  4. Write code
  5. Write tests for your code
  6. Ensure all tests pass
  7. Ensure test coverage does not decrease
  8. Document your changes
  9. Send pull request

Changelog

See CHANGES.

License

See LICENSE.

django-loci's People

Contributors

a09hopper avatar aryamanz29 avatar atb00ker avatar ayzhu avatar cking100 avatar codesankalp avatar daffytheduck avatar devkapilbansal avatar devprisha avatar haikalvidya avatar kidzmyujikku avatar kkreitmair avatar lehone-hp avatar marfgold1 avatar morrme avatar nemesifier avatar nepython avatar nishi1401 avatar niteshsinha17 avatar noumbissivalere avatar pandafy avatar ppabcd avatar purhan avatar strang1ato avatar waleko avatar wizanyx avatar yashikajotwani12 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

django-loci's Issues

[change] Rethink LOCI_FLOORPLAN_STORAGE

Maybe it's overkill to have a dedicated storage class for django-loci, we could investigate whether we can default to the global storage class.

I believe the purpose of the custom storage class was to:

  • delete the floorplan files when floorplan objects are deleted
  • ensure uploading a new floorplan overwrites the current floorplan successfully

[dependencies] Update django-leaflet

Upgrade django-leaflet to the latest available version, ensuring that the outdoor and indoor map features still work as expected.

Reserved for GCI.

[bug] ignores `is_mobile` False from device

In admin interface:

  1. Create a device with a location, set the marker, check is_mobile & save.
  2. Open it again, uncheck is_mobile, save
  3. Open the device again and notice is_mobile is checked.

[bug] NOT NULL constraint failed: django_loci_floorplan.floor

IntegrityError at /admin/testdeviceapp/device/add/
NOT NULL constraint failed: django_loci_floorplan.floor

Steps to reproduce:

  1. Create a device with new indoor environment, flag it as mobile
  2. In the "Indoor coordinates" section, do not fill a value for floor.
  3. Complete the rest of the form correctly.
  4. Submit

[docs] Add animated gif to README

Add a brief animated gif which quickly shows:

  • add an item to an existing indoor location
  • automatic conversion of point to address (after #46 is merged)

[build] Test multiple django and python versions

In the travis build that runs automated tests, we should be testing the different supported django versions and at least python 3.6 and python 3.7.

PS: we can drop support for django 1.11.x.

Reserved for GCI.

[ux] Automatically calculate geographic coordinates or address

When the position on the geographic map is updated:

  1. if the address field is empty we should automatically get the best address from the geographic coordinates and update the address field with the result
  2. if the address field is not empty we should ask the user if the user wants the map to be updated (for now with a simple javascript confirm dialog); If the user replies affirmatively we can do the same described in point 1

When address field is updated:

  1. If the geographic coordinates on the map are not set, we can get the geographic coordinates from the address and update the map accordingly
    2 if the geographic coordinates on the map are already set, we should ask the user if the user wants the map to be updated (for now with a simple javascript confirm dialog); If the user replies affirmatively we can do the same described in point 1

For the geocoding operations, we can use geopy.

We should implement an abstraction that allows to change the provider, trying to support at least Google Maps, Nominatim and ArcGIS in the first version fo this feature.

We should handle failures and retry, retry by default 3 times with a delay of 1 second in case of failures, the geocoding library should have a built-in implementation for this, see RateLimiter.

We will likely need the addition of a few django global settings that allow to set:

  • the chosen provider, defaults to ArcGIS
  • the API key, defaults to None
  • whether the django app may refuse to start if the testing of the geocoding operation fails, defaults to True, eg: DJANGO_LOCI_STRICT_GEOCODE_TEST
  • amount of retries after failures, default to 3
  • delay in seconds before trying again after a failure, default to 1

On startup, the app shall test an arbitrary geocoding operation (eg: get the coordinates of the address of the Python Foundation) and if it fails it will raise an ImproperlyConfigured exception if DJANGO_LOCI_STRICT_GEOCODE_TEST is True or it will log an error otherwise (using the python logging facility).

Since this app does not use django-rest-framework, the geocoding operations should be performed via a simple JSON view API that it is added to the admin as we do in django-netjsonconfig (see context_view as a reference), although the logic to perform the geocoding can be stored in a function or a class to improve reusability.

This feature needs to be documented in the README and good tests must be written for it.

[change] Use ReconnectingWebsocket to websocket connection

openwisp/ansible-openwisp2#321 sets the websocket_timeout of daphne to 1 hour (reducing from the 24 hours default value).
This means that the websocket connection will get terminated abruptly for users after the timeout. To prevent that use ReconnectingWebsocket from https://github.com/joewalnes/reconnecting-websocket which re-establishes the connection when the connection is terminated.

This library is already used in openwisp-notifications and openwisp-controller.

[model] Allow indoor location without indoor coords

When adding a location using the base class AbstractObjectLocationInline, we may want to flag that the object is indoor, but we may not have the floorplan image ready yet, in that case we are forced to set it to outdoor in order to save, but it's incorrect since it's not an outdoor location.

We could ask for confirmation to the user that they want to save the object even if they have not specified the indoor coordinates and if they confirm we can save the object anyway.

Changes to the JS and admin form will be necessary.

[dependencies] Upgrade Pillow

Upgrade pillow to the latest available version, ensuring that the feature to upload floorplans still works as expected.

Reserved for GCI.

[ux] Floorplan fields is shown when error occurs in new indoor environment

To produce this, follow these steps:

  1. Add device, leave some fields empty (especially the required one: name, organization and mac address) to produce error when saving.
  2. In Geographic Information secton, set Location selection to New and set Type to Indoor environment
  3. In Indoor Coordinates section, select file to the Image field, and set the position for Indoor (you can just use dummy image and set position anywhere) to bypass the client indoor position validation.
  4. Save the device.
  5. In Indoor Coordinates section, you can see the Floorplan section and Floorplan fields is showed.

The expected output: Floorplan fields should be hidden (You can change the type to Outdoor and change it again to Indoor, then you see Floorplan fields is hidden again).
Additional note: The Floorplan fields should be showed only in existing location

[ux] Location selection hidden when error occurs while isMobile toggled on

To produce this issue, follow this step:

  1. Add device, leave some fields empty. (MAC, Name)
  2. Set Location selection to New
  3. Set Type to Outdoor environment
  4. Toggle isMobile to true.
  5. Save device

On the form error page, we see "Location data not received yet" and the isMobile check box is hidden.

Expected output: Location selection & isMobile field should be showed.
Additional note: Leaving isMobile to false will show the field, though. It's also occurs in Indoor environment, existing location selection, and even in the change device form, as long as isMobile is toggled on and there is error when saving.

[docs] Outdated ASGI_APPLICATION directive in README

ASGI_APPLICATION = "django_loci.channels.routing.channel_routing"
CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels.layers.InMemoryChannelLayer",
    },
}

NO: django_loci.channels.routing.channel_routing
YES: django_loci.channels.asgi.channel_routing

[bug] Delete geometry in change view doesn't work

Steps to reproduce:

  1. Create mobile location with geometry set to random location
  2. Go to change view of location delete geometry and click save

By doing these steps and going to location change view you can see that geometry haven't been deleted

[change] Adjust or remove strict test

This strict test feature is backfiring quite a bit.

It's triggered every time django is initialized, also with scripts, management commands, celery, etc.

We should address this, maybe we can turn this into a management command to execute only some times.

[ux] errors made while adding a new device gets hidden

When adding a new device if someone makes an error of not entering Geometry for a Outdoor environment, the user will get error Please correct the errors below. at the top of the page but the actual field to make the correction is not shown.
The user needs to change the Type field then select Outdoor environment again to see the errored field.
This may cause confusion to users.

[ux] isMobile UI inconsistency

In add/change device, when user toggle on isMobile, then related fields (location, address, and geometry) should be hidden. But when user change the Type while leaving isMobile toggled on, it will show the related fields again, even though isMobile is still toggled on. The expected result is the related fields should be stay hidden.

[admin] Validate type before calling

the issue in #50 shows that a 500 error is gotten when adding a device with location set to new and type set to none. this happens because the django-loci throws a KeyError exception when trying to access the type field. Even though a fix was attempted by overriding the methods in the #86 PR, it will be good if this problem is handle at the root.
I think it can be solved by inserting the line which throws the exception in a try-except block.

[bug] Assigning an existing geographic location is broken

How to replicate:

  • create a device
  • create a geographic location in a separate step
  • go to the device created in the first step, then go to map tab, select "Existing"
  • select the geographic location just created
  • hit save

Expected result:

  • when the location is selected, the geo information should appear
  • device is saved normally

Actual result:

  • when the location is selected, the geo information does not appear
  • device is not saved because a validation error

[feature] Allow to configure leaflet referrerPolicy

When using tiles which have an access token tied to a particular domain or URL pattern, the leaflet library is not sending the http referer and the tile requests return 403 Forbidden if the web server is configured with strict referer policy (eg: we use "same-site" in OpenWISP deployments by default).

In order to fix this we need to find a way to set https://leafletjs.com/reference.html#tilelayer-referrerpolicy at JS level.
It's possible we may have to patch https://github.com/makinacorpus/django-leaflet to get this working.

[bug] Geocoding operations should not hit external services during tests but should be mocked

Try to run the tests of django-loci while offline.

Result: some tests fail because they perform real geocoding operations.

We need to fix this, we should use mocking, I have used the responses library for similar simple HTTP request mocking operations successfully in the past and we could reuse it here if applicable.

The tests that have to be fixed are:

  • test_geocode
  • test_reverse_geocode

Once the response from the geocoding server is mocked, the real web service won't be called anymore during tests, therefore we expect to run tests while being offline with no failures.

This issue is affecting also openwisp-controller, because openwisp-controller imports the test suite from django-loci and executes it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.