Giter VIP home page Giter VIP logo

osm_hall_monitor's Introduction

OpenStreetMap Hall Monitor

This module is a monitoring tool for use with OpenStreetMap. With OSM Hall Monitor, you can track edits made by specified users, made to specified objects, or made with certain tags. You can also enable notifications to receive emails when any of the flagged people or items are edited. Basic functionality for suspicious changeset monitoring is also included.

Requirements

OpenStreetMap Diff Tool is required. If you use the setup.py file, it should fetch the repository from pypi.

Also, psycopg2 is necessary at the moment for all the database work. Again, setup.py should fetch this.

Installation

Releases

$pip install osm_hall_monitor

Bleeding Edge

Method 1

  • Download the zip.
  • Unpack the zip to $somewhere.
  • Navigate to $somewhere.
  • $python setup.py install

Method 2

$pip install git+https://github.com/ethan-nelson/osm_hall_monitor.git

Database setup

A few comments on the database setup. Right now, this depends on a Postgres database for all the storage. Following the procedure of PaaSs, OSM Hall Monitor looks for the database information in the environment. This is set up (in a mostly orthodox way) as:

DATABASE_URL = "postgres://username:password@hostname:port/database_name

Users may often have this configured for something else, so the fetched database information can be configured within the program via the config module. (N.B. This must be done every time the module is imported.)

import osmhm

osmhm.config.database_url = "postgres://username:password@hostname:port/database_name"

#Continue with things.

Once the database is configured, you can begin building the tables necessary. This is accomplished via the tables module:

osmhm.tables.all_tables('create')

Next, the file_list table needs to be filled in with the most current state file. This can be automatically populated via:

osmhm.fetch.fetch_next(reset=True,time='minute') #'hour' is the default

You should be all good to go with running things now!

Use

To begin use, simply call import osmhm after installation.

Sample calls available for now:

import osmhm

osmhm.run(history=False, monitor=True, suspicious=True) # Does not log full history, does watch flagged objects and users, and does look for strange changesets

To add or remove watched users, use osmhm.db.add_watched_user(username, reason, author, email) (email can be None type) or osmhm.db.remove_watched_user(username), respectively.

To add or remove watched objects, similarly use osmhm.db.add_watched_object(element, reason, author, email) (email can be None type) or osmhm.db.remove_watched_object(element), respectively. Elements should be composed of a singleton character denoting node, way, or relation, followed by the OSM id number.

To add or remove users from the whitelist (used to ignore users in the suspicious filter), again use osmhm.db.add_whitelisted_user(username, reason, author) or osmhm.db.remove_whitelisted_user(username).

More information will be coming soon and the tool will continue to be improved, including the option to remove the dependency on psycopg2 and a postgres database to store everything.

osm_hall_monitor's People

Contributors

ethan-nelson avatar someoneelseosm avatar

Stargazers

Gabriel De Luca avatar Francesco de Virgilio avatar Emerson Rocha avatar J07H15H avatar Manoj Karingamadathil avatar Strubbl avatar humanitiesNerd avatar Mæve Rey avatar David Morais Ferreira avatar Asish Abraham Joseph avatar Andrew Harvey avatar Sam avatar Matthias Meißer avatar ImreSamu avatar  avatar Seth Fitzsimmons avatar Jubal Harpster avatar Blake Girardot avatar 거윈 | Win avatar @tomgertin avatar Guillaume Rischard avatar Taichi FURUHASHI avatar Martin Raifer avatar Norbert Renner avatar Jonah Adkins avatar Will Skora avatar Alan McConchie avatar Martijn van Exel avatar  avatar

Watchers

Aaron Lidman avatar Taichi FURUHASHI avatar James Cloos avatar Alan McConchie avatar ImreSamu avatar  avatar

osm_hall_monitor's Issues

Add table call that checks for existence

When pushing to Heroku, I have to manually pull tables from my local machine and then never deal with the osmhm.tables commands. There should be a table call that is "add if doesn't exist" so that I can call it in my worker script that runs every time I deploy.

auto-subscribe to Objects a user once touched

How about an option to automatically subscribe to all objects a user has touched? A new user to OSM could be subscribed to this and so they can see how things they added/edited have been further extended.

Add user gives error 500

i just wanted to try https://hallmonitor.info/user_watch/add, but it says

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at webmaster@localhost to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.
Apache/2.4.41 (Ubuntu) Server at hallmonitor.info Port 443

Digest option for email

Rather than barraging people with many emails when there are multiple events detected in an hour, the notification function should have a catch for and combine messages when an email address is listed more than once.

Add a 'mute monitor for x time' option?

Again, just an idea. When watching an individual user, if they suddenly get active, it might be nice to have an option to mute the monitor for some period of time.

This might just be me, maybe if someone is not interested in the monitor anymore they should turn it off, but I was thinking I'd like a "mute for 12 hours" so I am not notified of the rest of the edits they are doing for the day, but next time they start up I might be interested.

(this just gave me an idea for mentoring new users using a tool like this, validation related in the HOT workflow)

"Authorise access to your account" fails with "Internal Server Error"

Currently the front end "OSM Hall Monitor" on "develop" does not work currently - I'm guessing that that is an http vs https issue somewhere. It seems to have a problem with the callback from "Authorise access to your account" (other oauth stuff works OK):

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at [email protected] to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.
Apache/2.4.18 (Ubuntu) Server at enelson.dev.openstreetmap.org Port 443

Expand notification functionality

Right now we have emailing hardcoded in. It'd be nice if a function object could be supplied to be used for notification, in case IRC, a specific email service, or something else is desired.

What functions do we want for bare bones use?

Right now the program is focused on a very limited use: for my DWG monitoring. Instead, the tool should have multiple use cases with mine taking a back seat as a more applied case. So, functions a user should be able to perform when downloading this package are:

  • Fetch an hourly diff
  • Retrieve a list of all users in that hour and their actions
  • Retrieve a list of all objects in that hour and the action on them
  • Retrieve information about user

...

Provide better digests

Right now, digest communications consist of concatenated long-form single notification messages. Instead, it should provide a cursory overview of all the events occurring in an easily digestible format.

Include fetch error handling

A common problem experienced is temporary DNS resolution failure when fetching a diff file. The run() command should account for this and wait for a bit, then try again before failing. Code location.

Add certain tag monitors

Certain key values may deserve more scrutiny than others...for example, high admin level numbers.

Add monitoring over an area

If the bbox is any nontrivial size, and the system checks for changeset lat/lon extent, we're going to get a lot of notifications. We should probably scan individual objects specifically and also impose a limited area for the bbox.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.