watcheye / watcheye-collector Goto Github PK

Collector is a Django application to collect monitoring data samples through HTTP or SNMP GET interface. Collected data might be then visualized using e.g. Grafana.

Collector uses InfluxDB time series database for storing monitoring data samples and django supported database for storing configuration and Redis broker.

Assuming InfluxDB is properly configured (setting up authentication is recommended) and so is Redis broker just a few steps are required.

1. Install watcheye-collector:

   $ pip install watcheye-collector

2. Add collector application to INSTALLED_APPS setting.

   INSTALLED_APPS = [
       ...
       'collector',
   ]

3. Integrate Celery with your django project (see Celery documentation for further reference).

4. Add InfluxDB connection and Celery configuration in settings.py file:

   INFLUXDB_HOST = 'my_host'
   INFLUXDB_USERNAME = 'user'
   INFLUXDB_PASSWORD = 'secret'
   
   CELERY_BEAT_SCHEDULE = {
       'snmp-scheduler': {
           'task': 'collector.tasks.snmp_scheduler',
           'schedule': crontab(minute='*')
       }
   }
   CELERY_RESULT_BACKEND = 'redis://my_broker_host/0'
   CELERY_BROKER_URL = 'redis://my_broker_host/1'
   
   # to use non-default values configure also:
   INFLUXDB_PORT = 1234
   INFLUXDB_DATABASE = 'my_database'
   INFLUXDB_RETENTION_POLICY = 'my_policy'
   INFLUXDB_DURATION = '30d'

5. To set InfluxDB instance up run:

   python manage.py setupinfluxdb --username <admin_username>

6. To create the collector models run:

   python manage.py migrate

7. Include the collector URLconf in your project urls.py:

   path('collector/', include('collector.urls')),

8. Start the development server and visit http://127.0.0.1:8000/admin/ to create a collector configuration (the Admin application must also be enabled).

9. Run Celery worker with:

   celery --beat --app <my_project> worker

10. POST some samples:

    $ curl -i -X POST \
    -H "Content-Type: application/json" \
    -d '{"host":"test", "timestamp": 1500000000,
    "samples": [{"parameter":"CPU", "value": 10}]}' \
    http://127.0.0.1:8000/collector/