Giter VIP home page Giter VIP logo

mqtt-tools / mqttwarn Goto Github PK

View Code? Open in Web Editor NEW
950.0 75.0 183.0 3.08 MB

A highly configurable MQTT message router, where the routing targets are notification plugins, primarily written in Python.

Home Page: https://mqttwarn.readthedocs.io/

License: Eclipse Public License 2.0

Shell 0.87% Python 98.20% Dockerfile 0.31% Makefile 0.38% Jinja 0.24%
acquisition mosquitto mqtt mqtt-client mqtt-protocol mqtt-router notification notification-plugin notification-service notifications

mqttwarn's Introduction

mqttwarn

https://cloud.githubusercontent.com/assets/2345521/6320105/4dd7a826-bade-11e4-9a61-72aa163a40a9.png

To warn, alert, or notify.

https://raw.githubusercontent.com/mqtt-tools/mqttwarn/main/assets/google-definition.jpg

Status


https://pepy.tech/badge/mqttwarn/month

GitHub | PyPI | Documentation | Issues | Changelog

About

mqttwarn - subscribe to MQTT topics and notify pluggable services.

Description

mqttwarn subscribes to any number of MQTT topics and publishes received payloads to one or more notification services after optionally applying sophisticated transformations.

A picture says a thousand words.

https://raw.githubusercontent.com/mqtt-tools/mqttwarn/main/assets/mqttwarn.png

Notification service coverage

mqttwarn comes with over 70 notification handler plugins, covering a wide range of notification services and is very open to further contributions. You can enjoy the alphabetical list of plugins on the mqttwarn notifier catalog page.

On top of that, it integrates with the excellent Apprise notification library. Apprise notification services has a complete list of the 80+ notification services supported by Apprise.

Documentation

The mqttwarn documentation is the right place to read all about mqttwarn's features and integrations, and how you can leverage all its framework components for building custom applications. Its service plugins can be inspected on the mqttwarn notifier catalog page.

Installation

Using pip

Synopsis:

pip install --upgrade mqttwarn

You can also add support for a specific service plugin:

pip install --upgrade 'mqttwarn[xmpp]'

You can also add support for multiple services, all at once:

pip install --upgrade 'mqttwarn[apprise,asterisk,nsca,desktopnotify,tootpaste,xmpp]'

See also: Installing mqttwarn with pip.

OCI container image

For running mqttwarn on a container infrastructure like Docker or Kubernetes, corresponding images are automatically published to the GitHub Container Registry (GHCR).

  • ghcr.io/mqtt-tools/mqttwarn-standard:latest
  • ghcr.io/mqtt-tools/mqttwarn-full:latest

To learn more about this topic, please follow up reading the Using the OCI image with Docker or Podman documentation section.

Configuration

In order to learn how to configure mqttwarn, please head over to the documentation section about the mqttwarn configuration.

Usage

Interactive service

Just launch mqttwarn:

# Run mqttwarn
mqttwarn

To supply a different configuration file or log file, optionally use:

# Define configuration file
export MQTTWARNINI=/etc/mqttwarn/acme.ini

# Define log file
export MQTTWARNLOG=/var/log/mqttwarn.log

# Run mqttwarn
mqttwarn

System daemon

There are different ways to run mqttwarn as a system daemon. There are examples for systemd, traditional init, OpenRC, and Supervisor in the etc directory of this repository, for example supervisor.ini (Supervisor) and mqttwarn.service (systemd).

Standalone

In order to directly invoke notification plugins from custom programs, or for debugging them, see Running notification plugins standalone.

Development sandbox

For hacking on mqttwarn, please install it in development mode, using a mqttwarn development sandbox installation.

Project information

About

These links will guide you to the source code of mqttwarn and its documentation.

Requirements

You will need at least the following components:

  • Python 3.x or PyPy 3.x.
  • An MQTT broker. We recommend Eclipse Mosquitto.
  • For invoking specific service plugins, additional Python modules may be required. See setup.py file.

Contributing

We are always happy to receive code contributions, ideas, suggestions and problem reports from the community.

So, if you would like to contribute, you are most welcome. Spend some time taking a look around, locate a bug, design issue or spelling mistake, and then send us a pull request or create an issue.

Thank you in advance for your efforts, we really appreciate any help or feedback.

License

mqttwarn is copyright © 2014-2023 Jan-Piet Mens and contributors. All rights reserved.

It is and will always be free and open source software.

Use of the source code included here is governed by the Eclipse Public License 2.0, see LICENSE file for details. Please also recognize the licenses of third-party components.

Troubleshooting

If you encounter any problems during setup or operations or if you have further suggestions, please let us know by opening an issue on GitHub. Thank you already.

Attributions

Acknowledgements

Thanks to all the contributors of mqttwarn who helped to conceive it in one way or another. You know who you are.

Legal stuff

"MQTT" is a trademark of the OASIS open standards consortium, which publishes the MQTT specifications. "Eclipse Mosquitto" is a trademark of the Eclipse Foundation.


Have fun!

mqttwarn's People

Contributors

amotl avatar appzer avatar christiankniep avatar clach04 avatar davidventura avatar devsaurus avatar dgomes avatar dlangille avatar fabaff avatar halacs avatar jfarcher avatar jlrgraham avatar jpmens avatar jusplainmike avatar juzam avatar jwyse avatar koenvervloesem avatar kstaniek avatar lrizzi avatar marcelrv avatar matbor avatar mortenf avatar otdftr avatar padelt avatar przemas75 avatar psyciknz avatar qk4l avatar r41d avatar rgitzel avatar sumnerboy12 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mqttwarn's Issues

XBMC Notification - 401 Unauthorized

Hi,
I've just been trying out the XBMC service but there doesn't seem to be a method of passing a username or password.

My XBMC instances expect authentication so is there any way to include this in the xbmc service?

Thanks

mqttwarn.py and chdir

Is it possible to add this line to mqttwarn.py?

os.chdir(os.path.dirname(os.path.realpath(file)))

I start mqttwarn as a deamon from another folder and would love this to be a 'standard' feature. Unless this will hurt those that start from the same folder obviously.

Thanks,

Marcel

Add per/service Queue

The code currently uses one Queue to handle all incoming requests, irrespective of how many services are configured.

Enhance to add a per/service Queue so that "slow" targets don't hinder "fast" targets.

Condition to drop message

Can we get a condition that drops a message once a condition is true (use case: don't push me notifications for normal messages but just those with a JSON event attribute in them) ?

config:log and title

this one is more out of interest than need and fyi...

when you setup the target to log feature with a title and format, i will use my weather example config again;

[weatherforecast-alerts]
topic = /software/bom_forecast2mqtt/forecast/Victoria/Melbourne/json
targets = log:info,pushover:weather
title = Weather Forecast for {fcast_town}
format = {fcast_today_precis} with a top of {fcast_today_temperature}\n\n(Issued {fcast_date}  @{fcast_time})

all that gets logged to the log file is the message, the title gets ignored.

2014-03-18 11:27:14,009 DEBUG [log] *** MODULE=services/log.pyc: service=log, target=info
2014-03-18 11:27:14,009 INFO  [log] Shower or two developing with a top of 25\n\n(Issued Tuesday 18 March 2014  @5:15 am)

if this is meant to be the way it works, maybe something needs to be added to the readme file

XMPP target

A bit like the IRC target, not sure if useful (I don't need it).

Crash on publish

When mqtt2pushover receives a pub, it crashes with NoneType' object is not iterable.

I traced it back to `onMessagewhere users is initialized tonone``after which it is iterated in``for user in users``. Shouldn't it be filled with users somewhere before?

Pushover, with a priority of 2 doesn't work

If I use the priority map of 2 the service doesn't seem to send the notification....

2014-02-27 23:06:30,290 pushover *** MODULE=services/pushover.pyc: service=pushover, target=cfaalerts 2014-02-27 23:06:30,290 pushover Sending pushover notification to cfaalerts [{'priority': 2, 'retry': 60, 'expire': 3600}]... 2014-02-27 23:06:31,399 pushover Error sending pushover notification to cfaalerts [{'priority': 2, 'retry': 60, 'expire': 3600}]: HTTP Error 400: Bad Request 2014-02-27 23:06:31,400 mqttwarn Notification of pushover for /notifications/cfaalerts' FAILED`

But if I change it back to a 1 it works fine......

2014-02-27 23:06:55,930 pushover *** MODULE=services/pushover.pyc: service=pushover, target=cfaalerts 2014-02-27 23:06:55,930 pushover Sending pushover notification to cfaalerts [{'priority': 1, 'retry': 60, 'expire': 3600}]... 2014-02-27 23:06:57,062 pushover Successfully sent pushover notification

I tired sending the same message with another python script I wrote with a priority of 2 and the same retry and expire times and it works fine.

Matt.

different log output format

Hi,

I want to log every item change and event change from openHAB through mqttwarn but need a different format for the log than for the message I send to, for example, my mqtt server.

Is that possible or should I use the event trigger available in openHAB for this?

Thanks,

Marcel

Open url

To interface with URL-based APIs (my use case is delivering a json payload to a domoticz url) it would be nice to have the notification type "open url" where the URL is constructed by combinations of filters and/or transforms.
This may even be generalized to execute commands with arguments taken from filters/transforms.

Feature request, titlemap to accept json objects from the message payload.

If the message payload has json objects in it, and you want some of them to pass thru to the title/subject in 'titlemap'

For example I want to past the {fcast_town} json object thru to the title, this object exists in the payload message.

titlemap = {
    'monitoring/icinga/#'   : 'Info',
    'notification/warn'     : 'Warning',
    'notification/error'    : 'Error',
    '/software/bom_forecast2mqtt/forecast/#' : 'Weather Forecast for {fcast_town}',
}

No mqtt pub send

I'm having some issues with the following config.

# Logging
import logging
loglevel = logging.DEBUG
logformat = '%(asctime)-15s %(module)s %(message)s'

# MQTT Broker
broker = 'localhost'
port = 1884
username = 'redacted'
password = 'redacted'
lwt = None        # Where the MQTT LWT will be publied to. Ignore if None

# List of services we will be using. Omit services you won't be using
# This loads the respective service plugins from the services/ directory
services = ['mqtt', 'pushover']

# Outgoing MQTT publishes
mqtt_config = {
    'host'       : 'localhost',
    'username'   : 'redacted',
    'password'   : 'redacted',
    'port'       : 1884,
    'qos'        : 0,
    'retain'     : False
}
mqtt_targets = {
    'owntracks'    : [ 'events/notifications/owntracks' ]
}

pushover_config  = None    # This service requires no configuration
pushover_targets = {
    'owntracks'     : ['redacted', 'redacted'],
}

# Republishes
mqtt_targets = {
  'owntracks' : ['events/notifications/owntracks'],
}

# Subs
topicmap = {
    'owntracks/bucks/#'         : ['mqtt:owntracks'],
    'owntracks/sina/#'         : ['mqtt:owntracks'],
    'events/notifications/owntracks'        : ['pushover:owntracks']
}

def owntracksTopicDataMap(topic):
    if type(topic) == str:
        try:
            # owntracks/username/device
            parts = topic.split('/')
            username = parts[1]
            deviceid = parts[2]
        except:
            deviceid = 'unknown'
            username = 'unknown'
            return dict(username=username, device=deviceid)
    return None

topicdatamap = {
    'owntracks/+/+' : owntracksTopicDataMap
}

# Used by: pushover
prioritymap = {
    'events/notifications/owntracks'    : -1,
    'events/notifications/warn'    : 0,
    'events/notifications/alert'  : 1
}
formatmap = {
    ##'owntracks/bucks/+'    :  '{topic} at {lat}/{lon}\n'
}

titlemap = {
    'events/notifications/owntracks'   : 'OwnTracks',
    #'notification/warn'     : 'Warning',
    #'notification/error'    : 'Error'
}

I'm getting the following log when a message is published to owntracks/bucks/nexus4:

2014-02-16 11:26:15,074 mqttwarn Topic [owntracks/bucks/nexus4] going to ['mqtt:owntracks']
2014-02-16 11:26:15,075 mqttwarn New `mqtt:owntracks' job: owntracks/bucks/nexus4

It should (tm) format the message and then publish it to events/notifications/owntracks where it should be picked up by the pushover connector. However, no message is ever published. Instead the task just hangs and cannot be exited without a kill -9. Ideas?

Also the format map wasnt working and complaining about topic being an unknown key before.

samplefuncs.py and import json

I had to put in the samplefuncs.py file.

import json

to get it to work.

Otherwise I got the error.

2014-03-18 10:46:58,206 WARNING [mqttwarn] Cannot invoke filter function OwnTracksBattFilter defined in owntracks-battery: global name 'json' is not defined

pushbullet

I have tried to use pushbullet this morning but keep getting the following :-
2014-02-20 11:54:35,723 mqttwarn processor is handling: `pushbullet' for warnme
2014-02-20 11:54:35,723 pushbullet *** MODULE=services/pushbullet.pyc: service=pushbullet, target=warnme
2014-02-20 11:54:35,723 pushbullet Sending pushbullet notification to warnme...
2014-02-20 11:54:35,732 connectionpool Starting new HTTPS connection (1): api.pushbullet.com
2014-02-20 11:54:37,165 connectionpool "GET /api/devices HTTP/1.1" 200 None
2014-02-20 11:54:37,170 pushbullet Cannot notify pushbullet: PushBullet instance has no attribute 'pushNote'

is it something I'm doing wrong?

Pushbullet

pushbullet_config = None
pushbullet_targets = {
'warnme' : [ 'abcdefghi', '1234567890'],
}
topicmap = {
'/openhab/notification/info/#' : ['smtp:special', 'pushbullet:warnme'],
'/openhab/notification/alert' : ['smtp:special', 'pushbullet:warnme'],
'/openhab/notification/warn' : ['smtp:special', 'pushbullet:warnme'],
'/openhab/notification/alarm' : ['smtp:special', 'pushbullet:warnme'],
}

Multiple filters per topic

Reported/suggested by @sumnerboy12 and I had stumbled over this problem earlier too :)

I have just noticed an issue with my mqttwarn config however. I
already had a notification set up for enter/leave events using
OwnTracks. This was based on your example and involves adding a custom
filter to check for 'event' messages, and formatting to 'ben: enter =>
home' etc.

This works like a charm, however now I want to add a second
notification based on the 'batt' parameter.

Because the filters are applied at the start of the processing loop,
the first filter (in my case it seems to be the battery filter) is
applied to all OwnTracks publishes and so the event stuff never gets
hit.

Not sure of the easiest way to deal with this, but I am starting to
think it might be better to have a separate set of config for each
topic. This would allow multiple definitions of a given topic (or
allow overlapping topics) to all be processed one-by-one with their
own config, without fear of overlapping filters etc impacting on one
another.

Not sure how it should be defined, but the general idea would be
something like;

# topic name is the section header, followed by the config for that topic
[/owntracks/ben]
target = ['pushover', 'twitter']
format = 'Ben\'s phone battery getting low ({batt}%)!'
priority = 1
title = 'OwnTracks'
filter = owntracks_battfilter

Add Redis GET/SET

Inspired by #51

I don't yet know exactly how/what/why, but I think we need this. ;-)

typo?

in the readme, is it missing an apostrophe after pushover?

Now simply add your choice of target(s) to the topicmap and a nice format string and you are done;

topicmap = {
'/owntracks/#' : ['pushover, 'xbmc']
}

rss, target (enhancement)

Not sure if it is been mentioned before. RSS target.

Could be called via pipe or file i guess.

I don't need it.

deal with out of control publishing

Maybe add a feature that if x amount of the same messages is published within y amount of time, then just send one message saying that. Instead of publishing all the messages, then goto sleep on that message for z time.

As you could easily blow api limits on some services, or it just becomes annoying on your mobile device!

Take topicdata() and formatmap() out of the queue

topic processing and format functions are done in the job queue, i.e. they're invoked N times if a message has N targets.

Pull those up and invoke before the job is added to the queue.

Simultaneously, attempt to parse JSON from payload BEFORE the filter() function is invoked, and pass data{} to it as well. (Doesn't cost more, and adds functionality.) In order to not break existing filter(), add data=None to end of call.

format, newline?

previously I could do a newline with \n in the formatmap, can't seem to do it with the new format.

example.

[weatherforecast-alerts]
topic = /software/bom_forecast2mqtt/forecast/Victoria/Melbourne/json
targets = log:info,pushover:weather
title = Weather Forecast for {fcast_town}
format = {fcast_today_precis} with a top of {fcast_today_temperature}\n\n(Issued {fcast_date}  @{fcast_time})

will display the characters in the pushover message as \n\n not as two newlines.

Shower or two developing with a top of 25\n\n(Issued Tuesday 18 March 2014  @5:15 am)

Exception in plugin hangs processor -> can't shutdown

If a plug script fails (e.g. /services/xbmc.py) then the mqttwarn processor hangs. Attempting to shutdown the main mqttwarn loop gets to the;

2014-02-16 10:53:06,957 pushover Successfully sent pushover notification
2014-02-16 10:53:06,957 mqttwarn processor is handling: `xbmc' for living
2014-02-16 10:55:01,730 mqttwarn Disconnecting from MQTT broker...
2014-02-16 10:55:01,730 mqttwarn Clean disconnection
2014-02-16 10:55:01,730 mqttwarn Waiting for queue to drain

The queue never drains so the process never shuts down. Have to kill -9 to stop it.

Dealing with multiple messages

I was wondering how to handle dealing with several messages at the same time, or remember state when I was thinking about logging the time it took for my daily commute to work and back.

I would use owntracker to detect the timestamp when I leave location home, and then later with a new message the timestamp when I enter location work.

I would need something like:

If leave home, remember timestamp as LEFTHOME (perhaps publish as a message /commute/towork/leavingtimestamp)
if arrive at work, subtract LEFTHOME from ARRIVEWORK and publish as /commute/towork/duration

It seems mqttwarn only deals with a single message at a time, and perhaps that is its purpose, so I wonder if there is a way to do the above scenario?

Thank you very much for the work you put into mqttwarn.

(I was able to solve my earlier issue with a filter by the way, I ended up with the following filter:

def OwntracksFilter(topic, message):
data = dict(json.loads(message).items())
if 'desc' in data and 'event' in data:
if data['desc'] == "Werk" and data['event'] == "leave":
return False
return True

Failover to alternate notification service?

Pushover was unreacheable for 20m for me, and something @sumnerboy12 said made me think: Do we need failover in case a particular notification cannot be emitted?

Assuming mqttwarn should notify target xxxx:urgent, but that service cannot be reached, should we instead have some way of configuring a "failover" service for that particular target? In other words: "get the message to the user by whichever means possible"?

If so, we have to think up some clever way to configure that.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.