The smartmeter-datacollector
tool is a Python3 software which allows you to continuously retrieve data from supported smart meters (listed below).
The acquired values can be forwarded to one or more data sinks like a MQTT broker or logger.
This tool is an open source software funded by EKZ (Elektrizitätswerke des Kantons Zürich) and developed by Supercomputing Systems AG. The goal of this voluntary initiative is to create a tool that can read, decode and decrypt data (if necessary) from the local interface of various smart meters and convert it into human-readable and simple formats. This should eliminate the complexity of decoding DLMS and similar protocols for the user. The user has the data in a format, which he can easily use in various use-cases ranging from simple vizualization (see Method 1: Raspberry Pi image with demo) to integration into energy management algorithms or into custom home automation software.
The following smart meters are supported:
- Landis+Gyr E450:
Data pushed by smart meter over CII interface (wired M-Bus). Encoded with HDLC and DLMS/COSEM.- Unencrypted data
- Encrypted data
- Iskraemeco AM550:
Data pushed by smart meter over P1 interface (DSMR). Encoded with HDLC (IEC 62056-46) and DLMS/COSEM.- Unencrypted data
- Encrypted data
The following data sinks are implemented:
- MQTT (v3.1.1):
- Encryption
- Unencrypted connection
- Encrypted connection
- TLS
- optional custom CA certificate
- Authentication
- Unauthenticated
- Authenticated with username / password
- Authenticated with client certificate
- Encryption
- Logger to STDOUT
smartmeter-datacollector
is fully configurable through a .ini
configuration file. The smartmeter-datacollector-configurator
webinterface can help to create and modify the configuration.
There are different methods how to use smartmeter-datacollector
.
- Raspberry Pi image with demo
- Python package
- Debian package
For a very easy first time usage of smartmeter-datacollector
we provide a Raspberry Pi image (based on Raspberry Pi OS) which contains the following parts:
smartmeter-datacollector
as a systemd servicesmartmeter-datacollector-configurator
webinterface- demo
- mosquiotto as a local MQTT broker to publish the measurements from
smartmeter-datacollector
- Telegraf to collect measurements published by
smartmeter-datacollector
over MQTT and store them in InfluxDB - InfluxDB to store the measurements
- Grafana to visualize the measurements
- mosquiotto as a local MQTT broker to publish the measurements from
This setup allows a (first time) user to install the image on a Raspberry Pi and immediately get started with smartmeter-datacollector
. The following steps are required:
- Download the Raspberry Pi image from the latest release.
- Install the image on a (micro)SD card, e.g. using Raspberry Pi Imager.
- Prepare the Raspberry Pi.
- Push the (micro)SD card into the proper slot.
- Use an RJ-45 cable (or configure WiFi later) to connect to a network.
- Connect the power cable to the Micro USB port.
- Open the
smartmeter-datacollector-configurator
webinterface usinghttp://<Raspberry Pi IP>:8000/
in your favorite browser.- Load the pre-defined configuration.
- Adjust the settings of the connected smart meter.
- Deploy the adjusted configuration.
- Restart the
smartmeter-datacollector
service.
- Open
Grafana
usinghttp://<Raspberry Pi IP>:3000/
in your favorite browser. - Observe the measurements from your smart meter while they arrive periodically.
See Wiki/Demo for more details on the provided Raspberry Pi image with the demo.
smartmeter-datacollector
can be installed as a Python3 package either from PyPi or manually using a released wheel.
The Python3 package does not contain any infrastructure to run smartmeter-datacollector
in the background or to automatically start it during a boot sequence. If such infrastructure is required either see Method 3: Debian package or provide it yourself.
- Python >= 3.7 (tested with 3.8)
Install the package either as global Python package or in a virtualenv with
python3 -m pip install smartmeter-datacollector
Similarly the smartmeter-datacollector-configurator
webinterface can be installed with
python3 -m pip install smartmeter-datacollector-configurator
smartmeter-datacollector
is also available as a Debian (.deb
) package from the releases which installs it system wide. The Debian package includes a systemd service file which enables smartmeter-datacollector
to automatically start after booting the system.
- Distribution
- Debian
- Release
- Buster
- CPU architecture
- amd64
- armhf
Download the Debian package from releases and install it with
sudo apt install ./python3-smartmeter-datacollector_*.deb
Similarly the smartmeter-datacollector-configurator
webinterface can be installed with a Debian package from its releases with
sudo apt install ./python3-smartmeter-datacollector-configurator_*.deb
The usage of smartmeter-datacollector
depends on the installation method. Independent of the installation method a .ini
configuration file is required to properly run smartmeter-datacollector
.
The .ini
configuration file supports the following format
[reader0]
# type of the connected smart meter
type = lge450
# serial port to which the smart meter is connected
port = /dev/ttyUSB0
# optional encryption key
key =
[sink0]
# type of the sink
type = mqtt
# host / IP address of the MQTT broker
host = localhost
# port of the MQTT broker
port = 1883
# whether to use TLS (Transport Layer Security)
tls = False
# optional path to the CA certificate used to validate the MQTT broker's certificate
ca_file_path =
# whether to check the hostname of the MQTT broker against the certificate
check_hostname = False
# optional username for authentication with the MQTT broker
username =
# optional password for authentication with the MQTT broker
password =
# optional path to the client certificate for authentication with the MQTT broker
client_cert_path =
# optional path to the private key for the client certificate for authentication with the MQTT broker
client_key_path =
[sink1]
# type of the sink
type = logger
# name of the logger
name = DataLogger
[logging]
# log level configuration: DEBUG / INFO / WARNING / ERROR
default = WARNING
smartmeter = WARNING
collector = WARNING
sink = WARNING
There can be multiple [readerX]
and [sinkY]
sections for every connected smart meter and / or data sink.
See Wiki/Configuration for more details on the available configuration options.
To simplify the process of generating a valid .ini
configuration for smartmeter-datacollector
the companion smartmeter-datacollector-configurator
webinterface can be used. It supports
- a graphical approach to manage the configuration
- input validation to avoid invalid configurations
- loading / saving / discarding a configuration
- restarting
smartmeter-datacollector
(only if installed as a Debian package)
See Wiki/smartmeter-datacollector-configurator for more details on the webinterface and the configuration possibilities.
Run smartmeter-datacollector
with the following command:
smartmeter-datacollector --config datacollector.ini
The following command line arguments are supported:
-h, --help
: Shows the help output ofsmartmeter-datacollector
.-c, --config CONFIG
: Path to the.ini
configuration file.-s,--saveconfig
: Cteate a default.ini
configuration file.-d, --dev
: Enable development mode.
The configuration file can be located anywhere and use any filename. If no .ini
configuration file is specified a default configuration with the following options is used:
- Landys+Gir E450 smart meter in unencrypted mode connected to
/dev/ttyUSB0
LOGGER
sinkMQTT
sink connected to a local broker without encryption or authentication
When smartmeter-datacollector
has been installed as a Debian package it provides a systemd .service file named python3-smartmeter-datacollector.service
. Therefore the service can be managed using the systemctl
command:
sudo systemctl start|stop|restart python3-smartmeter-datacollector
Enabling or disabling the service can also be managed using the systemctl
command:
sudo systemctl enable|disable python3-smartmeter-datacollector
Inspecting the log messages can be done using the journalctl
command:
journalctl -u python3-smartmeter-datacollector [-f]
The systemd service expectes the .ini
configuration to be located at
/var/lib/smartmeter-datacollector/datacollector.ini
. This is also the default location used by the smartmeter-datacollector-configurator
webinterface to load and save the managed configuration.
Help from the community for the smartmeter-datacollector
project is always welcome. Please follow the next few chapters to setup a working development environment.
- Python >= 3.7
pipenv
- Optional software packages (Debian / Ubuntu)
- python3-all
- debhelper
- dh-python
- dh-systemd
To install the listed minimal requirements run the following command:
sudo apt install git python3 pipenv
To install the optional requirements also run the following command:
sudo apt install python3-all debhelper dh-python dh-systemd
Use git
to clone / checkout smartmeter-datacollector
from GitHub using
git clone https://github.com/scs/smartmeter-datacollector.git
smartmeter-datacollector
uses pipenv
to manage its dependencies and setup a virtual environment. Run the following command to setup the initial development environment:
pipenv install --dev
This will install all runtime and development dependencies for smartmeter-datacollector
in a new virtual environment. Now you are ready to start working on smartmeter-datacollector
.
smartmeter-datacollector
offers a few custom pipenv run
commands to simplify certain development workflows:
format_check
checks if the code follows theautopep8
code formatting rules.format
automatically adjusts the code to follow theautopep8
code formatting rules.isort_check
checks if the order of the import statements is correct usingisort
.isort
automatically re-orders the import statements usingisort
.lint_check
checks if the code follows thepylint
rules defined inpyproject.toml
.lint
automatically adjust the code to follow thepylint
rules defined inpyproject.toml
.license
makes sure every Python (*.py
) file contains the proper license header.build
builds a Python package which can be uploaded toPyPI
usingtwine
.build_check
usestwine
to check if the built Python package will be accepted byPiPI
.setup_check
checks whether the dependencies defined inPipfile
/Pipfile.lock
are in sync withsetup.py
.setup
synchronizes the dependencies defined inPipfile
/Pipfile.lock
withsetup.py
.debianize
creates adebian/
directory used to build Debian source / binary packages.build_srcdeb
builds a Debian source package which can be used to build a Debian (binary) package for any platform (e.g. usingpbuilder
)build_deb
builds a Debian package for the current development plattform.
Make sure to run format_check
/ format
, isort_check
/ isort
, lint_check
/ lint
, license
, setup_check
/ setup
before commiting changes to the repository to avoid unnecessary development cycles. smartmeter-datacollector
uses GitHub Actions to check if these rules apply.
smartmeter-datacollector
and its companion project smartmeter-datacollector-configurator
have been developed by Supercomputing Systems AG on behalf of and funded by EKZ.