Giter VIP home page Giter VIP logo

modbus4mqtt's Introduction

Modbus4MQTT

https://github.com/tjhowse/modbus4mqtt

https://pypi.org/project/modbus4mqtt/

codecov

This is a gateway that translates between modbus and MQTT.

The mapping of modbus registers to MQTT topics is in a simple YAML file.

The most up-to-date docs will always be on Github.

Similar software

There is already good software out there that can do what Modbus4MQTT does, but none that I could find that has this functionality as its focus. Do one thing and do it well. Modbus4mqtt is designed to fit in as a component of other systems, rather than trying to be a complete solution.

Installation

Python module

$ pip3 install --user modbus4mqtt
$ modbus4mqtt --help

Docker container

Alternatively you can run Modbus4MQTT in a Docker container. A Dockerfile example is provided.

$ docker pull tjhowse/modbus4mqtt:latest
$ docker run modbus4mqtt --help

When launching inside the docker container you will either need to use one of the built-in YAMLs like /modbus4mqtt/modbus4mqtt/Sungrow_SH5k_20.yaml, or map your custom YAML into the container in a volume.

YAML definition

Look at the Sungrow SH5k-20 configuration YAML for a working example.

Modbus device settings

ip: 192.168.1.89
port: 502
update_rate: 5
address_offset: 0
variant: sungrow
scan_batching: 100
word_order: highlow
Field name Required Default Description
ip Required N/A The IP address of the modbus device to be polled. Presently only modbus TCP/IP is supported.
port Optional 502 The port on the modbus device to connect to.
device_address Optional 1 The modbus device address ("unit") of the target device
update_rate Optional 5 The number of seconds between polls of the modbus device.
address_offset Optional 0 This offset is applied to every register address to accommodate different Modbus addressing systems. In many Modbus devices the first register is enumerated as 1, other times 0. See section 4.4 of the Modbus spec.
variant Optional 'tcp' Allows modbus variants to be specified. See below list for supported variants.
write_mode Optional 'single' Which modbus write function code to use single for 06 or multi for 16
scan_batching Optional 100 Must be between 1 and 100 inclusive. Modbus read operations are more efficient in bigger batches of contiguous registers, but different devices have different limits on the size of the batched reads. This setting can also be helpful when building a modbus register map for an uncharted device. In some modbus devices a single invalid register in a read range will fail the entire read operation. By setting scan_batching to 1 each register will be scanned individually. This will be very inefficient and should not be used in production as it will saturate the link with many read operations.
word_order Optional 'highlow' Must be either highlow or lowhigh. This determines how multi-word values are interpreted. highlow means a 32-bit number at address 1 will have its high two bytes stored in register 1, and its low two bytes stored in register 2. The default is typically correct, as modbus has a big-endian memory structure, but this is not universal.

Modbus variants

The variant is split into two: The connection variant and the framer variant using the format <framer>-over-<connection> or just <connection>. For example rtu-over-tcp or ascii-over-tls. The framer is optional allowing to simply specify tcp, which makes it use the default modbus-TCP framer. Supported framer variants are: ascii, binary, rtu and socket. The following connection variants are supported: tcp, udp, tls, sungrow, with the latter one transparently decrypting traffic from sungrow SH inverters running newer firmware versions.

Register settings

registers:
  - pub_topic: "forced_charge/mode"
    set_topic: "forced_charge/mode/set"
    retain: true
    pub_only_on_change: false
    table: 'holding'
    address: 13140
    value_map:
      enabled: 170
      disabled: 85
  - pub_topic: "forced_charge/period_1/start_hours"
    set_topic: "forced_charge/period_1/start_hours/set"
    pub_only_on_change: true
    table: 'holding'
    address: 13142
  - pub_topic: "voltage_in_mv"
    address: 13000
    scale: 1000
  - pub_topic: "first_bit_of_second_byte"
    address: 13001
    mask: 0x0010
  - pub_topic: "load_control/optimized/end_time"
    address: 13013
    json_key: hours
  - pub_topic: "load_control/optimized/end_time"
    address: 13014
    json_key: minutes
  - pub_topic: "external_temperature"
    address: 13015
    type: int16
  - pub_topic: "minutes_online"
    address: 13016
    type: uint32

This section of the YAML lists all the modbus registers that you consider interesting.

Field name Required Default Description
address Required N/A The decimal address of the register to read from the device, starting at 0. Many modbus devices enumerate registers beginning at 1, so beware.
pub_topic Optional N/A This is the topic to which the value of this register will be published.
set_topic Optional N/A Values published to this topic will be written to the Modbus device. Cannot yet be combined with json_key. See #23 for details.
retain Optional false Controls whether the value of this register will be published with the retain bit set.
pub_only_on_change Optional true Controls whether this register will only be published if its value changed from the previous poll.
table Optional holding The Modbus table to read from the device. Must be 'holding' or 'input'.
value_map Optional N/A A series of human-readable and raw values for the setting. This will be used to translate between human-readable values via MQTT to raw values via Modbus. If a value_map is set for a register the interface will reject raw values sent via MQTT. If value_map is not set the interface will try to set the Modbus register to that value. Note that the scale is applied after the value is read from Modbus and before it is written to Modbus.
scale Optional 1 After reading a value from the Modbus register it will be multiplied by this scalar before being published to MQTT. Values published on this register's set_topic will be divided by this scalar before being written to Modbus.
mask Optional 0xFFFF This is a 16-bit number that can be used to select a part of a Modbus register to be referenced by this register. For example a mask of 0xFF00 will map to the most significant byte of the 16-bit Modbus register at address. A mask of 0x0001 will reference only the least significant bit of this register.
json_key Optional N/A The value of this register will be published to its pub_topic in JSON format. E.G. { key: value } Registers with a json_key specified can share a pub_topic. All registers with shared pub_topics must have a json_key specified. In this way, multiple registers can be published to the same topic in a single JSON message. If any of the registers that share a pub_topic have the retain field set that will affect the published JSON message. Conflicting retain settings are invalid. The keys will be alphabetically sorted.
type Optional uint16 The type of the value stored at the modbus address provided. Only uint16 (unsigned 16-bit integer), int16 (signed 16-bit integer), uint32, int32, uint64 and int64 are currently supported.

modbus4mqtt's People

Contributors

iconnor avatar icypete avatar jmkelly avatar m4gnv5 avatar peterpedron avatar tjhowse avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

modbus4mqtt's Issues

ERROR 'ExceptionResponse' object has no attribute 'registers

I try to test your script (it would be soooo useful to me).

My Inverter: SG5K-D
With latest firmware: M_WIFI_RAK475_V31_V01_Q

modbus4mqtt --hostname mqttserver --port 1883 --username username --password password --mqtt_topic_prefix test --config ./modbus4mqtt/Sungrow_SH5k_20.yaml

2020-10-28 13:50:29 INFO     Connected to MQTT.
Subscribed to test/no_export/partial/limit/set
Subscribed to test/forced_charge/mode/set
Subscribed to test/forced_charge/weekdays/set
Subscribed to test/forced_charge/period_1/start_hours/set
Subscribed to test/forced_charge/period_1/start_minutes/set
Subscribed to test/forced_charge/period_1/end_hours/set
Subscribed to test/forced_charge/period_1/end_minutes/set
Subscribed to test/forced_charge/period_1/target_soc/set
Subscribed to test/forced_charge/period_2/start_hours/set
Subscribed to test/forced_charge/period_2/start_minutes/set
Subscribed to test/forced_charge/period_2/end_hours/set
Subscribed to test/forced_charge/period_2/end_minutes/set
Subscribed to test/forced_charge/period_2/target_soc/set
Subscribed to test/bat_usage_time/weekday_usage/end_time_1_hours/set
Subscribed to test/bat_usage_time/weekday_usage/end_time_2_hours/set
Subscribed to test/bat_usage_time/weekend_usage/set
2020-10-28 13:50:30 ERROR    Failed to poll modbus device, attempting to reconnect: 'ExceptionResponse' object has no attribute 'registers'
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus4mqtt.py", line 71, in poll
    self._mb.poll()
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 128, in _scan_value_range
    return result.registers
AttributeError: 'ExceptionResponse' object has no attribute 'registers'
2020-10-28 13:50:35 ERROR    Failed to poll modbus device, attempting to reconnect: 'ExceptionResponse' object has no attribute 'registers'
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus4mqtt.py", line 71, in poll
    self._mb.poll()
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 128, in _scan_value_range
    return result.registers
AttributeError: 'ExceptionResponse' object has no attribute 'registers'
2020-10-28 13:50:42 ERROR    Failed to read holding table registers from 13000 to 13100
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 127, in _scan_value_range
    raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
ValueError: Failed to read holding table registers from 13000 to 13100
2020-10-28 13:50:42 ERROR    Unable to decode response Modbus Error: Unknown response 0
2020-10-28 13:50:42 ERROR    Modbus Error: [Input/Output] Unable to decode request
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/pymodbus/transaction.py", line 190, in execute
    self.client.framer.processIncomingPacket(response,
  File "/home/abadonna/.local/lib/python3.8/site-packages/pymodbus/framer/socket_framer.py", line 165, in processIncomingPacket
    self._process(callback, error=True)
  File "/home/abadonna/.local/lib/python3.8/site-packages/pymodbus/framer/socket_framer.py", line 175, in _process
    raise ModbusIOException("Unable to decode request")
pymodbus.exceptions.ModbusIOException: Modbus Error: [Input/Output] Unable to decode request
2020-10-28 13:50:42 ERROR    Failed to read holding table registers from 13100 to 13200
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 127, in _scan_value_range
    raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
ValueError: Failed to read holding table registers from 13100 to 13200
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13073 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13139 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13140 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13141 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13142 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13143 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13144 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13145 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13146 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13147 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13148 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13149 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13150 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13001 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13002 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13003 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13004 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13005 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13006 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13007 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13008 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13009 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13010 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13011 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13012 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13013 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13014 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13015 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13122 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13123 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13124 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13125 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13126 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13127 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13128 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13129 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13130 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13131 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13132 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13133 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13134 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13135 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13136 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13137 in table holding
2020-10-28 13:50:42 WARNING  Couldn't get value from register 13138 in table holding
2020-10-28 13:50:48 ERROR    Failed to read holding table registers from 5000 to 5100
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 127, in _scan_value_range
    raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
ValueError: Failed to read holding table registers from 5000 to 5100
2020-10-28 13:50:48 ERROR    Unable to decode response Modbus Error: Unknown response 0
2020-10-28 13:50:48 ERROR    Modbus Error: [Input/Output] Unable to decode request
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/pymodbus/transaction.py", line 190, in execute
    self.client.framer.processIncomingPacket(response,
  File "/home/abadonna/.local/lib/python3.8/site-packages/pymodbus/framer/socket_framer.py", line 165, in processIncomingPacket
    self._process(callback, error=True)
  File "/home/abadonna/.local/lib/python3.8/site-packages/pymodbus/framer/socket_framer.py", line 175, in _process
    raise ModbusIOException("Unable to decode request")
pymodbus.exceptions.ModbusIOException: Modbus Error: [Input/Output] Unable to decode request
2020-10-28 13:50:48 ERROR    Failed to read holding table registers from 13000 to 13100
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 127, in _scan_value_range
    raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
ValueError: Failed to read holding table registers from 13000 to 13100
2020-10-28 13:50:49 ERROR    Failed to read holding table registers from 13100 to 13200
Traceback (most recent call last):
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 58, in poll
    values = self._scan_value_range(table, group, DEFAULT_SCAN_BATCHING)
  File "/home/abadonna/.local/lib/python3.8/site-packages/modbus4mqtt/modbus_interface.py", line 127, in _scan_value_range
    raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))

[...]

and so on, and so on...

cat modbus4mqtt/Sungrow_SH5k_20.yaml

ip: 172.16.50.116
port: 502
update_rate: 5
address_offset: 0
variant: sungrow
registers:
  - pub_topic: "no_export/partial/limit"
    set_topic: "no_export/partial/limit/set"
    address: 13073
[...]
# The rest of lines are the same as provided in repo

Change unit to slave in modbus_interfaces

Hello!

I've been using a modified version of modbus4mqtt (applied the modbus RTU patches and added some custom debugging). When running on Debian 12 (Bookworm) the modbus client wasnยดt answering at all, so after some low-level debugging, the solution was to change in modbus_interfaces.py all ocurrences of unit=slave for slave=slave.

It seems that Pymodbus completed the transition from unit to slave.

I apologize for not providing a proper patch.

Best regards!

Needs register masks

It would be good to be able to assign a mask to registers, like 0x0f, such that the values are only written or read from a part of the 16-bit Modbus register.

Getting modbus4mqtt off the ground with or without docker

I am looking for a way to move data from a PLC via modbus via mqtt to a online dashboard/portal.
I followed the bouncing ball...
When I ran the pip3 install --user modbus4mqtt I got a screenfull of stuff about
externally managed environment,

It suggested (sudo) apt install python3-modbus4mqtt
which, after giving my request some consideration, respectfully declined with a "unable to locate package" error

So I thought I'll venture into the mysterious Domain of Docker

Yesterday when I ran: docker pull tjhowse/modbus4mqtt:latest
i was told "permission denied"
Today when I naively and foolishly decided to try again...
it successfully pulled about a dozen "things" and appeared to have zero Cows...
When I then ran: docker run modbus4mqtt --help
I was told Unable to find image 'modbus4mqtt:latest' locally
pull access denied
repository does not exist or may require 'docker login'
I did docker run --help and got a screenfull of stuff which is incomprehensible at this point

There is a link to https://pypi.org/project/modbus4mqtt/Dockerfile - which 404's out
ditto for https://pypi.org/project/modbus4mqtt/modbus4mqtt/Sungrow_SH5k_20.yaml

So I am equally out of brilliant and dumb ideas.

I get the sense that docker is a little microcosm into which stuff can be installed without fouling the larger
pi-nest, but naively assumed that the docker pull would magically "pull it together'

The overall conceptual question I have is:
Is there a way to manually interact as a man-in-the-middle ?
i.e. issue some manual commands which test whether the modbus TCP connection is up and running
and coughing up data - kinda like using ModbusPoll ?
and conversely, to manually simulate the generation of "manual-modbus" data which would then be
mqtt-shovelled up into the dashboard
thereby segmenting a three-node commissioning into two two-nodes commissionings ?

Of course, any clues, hints, over-the-top spoonfeeding for pulling all the bits of string together
would be most welcome ;-)

best regards

Please add Float32 and float64 Support

Would be Nice to have the ability to publish the Float32/float64 valzes of Energy Meter (kWh/Power/Vorlage/.....). to mqtt Server.

Would even donate for this feature.

Thank you for your good work.

No type hinting

I would love to add type hinting across this project. Duck-typed languages make me nervous after spending a while working on Go-centric projects.

Modbus writes aren't batched

Modbus registers are written directly in response to receiving an MQTT message. They should instead be written to a queue in the modbus_interface and written to the device in a controlled way rather than letting MQTT set the pace. Modbus isn't very fast.

modbus4mqtt errors out on SG5K-D

This is a different issue to #13 but related.

The module constantly throws errors:

solar2mqtt | 2020-11-05 20:21:32 ERROR    Failed to read input table registers from 5000 to 5100
solar2mqtt | Traceback (most recent call last):
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 72, in poll
solar2mqtt |     values = self._scan_value_range(table, group, self._scan_batching)
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 143, in _scan_value_range
solar2mqtt |     raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
solar2mqtt | ValueError: Failed to read input table registers from 5000 to 5100
solar2mqtt | 2020-11-05 20:21:38 ERROR    Unable to decode response Modbus Error: Unknown response 0
solar2mqtt | 2020-11-05 20:21:38 ERROR    Modbus Error: [Input/Output] Unable to decode request
solar2mqtt | Traceback (most recent call last):
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/pymodbus/transaction.py", line 190, in execute
solar2mqtt |     self.client.framer.processIncomingPacket(response,
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/pymodbus/framer/socket_framer.py", line 165, in processIncomingPacket
solar2mqtt |     self._process(callback, error=True)
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/pymodbus/framer/socket_framer.py", line 175, in _process
solar2mqtt |     raise ModbusIOException("Unable to decode request")
solar2mqtt | pymodbus.exceptions.ModbusIOException: Modbus Error: [Input/Output] Unable to decode request
solar2mqtt | 2020-11-05 20:21:38 ERROR    Failed to read input table registers from 5000 to 5100
solar2mqtt | Traceback (most recent call last):
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 72, in poll
solar2mqtt |     values = self._scan_value_range(table, group, self._scan_batching)
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 143, in _scan_value_range
solar2mqtt |     raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
solar2mqtt | ValueError: Failed to read input table registers from 5000 to 5100
solar2mqtt | 2020-11-05 20:21:44 ERROR    Failed to read input table registers from 5000 to 5100
solar2mqtt | Traceback (most recent call last):
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 72, in poll
solar2mqtt |     values = self._scan_value_range(table, group, self._scan_batching)
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 143, in _scan_value_range
solar2mqtt |     raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
solar2mqtt | ValueError: Failed to read input table registers from 5000 to 5100
solar2mqtt | 2020-11-05 20:21:50 ERROR    Failed to read input table registers from 5000 to 5100
solar2mqtt | Traceback (most recent call last):
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 72, in poll
solar2mqtt |     values = self._scan_value_range(table, group, self._scan_batching)
solar2mqtt |   File "/usr/local/lib/python3.9/site-packages/solar2mqtt/modbus_interface.py", line 143, in _scan_value_range
solar2mqtt |     raise ValueError("Failed to read {} table registers from {} to {}".format(table, start, start+count))
solar2mqtt | ValueError: Failed to read input table registers from 5000 to 5100

and finally stops reporting (most of the cases at night). Memory leak?

My config SG5K-D.yaml:

ip: 172.16.50.116
port: 502
update_rate: 5
address_offset: 0
variant: sungrow
scan_batching: 100
registers:
  - pub_topic: "5016"  # Active power generated
    address: 5016
    table: 'input'
  - pub_topic: "5002"  # Daily yield
    address: 5002
    table: 'input'
  - pub_topic: "5003"  # Total yield
    address: 5003
    table: 'input'

self._mb.write_register is only valid for 16 bit values - 32 bit values require writing 2 addresses at once

In

self._mb.write_register(addr, value, unit=0x01)
self._mb.write_register is used. But this writes only one address at once, which is only valid for devices expecting 16 bit values. Devices with values like int32 or int64 expect writing all relevant addresses at once and not in sequence. Hence self._mb.write_registers (plural) is required. Sure, this might require code changes at many other positions.

Wrong address for total running time in SG5K-D.yaml

In the example file SG5K-D.yaml the address for total running time is incorrect. The file has

- pub_topic: "total_yield" #Total yield kWh address: 5003 table: 'input' - pub_topic: "total_running_time" #Total running time (h) address: 5003 table: 'input'

I think 5003 should be replaced by 5005 for total_running_time

Add json_key field to YAML configuration

If a register has a json_key field then its key and value should be published to its pub_topic MQTT topic as JSON rather than the raw value. Multiple registers can share the same pub_topic, and this topic will be published at most once per scan populated with all relevant key/value pairs for those registers.

It is not allowed for more than one register to share a pub_topic unless they all have a json_key field and that value is not duplicated amongst registers that share a pub_topic.

Graceless handling of the MQTT broker disconnecting

2020-11-20 09:59:18 INFO     Connected to MQTT.
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 3452, in _thread_main
    self.loop_forever(retry_first_connection=True)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 1779, in loop_forever
    rc = self.loop(timeout, max_packets)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 1181, in loop
    rc = self.loop_read(max_packets)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 1574, in loop_read
    return self._loop_rc_handle(rc)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 2227, in _loop_rc_handle
    self._do_on_disconnect(rc, properties)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 3360, in _do_on_disconnect
    self.on_disconnect(self, self._userdata, rc)
TypeError: _on_disconnect() missing 1 required positional argument: 'rc```

We shouldn't crash on disconnect, and we should try to reconnect.

Modbus TCP connection handling

I've noticed that when I initially connect modbus4mqtt on my SG-5K-D, I get a readout for all the registers, and unless I've got a register in there that is wrong, no errors. As it keeps running though, I'm only getting readouts on a few registers that regularly update and lots of exceptions on less frequently changing registers.

I was wondering if this could be something to do with a single tcp connection staying alive for the whole time that modbus5mqtt is running? I've looked at the code, and I'm not all that familiar with python, but am I correct in assuming that once a tcp connection is made to the inverter, it is kept open? If this is the case, then I was wondering what your thoughts on tearing down the connection and then re establishing it each tcp call to the inverter?

Add a modbus device IP override to the launch arguments

The IP addresses in the example YAMLs are not useful to the majority of people. It would make it easier for people to use the bundled YAMLs directly if the IP address of the modbus device could be overridden in a launch argument.

Scan batching misbehaves with modbus address zero/one issue.

Minimal config:

ip: 192.168.1.89
port: 502
update_rate: 5
address_offset: 0
variant: sungrow
scan_batching: 100
registers:
  - pub_topic: "inverter_model"
    address: 4999
    table: input

Fails: 2021-01-13 20:51:28 ERROR Failed to read 100 input table registers starting from 4900: Modbus Error: [Input/Output] No Response received from the remoteunit/Unable to decode response

This is due to the long-ignored issue of modbus4mqtt addresses starting from zero, but actual modbus addresses starting from 1. When the yaml specifies address 4999 we're actually reading address 5000 from modbus' perspective. This means when we try to batch reads on 100-register-aligned boundaries we send a request for address 4900 (4901) which is invalid.

We should fix the addresses in the example YAMLs and teach the modbus interface to account for the zero/one problem.

Strange issues with version 0.5.0

Yesterday I have updated the version to 0.5.0 and it stopped working. It is throwing errors:

2021-10-10 23:20:52 INFO     Starting modbus4mqtt v0.5.0
2021-10-10 23:20:52 INFO     Connected to MQTT.
2021-10-10 23:20:52 ERROR    Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey
    self._setup()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup
    self._aes_ecb = AES.new(self._key, AES.MODE_ECB)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
2021-10-10 23:21:52 ERROR    Failed to poll modbus device, attempting to reconnect: Modbus Error: [Connection] ModbusTcpClient(172.xxx.xxx.xxx:502): Connection unexpectedly closed 0.001899 seconds into read of 25 bytes without response from unit before it closed connection
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 40, in _getkey
    self._key_packet = self._recv(25)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 298, in _recv
    return self._handle_abrupt_socket_close(
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 343, in _handle_abrupt_socket_close
    raise ConnectionException(msg)
pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] ModbusTcpClient(172.xxx.xxx.xxx:502): Connection unexpectedly closed 0.001899 seconds into read of 25 bytes without response from unit before it closed connection
2021-10-10 23:22:52 ERROR    Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey
    self._setup()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup
    self._aes_ecb = AES.new(self._key, AES.MODE_ECB)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
2021-10-10 23:23:52 ERROR    Failed to poll modbus device, attempting to reconnect: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 87, in poll
    self._mb.poll()
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 71, in poll
    values = self._scan_value_range(table, group, self._scan_batching)
  File "/usr/local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 138, in _scan_value_range
    result = self._mb.read_input_registers(start, count, unit=0x01)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/common.py", line 125, in read_input_registers
    return self.execute(request)
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 107, in execute
    if not self.connect():
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 54, in connect
    self._getkey()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 43, in _getkey
    self._setup()
  File "/usr/local/lib/python3.10/site-packages/SungrowModbusTcpClient/SungrowModbusTcpClient.py", line 23, in _setup
    self._aes_ecb = AES.new(self._key, AES.MODE_ECB)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

When running directly from console, it is throwing errors but occasionally reads and publishes values. Once every 20-40 minutes.
When running inside container I am still waiting for a single happy read - none so far for over an hour.

I call it like that:

modbus4mqtt --hostname mqtt.mydomain.net --port 1883 --username mqtt_user --password password123 --mqtt_topic_prefix solar --config ./SG5K-D.yaml

Please help, the previous version worked fine...

Huawei SUN2000

I'd like to use modbus4mqtt to read my data from my two HUAWEI SUN2000 inverters because I am not happy with my current solution. The problem is that most of the interesting registers are of type INT32 or UINT32. When do you think you are able to merge #29 ?

Add support for float32

I've got some devices that output everything as float32, and I do mean everything, including baud rate, device address, etc. I've tried added float32 support to the code but since I'm not too familiar with python I'm still getting integer values even if I specify 'float32' as the type. Could support for float32 be added?

Scan batching and modbus start address

Hi,

I got the following issue with my solar inverter:
scan batching causes the first modbus read address to start at a boundary that is a multiple of scan batching value.
I have a SMA tripower solar inverter that is somewhat picky on the modbus side: It has 32 bit modbus registers that need to be read within one single modbus command, so I have to set scan_batching=2. However the starting address some registers is an odd value (e.g. 30843 for battery current). So if I set scan_batching = 2 or 4, the script always changes the starting address to an even value (e.g.. 30842). Because this register does not exist, the solar inverter aborts the read command. scan_batching=1 does also not work because the 32 bits must be read within one command.
Workaround is to change line 128 of modbus_read.py in order not to allow to reduce the starting address:
group = int(k) # - int(k) % self._scan_batching
I did not understand the reason why the starting address is reduced to a boundary of scan_batching multiples. Is this somewhere required in the modbus spec or is there another reason to do this?

[v0.3.0] TypeError on run

installed 0.3.0 and got the following error

pi@TBSH02:~ $ /home/pi/.local/bin/modbus4mqtt --hostname 192.168.1.63 --username mqtt --password mqtt --config config.yaml
Traceback (most recent call last):
  File "/home/pi/.local/bin/modbus4mqtt", line 10, in <module>
    sys.exit(main())
  File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/pi/.local/lib/python3.7/site-packages/modbus4mqtt/modbus4mqtt.py", line 169, in main
    i.connect()
  File "/home/pi/.local/lib/python3.7/site-packages/modbus4mqtt/modbus4mqtt.py", line 29, in connect
    self.connect_modbus()
  File "/home/pi/.local/lib/python3.7/site-packages/modbus4mqtt/modbus4mqtt.py", line 38, in connect_modbus
    while self._mb.connect():
  File "/home/pi/.local/lib/python3.7/site-packages/modbus4mqtt/modbus_interface.py", line 37, in connect
    RetryOnEmpty=True, retries=1)
TypeError: 'module' object is not callable

Error with fresh install due to PAHO-Client incomtability

Hi,

I just wanted to try out this library and before using it with docker I like to use it from the CLI in order to better understand what is happening. So I created a folder with a venv and installed modbus4mqtt into it with pip3 install modbus4mqtt. Then I tried to create a really simple config.yaml.

To test the program I just called the help text with modbus4mqtt --help which worked fine.
When I tried to run the program against the config I got:

modbus4mqtt --hostname 192.168.10.5 --config EBYTE_ME31_AAAX2240.yaml

RESULT:

2024-03-28 18:13:21 INFO     Starting modbus4mqtt v0.6.1
Exception ignored in: <function Client.__del__ at 0x0000029263C1C7C0>
Traceback (most recent call last):
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\paho\mqtt\client.py", line 874, in __del__
    self._reset_sockets()
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\paho\mqtt\client.py", line 1133, in _reset_sockets
    self._sock_close()
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\paho\mqtt\client.py", line 1119, in _sock_close
    if not self._sock:
           ^^^^^^^^^^
AttributeError: 'Client' object has no attribute '_sock'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\_coding_\modbus4mqtt_testing\venv\Scripts\modbus4mqtt.exe\__main__.py", line 7, in <module>
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\modbus4mqtt\modbus4mqtt.py", line 292, in main
    i.connect()
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\modbus4mqtt\modbus4mqtt.py", line 42, in connect
    self.connect_mqtt()
  File "D:\_coding_\modbus4mqtt_testing\venv\Lib\site-packages\modbus4mqtt\modbus4mqtt.py", line 75, in connect_mqtt
    self._mqtt_client = mqtt.Client()
                        ^^^^^^^^^^^^^
TypeError: Client.__init__() missing 1 required positional argument: 'callback_api_version'

So I guessed there has to be something wrong with the underlying mqtt library. I removed PAHO 2.0.0 with pip uninstall paho-mqtt and installed and earlier version of PAHO with pip3 install paho-mqtt==1.6.1.

Afterwards modbus4mqtt is running right away, so you should consider either updating your program to the new API of PAHO-mqtt 2.0.0 or change your requirements.txt to use 1.6.1 version during installation.

P.S.: For me it is difficult to understand how the addresses for the modbus registers are configured in the right way. Is the first number of the address the modbus function? When is the holding and when is the input table used? I think this is the core of modbus configuration and the documentation should be a little more extensive.

HTH

regards from germany

Chris

IndexError: list index out of range

When I poll a device the code runs for a minute or two and then starts throwing 'IndexError: list index out of range' errors:

ERROR    Failed to poll modbus device, attempting to reconnect: list index out of range
Traceback (most recent call last):
  File "/home/odroid/.local/lib/python3.10/site-packages/modbus4mqtt/modbus4mqtt.py", line 100, in poll
    self._mb.poll()
  File "/home/odroid/.local/lib/python3.10/site-packages/modbus4mqtt/modbus_interface.py", line 134, in poll
    self._values[table][key] = values[x]
IndexError: list index out of range

I've had a quick look into the code, but see my previous post about lack of familiarity with Python.

Cannot convert signed 32 bit values

Some addresses like Energy Meter Power (address 5083-5084) are signed 32 bit values. There doesn't seem to be a way to read these values (type: int32 did not work). Also, the code looks to read all values as uint16 first and then try and convert them to an actual type so maybe 32 values can't be converted at the moment.

Unit ID = device_address not working?

Huawei inverters use a dongle as bridge to accept modbus tcp requests and map them to modbus rtu. With this more than one inverter can be connected to the dongle.

When wanting to read registers from one or the other inverter, the unit ID is needed to identify the correct inverter.

Please add support for that, many thanks in advance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.