spacemanspiff2007 / smllib Goto Github PK

View Code? Open in Web Editor NEW

27.0 27.0 9.0 69 KB

A library for the SML (Smart Message Language) protocol

License: GNU General Public License v3.0

Python 100.00%

smllib's People

Contributors

Stargazers

Watchers

Forkers

dokuhn eng1neer-com cgommel h3tz j-a-u bubuot f41ardu pvogt09

smllib's Issues

parse_frame fails

get_obis is working fine but parse_frame is failing with my data:

from smllib import SmlStreamReader

file = open("sml.stream", "rb")

daten = file.read(396)

stream = SmlStreamReader()
stream.add(daten)
sml_frame = stream.get_frame()
if sml_frame is None:
    print('Bytes missing')

# Shortcut to extract all values without parsing the whole frame
obis_values = sml_frame.get_obis()
print(obis_values)

# return all values but slower
parsed_msgs = sml_frame.parse_frame()
for msg in parsed_msgs:
    # prints a nice overview over the received values
    print(msg.format_msg())

Output:

`[<obis: 0100000009ff, value: 06454d480104c56c97db>, <obis: 0100010800ff, status: 65954, unit: 30, scaler: -1, value: 117854333>, <obis: 0100020800ff, status: 65954, unit: 30, scaler: -1, value: 325350196>, <obis: 0100010801ff, unit: 30, scaler: -1, value: 117854333>, <obis: 0100020801ff, unit: 30, scaler: -1, value: 325350196>, <obis: 0100010802ff, unit: 30, scaler: -1, value: 0>, <obis: 0100020802ff, unit: 30, scaler: -1, value: 0>, <obis: 0100100700ff, unit: 27, scaler: -1, value: -1772>]
Traceback (most recent call last):
  File "/home/eckhard/smllibtest.py", line 24, in <module>
    parsed_msgs = sml_frame.parse_frame()
  File "/usr/local/lib/python3.9/dist-packages/smllib/sml_frame.py", line 82, in parse_frame
    raise ValueError(
ValueError: No start of SML Message found at 281: 0x77
7607000e1ef6d82a620062007263010176010107000e10e4480e0b06454d480104c56c97db0101635584007607000e1ef6d82b620062007263070177010b06454d480104c56c97db070100620affff7262016510e460887a77078181c78203ff0101010104454d480177070100000009ff010101010b06454d480104c56c97db0177070100010800ff640101a201621e52ff56000706507d0177070100020800ff640101a201621e52ff5600136473340177070100010801ff0101621e52ff56000706507d0177070100020801ff0101621e52ff5600136473340177070100010802ff0101621e52ff5600000000000177070100020802ff0101621e52ff5600000000000177070100100700ff0101621b52ff55fffff9140177078181c78205ff017262016510e46088010183026bcb474c058afe4d88b009910a958432f7c76ef11e1ba5d13d047051d5b189e1263e62d73058e3f03e219b24804ecac4010101632538007607000e1ef6d82e62006200726302017101639eda00

Is this a SML data problem?

Still open: InvalidBufferPos: Start pos bigger than buffer

Even in version 1.2 I get the following error:
File "/opt/daemon/ehz-daemon5.py", line 78, in <module> obis_values = sml_frame.get_obis() File "/usr/local/lib/python3.9/dist-packages/smllib/sml_frame.py", line 123, in get_obis self._parse_msg(data) File "/usr/local/lib/python3.9/dist-packages/smllib/sml_frame.py", line 108, in _parse_msg _lst[i] = v = self.get_value() File "/usr/local/lib/python3.9/dist-packages/smllib/sml_frame.py", line 27, in get_value raise InvalidBufferPos(f'Start pos bigger than buffer: {pos} > {self.buf_len}') smllib.errors.InvalidBufferPos: Start pos bigger than buffer: 244 > 244

ValueError: Last Entry is not <EndofSmlMsg>! Something went wrong!

Hello,
Thanks for your work.

I get the following error message:

[2021-05-30 14:27:00,675] [sml.device.ttyUSB0 ] ERROR | Traceback (most recent call last):
[2021-05-30 14:27:00,676] [sml.device.ttyUSB0 ] ERROR | File "/home/pi/.local/lib/python3.9/site-packages/sml2mqtt/sml_device.py", line 81, in read
[2021-05-30 14:27:00,676] [sml.device.ttyUSB0 ] ERROR | for obj in frame.parse_frame():
[2021-05-30 14:27:00,677] [sml.device.ttyUSB0 ] ERROR | File "/home/pi/.local/lib/python3.9/site-packages/smllib/sml_frame.py", line 114, in parse_frame
[2021-05-30 14:27:00,678] [sml.device.ttyUSB0 ] ERROR | raise ValueError(f'Last Entry is not {EndofSmlMsg}! Something went wrong!')
[2021-05-30 14:27:00,679] [sml.device.ttyUSB0 ] ERROR | ValueError: Last Entry is not ! Something went wrong!

My Data is

b'v\x04\x00\x00\x01b\x00b\x00re\x00\x00\x01\x01v\x01\x01\x07\x00\x00\x02\xdaTM\x0b\n\x01HLY\x02\x00\x04$\xa0\x01\x01c~h\x00v\x04\x00\x00\x02b\x00b\x00re\x00\x00\x07\x01w\x01\x0b\n\x01HLY\x02\x00\x04$\xa0\x01\x01\xf1\x04w\x07\x01\x00`2\x01\x01\x01\x01\x01\x01\x04HLY\x01w\x07\x01\x00`\x01\x00\xff\x01\x01\x01\x01\x0b\n\x01HLY\x02\x00\x04$\xa0\x01w\x07\x01\x00\x01\x08\x00\xffe\x00\x1c\x81\x04e\x02\xdaTMb\x1eR\xffe\x02\xad\x94T\x01w\x07\x01\x00\x02\x08\x00\xffe\x00\x1c\x81\x04e\x02\xdaTMb\x1eR\xffb\x00\x01w\x07\x01\x00\x10\x07\x00\xff\x01\x01b\x1bR\x00S\x00\x9d\x01w\x07\x01\x00 \x07\x00\xff\x01\x01b#R\xffc\t*\x01w\x07\x01\x004\x07\x00\xff\x01\x01b#R\xffc\t&\x01w\x07\x01\x00H\x07\x00\xff\x01\x01b#R\xffc\t-\x01w\x07\x01\x00\x1f\x07\x00\xff\x01\x01b!R\xfeb*\x01w\x07\x01\x003\x07\x00\xff\x01\x01b!R\xfebW\x01w\x07\x01\x00G\x07\x00\xff\x01\x01b!R\xfebF\x01w\x07\x01\x00Q\x07\x01\xff\x01\x01b\x08R\x00b\xf0\x01w\x07\x01\x00Q\x07\x02\xff\x01\x01b\x08R\x00bx\x01w\x07\x01\x00Q\x07\x04\xff\x01\x01b\x08R\x00c\x01\x14\x01w\x07\x01\x00Q\x07\x0f\xff\x01\x01b\x08R\x00c\x018\x01w\x07\x01\x00Q\x07\x1a\xff\x01\x01b\x08R\x00c\x01\x1e\x01w\x07\x01\x00\x0e\x07\x00\xff\x01\x01b,R\xffc\x01\xf4\x01w\x07\x01\x00\x00\x02\x00\x00\x01\x01\x01\x01\t1.02.007\x01w\x07\x01\x00`Z\x02\x01\x01\x01\x01\x01\x05A01A\x01w\x07\x01\x00`\x05\x00\xff\x01\x01\x01\x01e\x00\x1c\x81\x04\x01\x01\x01c#b\x00v\x04\x00\x00\x03b\x00b\x00re\x00\x00\x02\x01q\x01c\xe8#\x00'

The EndofSMLMsg comes in the wrong place.

['000002', 0, 0, [1793, [None, '0a01484c5902000424a0', None, None, ['770701', <EndofSmlMsg>, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 0, 0]], 0, 0]

Can you help ?

Trying to get it running; getting AttributeError

Hello,

I'm trying to get your SmlLib running on my Raspberry Pi with Python 3.9 and a DZG DWZ7412.2 smart meter. When parsing an incoming SML message I'm getting the following error:

Traceback (most recent call last):
  File "/home/pi/.asdf/installs/python/3.9.2/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/pi/.local/share/virtualenvs/sml2mqtt-h7SpFYOC/lib/python3.9/site-packages/serial_asyncio/__init__.py", line 119, in _read_ready
    self._protocol.data_received(data)
  File "/home/pi/my_sml2mqtt/main.py", line 59, in data_received
    self.parser.parse(data)
  File "/home/pi/my_sml2mqtt/main.py", line 32, in parse
    parsed_msgs = sml_frame.parse_frame()
  File "/home/pi/.local/share/virtualenvs/sml2mqtt-h7SpFYOC/lib/python3.9/site-packages/smllib/sml_frame.py", line 93, in parse_frame
    ret.append(SmlMessage.from_list(val))
  File "/home/pi/.local/share/virtualenvs/sml2mqtt-h7SpFYOC/lib/python3.9/site-packages/smllib/sml_fields.py", line 102, in from_list
    _in[3] = get_msg_cls(_in[3])
  File "/home/pi/.local/share/virtualenvs/sml2mqtt-h7SpFYOC/lib/python3.9/site-packages/smllib/sml_fields.py", line 241, in get_msg_cls
    return cls.from_list(_in[1])
  File "/home/pi/.local/share/virtualenvs/sml2mqtt-h7SpFYOC/lib/python3.9/site-packages/smllib/sml_fields.py", line 47, in from_list

I'm not sure if there is personal data in the SML message. Can I send it to you too without having to worry about that?

Thanks! Seems like your lib is the only available for Python. I hope I can make it run.

Debugging CRC Errors [while using stream.get_frame() method] - How2?

I AM JUST A PYTHON NOOB - sorry!

Obis Code 2.8.0 umbenennen?

In der const.py existiert im dict eine Zeile für 2.8.0

'0100020800ff': 'Wirkenergie Total',

Ist das vielleicht eher die eingespeiste Energie im Gegensatz zur bezogenen Energie (1.8.0)?

Gruß,
Bernd

is it possible to integrate serialnumber decoding?

wäre es für dich möglich, ein decodierung der hex seriennummer in decimal einzufügen?

Zitat (https://tasmota.github.io/docs/Smart-Meter-Interface/):
If you want you can also convert it to your Identifikationsnummer with some ASCII, DEC and HEX encoding. 0A-01-45-46-52-ST-UV-WX-YZ = 1EFR + string(S) + string(T) + hexToDec(UVWXYZ)

Danke, Mart

Negative current consumption; no feed-in

Hey, it's me again.

The sml2mqtt service I'm running on my Raspi Zero W works now, but I've noticed one thing I feel is still off. The service reports negative consumption values (aktuelle Wirkleistung; I suppose you're from Germany, like me?) even though I've got no feed-in (Einspeisung) and the smart meter shows the correct (positive) value on its display.

I'm not sure but it seems like the scaler isn't evaluated under some circumstances. The reported value multiplied by the given scaler gives a much more realistic consumption (close to what the Smart Meter shows on its display; I'm unfortunately not fast enough to check all three values simultaneously to be sure).

It might also be a faulty smart meter firmware as described here: https://wiki.volkszaehler.org/hardware/channels/meters/power/edl-ehz/dzg_dvs74 (the second yellow box).

If you want to check, here's the message I'm getting:

b'76051b105102620062007263010176010102310b0a01445a47000282c0b0726201650548bbeb62026339c20076051c105102620062007263070177010b0a01445a47000282c0b0070100620affff726201650548bbeb7577070100603201010172620162006200520004445a470177070100600100ff017262016200620052000b0a01445a47000282c0b00177070100010800ff641c01047262016200621e52ff650800a9e40177070100020800ff017262016200621e52ff62000177070100100700ff017262016200621b52fe53aefe010101638c670076051d1051026200620072630201710163e39a00'

smllib on Raspberry Pi?

After testing on Ubuntu I am tried to use SmlLib on a Raspberry Pi. But the import already failed:

i@raspi:~ $ sudo python3 ./ehz2.py 
Traceback (most recent call last):
  File "./ehz2.py", line 6, in <module>
    from smllib import SmlStreamReader
  File "/usr/local/lib/python3.7/dist-packages/smllib/__init__.py", line 3, in <module>
    from smllib.sml_frame import SmlFrame                # noqa: F401
  File "/usr/local/lib/python3.7/dist-packages/smllib/sml_frame.py", line 107
    while (start := self.bytes.find(b'\x77\x07\x01', start + 1)) != -1:
                 ^
SyntaxError: invalid syntax
pi@raspi:~ $

Does SmlLib require Python 3.8?

OBIS Codes

Hello,

Great SML module for python. I´ll be using it soon for one of my scripts.

One improvement I´ve added in my script is to decoded the OBIX Hex strings you output.

https://www.promotic.eu/en/pmdoc/Subsystems/Comm/PmDrivers/IEC62056_OBIS.htm
If looking at the whole HEX string is contains: A-B:C.D.E*F

Personally I think C.D.E is sufficient.

In my testing today simply decoded it with:
obis_C = int(value[4:6], 16)
obis_D = int(value[6:8], 16)
obis_E = int(value[8:10], 16)
obis_string = "{}.{}.{}".format(obis_C, obis_D, obis_E)

Would be great if your library would decode it directly instead of giving the RAW Hex value.

Thanks.

smllib.sml_frame.InvalidBufferPos: Start pos bigger than buffer: 333 > 333.

Hi,

thanks for creating this awesome library. I have been using it for quite some time now and it works really well. Unfortunately there is one issue which pops up every few hours if the library is used for continuous telegram reading of my electric meter.

Error:
smllib.sml_frame.InvalidBufferPos: Start pos bigger than buffer: 333 > 333.
sml_frame.py", line 37, in get_value
(occurs while calling "sml_frame.get_obis()")

I tried to investigate why it only occurs for certain telegrams but not all of them.

It seems that it is due to a bug in the get_obis() function in this line:
while (start := self.bytes.find(b'\x77\x07\x01', start + 1)) != -1:

This line seems to be searching for OBIS list entries. But if the 'value' entry of some of the OBIS register itself (e.g. the total energy register) ends with value 0x7707 this will lead to the issue since the next byte after the register is usually a 0x01. In combination a new OBIS is detected by mistake which will screw up the array.

I created the pull request #7 as an example of how I fixed it locally. Basically I am just adding up all the length byte nibbles of each of the OBIS entries and add this as an offset for the next OBIS occurence. That way I prevent a false detection of a new OBIS.

How to reproduce the issue

I attached a few example telegrams:
telegrams_to_replicate_issue.txt
usual_data: regular telegram which is working fine
not_working: telegram which caused the issue
fixed_data: issue fixed by replacing the total energy register value 0x7707 with zeros
(The telegrams are slightly modified. I removed private data and updated the CRC)

Example code (as in your readme) using the "not_working" telegram

stream = SmlStreamReader()
stream.add(a2b_hex(not_working))
sml_frame = stream.get_frame()
if sml_frame is None:
    print('Bytes missing')

# Shortcut to extract all values without parsing the whole frame
obis_values = sml_frame.get_obis()

# return all values but slower
parsed_msgs = sml_frame.parse_frame()
for msg in parsed_msgs:
    # prints a nice overview over the received values
    print(msg.format_msg())

[INFO] act_sensor_time can also be the timestamp of meter installation

info only:
in my case it's showing the "time" (=seconds) since the smartmeter was installed and activated.
act_sensor_time / SML_timestamp

CRC Errors

Hi there,

I'm using smllib to monitor my smartmeter.
Occasionally (rather frequently lately) I'm seeing CRC errors like the one below:

monitoring-smartmeter-exporter-1  |   File "/usr/local/lib/python3.10/site-packages/smllib/reader.py", line 58, in get_frame
monitoring-smartmeter-exporter-1  |     raise CrcError(msg, crc_msg, crc_calc)
monitoring-smartmeter-exporter-1  | smllib.errors.CrcError: (b'\x1b\x1b\x1b\x1b\x01\x01\x01\x01v\x05\tPA\xbab\x00b\x00rc\x01\x01v\x01\x01\x05\x03\x1a\xc0\x94\x0b\t\x01ISK\x00\x04i\xb2\xb8\x01\x01c\x08\xf3\x00v\x05\tPA\xbbb\x00b\x00rc\x07\x01w\x01\x0b\t\x01ISK\x00\x04i\xb2\xb8\x07\x01\x00b\n\xff\xffrb\x01e\x05;\x82\x87ww\x07\x81\x81\xc7\x82\x03\xff\x01\x01\x01\x01\x04ISK\x01w\x07\x01\x00\x00\x00\t\xff\x01\x01\x01\x01\x0b\t\x01ISK\x00\x04i\xb2\xb8\x01w\x07\x01\x00\x01\x08\x00\xffe\x00\x00\x01\x82\x01b\x1eR\xffY\x00\x00\x00\x00\x05\xab\xdb\xdd\x01w\x07\x01\x00\x01\x08\x01\xff\x01\x01b\x1eR\xffY\x00\x00\x00\x00\x05\xaboW\x01w\x07\x01\x00\x01\x08\x02\xff\x01\x01b\x1eR\xffY\x00\x00\x00\x00\x00\x00l\x86\x01w\x07\x01\x00\x10\x07\x00\xff\x01\x01b\x1bR\x00U\x00\x00\x01\'\x01w\x07\x81\x81\xc7\x82\x05\xff\x01\x01\x01\x01\x83\x02\xc4\xd3$\xa0\xa77\xda\x15B\xe0j\xbf\xdd\x95\xd4\xf1\xd4\xe4\x1aL\xa4\xa5g\x05\xc9"_\xeeT\x87\x91\x80\xdcn\x1b\x1b\x1b\x1b\x01\x01\x01\x01v\x05\tPA\xbdb\x00b\x00rc\x01\x01v\x01\x01\x05\x03\x1a\xc0\x95\x0b\t\x01ISK\x00\x04i\xb2\xb8\x01\x01c;o\x00v\x05\tPA\xbeb\x00b\x00rc\x07\x01w\x01\x0b\t\x01ISK\x00\x04i\xb2\xb8\x07\x01\x00b\n\xff\xffrb\x01e\xc7\x82\x05\xff\x01\x01\x01\x01\x83\x02\xc4\xd3$\xa0\xa77\xda\x15B\xe0j\xbf\xdd\x95\xd4\xf1\xd4\xe4\x1aL\xa4\xa5g\x05\xc9"_\xeeT\x87\x91\x80\xdcn:\x93\x8a\xcf\x88z\x9a\xdf}\xb4h#\xe2_\x01\x01\x01c\xaaD\x00v\x05\tPA\xbfb\x00b\x00rc\x02\x01q\x01c\xbe\x7f\x00\x1b\x1b\x1b\x1b\x1a\x00vH', 30280, 23196)

Some frames in between always seem fine and yield proper data.
Can I expect this to be a "hardware" issue possibly related to the IR reader (in which case I'd probably catch & ignore them)? Or could this be some other issue?