dennisosrm / hms-mqtt-publisher Goto Github PK

it would be great if release binaries were published after a tag/release was created. preferably for multiple architectures. that would make it easier to simply run hms-mqtt-publish on a raspberry pi for example, without having to cross-compile on your own or installing the whole rust toolchain on the pi.

if that's something that you're generally interested in, i'd start working on it.

Hi, just tested this library and it works great!
But I do have a question: Is there a possibility to configure the power ratio? I saw that you're working with a TCP connection to read values from the DTU and I was wondering if it's also possible to adjust the maxmimum generated power through this connection.
I was planning to handle this automatically but I'm currently struggling to set it up.

Thanks for your work!

The config.toml file can only be read, if it's in the executed directory. if you try to call the program from other directory in e.g. /usr/bin, the program stops with an error.
Following code for main.rs establish program to read config.toml in the same directory as the executable file is:

use std::path::PathBuf;

fn main() {
    let mut path = std::env::current_exe().expect("Unable to get current executable path");
    path.pop(); // Remove executable name
    path.push("config.toml");

    let contents = fs::read_to_string(&path).expect("Could not read config.toml");
    // Rest of your code using 'contents' variable
} 

I would like to power adjust my two HMS 800W-2T to achieve a "Nulleinspeisung".

For that it would be perfect if one could write into the mqtt the %/power level.

It would be great to have the option to define the name of the MQTT topics using a variable in the config.toml

I now have two containers running with hms-mqtt-publisher inside and everything seems to be working ok.
I just get a lot (3-4 per second) of the following messages in my mosquitto log:
1709505662: Client hms800wt2-mqtt-publisher-ha already connected, closing old connection.

I think this is because both instances are using the same client name when connecting to mqtt. I see two possibilities:

• Add the inverter id to the client name
• Let the user configure the client name (or at least part of it)

Is it possible to do this?

The tool should be able to support more than one output channel, e.g. home assistent auto discovery + publishing, a simple independent MQTT publisher, and InfluxDB publishing come to mind.

For the two ports there are infos 'pv_energy_total', 'pv_daily_yield' available.

for the inverter only 'pv_daily_yield'.

Making 'pv_energy_total' of the inv would be nice!

• temperature
• total yield
• mpp tracker specific metrics

It would be great to have an option that the publisher can handle multiple inverters at the same time.

As mentioned in #17, the tool should support auto discovery in a home assistant setup. This is achieved by publishing a certain topic to MQTT. More details:

• discussion: #17 (comment)
• HA documentation
• OpenAI generated usage information (unverified)

Hi,

I have two of the HMS-800W-2T and want's to request both inverters.
I tried it with configuring the normal and the nightly version. Is there another possibility?

The other thing is, it overwrites the other in HA, because of the serial is not complete in the naming:

Where I can edit it, that I have only the last 6 for example of the serial?

Thanks

This will improve future flexibility

Discussed in #89

One possibility is to announce the state change only. Right now, the information that a state change happened is available in the inverter struct when fetching the update. This is not exposed to the outside though.

Hi,

I setup the hms_mqtt_publish as docker on a debian vm. I configuriered the ip of the inverter and my mqtt broker.

I start the container with this command docker run --rm -it -v ./config.toml:/config.toml dennisosrm/hms-mqtt-publisher

In the log it shows that the inverter is offline, but in a tcpdump I can see connections between the vm and the inverter. What can be the reason for this?

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:52:14.150276 IP 10.12.30.7.58834 > 10.12.11.100.10081: Flags [S], seq 1823358672, win 64240, options [mss 1460,sackOK,TS val 594905342 ecr 0,nop,wscale 7], length 0
13:52:14.154111 IP 10.12.11.100.10081 > 10.12.30.7.58834: Flags [S.], seq 71257, ack 1823358673, win 8760, options [mss 1460,nop,nop,sackOK], length 0
13:52:14.154167 IP 10.12.30.7.58834 > 10.12.11.100.10081: Flags [.], ack 1, win 64240, length 0
13:52:14.154195 IP 10.12.30.7.58834 > 10.12.11.100.10081: Flags [P.], seq 1:11, ack 1, win 64240, length 10
13:52:14.358784 IP 10.12.30.7.58834 > 10.12.11.100.10081: Flags [P.], seq 1:11, ack 1, win 64240, length 10
13:52:14.362424 IP 10.12.11.100.10081 > 10.12.30.7.58834: Flags [.], ack 11, win 8750, length 0
13:52:19.310971 IP 10.12.30.7.58834 > 10.12.11.100.10081: Flags [F.], seq 11, ack 1, win 64240, length 0
13:52:19.315352 IP 10.12.11.100.10081 > 10.12.30.7.58834: Flags [.], ack 12, win 8749, length 0
13:52:19.315978 IP 10.12.11.100.10081 > 10.12.30.7.58834: Flags [F.], seq 1, ack 12, win 8749, length 0
13:52:19.316073 IP 10.12.30.7.58834 > 10.12.11.100.10081: Flags [.], ack 2, win 64240, length 0
13:52:49.811249 IP 10.12.30.7.49678 > 10.12.11.100.10081: Flags [S], seq 1505675049, win 64240, options [mss 1460,sackOK,TS val 594941003 ecr 0,nop,wscale 7], length 0
13:52:49.815124 IP 10.12.11.100.10081 > 10.12.30.7.49678: Flags [S.], seq 77013, ack 1505675050, win 8760, options [mss 1460,nop,nop,sackOK], length 0

I have a hms-800w-2t.

config.toml:

inverter_host = "10.12.11.100"
update_interval = 30500

[home_assistant]
host = "10.12.30.2"
username = "USER"
password = "PW"
port = 1883

I am using this project as anyone else, but am also exporting Home Assistant data via Prometheus and graphing it via Grafana. It seems that the grafana graph is quite pointy with lots of spikes.

I am not sure if this is usual for balcony solar devices, since the sun isn't rapidly switching on and off, so i am opening this issue to confirm if this is the usual behaviour for the metric, or if there is something to be improved.

also publish crate.

This will make the code reusable in an embedded implementation.

Hi,
in the dockerfile for the home assistant addon is still fixed to version 0.3.
Can you please correct this to make the latest release available as home assistant addon as well.
Thanks

Build a tool that simulates the inverter behavior. This will enable offline development and testing.

As mentioned first here: lumapu/ahoy#1189 (comment)

• merge #13
• create credentials for Dockerhub
• set up auto-deployment via a GitHub action

Hello, will this code find a way in openDTU? this was very nice ..
users without a Display, Grafana, and DB can show via URL the data from the Wifi WR

Hi,

I've got an issue running the has-mqtt-publisher container (from repo) on Raspi3 (armv7l). It starts properly but doesn't deliver all values:

[2024-03-15T15:12:55Z INFO hms_mqtt_publish] Running revision: b3759da
[2024-03-15T15:12:55Z INFO hms_mqtt_publish] loading configuration from /config.toml
[2024-03-15T15:12:55Z INFO hms_mqtt_publish] using non-default update interval of 300.00s
[2024-03-15T15:12:55Z INFO hms_mqtt_publish] inverter host: 192.168.0.124
[2024-03-15T15:12:55Z INFO hms_mqtt_publish] Publishing to simple MQTT broker
[2024-03-15T15:12:55Z INFO hms2mqtt::inverter] Inverter is Online
thread 'main' panicked at library/std/src/sys/unix/thread.rs:245:21:
assertion left == right failed
�
1
�
4
Would really appreciate if this can be fixed.

This will prohibit the following artifact:

Hi,
at first Thank You for sharing this project.

I've compiled the sources using RaspberryPi, Ubuntu 20.04 and cargo/rust 1.73.0 for a friend. Starting the binary with given ip-adresses of inverter and mqtt broker tells me, that the inverter is online. But nothing else happens, no data are published. The inverter is a HMS 800W-T2, checking the version in the hoymiles app it tells V1.1.15, I suppose its the version number of the DTU.

What I didn't do was setting the environment variables.

Any ideas?

[edit]
additional: normally I am connection anonymous to the broker, so I didn't deliver neither mqttuser/pwd nor portno. to the tool
[/edit]

Hi,
I have updated the docker image from v.01 to v.02. Now it is no longer working and i get following errors:

toml config unparsable: Error { inner: Error { inner: TomlError { message: "invalid string\nexpected ", '", original: Some("inverter_host = \"\" \n\n[home_assistent] \nhost = \"\"\nusername = \"\"\npassword = \"\"\nport = \n\n"), keys: [], span: Some(84..85) } } } thread 'main' panicked at src/main.rs:46:52:

I changed back to v.01 and it worked again. Are there any new settings required or is it a bug?

I have a MQTT server which allows access via TLS only. Just setting the port to 8883 results in a lot of complaining by the server as hms-mqtt-publisher seems to reconnect really fast and often, but not with TLS enabled.

Is there a way this could be implemented and exposed to the config.toml?

I just tested the docker image and the configuration through env variables is not working yet.

Looking at your dockerfile (https://github.com/DennisOSRM/hms-mqtt-publisher/blob/main/Dockerfile#L38) it looks like this line should be part of an entrypoint script instead of being executed during build with RUN.

I can also create a PR if you are interested.

$ docker run --rm -it dennisosrm/hms-mqtt-publisher cat config.toml

inverter_host = ""

[home_assistent]
host = ""
username = ""
password = ""
port =

As surfaced in PR20:

Maybe long term we can also come up with some means of automatic testing. For this we would need:

a docker service emulating a HMS
a docker service running an mqtt server
a docker service running some tests and validates mqtt messages against some predefined schema
Such a test framework would of course also be useful for testing the rest of the codebase

• What's necessary to have a barebones implementation that runs on ESP32 chips
• Break down implementation plan into milestones

 Add-on version: v0.5
 You are running the latest version of this add-on.
 System: Home Assistant OS 12.3  (aarch64 / raspberrypi4-64)
 Home Assistant Core: 2024.5.4
 Home Assistant Supervisor: 2024.05.1

Although update_interval is set to 60500 (I even tried bigger values like 90000) I can see the data values of the hoymiles device in HA refreshing every 30 seconds.
That alone is not a big problem but I also can see the DTA going offline most of the time in Hoymiles cloud since the add-on is running in HA. Maybe related to the (too) frequent updates in HA.

Hello,

I've noticed that the MQTT port is currently hardcoded in the application. For improved flexibility and to accommodate various deployment environments, it would be beneficial to have the MQTT port settable through a configuration parameter.

Thanks

The current [A] for Pv x provided by MQTT in HA is factor 10 too high, e.g. 80A instead of 8A.
Please let us know whether it will be corrected in future updates.

Since Docker v0.2 there seem to be username und password required arguments for the MQTT server connection. The container crashes while starting if I don't provide any user credentials. But I cannot provide correct user credentials as my MQTT server has no authentication requirements.

I would like to run this application directly within home assistant as an addon. I do have a little experience with writing addons and think it shouldn't be too complicated. Has someone already started with that? Does it make sense to have a separate repository for the home assistant addon or directly create a PR for this one?

Great work btw, I really appreciate it!

The number of parameters is growing, things are becoming unwieldy.

hello,
my hms800w-2t stops acception connections after some days.
it will work perfectly for 3-4 days then it stops.
someone has a fix for that ?

you can even restart the inverter via the app, nothing happens.
plugging it out of the wall and back in, nothing happens.
the next day everything is fine again.

btw. the hoymiles cloud is updating fine

config.toml:

inverter_host = ""
coop_mode = true
update_interval = 60500

[simple_mqtt]
host = ""
port = 1883

Logs:

[2024-03-03T08:43:11Z DEBUG hms2mqtt::simple_mqtt] dtu_sn: time: 1709455397 device_nub: 1 package_nub: 1 inverter_state {inv_id: port_id: 1 grid_voltage: 2369 grid_freq: 5005 pv_current_power: 2369 unknown1: 99 unknown2: 999 temperature: 215 unknown3: 5 unknown4: 1 bit_field: 7143442 11: 59656} port_state {pv_sn: pv_port: 1 pv_vol: 329 pv_cur: 758 pv_power: 2502 pv_energy_total: 29064 pv_daily_yield: 254 bitfield: 50921472} port_state {pv_sn: pv_port: 2 pv_vol: 4 pv_cur: 1 bitfield: 50331648} pv_current_power: 2369 pv_daily_yield: 254

[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/inverter_local_time, Pkid = 0, Payload Size = 29
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_current_power, Pkid = 0, Payload Size = 5
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_daily_yield, Pkid = 0, Payload Size = 3
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_current_power, Pkid = 0, Payload Size = 5
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_daily_yield, Pkid = 0, Payload Size = 3
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_grid_voltage, Pkid = 0, Payload Size = 5
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_grid_freq, Pkid = 0, Payload Size = 5
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_inv_temperature, Pkid = 0, Payload Size = 4
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port1_voltage, Pkid = 0, Payload Size = 4
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port1_curr, Pkid = 0, Payload Size = 4
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port1_power, Pkid = 0, Payload Size = 5
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port1_energy, Pkid = 0, Payload Size = 5
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port1_daily_yield, Pkid = 0, Payload Size = 3
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port2_voltage, Pkid = 0, Payload Size = 3
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port2_curr, Pkid = 0, Payload Size = 4
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port2_power, Pkid = 0, Payload Size = 1
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port2_energy, Pkid = 0, Payload Size = 1
[2024-03-03T08:43:11Z DEBUG rumqttc::state] Publish. Topic = hms800wt2/pv_port2_daily_yield, Pkid = 0, Payload Size = 1
[2024-03-03T08:43:13Z DEBUG rumqttc::state] Pingreq,
last incoming packet before 5000 millisecs,
last outgoing request before 2631 millisecs

... some pings ...

[2024-03-03T08:44:08Z DEBUG rumqttc::state] Pingreq,
last incoming packet before 5000 millisecs,
last outgoing request before 5001 millisecs
[2024-03-03T08:44:11Z DEBUG hms2mqtt::inverter] could not connect: Connection refused (os error 111)
[2024-03-03T08:44:11Z INFO hms2mqtt::inverter] Inverter is Offline`

Some Grafana Stats:

I'm not sure where this issue stems from, but it seems that either the inverter stops sending updates or the application gets stuck after a while (usually after a couple of hours). I'm still at the beginning of investigating why this happens. For now, I've increased the delay between updates to one minute to see if that fixes things.

Hi,

I can't compile it and don't find an solution:

Updating crates.io index
   error: failed to select a version for `env_logger`.
   ... required by package `hms-mqtt-publish v0.2.0 (/root/hms-whg1)`
versions that meet the requirements `^0.11.2` are: 0.11.3, 0.11.2

the package `hms-mqtt-publish` depends on `env_logger`, with features: `anstream` but `env_logger` does not have these features.
It has an optional dependency with that name, but that dependency uses the "dep:" syntax in the features table, so it does not have an implicit feature with that name.

failed to select a version for `env_logger` which could resolve this conflict

Thanks for your help.

it was also nice when this software can run standalone or with openDTU on the Fusion 2.x

https://shop.allianceapps.io/products/opendtu-fusion-community-edition
thanks for your work

hi guys, this is a great project and I have it working against my new inverter and sending mqtt data. However, I see that it seems to stop the upload to the s-miles cloud. Is this the case that you have experienced?

I'm attached to the inverter via my normal wlan and not the inverter's internal wifi.

Hope someone can shed some light on this one,

Thanks

Should be a matter of retrieving a GetConfig response, encode the changed information and play it back in a SetConfig request

Hi,
really like the project, as I bought a HMS two weeks ago^^
I run it as a service and it works like a charm :)
It would be nice to have an option in the config file to specify the update interval.
Cheers iro