Reduce update frequency of plugin about ha-knmi HOT 30 CLOSED

I've got a PR ready where you can configure the scan interval, see #108
Feel free to try it out, I'll run it in my production instance and if everything is fine I'll try to release it in the coming days.

from ha-knmi.

The forecast attributes are deprecated and will be removed in 2023.3, if I remember correctly, so I wouldn't rely on those anymore.
weather.get_forecasts service only pulls the data out of the weather entity, it doesn't fetch any data from the API.

from ha-knmi.

i see, that didnt show in the screenshots.
thx for explaining

from ha-knmi.

Even without using the key somewhere else i get regular API limit exceeded errors.

from ha-knmi.

from ha-knmi.

Klinkt goed, kan ik een PR inschieten?

from ha-knmi.

from ha-knmi.

Even without using the key somewhere else i get regular API limit exceeded errors.

You might get this if you do multiple restarts a day, like 10+

from ha-knmi.

Personally, I would sign up for multiple keys, the website doesn't mention that that isn't allowed.

from ha-knmi.

Even without using the key somewhere else i get regular API limit exceeded errors.

You might get this if you do multiple restarts a day, like 10+

I don't. At most once a week. In the mean time, for today the API key exceeded its limit again. No reboots in between. I'll see what I can edit in the code 💪🏻

from ha-knmi.

Even without using the key somewhere else i get regular API limit exceeded errors.

You might get this if you do multiple restarts a day, like 10+

I don't. At most once a week. In the mean time, for today the API key exceeded its limit again. No reboots in between. I'll see what I can edit in the code 💪🏻

Are you using the same key multiple times? With the default config the API is being called 288 times (within the 300 limit)

from ha-knmi.

Not to my knowledge, but I could always get a fresh one just in case.

from ha-knmi.

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

from ha-knmi.

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

from ha-knmi.

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

from ha-knmi.

having disabled automatic. polling

and using an automation to update the integration and a rest sensor targeting the same resource

  - id: update_weerlive_knmi
    trigger:
      platform: time_pattern
      minutes: '/12'
    condition:
      condition: state
      entity_id:
        - binary_sensor.ha_restarted
        - binary_sensor.automation_integration_reloaded
        - binary_sensor.template_integration_reloaded
      state: 'off'
#       condition: numeric_state
#       entity_id: sensor.sun_solar_elevation
#       above: 10
    action:
      - service: homeassistant.update_entity
        target:
          entity_id:
  #           - sensor.weerlive_voorspelling_morgen
  #           - sensor.weerlive_voorspelling_vandaag
  #           - sensor.weerlive_actueel
          - sensor.weerlive_alarm
      - service: homeassistant.reload_config_entry
        data: {}
        target:
          device_id: 283263df2cf63301888cf4096178ffb7

I still receive this error:

Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.knmi
Source: custom_components/knmi/coordinator.py:37
Integration: KNMI (documentation, issues)
First occurred: 5 december 2023 om 23:06:03 (46 occurrences)
Last logged: 12:00:20

Update failed! - API key daily limit exceeded, try again tomorrow
Update failed! - Timeout error fetching information

which shouldnt be happening because these updates take place 5 times per hour = 120 a day, per service, so that would make 240 calls..

my api page at weerlive states:

Je hebt vandaag 117 API-requests gedaan.

which is well within the limits and makes the error in HA even more mysterious

any thought how this could occur? (also consider Ive disabled triggering upon HA restart, and reload of the automatic. and template integrations

from ha-knmi.

The best way to debug that is by calling the link directly, you can do that in your browser:
https://weerlive.nl/api/json-data-10min.php?key=KEY&locatie=LAT,LON

Update failed! - Timeout error fetching information

This is an actual error on the server where the API takes too long to respond, I got a few of those as well.

When you use homeassistant.update_entity I think you should only add one entity, this would fetch new data and update all entities in this integration. If you add multiple, all of them will trigger a call to the backend.

What is the idea of using the homeassistant.reload_config_entry? And what are you trying to achieve in general?

from ha-knmi.

What is the idea of using the homeassistant.reload_config_entry?

Simply to automate updating the integration entities, as Ive disabled the auto polling feature

And what are you trying to achieve in general?

ensuring that my rest sensor and your integration dont go over the Api request threshold

since the error was displayed: Update failed! - API key daily limit exceeded, try again tomorrow

somehow that must have been caused by the integration, because my api said it was only at 117. Weerlive support also confirmed this, and that the api I use on my system was still valid for the day without a problem...

When you use homeassistant.update_entity I think you should only add one entity, this would fetch new data and update all entities in this integration.

so this would do it?

    action:
      - service: homeassistant.update_entity
        target:
          entity_id:
  #           - sensor.weerlive_voorspelling_morgen
  #           - sensor.weerlive_voorspelling_vandaag
  #           - sensor.weerlive_actueel
          - sensor.weerlive_alarm. # rest sensor
          - sensor.knmi_gevoelstemperatuur # knmi integration to update all entities ?
#       - service: homeassistant.reload_config_entry
#         data: {}
#         target:
#           device_id: 283263df2cf63301888cf4096178ffb7

from ha-knmi.

What is the idea of using the homeassistant.reload_config_entry?

Simply to automate updating the integration entities, as Ive disabled the auto polling feature

If you use homeassistant.update_entity I don't see why reloading would be needed.

And what are you trying to achieve in general?

ensuring that my rest sensor and your integration dont go over the Api request threshold

I'm not sure why you have a rest sensor and this integration at the same time, maybe you should try to create your own weather entity from a rest sensor using https://www.home-assistant.io/integrations/weather.template/ to take full control of what you want.

since the error was displayed: Update failed! - API key daily limit exceeded, try again tomorrow

somehow that must have been caused by the integration, because my api said it was only at 117. Weerlive support also confirmed this, and that the api I use on my system was still valid for the day without a problem...

Or the rest sensor... ;-) What scan_interval do you have set there? The default is 30 seconds if I'm not mistaken.
If you see this again, check the URL manually to see if it returns json data or the error message.

When you use homeassistant.update_entity I think you should only add one entity, this would fetch new data and update all entities in this integration.

so this would do it?

    action:
      - service: homeassistant.update_entity
        target:
          entity_id:
  #           - sensor.weerlive_voorspelling_morgen
  #           - sensor.weerlive_voorspelling_vandaag
  #           - sensor.weerlive_actueel
          - sensor.weerlive_alarm. # rest sensor
          - sensor.knmi_gevoelstemperatuur # knmi integration to update all entities ?
#       - service: homeassistant.reload_config_entry
#         data: {}
#         target:
#           device_id: 283263df2cf63301888cf4096178ffb7

Why do you have your rest sensor here as well?

from ha-knmi.

I made the rest sensor, to have more detailed info than the integration. It returns several other data not available in the integration.
Tbh, I was hoping it would also return the alarmcode if active, that's the main reason why I created the rest sensor. But I dont think it does. It's the essential feature not available ;-) Geel/Oranje/Rood would allow us to represent that in th HA frontend...

anyways, my rest sensor, ofc, is setup like:

rest:

  - resource: !secret weerlive_rest_resource
    scan_interval: 86400 # <---- no auto updating here

so we can finetune that using a automation . As I do, updating it every 12 minutes.
btw, Ive now setup a second Api key dedicated per service, to be sure they wont frustrate the api limit either way. But as you can see that shouldnt have happened in the first place.

from ha-knmi.

What are you missing from the API in this integration? Most of the values should be here or you should be able to derive them. As you mention there is indeed no alarm code field in the API.

Your rest sensor still updates once per day and I do think also on startup

from ha-knmi.

yes, I guess you are right about the updating of the rest sensor, I am trying to get to the bottom of that. It doesnt cause the api to be exceeded though.

for the entities in the integration:
I'd love to see the actual temp (and not only feel temp) and visibility. As for the wind speed, I dont believe all available fields are displayed?

airpressure (the ld* values)

I am aware some of these are in the weather entity, having them as separate entities would be nice though, and make for easier front-end representation (or requiring templates on the weather entity attributes)

not sure how to add the many forecast details in the weather card, but at least allowing to exist in the sensor entities section would help, so we can enable them if so desired.

cut it short: why not simply use all available data the api provides.

it would save us the trouble of a separate rest sensor and finetuning those in updating ;-)

anyways, let me thank you again for your integration in the first place, and your efforts in responding, much appreciated!

from ha-knmi.

A template sensor is much more efficient than making a rest sensor.

Some built-in cards can show an attribute on a dashboard, for example Tile (state_content) and Entity (attribute) and, Entities Card (attribute). Than it isn't required to make a template sensor.

For forecast data, you need to make template sensors to extract values through the service that was introduced in 2023.9, see my example: https://github.com/golles/Home-Assistant-Config/blob/main/packages/weer/forecast.yaml

from ha-knmi.

For forecast data, you need to make template sensors to extract values through the service that was introduced in 2023.9, see my example: https://github.com/golles/Home-Assistant-Config/blob/main/packages/weer/forecast.yaml

yes thanks, I do that for some others too, but I am still not sure I fully understand this. Your weather entity still holds those forecast attributes (which I believe the whole idea behind the move was to get rid of in the state machine). Probably for some limited time just ,lime HA's 6 months?

yet, it is polling the resource as it stands.
How would adding that get.weather_forecasts service add to that? because if we'd trigger that automation, it would exceed to the count of the api limits, as that is already consumed by the integration itself?

from ha-knmi.

Ive updated to your configurable scan_interval and all seems well. thanks for that.

suppose there is no way of telling the direct number of calls the integration has done so far? might be a useful sensor. if not directly available from the api as a ratelimit maybe the integrations itself could count. Just so we always know.

from ha-knmi.

I discussed this with the API supplier a while ago, if they are going to decide to implement that I'll add it for sure.
Counting it manually is pointless, as the integration doesn't allow to make more than the allowed calls (scan interval cannot be set lower than 300). If people (like you) decide to make manual calls to the API this integration is unaware of that, until it runs out of calls.

from ha-knmi.

understood. btw I have a separate api for the rest sensor (which I have disabled after all), so I am only using your nice integration.
I was merely thinking of the people that still see limits exceeded, and stating their integration is not using that amount yet. Or so they think. For that scenario it would be nice if the integration registered/counted its own updates.

Not a big deal though. Ive set it to 500, and suppose I never see that error again.
thanks!

from ha-knmi.

If you want to use a customized API call frequency, you can use my custom integration, which includes a setting in the config flow.
It also got sensors for all data and many more.

https://github.com/HiDiHo01/ha-knmi/tree/main/custom_components/knmi

from ha-knmi.

Seems to be a direct copy of this repo...

from ha-knmi.

Seems to be a direct copy of this repo...

It is a copy with many adjustments
It gets initial coordinates from HA, which you can edit
API refresh interval setting (sets both HA and API update interval)
32 entitys
sensors for todays high an low temperatures
sensor barometer
sensor place
sensors with descriptions for wind, pressure, temperatures, etc
wind sensor icon with a arrow pointing to wind direction
it only updates if time sinds last update is 10 minutes or more
Timestamp attribute for every sensor
and other fixes/adjustments

from ha-knmi.