anand-p-r / fmi-hass-custom Goto Github PK

Hi,
I noticed that the sea level sensor is not updating in Home Assistant. It updates values on restart / initialisation, but the values that the component update pulls from FMI every 30mins don't seem to make their way all the way to be displayed in Home Assistant, so the sensor remains at the initial values.

I got the updating working by editing sensor.py, so that if the 'update(self)' function for the sea level sensor replaces 'self.mareo_data' with 'self._fmi.mareo_data.sea_levels' then it works, e.g. self._state = self._fmi.mareo_data.sea_levels[0][1]

Home Assistant 2021.3.4 informs that the manifest is missing a version key, which is not supported in future versions.

Adding

"version": "0.1"

to the manifest file fixes things right up.

Hi @anand-p-r

Awesome project and thanks for publishing!

To make installation & updating easier for users it could make sense to structure the repo so that the integration can be installed with HACS (https://hacs.xyz/docs/faq/custom_repositories). What do you think?

If you agree I'm happy to have a look what would be needed to move around and submit a PR.

Cheers from Turku,
Aarno

https://www.home-assistant.io/blog/2023/09/06/release-20239/#weather-forecast-service

So the warning reported on my setup:

Logger: homeassistant.components.weather
Source: components/weather/__init__.py:339
Integration: Weather (documentation, issues)
First occurred: 08:54:47 (1 occurrences)
Last logged: 08:54:47

custom_components.fmi.weather::None is using a forecast attribute on an instance of WeatherEntity, this is deprecated and will be unsupported from Home Assistant 2024.3. Please report it to the custom integration author.

I think you noticed this issue yourself so I decided to create the ticket for tracking purposes only :)

Hi!

I get the following error when installing the fmi integration:

"Error occurred loading configuration flow for integration fmi: No module named 'geopy'"

Running latest HassWP

https://github.com/AlexxIT/HassWP v2021.1.1

Br Marenl01

I noticed that there are differences in weather data between this custom component and FMI data. How does this component define what weather station is being used when GPS coordinates has been used to define place for weather data? For example there are 3 different weather stations in FMI (Pori) and the integration uses which one of those? Here´s some of the data that are not consistent:

• Temperature in custom component in Pori: -0,3C
• Temperature in Pori Rautatieasema station: -1,8C
• Temperature in Pori lentoasema: -1,9C
• Temperature in Pori Tahkoluoto satama: -1,2C

I tried to search where this -0,3C value is coming from but was unable to find anything. Also the forecast temperatures are different.

UPDATE I just found out that FMI API data is different than in https://www.ilmatieteenlaitos.fi/. Is this known issue here?

API calls are done, but sensor/weather attributes are not updating. Reported on HASS Forums.

/config/logs says:

custom_components.fmi.weather::FMIWeatherEntity is overriding deprecated methods on an instance of WeatherEntity, this is not valid and will be unsupported from Home Assistant 2023.1. Please report it to the custom integration author

In Ha 0.115 UNIT_PERCENTAGE is changed to PERCENTAGE.

home-assistant/core#40207

Looks like a simple change in sensor.py helps. UNIT_PERCENTAGE -> PERCENTAGE.

Component is not showing latest observation data as a current weather condition. Seems to be using fmi forecast data as a current temperature/weather. Would it be possible to use fmi's observation data to show current conditions because observation data is more accurate for current weather conditions?

I noticed that on the latest HA the FMI integration refreshes the forecast many times a second.
I think this is caused by 'async_request_refresh' called from update methods.
This call also produces a lot of warnings about coroutine not been awaited.

I'll try my fix and create a PR when it is ready.

Hello and thank You for this custom component! I am writing to notify You that the python library fmi-weather-client, used by this custom component, is using old Hirlam weather model.

From FMI website (https://en.ilmatieteenlaitos.fi/hirlam-opendata-on-aws-s3):

The use of the HIRLAM weather forecasting model at the Finnish Meteorological Institute will be discontinued in September 2022, after which its forecasts are no longer available through the open data services.

During the past fifteen years, the main focus in forecast model development has been on a more accurate model with improved resolution. The resulting HARMONIE (MEPS) model is capable of 2.5 km horizontal resolution (compared to 7.5 km of HIRLAM) and able to predict small scale phenomena more accurately. HARMONIE (MEPS) forecasts are now available through the open data services.

More information will be made available later, but already now it is worth switching to use HARMONIE (MEPS) model if possible.

It seems that the library is using soon to be removed Hirlam model: https://github.com/saaste/fmi-weather-client/blob/master/fmi_weather_client/http.py#L100

I have already opened saaste/fmi-weather-client#39 and saaste/fmi-weather-client#40 .

Hi, wondering if I'm doing something wrong or if there is potentially a bug in the custom component. I'm running this component in RPi 3B in docker, HA version 2021.4.6 and version 0.4.0 of this custom component. After several reboots and now after having it been running for few days, this sensor is showing nothing but "Unavailable". All the other sensors seem to work correctly as they provide sensible values for my location. I have been eagerly waiting for it to start working as the lightning season is going to start this night in my place. I remember this sensor having been working few versions back, but haven't seen it being reporting any values since last few versions. Any idea what might be wrong? Thanks

Just what the title says :) For the precipitation value, I always receive 0 even for rainy days. All the other values seem to work fine

It seems that FMI have made some change in the open data query for OAAS sea water height data, and the existing code stopped providing sea water level data around the 10th January. I cannot find any explanation on FMI Open Data website to OAAS data provision, but by trial and error I noticed that a slight change in the query will make it work again. I have also queried with FMI Open Data if they can provide more information.

In the meantime, it seems that the WFS queries for "fmi::forecast::oaas::sealevel::point::simple" now result in a "handler not found" error in the xml response to the query. But if that is amended to fmi::forecast::sealevel::point::simple" then it works.

Therefore, at least for now, this issue can be addressed by modifying row 66 in const.py to the following, i.e. removing "OAAS" from the query format:
BASE_MAREO_FORC_URL = "http://opendata.fmi.fi/wfs?service=WFS&version=2.0.0&request=getFeature&storedquery_id=fmi::forecast::sealevel::point::simple&timestep=30&"

Current weather.py is writing the timestamp of each forecast with datetime object/function call (time = forecast.time.astimezone(tz.tzlocal())) in place of a str. This is not compatible with HASS weather template which uses dict to travel the forecast, and maybe not a good way to save the timestamp anyway.
One fix is to add a call to isoformat(): time.isoformat() on lines 199 and 217.

In developer-tools/template this can be visualized with {{ states.weather.fmi.attributes.forecast }}
Compatible forecast can be tested with {{ states.weather.fmi.attributes.forecast | to_json }}

Hi,

You have done excellent job with your component. I'm hoping that you would at some stage find time to integrate it to HACS for more streamlined handling of your component.

I do admit that it does most likely require some additional work from your side unfortunately.

https://hacs.xyz/docs/publish/start

Happy new year!

P.S. First time I'm creating issue in public github so hopefully this was good enough.

#60 is closed but none of the sensors have a state_class: measurement attribute. Statistics don't work without it.

Hi, there were today couple of warnings and one possible error (?) for fmi integration. Not sure what could be done.

2021-04-25 21:06:51 WARNING (MainThread) [homeassistant.loader] You are using a custom integration fmi which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2021-04-25 21:06:51 WARNING (MainThread) [homeassistant.loader] No 'version' key in the manifest file for custom integration 'fmi'. As of Home Assistant 2021.6, this integration will no longer be loaded. Please report this to the maintainer of 'fmi'
2021-04-25 21:07:03 WARNING (MainThread) [homeassistant.loader] You are using a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2021-04-25 21:22:45 ERROR (MainThread) [aiohttp.server] Error handling request

Thanks for this project. As a Finnish person this is very useful indeed.

Otherwise working great, however I encountered one slight issue with configuration panel not showing any details for the items, see attached screenshot. Defaults seem to be good for my use, so I didn't need to change these (for now at least).

Home Assistant 2021.10.5
Python 3.9.7
Hassio 5.10.17-v8

On HA I can see wind speed is just offered as a sensor value. Wind bearing would also be nice like doing automation if Wind is blowing SE and is > 5m/s trigger something.

Also it is not clear if wind speed is some kind of forecast or where it is coming. Please also use API to read actual FMI weather station values. For example "aaltopoiju.fi" service is using those actual sensor values.

Hi,

Component worked fine with 115.x until 115.5 I got error

Logger: homeassistant.setup
Source: custom_components/fmi/init.py:171
First occurred: 12:37:24 PM (1 occurrences)
Last logged: 12:37:24 PM

Error during setup of component fmi
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/setup.py", line 213, in _async_setup_component
result = await task
File "/config/custom_components/fmi/init.py", line 171, in async_setup
for index, entity in enumerate(config[DOMAIN], start=0):
KeyError: 'fmi'

Currently shows weather as sunny even at night time. Is this something that is fixable?

The statistics card in HA doesn't currently work for temperature, wind speed, and other additional sensors. I think the state_class: measurement attribute is required for statistics to work.

With the recent heatwave-related thunderstorms in Finland (in June 2021), the lightning query pulls through quite a big dataset. As a result, the fetching of lightning data hits the timeout limit, and ends up in a loop of retrying every 90 secs or so, and as a result all of the sensors fail to be set up e.g. on reboot.

Should the timeout limit for lightning data fetch be extended, or should the size of the box be reduced (again/further)? Or should the code be modified that even if lightning fetch times out, it does not stop the entire FMI set of sensors being set-up / updated?

As per title, currently any time when the weather is clear the displayed state is Sunny. This would be improved if the integration checked whether the sun is up at a given hour, and "Clear" would be displayed instead if it's night time.

Hi,
tried to install FMI component but I'm getting the following errors as seen at the bottom of this message. In this specific Rasperry Pi 3 I didn't have any earlier installations of FMI. I'm pretty sure I have followed exactly the instructions on how to install this custom component, but so far no success in fetching any measurements. Any idea what I'm doing wrong?

(I also tried to upgrade the earlier version of the FMI sensor in a completely different RPi3, and experienced similar problems despite having cleared the cache according to the instructions)

I don't have any issues in using an earlier version of the sensor, though.

Thanks
BR, Mika

-->

2021-04-29 21:09:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration fmi which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2021-04-29 21:12:34 ERROR (MainThread) [custom_components.fmi] Unexpected error fetching fmi data: list index out of range
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
self.data = await self._async_update_data()
File "/config/custom_components/fmi/init.py", line 365, in _async_update_data
await self._hass.async_add_executor_job(
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/fmi/init.py", line 335, in update_mareo_data
if root_mareo[n][0][2].text == 'SeaLevel':
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 216, in getitem
return self._children[index]
IndexError: list index out of range
2021-04-29 21:12:34 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Viitaila' for fmi integration not ready yet; Retrying in background
2021-04-29 21:12:43 ERROR (MainThread) [custom_components.fmi] Unexpected error fetching fmi data: list index out of range
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
self.data = await self._async_update_data()
File "/config/custom_components/fmi/init.py", line 365, in _async_update_data
await self._hass.async_add_executor_job(
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/fmi/init.py", line 335, in update_mareo_data
if root_mareo[n][0][2].text == 'SeaLevel':
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 216, in getitem
return self._children[index]
IndexError: list index out of range
2021-04-29 21:12:57 ERROR (MainThread) [custom_components.fmi] Unexpected error fetching fmi data: list index out of range
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
self.data = await self._async_update_data()
File "/config/custom_components/fmi/init.py", line 365, in _async_update_data
await self._hass.async_add_executor_job(
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/fmi/init.py", line 335, in update_mareo_data
if root_mareo[n][0][2].text == 'SeaLevel':
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 216, in getitem
return self._children[index]
IndexError: list index out of range
2021-04-29 21:13:20 ERROR (MainThread) [custom_components.fmi] Unexpected error fetching fmi data: list index out of range
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
self.data = await self._async_update_data()
File "/config/custom_components/fmi/init.py", line 365, in _async_update_data
await self._hass.async_add_executor_job(
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/fmi/init.py", line 335, in update_mareo_data
if root_mareo[n][0][2].text == 'SeaLevel':
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 216, in getitem
return self._children[index]
IndexError: list index out of range
2021-04-29 21:14:04 ERROR (MainThread) [custom_components.fmi] Unexpected error fetching fmi data: list index out of range
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
self.data = await self._async_update_data()
File "/config/custom_components/fmi/init.py", line 365, in _async_update_data
await self._hass.async_add_executor_job(
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/fmi/init.py", line 335, in update_mareo_data
if root_mareo[n][0][2].text == 'SeaLevel':
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 216, in getitem
return self._children[index]
IndexError: list index out of range
2021-04-29 21:15:27 ERROR (MainThread) [custom_components.fmi] Unexpected error fetching fmi data: list index out of range
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
self.data = await self._async_update_data()
File "/config/custom_components/fmi/init.py", line 365, in _async_update_data
await self._hass.async_add_executor_job(
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/fmi/init.py", line 335, in update_mareo_data
if root_mareo[n][0][2].text == 'SeaLevel':
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 216, in getitem
return self._children[index]
IndexError: list index out of range
2021-04-29 21:16:52 ERROR (MainThread) [custom_components.fmi] Error requesting fmi data: HTTPConnectionPool(host='opendata.fmi.fi', port=80): Max retries exceeded with url: /wfs?service=WFS&version=2.0.0&request=getFeature&storedquery_id=fmi%3A%3Aforecast%3A%3Ahirlam%3A%3Asurface%3A%3Apoint%3A%3Amultipointcoverage&timestep=10&starttime=2021-04-29T18%3A06%3A47%2B00%3A00&endtime=2021-04-29T18%3A16%3A47%2B00%3A00&latlon=61.14532384470369%2C25.347302332520492 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x6a7a7898>: Failed to establish a new connection: [Errno -3] Try again'))

Request from @sunam:

Do you see it feasible to add an attribute to the FMI sensor of the distance to closest lightning strike based on the location set - and when that occured? I understood that FMI has the the data but not sure if its there in the same db that you pull this forecast info from?

Would it be possible to add the possibility to have FMI mareograph observations (or forecast) included in the data the add-on fetches, i.e. sea water height and water temperature?

Manually those observations can be queried using the following e.g. for the Helsinki mareograph:
http://opendata.fmi.fi/wfs?service=WFS&version=2.0.0&request=getFeature&storedquery_id=fmi::observations::mareograph::simple&fmisid=132310&timestep=30

The daily forecast shows the icon from the corresponding day's midnight, not the one that represents the day. For example at the moment saturday is shown as cloudy in helsinki although the FMI forecast says saturday will be sunny.

After installing Home Assistant core-2021.5.4 the integration breaksi:
2021-05-15 17:10:40 ERROR (MainThread) [custom_components.fmi-hass-custom] Timeout fetching fmi data
I do not know if this is caused by the update or if this is a FMI problem. There are however several reports of the problem after update in https://community.home-assistant.io/t/custom-fmi-finnish-meteorological-institute-weather-sensor-platform/186839/23

2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_place (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_condition (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_temperature (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_wind_speed (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_wind_direction (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_wind_gust (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_humidity (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_cloud_coverage (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_rain (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_time (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_best_time_of_day (<class 'custom_components.fmi.sensor.FMIBestConditionSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_lightning_strikes (<class 'custom_components.fmi.sensor.FMILightningStrikesSensor'>) implements device_state_attributes. Please report it to the custom component author.
2021-12-26 21:45:57 WARNING (MainThread) [homeassistant.helpers.entity] Entity sensor.merttila_sea_level (<class 'custom_components.fmi.sensor.FMIMareoSensor'>) implements device_state_attributes. Please report it to the custom component author.

Sensors should be updated to instead implement extra_state_attributes as per Core PR:
home-assistant/core#47304

Is it possible somehow?

Configuration -> Logs reports this:

"Detected code that uses str for device registry entry_type. This is deprecated and will stop working in Home Assistant 2022.3, it should be updated to use DeviceEntryType instead. Please report this issue."

It looks like this type of definition is found in weather.py file.