home-assistant / plugin-observer Goto Github PK

View Code? Open in Web Editor NEW

21.0 6.0 16.0 287 KB

Supervisor plugin observare to monitor host part of Supervisor

License: Apache License 2.0

Dockerfile 7.96% Go 54.87% HTML 14.63% CSS 10.04% JavaScript 12.50%

plugin-observer's Introduction

plugin-observer

Supervisor plugin observer to monitor host part of Supervisor on Port 4357

API endpoints

GET /logs
GET /ping

plugin-observer's People

Contributors

Stargazers

Watchers

Forkers

beardedtinker ludeeus uncleweb b46ha tkdrob liuleidp pnjongang mynotic irakhlin xpcom-bsd caraar12345 tomkusz tomkusze l3aalteshuva lgc-shs chrisduerr

plugin-observer's Issues

Pleas update readme with instructions how to install the observer

I was not able to find out how to install the observer add-on. It is not on the add-on page in the supervisor tab of HA

Can't start hassio_observer: 500 Server Error

Hi, I use hassio on proxmox. Today I found it non responsive, restarted the VM and now I have this error showing up over and over again. What's happening? Thank you.

Supervisor
Version
2020.11.0
Operating System
HassOS 4.17
Home Assistant 0.118.4

20-12-02 14:08:40 INFO (MainThread) [supervisor.plugins.observer] Starting observer plugin
20-12-02 14:08:40 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning hassio_observer application
20-12-02 14:08:43 ERROR (SyncWorker_1) [supervisor.docker] Can't start hassio_observer: 500 Server Error: Internal Server Error ("driver failed programming external connectivity on endpoint hassio_observer (f16a6062331efa497a8a628c3ed567aa13dd7bdb55a561315e9dab8072186307): Bind for 0.0.0.0:4357 failed: port is already allocated")
20-12-02 14:08:43 ERROR (MainThread) [supervisor.plugins.observer] Can't start observer plugin
20-12-02 14:08:43 ERROR (MainThread) [supervisor.misc.tasks] Observer watchdog reanimation failed!
20-12-02 14:09:43 WARNING (MainThread) [supervisor.misc.tasks] Watchdog/Docker found a problem with observer plugin!

Note in case of connection problem

Hello, there seems to be an error in communication between observer and supervisor on my system. On the status page under port 4357 there is no indication, the supervisor is shown as "connected" and "Healthy" as "Unhealthy" although the logs are full with error 403.

I think it would be good to show on the status page if there is a problem, even if it somehow results in an "Unhealthy".

I only see error messages at the moment by looking at the log of this container with Portainer.

How to use observer (API?)

I wonder if it's possible to use the Observer (trough port 4357) for sending command to HA instance (in my case running on RPI) to force a restart or to download logs..
I see in the Readme page that it mentions 2 Api command but it is not clear how to use them .. and I noticed that a /restart command was available in the past but apparently it is now removed..

I have sometimes my RPI tha doesn't respond on 8123 port (main GUI), even with local IP, but the Observer is running and it reports connected and healthy.. so my idea was to use it to restart the Raspberry.. but I need instructions (I don't keep SSH enabled for security)

I think that this can be usefull to all users in the same situation

Port 80 already in use when using Let's Encrypt plugin

When trying to enable the let's encrypt plugin I get this one:
20-10-12 16:33:44 ERROR (SyncWorker_3) [supervisor.docker] Can't start addon_core_letsencrypt: 500 Server Error: Internal Server Error ("driver failed programming external connectivity on endpoint addon_core_letsencrypt (e5510de186f61f35cbb37e1f7201db0de7da4c061b1453d92aff5846038c4c7e): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use")

Which is logical since the Observer is listening on 80...

How to disable hassio_observer?

It's useless for me, but it takes up too much memory. My device has just experienced a run out of memory and I suspect it has something to do with plugin-observer.

  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
  177   173 root     S     695m  40%   1   0% observer
  201     0 root     S     2600   0%   1   0% bash
  206   201 root     R     1636   0%   3   0% top
    1     0 root     S      192   0%   3   0% s6-svscan -t0 /var/run/s6/services
   31     1 root     S      192   0%   0   0% s6-supervise s6-fdholderd
  173     1 root     S      192   0%   1   0% s6-supervise observer

# docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
homeassistant/qemuarm-64-homeassistant    0.118.1             c4953f3cb5e5        5 days ago          1.14GB
hassioaddons/node-red-aarch64             7.2.10              4fe2d0edbbd1        7 days ago          252MB
hassioaddons/adguard-aarch64              2.6.1               fee7e04d39e5        13 days ago         47.1MB
hassioaddons/nginxproxymanager-aarch64    0.8.2               a8c81417dee2        2 weeks ago         401MB
dwelch2101/zigbee2mqtt-aarch64            1.16.1              637938b4fb29        2 weeks ago         456MB
homeassistant/aarch64-hassio-supervisor   2020.11.0           d933ed23bed8        3 weeks ago         317MB
homeassistant/aarch64-hassio-dns          2020.11.0           af5ecb447947        3 weeks ago         130MB
homeassistant/aarch64-hassio-cli          2020.10.1           0f05fe76c14f        4 weeks ago         98.5MB
homeassistant/aarch64-hassio-observer     2020.10.1           2276cf8c8343        6 weeks ago         98.1MB
hassioaddons/phpmyadmin-aarch64           0.1.4               8f206347de6a        3 months ago        123MB
homeassistant/aarch64-hassio-multicast    3                   272cc794b62f        3 months ago        88.3MB
homeassistant/aarch64-hassio-audio        17                  7fa0e2cc52fe        3 months ago        110MB
portainer/portainer                       latest              1fdf590d1a36        4 months ago        68.5MB
homeassistant/aarch64-addon-mariadb       2.2.1               da3a7e5f170c        4 months ago        297MB
homeassistant/aarch64-addon-mosquitto     5.1                 56b6964bab3a        14 months ago       37.2MB

20-11-25 19:02:48 INFO (MainThread) [supervisor.api.proxy] Home Assistant WebSocket API connection is closed
20-11-25 19:04:05 WARNING (MainThread) [supervisor.misc.tasks] Watchdog/Application found a problem with observer plugin!
20-11-25 19:04:27 INFO (SyncWorker_6) [supervisor.docker.interface] Stopping hassio_observer application

20-11-25 19:07:54 INFO (SyncWorker_6) [supervisor.docker.interface] Cleaning hassio_observer application
20-11-25 19:09:07 INFO (MainThread) [supervisor.plugins.observer] Starting observer plugin
20-11-25 19:11:00 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
20-11-25 19:11:30 WARNING (MainThread) [supervisor.updater] Can't fetch versions from https://version.home-assistant.io/stable.json: 

20-11-25 19:14:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
20-11-25 19:14:40 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/hassio-addons repository

20-11-25 19:28:32 ERROR (SyncWorker_4) [supervisor.docker] Dockerd connection issue for hassio_observer: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=900)
20-11-25 19:29:51 ERROR (MainThread) [supervisor.plugins.observer] Can't start observer plugin
20-11-25 19:30:16 ERROR (MainThread) [supervisor.misc.tasks] Observer watchdog reanimation failed!
20-11-25 19:31:39 ERROR (MainThread) [asyncio] Task exception was never retrieved

future: <Task finished name='Task-583242' coro=<Scheduler._run_task.<locals>._wrap_task() done, defined at /usr/src/supervisor/supervisor/misc/scheduler.py:58> exception=DockerRequestError()>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/stdlib.py", line 102, in getresponse
    rv = real_getresponse(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 410, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=900)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/docker/interface.py", line 164, in _is_running
    docker_container = self.sys_docker.containers.get(self.name)
  File "/usr/local/lib/python3.8/site-packages/docker/models/containers.py", line 887, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/docker/api/container.py", line 771, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.8/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/docker/api/client.py", line 228, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=900)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/misc/scheduler.py", line 62, in _wrap_task
    await task.coro_callback()
  File "/usr/src/supervisor/supervisor/misc/tasks.py", line 339, in _watchdog_observer_docker
    await self.sys_plugins.observer.is_running()
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/supervisor/supervisor/docker/interface.py", line 170, in _is_running
    raise DockerRequestError() from err
supervisor.exceptions.DockerRequestError
20-11-25 19:36:52 WARNING (MainThread) [supervisor.misc.tasks] Watchdog/Application found a problem with observer plugin!

Running clean install supervised, with error installing Observer?

I got this error on portainer when installing the HA

21-06-15 12:07:24 INFO (SyncWorker_1) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/amd64-hassio-observer with tag 2020.10.1.

21-06-15 12:07:25 ERROR (SyncWorker_1) [supervisor.docker.interface] Can't install ghcr.io/home-assistant/amd64-hassio-observer:2020.10.1: 500 Server Error for http+docker://localhost/v1.41/images/create?tag=2020.10.1&fromImage=ghcr.io%2Fhome-assistant%2Famd64-hassio-observer: Internal Server Error ("name unknown")

Any help?

Watchdog/Application found a problem with observer plugin!

Hello,

I've noticed some weird behavior of my Home Assistant instance that decided to check the logs. It turned out that the Observer plugin fails many times per day and it seems it also affect other addons (like mosquitto) sometimes.

Home Assistant is running on Raspberry PI 3 with 32Gb SD card:

arch	armv7l
chassis	embedded
dev	false
docker	true
docker_version	19.03.12
hassio	true
host_os	HassOS 4.16
installation_type	Home Assistant OS
os_name	Linux
os_version	4.19.127-v7
python_version	3.8.5
supervisor	2020.12.6
timezone	Europe/Minsk
version	0.116.4
virtualenv	false

The issue in the logs always look like this:

20-12-10 16:38:03 WARNING (MainThread) [supervisor.misc.tasks] Watchdog/Application found a problem with observer plugin!
20-12-10 16:38:09 INFO (SyncWorker_5) [supervisor.docker.interface] Stopping hassio_observer application
20-12-10 16:38:38 INFO (SyncWorker_7) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-mosquitto with version 5.1
20-12-10 16:38:55 INFO (SyncWorker_5) [supervisor.docker.interface] Cleaning hassio_observer application
20-12-10 16:38:56 INFO (MainThread) [supervisor.plugins.observer] Starting observer plugin
20-12-10 16:38:59 INFO (MainThread) [supervisor.services.modules.mqtt] Set core_mosquitto as service provider for mqtt
20-12-10 16:39:32 INFO (MainThread) [supervisor.auth] Auth request from 'core_mosquitto' for 'ha'
20-12-10 16:39:35 INFO (MainThread) [supervisor.auth] Successful login for 'ha'
20-12-10 16:39:56 INFO (SyncWorker_7) [supervisor.docker.observer] Starting Observer homeassistant/armv7-hassio-observer with version 2020.10.1 - 172.30.32.6
20-12-10 16:42:34 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
20-12-10 16:42:34 INFO (MainThread) [supervisor.resolution.check] System checks complete

Could you help to fix this issue, please?

Invalid token for access

Hi,

For me the observer has a wrong token for the supervisor which leads to the following error messages:

observer:

2022/02/21 13:12:50 Supervisor API call failed with status code 403
2022/02/21 13:12:51 Supervisor API call failed with status code 403
2022/02/21 13:12:52 Supervisor API call failed with status code 403

supervisor:

22-02-21 13:12:50 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /supervisor/info
22-02-21 13:12:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /supervisor/info
22-02-21 13:12:52 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /supervisor/info

I don't know how to find out if the observer or the supervisor is to blame for this, so feel free to move if this belongs to the supervisor.

Kind regards
Sören

Can't attach hassio_observer to hassio-network!

Setup:

arch	armv7l
chassis
dev	false
docker	true
docker_version	19.03.13
hassio	true
host_os	Raspbian GNU/Linux 10 (buster)
installation_type	Home Assistant Supervised
os_name	Linux
os_version	5.4.51-v7+
python_version	3.8.6
supervisor	2020.11.0
timezone	Europe/Sofia
version	0.117.5
virtualenv	false

Error message:

20-11-04 11:52:06 WARNING (SyncWorker_6) [supervisor.docker] Can't attach hassio_observer to hassio-network!
20-11-04 11:52:08 ERROR (SyncWorker_6) [supervisor.docker] Can't start hassio_observer: 500 Server Error: Internal Server Error ("driver failed programming external connectivity on endpoint hassio_observer (3b4a113cea453c5aeb4fcfddf242fc94a45551c8e0a7a03de452228efcd04c6e): Bind for 0.0.0.0:4357 failed: port is already allocated")
20-11-04 11:52:08 ERROR (MainThread) [supervisor.plugins.observer] Can't start observer plugin
20-11-04 11:52:08 ERROR (MainThread) [supervisor.misc.tasks] Observer watchdog reanimation failed!
20-11-04 11:53:02 WARNING (MainThread) [supervisor.misc.tasks] Watchdog/Application found a problem with observer plugin!
20-11-04 11:53:02 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_observer application
20-11-04 11:53:03 INFO (MainThread) [supervisor.plugins.observer] Starting observer plugin
20-11-04 11:53:04 WARNING (SyncWorker_4) [supervisor.docker] Can't attach hassio_observer to hassio-network!
20-11-04 11:53:05 ERROR (SyncWorker_4) [supervisor.docker] Can't start hassio_observer: 500 Server Error: Internal Server Error ("driver failed programming external connectivity on endpoint hassio_observer (57c5dab940a170e8fed4a21f7de0acd7c80f70373dcbf20271c2490d7fa8a5d6): Bind for 0.0.0.0:4357 failed: port is already allocated")
20-11-04 11:53:05 ERROR (MainThread) [supervisor.plugins.observer] Can't start observer plugin
20-11-04 11:53:05 ERROR (MainThread) [supervisor.misc.tasks] Observer watchdog reanimation failed!

It happens at least twice a day and The ZigBee2MQTT plugin needs to be restarted so that the sensors can report to the MQTT broker.

From time to time the whole system becomes unresponsive and I have to power cycle the RPI.