Hi, recently some exceptions occur in ht_rpxy <cod

<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clip

ht_proxy exceptions lead to RuntimeError: can't start new thread about hometop_ht3 HOT 4 CLOSED

norberts1 commented on August 18, 2024

ht_proxy exceptions lead to RuntimeError: can't start new thread

from hometop_ht3.

Comments (4)

norberts1 commented on August 18, 2024

Hi timmmmey,
it looks like a memory-leak. Reason could be the many connection - requests of socket-clients.
Please look for that portnumber: 8088 in your system. It could be that it is already used for something else.
So change that port-numer (for server and client) in the configuration-File: ~/HT3/sw/etc/config/ht_proxy_cfg.xml to any other unused port-number and start the Pi again.

from hometop_ht3.

timmmmmey commented on August 18, 2024

Hi Norbert,

the port is only used by ht_proxy and and the only connection initiated by the fhem heatronic plugin. Changing the port didn't help (as long I change it in fhem as well).

Every ~10 minutes there is also a short time connection from ht_netclient (cronjob, because fhem plugin sending temperatures is not working well and commands from the ht_netclient sometimes got ignored from the heater, so I put this in a cron job)

Your memory-leak theorie seems to fit, my ht_proxy is occupying 1,6GB virtual memory right now. I know virtual memory is not the real used memory, so I will try to run it with valgrid/massif and post the results afterwards.

from hometop_ht3.

timmmmmey commented on August 18, 2024

--------------------------------------------------------------------------------
Command:            python3 /opt/ht3/ht_proxy.py
Massif arguments:   --massif-out-file=/opt/ht3/mem_prof
ms_print arguments: /opt/ht3/mem_prof
--------------------------------------------------------------------------------


    MB
1.158^                     #
     |                     #
     |                     #:::::::::::::@@@::::::::::::::::::@::::@:::::@::::
     |                 :  @#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |                 :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |              ::::::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |              :: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |              :: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |             ::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |            :::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |       : @:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |       ::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |       ::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |      :::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |      :::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |    :::::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |  @::::::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |  @::::::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     |  @::::::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
     | :@::::::@:::::: :::@#::: ::: : :  @  : :: ::: :::::::::@: ::@:::::@::::
   0 +----------------------------------------------------------------------->Mi
     0                                                                   368.9

Number of snapshots: 67
 Detailed snapshots: [2, 9, 19, 20 (peak), 29, 46, 50, 60]

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  0              0                0                0             0            0
  1      7,910,915          118,320          115,806         2,514            0
  2     12,608,269          283,384          279,722         3,662            0
98.71% (279,722B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->53.11% (150,496B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->13.65% (38,686B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->12.52% (35,472B) 0xDBCFA: _PyBytes_Resize (in /usr/bin/python3.4)
|
->05.74% (16,270B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->05.10% (14,448B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->04.26% (12,062B) in 47 places, all below massif's threshold (01.00%)
|
->02.89% (8,192B) 0xC76E2: _PyFaulthandler_Init (in /usr/bin/python3.4)
|
->01.45% (4,096B) 0xD350A: ??? (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  3     17,035,122          262,976          258,584         4,392            0
  4     24,699,125          339,792          333,949         5,843            0
  5     29,400,944          361,952          355,150         6,802            0
  6     34,929,209          469,112          461,906         7,206            0
  7     41,417,527          616,424          607,192         9,232            0
  8     46,100,465          604,240          594,325         9,915            0
  9     51,661,824          642,496          631,829        10,667            0
98.34% (631,829B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->54.07% (347,392B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.42% (73,392B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->11.37% (73,078B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->03.52% (22,624B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.97% (19,088B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.63% (16,896B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->02.55% (16,396B) in 64 places, all below massif's threshold (01.00%)
|
->02.39% (15,387B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->01.91% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.48% (9,512B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.46% (9,376B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.28% (8,208B) 0xE30D2: PyArena_New (in /usr/bin/python3.4)
|
->01.28% (8,192B) 0xC76E2: _PyFaulthandler_Init (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 10     55,333,195          657,464          646,550        10,914            0
 11     60,258,708          665,904          654,460        11,444            0
 12     66,568,795          714,608          702,036        12,572            0
 13     71,670,640          760,240          746,923        13,317            0
 14     78,582,509          928,584          914,396        14,188            0
 15     84,587,520          964,208          949,291        14,917            0
 16     92,100,037        1,049,624        1,033,666        15,958            0
 17     97,332,672          992,296          975,640        16,656            0
 18    104,197,582        1,027,240        1,009,175        18,065            0
 19    110,490,727        1,040,216        1,021,899        18,317            0
98.24% (1,021,899B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->57.22% (595,168B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->12.44% (129,446B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->08.29% (86,198B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->04.14% (43,024B) in 92 places, all below massif's threshold (01.00%)
|
->03.62% (37,694B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.18% (33,040B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.14% (22,304B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.09% (21,760B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.45% (15,120B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.33% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.18% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.16% (12,032B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 20    115,912,173        1,214,728        1,194,864        19,864            0
98.36% (1,194,864B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->49.88% (605,856B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->12.54% (152,267B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->10.66% (129,446B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->05.57% (67,601B) 0xAEF42: ??? (in /usr/bin/python3.4)
|
->05.28% (64,116B) in 96 places, all below massif's threshold (01.00%)
|
->02.97% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->02.85% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.79% (33,936B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.36% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.33% (16,208B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.14% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.01% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 21    121,511,018        1,094,256        1,074,388        19,868            0
 22    127,653,812        1,094,256        1,074,388        19,868            0
 23    133,792,713        1,094,256        1,074,388        19,868            0
 24    139,928,584        1,094,256        1,074,388        19,868            0
 25    146,075,437        1,094,256        1,074,388        19,868            0
 26    150,984,717        1,094,256        1,074,388        19,868            0
 27    162,616,889        1,098,368        1,078,212        20,156            0
 28    176,263,811        1,098,344        1,078,197        20,147            0
 29    192,230,493        1,094,576        1,074,444        20,132            0
98.16% (1,074,444B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->55.66% (609,232B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->12.04% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->08.03% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->05.18% (56,748B) in 101 places, all below massif's threshold (01.00%)
|
->03.29% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.18% (34,832B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->03.16% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.62% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.52% (16,688B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.26% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.12% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.08% (11,840B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 30    209,428,136        1,094,568        1,074,444        20,124            0
 31    216,061,069        1,107,736        1,087,072        20,664            0
 32    221,020,184        1,107,392        1,086,724        20,668            0
 33    225,220,705        1,107,408        1,086,736        20,672            0
 34    233,501,124        1,107,328        1,086,660        20,668            0
 35    239,678,908        1,107,312        1,086,656        20,656            0
 36    245,818,744        1,107,312        1,086,656        20,656            0
 37    254,080,310        1,107,376        1,086,720        20,656            0
 38    262,279,318        1,107,424        1,086,740        20,684            0
 39    268,565,366        1,107,280        1,086,640        20,640            0
 40    272,750,390        1,107,280        1,086,640        20,640            0
 41    276,853,362        1,107,352        1,086,676        20,676            0
 42    283,014,570        1,107,328        1,086,660        20,668            0
 43    287,158,030        1,107,296        1,086,644        20,652            0
 44    295,381,273        1,107,576        1,086,904        20,672            0
 45    299,485,221        1,107,560        1,086,900        20,660            0
 46    305,649,852        1,107,544        1,086,896        20,648            0
98.14% (1,086,896B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->55.06% (609,840B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.90% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->07.93% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->05.79% (64,096B) in 102 places, all below massif's threshold (01.00%)
|
->03.25% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.59% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.57% (17,440B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.48% (16,384B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.25% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.06% (11,712B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 47    309,826,025        1,107,584        1,086,916        20,668            0
 48    318,071,099        1,107,608        1,086,932        20,676            0
 49    324,240,506        1,107,600        1,086,920        20,680            0
 50    327,920,023        1,107,544        1,086,896        20,648            0
98.14% (1,086,896B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->55.06% (609,840B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.90% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->07.93% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->05.79% (64,096B) in 102 places, all below massif's threshold (01.00%)
|
->03.25% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.59% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.57% (17,440B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.48% (16,384B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.25% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.06% (11,712B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 51    331,614,300        1,120,536        1,099,893        20,643            0
 52    335,296,269        1,115,928        1,095,480        20,448            0
 53    338,984,448        1,115,928        1,095,480        20,448            0
 54    342,659,845        1,115,888        1,095,460        20,428            0
 55    346,348,334        1,115,888        1,095,460        20,428            0
 56    350,046,237        1,115,928        1,095,468        20,460            0
 57    353,759,198        1,115,928        1,095,480        20,448            0
 58    357,437,992        1,115,912        1,095,476        20,436            0
 59    361,126,476        1,115,928        1,095,480        20,448            0
 60    364,801,506        1,115,912        1,095,476        20,436            0
98.17% (1,095,476B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->54.65% (609,840B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.81% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->07.87% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->06.59% (73,524B) in 102 places, all below massif's threshold (01.00%)
|
->03.23% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.10% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->03.09% (34,496B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.57% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.50% (16,704B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.47% (16,384B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.24% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.05% (11,712B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 61    368,491,931        1,115,928        1,095,480        20,448            0
 62    372,163,591        1,115,904        1,095,464        20,440            0
 63    375,849,820        1,115,928        1,095,468        20,460            0
 64    379,525,175        1,115,904        1,095,464        20,440            0
 65    383,196,627        1,115,936        1,095,480        20,456            0
 66    386,871,544        1,116,632        1,096,168        20,464            0

Scheint ein memory-leak zu sein. Jedoch nutzt definitiv nichts anderes als das FHEM-Plugin und der ht_netclient (wobei der häufig gestartet wird ~ alle 10 Minuten). Irgendwie scheint die Verbindung nicht richtig zu terminieren und daher die Ressourcen nicht freigegeben oder so. Kann ich noch irgendwas prüfen?

from hometop_ht3.

norberts1 commented on August 18, 2024

Hi timmmmmey,
it seems to be a bit strange for me. This lack of memory behaviour running the 'ht_proxy.server' I haven't seen on my maschine. On my RPI B it runs years so far. On other maschines too.
To fix that problem on your maschine please do following:
1.Deactivate the HT3-applikation under your folder /opt. It runs there as user 'root' and that environment could be a bit different.
2. Deaktivate the fhem-applikation or stop it by hand.
3. Install the HT3-Applikation again as user: pi (git clone....)
4. Move the folder ~/hometop_HT3/HT3/ to ~/HT3.
5. Install the startscripts using sudo as defined in documentation (only this as user: root)
6. Modify the config-file(s ) 'ht_proxy_cfg.xml' for your used serial ht_adapter (baudrate, portname etc must be modified. This depends on your system and the adapter.
7. create the databases (only if you are using them).
8. restart the RPi again and stop that running fhem.
Let's run a time wating for any wrong behaviour.
After all you can check the receiving of Heaterbus-data with:
http://IP-adr_of_your_RPI:8088
The data are depending on the ht_adapter and heater-system, see picture.

Let me ask:
Which type of RPI you are using?
Which type of ht_adapter you are using?

from hometop_ht3.

ht_proxy exceptions lead to RuntimeError: can't start new thread about hometop_ht3 HOT 4 CLOSED

Comments (4)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent