Giter VIP home page Giter VIP logo

roe-dl / weewx-gts Goto Github PK

View Code? Open in Web Editor NEW
12.0 5.0 1.0 263 KB

XType extension for WeeWX to provide solar energy, "Grünlandtemperatursumme" (a kind of growing degree days) and "dayET" and "ET24" as some kind of opposite to "dayRain" and "rain24"

License: GNU General Public License v3.0

Python 100.00%
weewx xtypes gts growing-degree-days gardening vegetation climate-data sun solar-energy

weewx-gts's Introduction

Deutsche Version dieses Textes hier

weewx-GTS

XType extension for WeeWX to provide

  • "Grünlandtemperatursumme" (a kind of growing degree days)
  • observation types 'dayET' and 'ET24' as the opposite to 'dayRain' and 'rain24'
  • observation types for equivalent (potential) temperature, mixing ratio, absolute humidity, and (saturation) vapor power
  • additional aggregation type for 'radiation' to calculate the total energy received during the aggregation interval
  • tags for day boundaries other than midnight local timezone time
  • observation types 'yearGDD' and 'seasonGDD'
  • aggregation type 'GDD'

Creating issues

If you want to report bugs or need help, please, refer to Help! Posting to weewx user for instructions. It applies here, too.

Please note, that GTS and yearGDD require temperature data from January 1st on to be calculated. If your database does not cover the whole timespan, those values cannot be calculated. You will have to wait for the next year.

If skin creation lasts too long for you, please, read the WeeWX wiki article XTypes performance first before creating an issue here. We can try to optimize things, but we cannot do magic.

Installation instructions:

  1. download

    wget -O weewx-GTS.zip https://github.com/roe-dl/weewx-GTS/archive/master.zip
  2. run the installer

    WeeWX up to version 4.X

    sudo wee_extension --install weewx-GTS.zip

    WeeWX from version 5.0 on and WeeWX packet installation

    sudo weectl extension install weewx-GTS.zip

    WeeWX from version 5.0 on and WeeWX pip installation into an virtual environment

    source ~/weewx-venv/bin/activate
    weectl extension install weewx-GTS.zip
  3. check configuration in weewx.conf

    [StdWXCalculate]
        [[Calculations]]
            ...
            GTS = software,archive
            GTSdate = software, archive
            utcoffsetLMT = software, archive
            dayET = prefer_hardware, archive
            ET24 = prefer_hardware, archive
            yearGDD = software, archive
            seasonGDD = software, archive
            outVaporP = software,loop
            outSVP    = software,loop
            outMixingRatio = software,loop
            outEquiTemp = software,loop
            outThetaE = software,loop
            outHumAbs = software,loop
    ...
    [Engine]
        [[Services]]
            ...
            xtype_services = ... ,user.GTS.GTSService
    
  4. restart weewx

    for SysVinit systems:

    sudo /etc/init.d/weewx stop
    sudo /etc/init.d/weewx start

    for systemd systems:

    sudo systemctl stop weewx
    sudo systemctl start weewx

Including in skins:

You can use the values provided by this extensions in all skins of WeeWX. You can show the values, and you can create a diagram. The following observation types are provided:

Grünlandtemperatursumme

"Grünlandtemperatursumme" is a kind of growing degree days that is used to estimate the start of growing of the plants. For the algorithm see below.

Display values (CheetahGenerator)

  • GTS: the value of "Grünlandtemperatursumme" itself (example tag: $current.GTS)
  • GTSdate: the date when the GTS value exceeds 200, which is considered the beginning of real spring (example tag: $day.GTSdate.last.format("%Y-%m-%d"))
  • utcoffsetLMT: offfset of the local mean time (Ortszeit) at the station's location
  • LMTtime: a string showing the local mean time (Ortszeit) at the station's location (can only be used with ".raw", example tag: $current.LMTtime.raw)

The values can be used together with every time period defined in the customization guide of WeeWX. There can be used aggregations as well. The following aggregations are defined: avg, min, max, last, has_data, not_null. Not all time spans are possible.

See http://weewx.com/docs/customizing.htm#Tags for details on how to use tags in skins.

Caution: The aggregation exists only reports whether the observation type is included in the database or not. For pure XTypes like the observation types provided here it reports False. The behavior of has_data changed. It reports False for WeeWX up to version 4.10 and True for WeeWX from version 5.0 on.

Diagrams (ImageGenerator)

To create diagrams you need to include additional sections into the [ImageGenerator] section of skin.conf. What follows are examples. There are more possibilities than that.

Within [[month_images]]:

        [[[monthGTS]]]
            line_gap_fraction = 0.04
            yscale = 0,None,None
            aggregate_type = avg
            [[[[GTS]]]]
                aggregate_interval = 86400
                label = Grünlandtemperatursumme

Within [[year_images]]:

        [[[yearGTS]]]
            aggregate_type = avg
            [[[[GTS]]]]
                label = Grünlandtemperatursumme

These examples create image files named 'monthGTS.png' or 'yearGTS.png', respectively. To display them within the web page appropriate <img> tags need to be included for example in index.html.tmpl:

<img src="monthGTS.png" />
<img src="yearGTS.png" />

Diagrams (WDC skin)

see Using weewx-GTS in WDC wiki

Diagrams (Belchertown skin)

configuration in graphs.conf:

    [[Vegetation]]
        #title = "Gr&uuml;nlandtemperatursumme"
        title = "Vegetation"
        aggregate_type = avg
        yAxis_label = "Wachstumsgradtage"
        yAxis_softMin = 100
        yAxis_softMax = 200
        exporting = 1
        [[[GTS]]]
            observation_type = GTS
            name = "Gr&uuml;nlandtemperatursumme"
        [[[yearGDD]]]
            observation_type = yearGDD
            name = "Wachstumsgradtage"
        [[[seasonGDD]]]
            observation_type = seasonGDD
            name = "Wachstumsgradtage Saison"

Growing degree days

Display values (CheetahGenerator)

  • yearGDD: the sum or integral of growing degrees from the beginning of the actual year up to the current time
  • seasonGDD: the sum or integral of growing degrees from GTSdate up to the current time. Before GTSdate the value is undefined. After October 31st it is undefined, too.
  • aggregation type GDD (or growdeg): Used to calculate growing degree days for observation types other than outTemp. This can be every temperature value, say greenhouse temperature.

Diagrams (ImageGenerator)

Within [[year_images]]:

        [[[yearGDD]]]
            aggregate_type = avg
            [[[[yearGDD]]]]
                label = Growing degree days
            [[[[seasonGDD]]]]
                label = Season growing degree days

This example creates an image files named 'yearGDD.png'. To display it within the web page appropriate <img> tags need to be included for example in index.html.tmpl:

<img src="yearGDD.png" />

You can combine that with GTS:

        [[[yearGTS]]]
            aggregate_type = avg
            [[[[GTS]]]]
                label = Grünlandtemperatursumme
            [[[[yearGDD]]]]
                label = Growing degree days
            [[[[seasonGDD]]]]
                label = Season growing degree days

To display it within the web page include the appropriate <img> tag for example into index.html.tmpl:

<img src="yearGTS.png" />

Diagrams (WDC skin)

see Using weewx-GTS in WDC wiki

Diagrams (Belchertown skin)

see above

Evapotranspiration

Display values (CheetahGenerator)

  • dayET: the sum of ET from the beginning of the archive day on, like dayRain does for rain
  • ET24: the sum of ET over the last 24 hours, like rain24 does for rain

Diagrams (ImageGenerator)

dayET and ET24 are not used in plots.

Special derived meteorological readings

WeeWX already includes the calculation of some special derived meteorological readings, but they are used internally, only. This extension makes them available for use in section [StdWXCalculate] and in tags.

Please note: WeeWX comes with an example extension, that calculates a value called "vapor pressure", but it is in fact saturation vapor pressure. The example uses another algorithm than WeeWX internally.

Caution: The calculation of these values is alpha code. It needs to be evaluated.

Display values (CheetahGenerator)

  • outSVP: saturation vapor pressure
  • outVaporP: actual vapor pressure
  • outMixingRatio: mixing ratio
  • outHumAbs: absolute humidity
  • outEquiTemp: equivalent temperature
  • outThetaE: equivalent potential temperature
  • boilingTemp: boiling temperature depending on station altitude and actual air pressure

Diagrams (ImageGenerator)

To show diagrams of these readings there is no need to store them in database. They will be calculated "on the fly" by the XTYPE system of WeeWX.

Example: absolute humidity

        [[[dayhumabs]]]
            unit = gram_per_meter_cubed
            [[[[outHumAbs]]]]

Diagrams (Belchertown skin)

Example: Display both the relative and absolute humidity in one diagram

    [[humidity]]
        title = "Humidity"
        [[[outHumidity]]]
            name = "relative
        [[[outHumAbs]]]
            name = "absolute"
            yAxis = 1
            unit = gram_per_meter_cubed
            [[[[numberFormat]]]]
                decimals = 1

Radiation energy

'radiation' and 'maxSolarRad' are built-in observation types of WeeWX. This extension only provides an additional aggregation type to them. It is called 'energy_integral' and calculates the total energy received during the aggregation interval.

Note: An Integral is not a sum of observation readings. See below for algorithm.

Display values (CheetahGenerator)

You need to use this aggregation type together with aggregation timespans like $day, $yesterday, $week, $month, and $year as well as timespans defined by some other extension to WeeWX.

energy_integral can be used like any other aggregation type like min, max, or sum.

Example: $yesterday.radiation.energy_integral displays the total sun energy received the day before.

To display the value in kWh/m^2 instead of Wh/m^2 use: $yesterday.radiation.energy_integral.kilowatt_hour_per_meter_squared

Diagrams (ImageGenerator)

Within [[month_images]]:

        [[[monthRadiationEnergy]]]
            line_gap_fraction = 0.04
            #y_label = "Wh/m²"
            [[[[radiation]]]]
                label = "Sonnenenergie (täglich gesamt)"
                data_type = radiation
                aggregate_type = energy_integral
                aggregate_interval = 86400

This example creates an image file called 'monthRadiationEnergy.png' To display it within the web page an appropriate <img> tag needs to be included for example in index.html.tmpl:

<img src="monthRadiationEnergy.png" />

Diagrams (Belchertown skin)

Belchertown skin uses another plot engine (Highcharts). Therefore the syntax is slightly different.

In section [month] or [year] of graphs.conf:

    [[Sonnenenergie]]
        title = "Sonnenenergie (täglich gesamt)"
        aggregate_type = energy_integral
        aggregate_interval = 86400
        yAxis_label = Energie
        yAxis_label_unit = "Wh/m&sup2;"
        [[[radiation]]]

No <img> tag is needed.

NOAA-like Table

There is an example template to create a text file showing monthly summeries of sun energy and sun radiation in the examples directory. To use it, copy the file to your skin directory and add the following to your skin.conf:

[CheetahGenerator]
    ...
    [[SummaryByYear]]
        ...
        [[[sun_year]]]
            encoding = strict_ascii
            template = sun-%Y.txt.tmpl

This creates a file for every year that data is available for.

Soil moisture

For soil moisture WeeWX defines the unit group group_moisture using the unit centibar. This presumably refers to the Davis Instruments soil moisture/temperature station 6345, which strictly speaking does not measure moisture but suction pressure.

Sometimes this pressure is expressed in a logarithmic value called pF value. This extension provides the pF value as an additional unit to group_moisture, called pF_value. It makes other pressure units than centibar available to group_moisture, too.

Special time spans

Sometimes day span used in meteorology are not from midnight to midnight but start and end at some arbitrary time of day like 09:00 a.m. To aggregate values with day boundaries other than midnight the following tags are provided. They are used like standard $hour, $day etc.

Arbitrary offset to UTC

  • $offsethour(data_binding=None, hours_ago=0, dayboundary=None)
  • $offsetday(data_binding=None, days_ago=0, dayboundary=None)
  • $offsetyesterday(data_binding=None, dayboundary=None)
  • $offsetmonth(data_binding=None, months_ago=0, dayboundary=None)
  • $offsetyear(data_binding=None, years_ago=0, dayboundary=None)

dayboundary is an offset to UTC in seconds, that sets the time of day to use as the day boundary for the given aggregation. In case of Python <3.7 the value is rounded to whole minutes.

Local Mean Time of the station location

  • $LMThour(data_binding=None, hours_ago=0)
  • $LMTday(data_binding=None, days_ago=0)
  • $LMTyesterday(data_binding=None)
  • $LMTmonth(data_binding=None, months_ago=0)
  • $LMTyear(data_binding=None, years_ago=0, month_span=None)

Day boundary is midnight Local Mean Time which is always near the antitransit of the sun at the station's location.

The optional parameter month_span is a time span of several months within the given year. For example $LMTyear(month_span=(6,8)).outTemp.avg is the average temperature of the summer of the actual year. $LMTyear(years_ago=1,month_span=(12,2)).outTemp.max is the maximum temparature of the last winter season.

The attribute days can be used together with $LMTmonth and $LMTyear for iteration.

Daylight timespan

  • $daylight(timestamp=None, data_binding=None, days_ago=0, horizon=None, use_center=False)

    timespan from sunrise to sunset

    If timestamp is None (the default), it is the timespan from sunrise to sunset of the current day or the day days_ago days ago.

    Otherwise timestamp can be a class TimespanBinder, a timespan or a timestamp. $daylight is then the timespan between sunrise and sunset of the day the specified timespan or timestamp is in. This is useful in #for loops over days.

  • $LMTweek(data_binding=None, months_ago=0).daylights(horizon=None, use_center=False)

    series of timespans describing the timespan from sunrise to sunset for each individual day. For use in #for loops

  • $LMTmonth(data_binding=None, months_ago=0).daylights(horizon=None, use_center=False)

    series of timespans describing the timespan from sunrise to sunset for each individual day. For use in #for loops

  • $LMTyear(data_binding=None, months_ago=0).daylights(horizon=None, use_center=False)

    series of timespans describing the timespan from sunrise to sunset for each individual day. For use in #for loops

For the options horizon and use_center see WeeWX customization guide. If omitted, defaults are used.

Examples:

  • average temperature while the sun is up
    $daylight.outTemp.avg
    
  • table with day of month and the average temperature while the sun is up on that day
    #for $span in $LMTmonth.daylights
    <p>$span.dateTime.format("%d"): $span.outTemp.avg</p>
    #end for
    
  • rain during day and night of the days of the current week
    #from weewx.units import ValueTuple, ValueHelper
    <table>
    <tr>
    <th>Day</th>
    <th>Day rain</th>
    <th>Night rain</th>
    </tr>
    #for $dd in $week.days
    #set $light=$daylight(timestamp=$dd)
    #set $nightrain=$dd.rain.sum.raw-$light.rain.sum.raw
    #set $nightrain_vh=ValueHelper(ValueTuple($nightrain,$unit.unit_type.rain,'group_rain'),formatter=$station.formatter)
    <tr>
    <td>$dd.start.format("%Y-%m-%d")</td>
    <td>$light.rain.sum</td>
    <td>$nightrain_vh</td>
    </tr>
    #end for
    </table>
    
  • sunrise, sunset, and duration of daylight using the timestamp option
    <table>
    <tr>
      <th>sunrise</th>
      <th>sunset</th>
      <th>daylight</th>
    </tr>
    #for $dd in $week.days
    <tr>
      <td>$dd.format("%A")</td>
      <td>$daylight(timestamp=$dd).start</td>
      <td>$daylight(timestamp=$dd).end</td>
      <td>$daylight(timestamp=$dd).length</td>
    </tr>
    #end for
    </table>
    

Algorithm:

Grünlandtemperatursumme (GTS)

  • GTS is calculated from the daily average temperatures. If the daily average temperature is above 0°C (32°F) it is used to add to the sum, otherwise it is discarded.
  • In January the daily average temperatures are multiplied by 0.5.
  • In February the daily average temperatures are multiplied by 0.75.
  • From March on the daily average temperatures are used as is.
  • To get the GTS value of a day all the values as described above are added from January 1st to the day in question. So the GTS value increases in time.
  • If the GTS value exceeds 200 this event is considered the beginning of growing of the plants in spring.
  • The GTS value itself is calculated up to May 31st. The end value is considered a statement about the spring.

Radiation energy

Radiation energy is calculated as follows: All the radiation readings within the aggregation interval are multiplied by their respective archive interval. That is based on the assumption that the radiation was constant during that interval. The error resulting form that is considered small enough to tolerate.

After that all the products of radiation and time interval are summarized together.

While the unit label of the radiation reading is W/m^2, the unit label of the radiation energy is Wh/m^2.

Daylight timespan

The $daylight tag uses the built-in module of WeeWX that provides the $almanac tag.

While $almanac.sunrise and $almanac.sunset calculate sunrise and sunset using the actual temperature and barometer of the calculation time, $daylight observes temperature and barometer of the time, the daylight timespan is calculated for if there are database records available. It first calculates approximate sunrise and sunset times for the ICAO standard atmosphere at 15°C and 1013.25 mbar. Then it looks up the real temperature and barometer for both those times. After that it calculates sunrise and sunset again, using the respective temperature and barometer.

That's why $daylight(timestamp=$X).start provides a more accurate time for sunrise than $almanac(timestamp=$X).sunrise if $X is a timestamp somewhere in the past and database records are available for that time. The same applies to $daylight(timestamp=$X).end instead of $almanac(almanac_time=$X).sunset for sunset and $daylight(timestamp=$X).length instead of $almanac(almanac_time=$X).sun.visible for the daylight duration.

Barometer

WeeWX includes several algorithms to calculate the barometer value out of the station pressure. Unfortunately those algorithms are not available to the user by default. And by 2023 there is no way to pass parameters to an XType. The barometer extension does a little bit of that and allows to add an algorithm name to the observation type barometer. This allows the following observation types:

  • barometerManBar
  • barometerDavisVp
  • barometerUnivie
  • barometerDWD

This is a temporary workaround only. It will be removed when WeeWX fully implements parameters to observation types.

Sources:

Links:

weewx-gts's People

Contributors

roe-dl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

hoetzgit

weewx-gts's Issues

Error "Unable to convert from unix_epoch to last" and no data calculated by GTS

I installed the GTS extension on my weewx 5.0.2. I see these infos in the log of weewx:

2024-03-03T13:35:20.319834+01:00 e45 weewxd[14737]: INFO user.GTS: Version 1.1
2024-03-03T13:35:20.319902+01:00 e45 weewxd[14737]: INFO user.GTS: Local mean time (LMT) UTC offset 0:34:47.880000
2024-03-03T13:35:20.319940+01:00 e45 weewxd[14737]: INFO user.GTS: PressureCooker True
2024-03-03T13:35:20.319976+01:00 e45 weewxd[14737]: INFO user.GTS: PressureCooker <user.barometer.PressureCooker object at 0x7fff8ab14310>
2024-03-03T13:35:20.321687+01:00 e45 weewxd[14737]: INFO main: Starting up weewx version 5.0.2
2024-03-03T13:37:11.558049+01:00 e45 weewxd[14737]: INFO user.belchertown: version 1.3.1
2024-03-03T13:37:12.707813+01:00 e45 weewxd[14737]: DEBUG weewx.units: Unable to convert from unix_epoch to last
2024-03-03T13:40:14.038725+01:00 e45 weewxd[14737]: DEBUG weewx.units: Unable to convert from unix_epoch to last
2024-03-03T13:43:15.941161+01:00 e45 weewxd[14737]: DEBUG weewx.units: Unable to convert from unix_epoch to last
2024-03-03T13:46:07.677514+01:00 e45 weewxd[14737]: DEBUG weewx.units: Unable to convert from unix_epoch to last

The error "Unable to convert from unix_epoch to last" is new after installing GTS and i don't see calculated values (e.g. GTS) in my skin with "$current.GTS". I can't see new data fields in my database either.

Is there something missing ?
I followed the install steps in the readme.

create observations and backfill?

This might be a slightly stupid situation, but perhaps there is a way to get this running?

I have a long-running weewx installation which I moved to v5 about a month ago. The upgrade was slightly bumpy and for some reason my user dir is still in /usr/share/weewx/user (not sure why and how to change it, but thats another issue...)

I now installed this extension (and manually copied the *.py files to my user dir). Upon restart, observations were not created in the database automatically and I have created them manually.

However, I don't see the new observations being populated in the database. Most of them are NULL, just dayET got set to 0. I don't see any error messages. I played a bit with restarting weewx and rebuild-daily, no success so far.

Update: After fiddling once more with weewx.conf and replacing the auto-created [[Calculations]] entries with the ones from this site, observations are now being populated.

Still,this question remains:

Is there a way to backfill the GTS observations for past years based on existing temperature/etc values?

Thanks! :-)

high CPU load of mariadb and I/O after update of GTS extension....

Hi,
as written in the weewx-user group I discovered high CPU and I/O load on my weewx system since October (mysql/mariadb). I upgraded the GTS extension at this time I could now reproduce.

I get this on mysql if I query for running processes:

MariaDB [(none)]> SHOW PROCESSLIST;
+-----+-------+-----------+-----------+---------+------+--------------+------------------------------------------------------------------------------------------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+-------+-----------+-----------+---------+------+--------------+------------------------------------------------------------------------------------------------------+----------+
| 121 | weewx | localhost | weewx_new | Query | 0 | Sending data | SELECT sum( CASE WHEN outTemp>1000.0 THEN 0.0 WHEN outTemp>86.0 THEN 36.0 WHEN `outTem | 0.000 |
| 122 | weewx | localhost | weewx_new | Sleep | 470 | | NULL | 0.000 |
| 123 | weewx | localhost | weewx_new | Sleep | 168 | | NULL | 0.000 |
| 124 | weewx | localhost | weewx_new | Sleep | 50 | | NULL | 0.000 |
| 219 | root | localhost | NULL | Query | 0 | starting | SHOW PROCESSLIST | 0.000 |
+-----+-------+-----------+-----------+---------+------+--------------+------------------------------------------------------------------------------------------------------+----------+

There is one Query sending data all the time:

| 121 | weewx | localhost | weewx_new | Query | 0 | Sending data | SELECT sum( CASE WHEN outTemp>1000.0 THEN 0.0 WHEN outTemp>86.0 THEN 36.0 WHEN `outTem | 0.000 |

A grep of this line tracks me to your GTS extension.
I disabled the xtype_service GTS and all is fine.
I think there is a problem....would be great if you could have a look in the code and if you could provide a fix as I'm very happy with this extension. :-)

Regards,

Christian

(GTS.py version is: VERSION = "1.0")

update to latest version....best way...?

Hi,

I noticed there are some fixes since 0.4.1.
What's the best way to update the nice addon you worte to 0.5.2!?
Should I re-run the installer or just put the most recent GTS.py into bin/usr...?
Won't break my weewx instance so I ask before I will try..

Thanks in andvance!

Regards,

Christian

weewx.UnknownType: outEquiTemp.max_ge: unknown aggregation type

Hallo,

weiß nicht, ob es an meiner Konfiguration liegt, aber mit aktueller GTS Version erscheint bei mir folgender Fehler (WeeWX 4.10.2):

Feb 27 16:00:33 weewx weewx[105352] ERROR weewx.reportengine:         ****  outEquiTemp.max_ge: outEquiTemp.max_ge: unknown aggregation type
Feb 27 16:00:33 weewx weewx[105352] ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Feb 27 16:00:33 weewx weewx[105352] ERROR weewx.reportengine:         ****    File "/home/weewx/bin/user/GTS.py", line 1006, in calc_derived
Feb 27 16:00:33 weewx weewx[105352] ERROR weewx.reportengine:         ****      elif aggregate_type=='avg':
Feb 27 16:00:33 weewx weewx[105352] ERROR weewx.reportengine:         ****  weewx.UnknownType: outEquiTemp.max_ge: unknown aggregation type

Grüße
Henry

Display values (CheetahGenerator)

Wenn ich die GTS Variablen in HTLM einbinde, dann bekomme ich folgende Fehler beim Erstellen der HTLM Seiten

HTLM Einbettung im WEEWX Mobile Template
image

image

Welchen "Aggregate interval " wird empfohlen?

Hallo,

gemäß der Readme habe ich alles soweit eingebaut, wobei wenn ich in der skin.conf nur:
[[year_images]] [[[yearGDD]]] aggregate_type = avg [[[[yearGDD]]]] label = Growing degree days [[[[seasonGDD]]]] label = Season growing degree days

benutze, erhalte zweimal die Fehlermeldung

ERROR weewx.imagegenerator: Aggregate interval required for aggregate type avg

welche Angabe von

aggregate_interval = ??

wird empfohlen? "86400" bzw. "day"? "month"? "year"?

Including the calculated Values in the Record

At first the values could be used in skins but they did not appear in the record. And they did not appear in the MQTT export. In order to include the values there, the observation type name are to be included in StdWXCalculate section of weewx. conf like this:

[StdWXCalculate]
    [[Calculations]]
        ...
        GTS = software
        GTSdate = software
        utcoffsetLMT = software

Unfortunately, there is no hint about that in the XType documentation

readme.md and install.py changed to reflect that, see commit 034c04e and commit fc147b7

Unrecoverable exception after install

Hello,

i get the following error when trying to restart weewx after installing your GTS extension:

May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: Caught unrecoverable exception:
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** offset must be a timedelta representing a whole number of minutes, not datetime.timedelta(0, 2445, 168000).
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** Traceback (most recent call last):
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** File "/usr/share/weewx/weewxd", line 151, in main
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** engine = weewx.engine.StdEngine(config_dict)
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** File "/usr/share/weewx/weewx/engine.py", line 93, in init
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** self.loadServices(config_dict)
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** File "/usr/share/weewx/weewx/engine.py", line 161, in loadServices
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** obj = weeutil.weeutil.get_object(svc)(self, config_dict)
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** File "/usr/share/weewx/user/GTS.py", line 765, in init
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** self.GTSextension=GTSType(__lat,__lon)
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** File "/usr/share/weewx/user/GTS.py", line 209, in init
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** self.lmt_tz=datetime.timezone(self.timeoffset,"LMT")
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** ValueError: offset must be a timedelta representing a whole number of minutes, not datetime.timedelta(0, 2445, 168000).
May 20 11:24:29 raspberrypi weewx[29526] CRITICAL main: **** Exiting.

Kompatibilitätsprobleme mit WeeWX 5.X

Bei meinem Test mit der Beta-Version von 5.0 hatte ich keine Probleme. Die Praxis ist dann eben immer noch etwas anderes.

Bekannt ist, daß WeeWX etwas an der Berechnung der X-Typen geändert hat. Die Frage ist, wie man herausfindet, was hier anders läuft als vorher. Als Workaround kann man die Größen in die Datenbank aufnehmen. Aber die endgültige Lösung ist das nicht. Vielleicht müßte man als erste die Implementation von get_series zwischen den beiden Versionen vergleichen.

Ein kurzer Vergleich von xtype.py zwischen den Versionen brachte eine Funktion has_data() zutage, die get_aggregate() mit not_null aufruft, so nach der Art $day.GTS.not_null. not_null ist in der GTS-Erweiterung nicht implementiert.

Originally posted by @roe-dl in #26 (comment)

get only time of the day where GTS was reached...?

Hi,

thanks for this great weewx-addon. I'm using it now and it's pretty sweet.
I have a problem:I added this in my weewx index file:

                      <tr>
                        <td>Grünlandtemperatur <200 erreicht am</td>
                        <td>$day.GTSdate.last</td>
                      </tr>

But I got only the time of the value of 200 if it was reached (in this case it was reached two days ago here)!?

`

Grünlandtemperatur <200 erreicht am 00:27

`
Do I something wrong? I would like to get the date too, like
3 Tage, 22 Stunden, 4 Minuten
, or
03.03.2021 14:28.

Any hint howt get something like this would be great!

Thanks in advance!

Christian

MySQL error in "calc_derived" function

hello,
I had MYSQL error in the function calc_derived each time I used an aggregate interval.
After some investigation, it appeared that at least on my system running Python 3.7.3 (Raspberry Pi) I needed to reformat the MySQL query to avoid any error :
for _result in db_manager.genSql( "SELECTdateTime,usUnits,outTemp,outHumidity,pressurefrom %s WHERE dateTime> %d AND dateTime<= %d ORDER BYdateTime" % (db_manager.table_name,timespan.start,timespan.stop)):

Unphysical values when some variables not measured

I recently had a power cut which lost 5 hours of all my meteorological readings. When the power came back, there was a gap in all my readings, as expected, except for outThetaE, which took a value of -5,555,555,573.3 C for the duration of the outage. Exactly 12 hours later, there was another spell of exactly this value for exactly the same length of time, even though there were no missing values during this interval. I have attached a png file to illustrate the effect.
Would it be possible, please, to include a quality check on the calculated value of outThetaE, and just leave a gap if the value falls outside some sort of sensible limits?

outThetaE issue

Reduce Decimal Places in Radiation KwH/msq

Hi,

Thanks for a great extension! I have some logic in my templates that changes the output of $timestamp.radiation.energy_integral from Wh/m² to kWh/m² if the value is >= 1000:

<div class="col-6">
            #if $month.radiation.energy_integral.raw >= 1000 
            <h4 class='h2-responsive '>$month.radiation.energy_integral.kilowatt_hour_per_meter_squared.round(ndigits=1)</h4>
            #else
            <h4 class='h2-responsive '>$month.radiation.energy_integral</h4>
            #end if
</div>

Although using the .round tag rounds the output to 1 decimal place it still displays the trailing zeros:

Screenshot 2023-01-31 at 11 53 49

Is there any way to display this value to a single decimal place?

Install.py - SyntaxError: Non-ASCII character '\xc3'

Wenn ich die Erweiterung installieren möchte kommt folgende Fehlermeldung:

sudo wee_extension --install weewx-GTS.zip
Request to install 'weewx-GTS.zip'
Extracting from zip archive weewx-GTS.zip
Traceback (most recent call last):
File "/usr/share/weewx/wee_extension", line 88, in
main()
File "/usr/share/weewx/wee_extension", line 80, in main
ext.install_extension(options.install)
File "/usr/share/weewx/weecfg/extension.py", line 125, in install_extension
self.install_from_dir(extension_dir)
File "/usr/share/weewx/weecfg/extension.py", line 147, in install_from_dir
extension_dir)
File "/usr/share/weewx/weecfg/init.py", line 1866, in get_extension_instal ler
import('install')
File "/var/tmp/weewx-GTS-master/install.py", line 15
SyntaxError: Non-ASCII character '\xc3' in file /var/tmp/weewx-GTS-master/instal l.py on line 15, but no encoding declared; see http://python.org/dev/peps/pep-02 63/ for details

UnknownAggregation error.... ?

Hi,

I updated GTS to 0.5.2 but get errors in my logfile:

Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: Caught unrecoverable exception in generator 'weewx.imagegenerator.ImageGenerator'
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** 1577834878
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** Traceback (most recent call last):
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/reportengine.py", line 196, in run
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** obj.start()
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/reportengine.py", line 281, in start
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** self.run()
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 41, in run
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** self.genImages(self.gen_ts)
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 181, in genImages
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** aggregate_interval=aggregate_interval)
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/xtypes.py", line 88, in get_series
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** aggregate_interval)
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/xtypes.py", line 143, in get_series
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** agg_vt = get_aggregate(obs_type, stamp, do_aggregate, db_manager)
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/xtypes.py", line 105, in get_aggregate
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** **option_dict)
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/user/GTS.py", line 656, in get_aggregate
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** raise weewx.UnknownAggregation("%s undefinded aggregation %s" % (obs_type,aggregation_type))
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/user/GTS.py", line 381, in get_gts
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** self.lock.release()
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** KeyError: 1577834878
Jun 12 06:50:32 weewx weewx[547] ERROR weewx.reportengine: **** Generator terminated

switching back to 0.4.1 I get:

Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: Caught unrecoverable exception in generator 'weewx.imagegenerator.ImageGenerator'
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** 1577834878
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** Traceback (most recent call last):
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/reportengine.py", line 196, in run
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** obj.start()
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/reportengine.py", line 281, in start
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** self.run()
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 41, in run
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** self.genImages(self.gen_ts)
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 181, in genImages
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** aggregate_interval=aggregate_interval)
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/xtypes.py", line 88, in get_series
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** aggregate_interval)
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/xtypes.py", line 143, in get_series
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** agg_vt = get_aggregate(obs_type, stamp, do_aggregate, db_manager)
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/weewx/xtypes.py", line 105, in get_aggregate
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** **option_dict)
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/user/GTS.py", line 656, in get_aggregate
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** __x=self.get_gts(obs_type,__b,_soye_ts)
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** File "/usr/share/weewx/user/GTS.py", line 381, in get_gts
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** __x=self.gts_values[soy_ts][dayOfGTSYear(sod_ts,soy_ts)]
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** KeyError: 1577834878
Jun 12 07:00:39 weewx weewx[547] ERROR weewx.reportengine: **** Generator terminated

Any idea?

Install problem

Hello, I have a problem installing your application:

sudo wee_extension --install weewx-GTS.zip
Request to install 'weewx-GTS.zip'
Extracting from zip archive weewx-GTS.zip
Traceback (most recent call last):
File "/usr/share/weewx/wee_extension", line 88, in
main()
File "/usr/share/weewx/wee_extension", line 80, in main
ext.install_extension(options.install)
File "/usr/share/weewx/weecfg/extension.py", line 125, in install_extension
self.install_from_dir(extension_dir)
File "/usr/share/weewx/weecfg/extension.py", line 147, in install_from_dir
extension_dir)
File "/usr/share/weewx/weecfg/init.py", line 1866, in get_extension_installer
import('install')
File "/var/tmp/weewx-GTS-master/install.py", line 21
'Calculations'{
^
SyntaxError: invalid syntax

Weewx .4.4.0 (/etc/weewx/... )

SQL statement results in error message with MySQL/MariaDB

            _result = db_manager.getSql(
                    "SELECT SUM(radiation*interval)/60.0, "
                    "MIN(usUnits), MAX(usUnits) FROM %s "
                    "WHERE dateTime >? AND dateTime<=?"
                    % db_manager.table_name, timespan)

results in

Mar 11 09:20:26 hesba weewx[16349] ERROR weewx.reportengine:         ****  weedb.DatabaseError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')/60.0, MIN(usUnits),MAX(usUnits) FROM archive WHERE dateTime>1612825200 AND ...' at line 1")

for MySQL/MariaDB.

Huge GDD value after year change

Syslog:

Jan  1 00:05:17 LokalWiki weewx[1142] INFO user.wns: GTS initialized 2022-01-01
Jan  1 00:05:17 LokalWiki weewx[1142] INFO user.wns: GTS None, 0 loops
Jan  1 00:05:17 LokalWiki weewx[1142] INFO user.GTS: other year 2022-06-01
Jan  1 00:05:17 LokalWiki weewx[1142] INFO user.GTS: GTS 0, 151 loops
Jan  1 00:10:16 LokalWiki weewx[1142] INFO user.GTS: GTS 2135.2096758175467, 214 loops
Jan  1 00:10:17 LokalWiki weewx[1142] INFO user.wns: GTS None, 0 loops
Jan  1 00:15:17 LokalWiki weewx[1142] INFO user.wns: GTS None, 0 loops
Jan  1 00:20:17 LokalWiki weewx[1142] INFO user.wns: GTS None, 0 loops

GTS Error on weewx starting

weewx[2453] ERROR user.GTS: dayET: no record (logged only once)
weewx[2453] INFO user.GTS: GTS initialized 2023-01-01 00:04:43
weewx[2453] INFO user.GTS: GTS 73.15416391976729, 67 loops
weewx[2453] ERROR weewx.cheetahgenerator: Evaluation of template /etc/weewx4/skins/Seasons/index.html.tmpl failed with exception '<class 'TypeError'>'
weewx[2453] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx4/skins/Seasons/index.html.tmpl
weewx[2453] ERROR weewx.cheetahgenerator: **** Reason: 'NoneType' object is not subscriptable
weewx[2453] ERROR weewx.cheetahgenerator: **** Traceback (most recent call last):
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/weewx/xtypes.py", line 77, in get_scalar
weewx[2453] ERROR weewx.cheetahgenerator: **** return xtype.get_scalar(obs_type, record, db_manager, **option_dict)
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/user/GTS.py", line 570, in get_scalar
weewx[2453] ERROR weewx.cheetahgenerator: **** return weewx.units.convertStd(__x,record['usUnits'])
weewx[2453] ERROR weewx.cheetahgenerator: **** TypeError: 'NoneType' object is not subscriptable
weewx[2453] ERROR weewx.cheetahgenerator: ****
weewx[2453] ERROR weewx.cheetahgenerator: **** During handling of the above exception, another exception occurred:
weewx[2453] ERROR weewx.cheetahgenerator: ****
weewx[2453] ERROR weewx.cheetahgenerator: **** Traceback (most recent call last):
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/weewx/cheetahgenerator.py", line 348, in generate
weewx[2453] ERROR weewx.cheetahgenerator: **** unicode_string = compiled_template.respond()
weewx[2453] ERROR weewx.cheetahgenerator: **** File "_etc_weewx4_skins_Seasons_index_html_tmpl.py", line 748, in respond
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/lib/python3/dist-packages/Cheetah/Template.py", line 1707, in _handleCheetahInclude
weewx[2453] ERROR weewx.cheetahgenerator: **** self._CHEETAH__cheetahIncludes[_includeID].respond(trans)
weewx[2453] ERROR weewx.cheetahgenerator: **** File "_etc_weewx4_skins_Seasons_current_inc.py", line 537, in respond
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/weewx/tags.py", line 599, in getattr
weewx[2453] ERROR weewx.cheetahgenerator: **** vt = weewx.xtypes.get_scalar(obs_type, self.record, db_manager)
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/weewx/xtypes.py", line 81, in get_scalar
weewx[2453] ERROR weewx.cheetahgenerator: **** return xtype.get_scalar(obs_type, record, db_manager)
weewx[2453] ERROR weewx.cheetahgenerator: **** File "/usr/share/weewx/user/GTS.py", line 570, in get_scalar
weewx[2453] ERROR weewx.cheetahgenerator: **** return weewx.units.convertStd(__x,record['usUnits'])
weewx[2453] ERROR weewx.cheetahgenerator: **** TypeError: 'NoneType' object is not subscriptable

Problem line in GTS.py
return weewx.units.convertStd(__x,record['usUnits'])

Best Practice for using yearGDD / seasonGDD

Hi,

i need some help again. What is the best practice for using yearGDD / seasonGDD with your gts plugin. When i run it with the following config, i already have 20 - 40 seconds to calculate.

time wee_reports
Using configuration file /etc/weewx/weewx.conf
Generating for all time
wee_reports  24,37s user 5,16s system 91% cpu 32,265 total

normal time to calculate without yearGDD / seasonGDD.

time wee_reports
Using configuration file /etc/weewx/weewx.conf
Generating for all time
wee_reports  5,47s user 0,40s system 74% cpu 7,872 total

here my config for testing with the belchertown skin.

[test]
title = Test
show_button = false
button_text = Test
start_at_midnight = true
time_length = year
[[yearGTS_]]
title = Test
line_gap_fraction = 0.04
yscale = 0,None,None
aggregate_type = avg
aggregate_interval = 86400
[[[GTS]]]
[[[yearGDD]]]
[[[seasonGDD]]]

or is only one calculation every 24h recommended?

Issue during installation

Hi,

during the installation I get an issue :-(

`pi@nodered:~ $ wget -O weewx-GTS.zip https://github.com/roe-dl/weewx-GTS/archive/master.zip
--2021-08-02 16:21:57-- https://github.com/roe-dl/weewx-GTS/archive/master.zip
Auflösen des Hostnamens github.com (github.com)… 140.82.121.3
Verbindungsaufbau zu github.com (github.com)|140.82.121.3|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 302 Found
Platz: https://codeload.github.com/roe-dl/weewx-GTS/zip/master [folgend]
--2021-08-02 16:21:57-- https://codeload.github.com/roe-dl/weewx-GTS/zip/master
Auflösen des Hostnamens codeload.github.com (codeload.github.com)… 140.82.121.9
Verbindungsaufbau zu codeload.github.com (codeload.github.com)|140.82.121.9|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: nicht spezifiziert [application/zip]
Wird in »weewx-GTS.zip« gespeichert.

weewx-GTS.zip [ <=> ] 50,30K --.-KB/s in 0,09s

2021-08-02 16:21:58 (560 KB/s) - »weewx-GTS.zip« gespeichert [51506]

pi@nodered:~ $ sudo wee_extension --install weewx-GTS.zip
Request to install 'weewx-GTS.zip'
Extracting from zip archive weewx-GTS.zip
Traceback (most recent call last):
File "/usr/share/weewx/wee_extension", line 88, in
main()
File "/usr/share/weewx/wee_extension", line 80, in main
ext.install_extension(options.install)
File "/usr/share/weewx/weecfg/extension.py", line 125, in install_extension
self.install_from_dir(extension_dir)
File "/usr/share/weewx/weecfg/extension.py", line 183, in install_from_dir
shutil.copy(source_path, destination_path)
File "/usr/lib/python3.7/shutil.py", line 245, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.7/shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/weewx-GTS-master/bin/usr/dayboundarystats.py'
`

Do you have any idea what is happening and how I can fix it?

Kind regards

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.