scrin / ruuvicollector Goto Github PK
View Code? Open in Web Editor NEWUtility to collect measurements from RuuviTags and store them in InfluxDB
License: MIT License
Utility to collect measurements from RuuviTags and store them in InfluxDB
License: MIT License
I'm working on a branch to allow the user to specify the exact fields they want to store into InfluxDB. This should work the same way as the current whitelisting/blacklisting of MAC addresses: I plan to use the storage.values
property, which currently supports the values raw
and extended
, and to add the new possible values of whitelist
and blacklist
. The actual list would be provided by some new configuration property, such as storage.values.list
.
Describe the feature you'd like to have implemented
Support for InfluxDB2
InfluxDB2 has a new java client library, a bit different internal structure with organizations and buckets, and a new connection setup which uses access token instead of username/password.
Describe alternatives you've considered
Alternative is of course continue using v1.x, which works perfectly fine.
Do you want to implement this feature?
I kinda already did
Additional info
I decided to see if I can make a db converter and connector with the new java client library, and it seems to run now. I used as much the same structure as possible with the existing InfluxDBConnector and Converter, and then added new configuration items for v2 support.
I did this only to see if I could still write something, as I haven't really programmed anything in a very long time. Anyway, if this would be useful in anyway, I could for once contribute something back
Describe the bug
Jul 25, 2020 12:38:49 AM org.influxdb.impl.BatchProcessor write
SEVERE: Batch could not be sent. Data will be lost
org.influxdb.InfluxDBIOException: java.net.UnknownHostException: darkball.net
at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:800)
at org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:455)
at org.influxdb.impl.OneShotBatchWriter.write(OneShotBatchWriter.java:22)
at org.influxdb.impl.BatchProcessor.write(BatchProcessor.java:321)
at org.influxdb.impl.BatchProcessor$1.run(BatchProcessor.java:269)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: darkball.net
at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
at java.net.InetAddress.getAllByName(InetAddress.java:1193)
at java.net.InetAddress.getAllByName(InetAddress.java:1127)
at okhttp3.Dns$1.lookup(Dns.java:40)
at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)
at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149)
at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:214)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at org.influxdb.impl.BasicAuthInterceptor.intercept(BasicAuthInterceptor.java:22)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at org.influxdb.impl.GzipRequestInterceptor.intercept(GzipRequestInterceptor.java:53)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:144)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
at okhttp3.RealCall.execute(RealCall.java:77)
at retrofit2.OkHttpCall.execute(OkHttpCall.java:180)
at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:788)
... 11 more
To Reproduce
It happens randomly
Expected behavior
This error shouldn't happen
Environment
OS and device (ie. Raspbian on RaspberryPI): OctoPi 0.17.0 on raspberry pi 4
Java version (check with java -version):
java version "1.8.0_212"
Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
Java HotSpot(TM) Client VM (build 25.212-b10, mixed mode)
Additional information
Ruuvi-collector is running in screen. InfluxDB and Grafana are hosted on my linode server and not locally on rpi.
RPI4 is connected to the internet through wifi. It is also running octoprint for my 3d printer, that's why octopi distro. It is based on raspbian
Hi,
currently the files in this repository have a mixture of LF and CRLF line-endings. Should all the files have LF line-endings?
Based on the last comment to this pull request PR 10 comment it seems that the desire would be to have only LF line-endings.
There are many instructions about how to convert repository to only have LF line-endings:
The simplest way to achieve this is to add .gitattributes file and then running "git add --renormalize ." to convert all files accordingly.
Describe the feature you'd like to have implemented
Today the MAC filtering is performed after the message has been parsed
Describe alternatives you've considered
MAC filtering should be before any subsequent message parsing
Do you want to implement this feature?
Yes
Additional info
This should leverage the existing code to whitelist/blacklist MAC addresses
Describe the bug
RuuviCollector crashes after couple of hours usage ''connection refused'' Java print this:
sudo systemctl status ruuvicollector.service
● ruuvicollector.service - RuuviCollector Service
Loaded: loaded (/etc/systemd/system/ruuvicollector.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-06-18 15:38:35 EEST; 1h 55min ago
Main PID: 466 (java)
CGroup: /system.slice/ruuvicollector.service
├─466 /usr/bin/java -jar /home/pi/ruuvi-collector-0.2.jar
├─728 hcitool lescan --duplicates --passive
└─744 hcidump --raw
Jun 18 15:38:59 raspberrypi java[466]: Caused by: java.net.ConnectException: Connection refused
Jun 18 15:38:59 raspberrypi java[466]: at java.net.PlainSocketImpl.socketConnect(Native Method)
Jun 18 15:38:59 raspberrypi java[466]: at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
Jun 18 15:38:59 raspberrypi java[466]: at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
Jun 18 15:38:59 raspberrypi java[466]: at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
Jun 18 15:38:59 raspberrypi java[466]: at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
Jun 18 15:38:59 raspberrypi java[466]: at java.net.Socket.connect(Socket.java:589)
Jun 18 15:38:59 raspberrypi java[466]: at okhttp3.internal.platform.Platform.connectSocket(Platform.java:129)
Jun 18 15:38:59 raspberrypi java[466]: at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:245)
Jun 18 15:38:59 raspberrypi java[466]:
Environment
raspberry 3, Rasbian
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) Client VM (build 25.65-b01, mixed mode)
Is there anything easy I could do for this project?
FYI ... a good link to humidity formulas is:
https://www.vaisala.com/sites/default/files/documents/Humidity_Conversion_Formulas_B210973EN-F.pdf
You mention that you lost the source of many of the formulas in measurements.md.
Currently RuuviCollector does not handle and/or log certain unexpected output from hcitool/hcidump. These kind of messages should be parsed and logged in a more user-friendly way. Examples:
device: disconnected
when the bluetooth adapter is either externally disabled or physically disconnected during operation
Can't attach to device hci0. No such device(19)
when trying to run hcidump with no available/enabled bluetooth adapters present
The latest version of the Weather Station DFU image has a single byte difference in the header to the sensortag firmware:
private static final String SENSORTAG_BEGINS = "> 04 3E 25 02 01 03 01 ";
I'm not sure exactly what difference in the DFU image causes this change in the packet, but I've now got a working Grafana dashboard looking at an InfluxDB data source with three tags logging successfully after changing this string.
No data received anymore after upgrade of Bluetooth related packages on Raspberry Pi3 Raspbian Buster.
Bluetooth packages were upgraded, history.log:
Start-Date: 2020-04-02 07:59:14
Commandline: apt-get upgrade
Requested-By: pi (1000)
Upgrade: bluez:armhf (5.50-1+rpt1, 5.50-1.2deb10u1+rpt1), bluez-hcidump:armhf (5.50-1+rpt1, 5.50-1.2deb10u1+rpt1), fi
rmware-realtek:armhf (1:20190114-1+rpt4, 1:20190114-1+rpt5), rpi-eeprom-images:armhf (5.0-2, 5.2-1), libicu63:armhf (63
.1-6, 63.1-6+deb10u1), rpi-eeprom:armhf (5.0-2, 5.2-1), firmware-atheros:armhf (1:20190114-1+rpt4, 1:20190114-1+rpt5),
firmware-libertas:armhf (1:20190114-1+rpt4, 1:20190114-1+rpt5), firmware-brcm80211:armhf (1:20190114-1+rpt4, 1:20190114
-1+rpt5), firmware-misc-nonfree:armhf (1:20190114-1+rpt4, 1:20190114-1+rpt5)
End-Date: 2020-04-02 08:01:19
After the upgrade no data has been received from RuuviTags.
Environment
Running Raspberry Pi3b+ Rasbian Buster latest.
Ruuvitag macs are visible in hcitool lescan
Restarting, running disbling/enabling scans through bluetoothctl/hcitool does not seem to help.
Describe the feature you'd like to have implemented
Replace deprecated bluetooth utilities with supported versions, maybe bluetoothctl?
Describe alternatives you've considered
N/A
Do you want to implement this feature?
No
Additional info
hcitool/hcidump deprecated since 2017
https://git.kernel.org/pub/scm/bluetooth/bluez.git/commit/?id=b1eb2c4cd057624312e0412f6c4be000f7fc3617
Been running for a few days - then this occured
2019-04-28 18:54:26.448 INFO [Main] Successfully reading data from hcidump
2019-05-01 11:22:55.300 WARN [Main] Uncaught exception while handling measurements from MAC address
"C9B7EB45C0EF", if this repeats and this is not a Ruuvitag, consider blacklisting it
java.lang.StringIndexOutOfBoundsException: String index out of range: 19
at java.lang.String.charAt(String.java:658)
at fi.tkgwf.ruuvi.utils.Utils.hexToBytes(Utils.java:33)
at fi.tkgwf.ruuvi.utils.HCIParser.readLine(HCIParser.java:51)
at fi.tkgwf.ruuvi.Main.run(Main.java:104)
at fi.tkgwf.ruuvi.Main.run(Main.java:80)
at fi.tkgwf.ruuvi.Main.main(Main.java:43)
2019-05-01 11:22:55.330 DEBUG [Main] Offending line: device: disconnected
2019-05-01 11:22:55.342 INFO [Main] Clean exit
I'm going to try and trap the error - then look at handling input - prefer the service to keep running and log the bad data. Thoughts?
A HttpDBConnection that will do a HTTP POST to a configurable http/https url, with the measurement data encoded as a JSON object with the field names as described in https://github.com/Scrin/RuuviCollector/blob/master/MEASUREMENTS.md
How are you running locally to test it? (mac or pc?)
Googling/searching around Bluez isnt supported on Mac or PC - have you stored locally some streamed data that your replay, via a test class back into the main class?
Trying to work out how you built this project :)
Create service script and something to build a .deb installer package which will install the collector and set up it as a service.
The installer package should have properly marked dependencies (ie. Java 8 or newer).
The installer package should set up a service user for the collector with permissions to run hcitool and hcidump as needed.
If there is a problem with influxdb (broken configuration, out of space..). RuuviCollector will crash continuously sucking CPU and perhaps filling log. See #8
Please add (something like)
RestartSec=300
to
/lib/systemd/system/ruuvicollector.service
++++++++++++++
When running influxDB on the same machine “requires=influxdb.service after=influxdb.service” should be specified.
PS I consider this as deserving a BUG label as "..Malfunctions that should be fixed to comply with the expected behavior" If influx is not running,
Currently there 3 different filter modes configurable:
# none = Allows any source to be stored (default)
# blacklist = Allows all sources EXCEPT those listed
# whitelist = Allows ONLY sources that are listed
and a 4th mode should be added called named
which will operate like the whitelist, but only allow tags (macs) that have an associated name in ruuvi-names.properties
.
It should ignore the filter.macs
configuration property (like none
does), and it should throw an IllegalStateException if no names are defined when using this filter mode, just like the whitelist
does now when using it and no macs are defined.
InfluxDB may not be available at all times, especially during startup.
Create retry/reconnect logic to InfluxDBConnection.java
The retry logic should retry a configurable number of times (default maybe 10) before giving up and throwing errors/exiting
Hi!
It would be nice to be able to fix temperature readings of a tag by some offset based on a reference temperature. I know that's not fully accurate either because the offset may vary by temperature, but this would be a start, doesn't it?
I can implement it if no one else wants to. :-)
How about a new configuration property: temperatureOffset.MAC=nn ?
I think it's better use MACs here rather than names because names can be changed by tag use but this property is strictly tag hardware -specific.
Cheers,
Anssi Kanninen
Describe the feature you'd like to have implemented
Ability to choose the BT adapter used, instead of defaulting to hci0.
Describe alternatives you've considered
I'm running this on an RPI3B+ with an integrated BT adapter to monitor a few Ruuvitags and a couple MiFlora plant sensors. Running 2 BTLE daemons simultaneously on the same adapter causes conflicts, and I added an extra usb BT dongle. I would very much like the ability to choose which adapter each collector daemon is running on, I would guess this would be fairly simple to implement with a config option.
Do you want to implement this feature?
No
Currently, by default, the app records new measurements every ten seconds. There should of course be some limiting strategy like this, as otherwise one would fill their database too quickly.
However, if you intend to track something that takes less than 10 second, such as some movements, there's a great chance you won't catch the packet(s) where the accelerometer values are interesting.
So I propose that the application should have different methods of limiting the number of saved measurements. Maybe the app could decide to store a measurement after all even if 10 seconds hasn't elapsed yet, provided that some threshold of some sensor values are exceeded. Or maybe it should remember, say, the maximum values of the 10 second period and record those at the end of it.
I volunteer myself for working on this improvement as I've already started on it in my own fork. :)
Hi! I got this warning around every 3 second and no data was stored in the Influxdb:
WARN [Main] Uncaught exception while handling measurements java.lang.ArrayIndexOutOfBoundsException: 5 at fi.tkgwf.ruuvi.handler.impl.AbstractEddystoneURL.handleMeasurement(AbstractEddystoneURL.java:74) at fi.tkgwf.ruuvi.handler.impl.AbstractEddystoneURL.read(AbstractEddystoneURL.java:36) at fi.tkgwf.ruuvi.Main.lambda$run$5(Main.java:122) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1235) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at fi.tkgwf.ruuvi.Main.run(Main.java:124) at fi.tkgwf.ruuvi.Main.main(Main.java:88)
Do you have any clues? Thanks!
Hi!
I edited ruuvi-names.properties with my MAC & new tags and placed it beside the .jar, but they don't seem to change the tag names in grafana from the MAC format? Service start/stop service does not seem to do the job either. Is there rebuild needed or how the configuration should work?
br,
Markus
I have retrieved Humidity_Conversion_Formulas_B210973EN-F.pdf and put it at
MyBeacons.info/Humidity_Conversion_Formulas_B210973EN-F.pdf
Please retrieve it from there and correct the link at:
https://github.com/Scrin/RuuviCollector/blob/master/MEASUREMENTS.md
Describe the feature you'd like to have implemented
When the database connection is lost, I'd like the software to exit with a non-zero return code.
Describe alternatives you've considered
Instead of exiting, just keep trying to reconnect forever. This was discussed in #8 but the decision was to exit uncleanly and allow the supervisor program (e.g., systemd) to deal with restarting failed services.
The scope of #8 was only if the service could not make the initial database connection (e.g., if the service were started before the database was ready). It does not handle the case where the database connection is established but then later lost.
Do you want to implement this feature?
I'd be happy to, but if someone else wanted to that'd be fine too.
Additional info
When a database connection currently drops, the error message printed is this one:
Uncaught exception while handling measurements from MAC address "00FF00AABBCC", if this repeats and this is not a Ruuvitag, try blacklisting it
This proposed change would have a more specific error message in the event that the database save was what failed.
Describe the feature you'd like to have implemented
It would be nice if the database contained information about the receiver (such as a custom string identifier or receiver MAC), in case a person has multiple receivers. Currently many receivers may receiver partially the same data except maybe for the RSSI value (barring manual white/blacklisting).
It would also allow interesting applications like estimating location via means of triangulating the RSSI values (now that it would be known where it was received).
Describe alternatives you've considered
An alternative to this would be using multiple databases, one for each receiver. While doable, I think this would complicate the query part needlessly. Perhaps this data can be optional similarly as other collected values are now configurable.
Another alternative would be to use different names on different hosts. Still, I think a global identifier would be easier.
Do you want to implement this feature?
Yes.
No.
Not until I find myself installing more tags, anyway..
It would be helpful to have the log include a startup message including the version in Main
something like
LOG.info("Version: 0.2.4 begins with args:"+args+":");
Additionally, since configuration only occurs at startup perhaps using
LOG.info in config/Config.java (instead of .debug) would be helpful and not clutter logs.
For that matter LOG.info of EVERY getProperty in readConfigFromProperties
LOG.info would not only document properties but also assist in diagnosing incorrect/misspelled property names.
Thanks
If I understand correctly, the retention policy is currently assumed to be "autogen". I assume this is the InfluxDB java library default since it doesn't seem to be set explicitly.
Following the helpful suggestion in MEASUREMENTS.md, I'm trying to set retention policy and continuous query to automatically downsample data in Influxdb. For this purpose, it would be useful to be able to set retention policy in the configuration file.
in ruuvi/config/Config.java/readConfig.java
, It would be helpful if item encountered in ruuvi-collector.properties
were output to LOG.info
.
Describe the bug
RuuviCollector does not work when run without root privileges after raspberry pi os 12/2022 updates.
To Reproduce
Expected behavior
RuuviCollector works.
Environment
OS and device (ie. Raspbian on RaspberryPI): Raspberry pi OS (32-bit) & Raspberry Pi ZeroW 2
Java version (check with java -version): openjdk version "17.0.4" 2022-07-19
Additional information
Issue was resolved with adding cap_sys_rawio
to capabilities.
I wonder if you would consider creating an option for ruuviCollector (or a totally independent application)
which very simply inserts any messages with the manufacturer ID of 0x0499 (or a list of configurable ID) into an influxDB without interpretation.
An option to blacklist (ignore) packets of certain formats intrepreted elsewhare would be nice.
This would "future proof" the RuuviCollector allowing new formats to be stored without need to modify the collector instead places the responsibility of the interpretation of the message on a subsequent application retrieving the data from the database.
This would also allow custom packets to be stored as created by custom firmware.
Consider :
The currently preliminary proposed format 08.
https://f.ruuvi.com/t/proposal-for-secure-data-format/3756
Your own proposal for dynamic data format
https://f.ruuvi.com/t/proposal-for-secure-data-format/3756
A lower resolution format reducing the size most measurement fields allowing for increase in size of count fields like movement counter, additional fields for additional sensors or previous values while still remaining within the advertisement length limit.
theBASTI0N proposed alternating PacketA and Packet B
Secure Shuffling schedules beacons to change the data that they broadcast (major, minor, MAC-address) on regular basis.
https://support.kontakt.io/hc/en-gb/articles/206762009
I am considering implementing :
A) A C0 format which reports the configuration to be transmitted every 100 packets.
as discussed at
https://f.ruuvi.com/t/packet-formats-new-tag-status-configuration-packet/3797
B) A 06 format a one byte extension to 05 containing temperature/humidity delta indicators as discussed at https://f.ruuvi.com/t/packet-formats-new-tag-status-configuration-packet/3797
C) A variant of an existing format with solar incidence as reported by a GA1A1s202wp log sensor replacing some combination of atmospheric pressure, batteryVoltage&eTXpower or most significant bytes of MAC (leaving remaining bytes for beacon identification)
https://cdn-learn.adafruit.com/downloads/pdf/adafruit-ga1a12s202-log-scale-analog-light-sensor.pdf
Thank you for your consideration
Newest version of ruuvitag_fw (as of Fix #157, 9/3/19, 2.5.5) will be reporting several INVALID values (Actually they indicate UNAVAILABLE sensors).
In addition the humidity will now include an invalid constant (120) when the compensated (i.e. reading adjusted by the correction table equations results in an error
(see https://github.com/ruuvi/ruuvitag_fw/blob/master/drivers/bme280/bme280.c compensate_H_int32 ( where 503316480 is actually 120<<10<<12 i.e. 120% ))
Can these values be (optionally?) treated as null?
I'd like to see (and possibly contribute to) documentation on the saved values, so I could understand what I'm actually storing. At least the following information would be helpful:
As the quality of this software feels like professional level, why not bring the documentation to the same level? :)
Describe the bug
named-filter does not recognise ruuvi-names.properties
To Reproduce
Configure ruuvi-names.properties:
miha@ruuvitag:~/RuuviCollector/target$ tail -2 ruuvi-names.properties
EA:D6:63:18:F3:82=Fridge
C7:60:16:8C:BB:4F=Freezer
Configure ruuvi-collector.properties:
#filter.mode=none
filter.mode=named
filter.macs=Fridge,Freezer
Expected behavior
Collector should send data to remote influxd-machine. That does not happen (followed with netstat -nputwc) However, with filter.mode=none and filer.macs commented out, collector works.
Environment
OS and device (ie. Raspbian on RaspberryPI):
NanoPI Neo AIr
Java version (check with java -version):
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-8u232-b09-1~deb9u1-b09)
OpenJDK Client VM (build 25.232-b09, mixed mode)
Additional information
Any other information that may help in diagnosing this issue
miha@ruuvitag:/RuuviCollector/target$ pwd
/home/miha/RuuviCollector/target
miha@ruuvitag:/RuuviCollector/target$ ls -la
total 4864
drwxr-xr-x 9 miha miha 4096 May 9 14:42 .
drwxr-xr-x 6 miha miha 4096 May 9 09:29 ..
drwxr-xr-x 3 miha miha 4096 May 9 09:29 classes
drwxr-xr-x 3 miha miha 4096 May 9 09:29 generated-sources
drwxr-xr-x 3 miha miha 4096 May 9 09:29 generated-test-sources
drwxr-xr-x 2 miha miha 4096 May 9 09:29 maven-archiver
drwxr-xr-x 3 miha miha 4096 May 9 09:29 maven-status
-rw-r--r-- 1 miha miha 53771 May 9 09:29 original-ruuvi-collector-0.2.jar
-rw-r--r-- 1 miha miha 4867034 May 9 09:29 ruuvi-collector-0.2.jar
-rw-r--r-- 1 miha miha 5328 May 9 14:41 ruuvi-collector.properties
-rw-r--r-- 1 miha miha 479 May 9 14:42 ruuvi-names.properties
drwxr-xr-x 2 miha miha 4096 May 9 09:29 surefire-reports
drwxr-xr-x 3 miha miha 4096 May 9 09:29 test-classes
miha@ruuvitag:/RuuviCollector/target$ tail -2 ruuvi-names.properties
EA:D6:63:18:F3:82=Fridge
C7:60:16:8C:BB:4F=Freezer
miha@ruuvitag:/RuuviCollector/target$ grep named ruuvi-collector.properties
filter.mode=named
miha@ruuvitag:/RuuviCollector/target$ sudo java -jar -Xdebug ruuvi-collector-0.2.jar
2020-05-09 14:48:09.184 DEBUG [Config] Tag names: /home/miha/RuuviCollector/target/ruuvi-names.properties
2020-05-09 14:48:09.217 DEBUG [Config] Config: /home/miha/RuuviCollector/target/ruuvi-collector.properties
Exception in thread "main" java.lang.ExceptionInInitializerError
at fi.tkgwf.ruuvi.Main.startHciListeners(Main.java:41)
at fi.tkgwf.ruuvi.Main.run(Main.java:63)
at fi.tkgwf.ruuvi.Main.main(Main.java:30)
Caused by: java.lang.IllegalStateException: You have set filter.mode=named but left ruuvi-names.properties empty. Please select a different filter.mode value or populate ruuvi-names.properties.
at fi.tkgwf.ruuvi.config.Config.parseFilterMode(Config.java:260)
at fi.tkgwf.ruuvi.config.Config.readConfigFromProperties(Config.java:145)
at fi.tkgwf.ruuvi.config.Config.readConfig(Config.java:127)
at fi.tkgwf.ruuvi.config.Config.reload(Config.java:88)
at fi.tkgwf.ruuvi.config.Config.reload(Config.java:81)
at fi.tkgwf.ruuvi.config.Config.(Config.java:77)
... 3 more
miha@ruuvitag:~/RuuviCollector/target$
Every time this appears in dmesg, RuuviCollector stops collecting data: [ke marras 27 20:51:50 2019] Bluetooth: hci0: hardware error 0x43
. The process itself does not crash, it is still there but nothing appears in logs about the situation.
Describe the feature you'd like to have implemented
tagSocket support for sharing RuuviTag poll data to multiple applications, ie. RuuviCollector and homebridge-ruuvitag receiving data at the same time.
Do you want to implement this feature?
No
Additional info
https://github.com/klaalo/ifData/tree/master/tagSocket
https://github.com/pakastin/homebridge-ruuvitag
It might be useful to provide a measurementUpdateLimit on a tag specific basis.
For example tags know to be located where environmental changes are infrequent might have an update limit of 30000ms (every 5 minutes) and others might be better with 10000(10 seconds).
Useful for very long term data samples and very small databases (on feeble processors like RaspberryPi Zero).
As the Ruuvi tags have some differences in the measured values, it would be good to be able to use some offset values to the data that is stored in the database. For example, when checking for the air pressure difference between tags outdoors and indoors, using normalized values is quite essential.
I have a bit more than 20 Ruuvitags. While in the same small container with a calibration pack (salt solution) for a few days they deviate up to around 5-6% relative humidity and about 0.8°C from eachother. The tags closest to the real RH still display 2% below the real relative humidity (using calibration pack with a salt solution and cross checking using other high end devices).
Wouldn't it make sense to have an option in RuuviCollector to calibrate these, by having the possibility to set an offset temp and/or humidity for each device in the config files?
Of course this would have to be updated or checked frequently, but that's not difference than doing that somewhere else. I understand (but didn't test) the RuuviStation mobile apps now have the same feature in the meantime.
It seems this Ruuvi collector app can't be run on OS X due to lack of Bluez on OS X? Right?
Describe the feature you'd like to have implemented
The core desire here is to ensure that the scanning command is always running. My proposed implementation to achieve this is to run the lescan
as a service, rather than being run by RuuviCollector.
I'm having troubles with the scan command (hcitool lescan --duplicates --passive
) dying on me. This results in no data coming in, no error messages being printed, and the service does not exit, which makes the recovery process difficult to automate.
Describe alternatives you've considered
The ProcessBuilder.start() method does return a Process object. It's be possible to spin up our own thread that would waitFor() the process to exit and then just immediately restart the process.
I have two objections to this alternative approach:
Do you want to implement this feature?
Yes
Additional info
I've already implemented this as a systemd service in my deployment. My question is if I can and how I should contribute this back to the project. I also have a .service file for RuuviCollector that I'd be willing to contribute as well.
Please let me know if these service files should go in a directory by themselves, whether you'd like an install shell script to set this up, if it's OK to update the README with instructions, and so forth. Also, systemd requires an absolute path to the executable, so if there are any suggestions on where that should go (/usr/local/bin?), I'd be happy to incorporate them.
This merge request would not be any changes to any existing code. The user could change their ruuvi-collector.properties file to make the scan command an empty string (or not, it wouldn't really hurt to have two lescan
s running at once, though it'd be unnecessary) and then set up and start the lescan service.
Steps to reproduce:
ruuvi-collector.properties
and ruuvi-names.properties
into the project rootmvn tests
Likely cause is that while running the tests, the collector picks up the "normal" config files rather than the test config files.
Compiling the project in case of failing tests can be worked around with -DskipTests
for now, ie.
mvn package -DskipTests
Describe the feature you'd like to have implemented
By default the collector writes to one influxUrl.
Describe alternatives you've considered
Making RuuviCollector to write two separate influxUrl's would help with displaying collected data on Grafana when ruuvitags are on two different geolocations. RuuvICollector on remote site would write to local influx and also to remote (Grafana-site influx. Local RuuvCollector would write to local influx. Remote RuuviCollector would continue writing to its local influx if the connection to remote influx is lost.
Sadly I don't have skills to make this happen
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.