[Bug]: GPS Checksum errors, HGLRC M100_MINI about firmware HOT 13 CLOSED

@GPSFan any ideas?

from firmware.

How did you connect the gps to your Heltec board? Does it have the GPS power switched by a mosfet?

from firmware.

Does this GPS module have a battery for config backup? I can't tell from the pictures on Amazon. If it does not, power cycling it will loose the config, but Meshtastic won't know this. The default baud rate for this module is 38400, so if the config is lost the baud rates will be out of sync. The same thing could happen even it the GPS module has a battery, but it hasn't had enough power on time to charge.
If it has a super-cap instead, it takes ~200 minutes to charge up enough to give 12 hours of backup.
The M10 series has no flash memory, so the only way to preserve the config through a power cycle or a PMREQ induced sleep is an onboard battery, a super-cap, or connecting the VCC_BACKUP pin to some unswitched source of power.
With a sealed all-in-one module like that it's hard to tell what's inside, one thing we do know is that VCC_BACKUP is not user connectable.
I have 3 different flavors of M10 modules, all came with backup batteries, and all were discharged when I got them.

from firmware.

I'll do some testing with my modules this weekend and see if I can replicate the issue.

from firmware.

I have the GPS connected as follows:

My understanding is that the 5V line comes (almost) directly off of the USB and doesn't go through a power switching mosfet (see schematic). I'm not able to probe the pin with my multimeter at the moment, but I believe the power should be constant since I had it connected to my PC the whole time to get the serial logs.

(For others following along: I plan to eventually move power over to the Heltec's Ve line (which I believe is switchable) but that's not how it is set up at the moment.)

The GPS PCB is entirely dominated by the antenna on one side and an RF can on the other. There is no sign of a battery or capacitor, but its possible something lurks under the can. Any energy storage would have to be rather small; the (what appears to be) battery/cap on a BN-180 module that I also have looks like it would take up a not-insignificant portion of the RF can's volume.

from firmware.

Some updates:

1. I'm not sure if this helps figure out if there is battery/capacitor backup, but if I remove the battery from my Heltec (leaving only USB as a power source) and let the GPS get a lock, I can disconnect USB (and thus all power) for 2 minutes, re-connect USB, and am able to get a fix in ~30 seconds. I don't know what the expected hot/warm/cold fix times for M10 chipsets are, but maybe this will help figure out what is going on.

2. My multimeter shows 5V being supplied at all times -- even when the GPS is put to sleep.

3. I tried moving the power source over to the Heltec's Ve pin. It supplies 3.3V instead of 5V, but the GPS docs indicate this is acceptable. When powered through Ve I don't notice a change in behavior. I'm able to get an initial fix, the GPS is put to sleep but power is not removed, and when the GPS is awoken I get endless checksum errors.

from firmware.

The complete power cycle re-initializes the GPS and thus sets the baud to 9600. If you leave the GPS powered for a while (several hours?) maybe the internal battery will charge enough for it to coast through the sleep period without losing the config.
Try that and see if the issue re-appears.
I'll be able to do more this weekend unless "Sheep Happens" as we say in the industry. ;>))

from firmware.

No luck leaving the GPS powered for a while.

I discovered that the problem would not occur if I turned the update interval all the way down to 10 seconds. The logs appear to indicate that the GPS is not put to sleep for such a short interval. Slightly longer intervals like 30 seconds do put it to sleep, however.

I let the system run with the 10 second interval yesterday for 19 hours. Over that time I did see random checksum failures, but no real problems. This morning I changed the config to 30 seconds and the problem immediately reappeared. If there is a battery or capacitor which needs to be charged, I assume 19 hours of charging should have been enough to handle a 30 second outage...

Logs just prior to rebooting with the new config

DEBUG | 17:01:51 70644 [GPS] WANT GPS=1
INFO  | 17:01:51 70644 [GPS] Setting GPS power=1
WARN  | 17:01:51 70644 [GPS] Warning, 1 new GPS checksum failures, for a total of 2710.
DEBUG | 17:01:52 70644 [GPS] WANT GPS=0
DEBUG | 17:01:52 70644 [GPS] GPS Lock took 0, average 0
DEBUG | 17:01:52 70644 [GPS] publishing pos@65e35b82:2, hasVal=1, Sats=6, GPSlock=1
DEBUG | 17:01:52 70644 [GPS] New GPS pos@65e35b82:3 lat=XXXXXX, lon=XXXXXX, alt=147, pdop=5.42, track=227.87, speed=0.02, sats=6
DEBUG | 17:01:52 70644 [GPS] onGPSChanged() pos@65e35b82, time=1709398912, lat=XXXXXX, lon=XXXXXX, alt=147
INFO  | 17:01:52 70644 [GPS] updatePosition LOCAL pos@65e35b82, time=1709398912, latI=XXXXXX, lonI=XXXXXX, alt=147
DEBUG | 17:01:52 70644 [GPS] Setting local position: latitude=XXXXXX, longitude=XXXXXX, time=1709398912
DEBUG | 17:01:52 70644 [GPS] Node status update: 1 online, 3 total
DEBUG | 17:02:02 70655 [GPS] WANT GPS=1
INFO  | 17:02:02 70655 [GPS] Setting GPS power=1
WARN  | 17:02:02 70655 [GPS] Warning, 1 new GPS checksum failures, for a total of 2711.
DEBUG | 17:02:03 70656 [GPS] WANT GPS=0
DEBUG | 17:02:03 70656 [GPS] GPS Lock took 1, average 0
DEBUG | 17:02:03 70656 [GPS] publishing pos@65e35b8d:2, hasVal=1, Sats=6, GPSlock=1
DEBUG | 17:02:03 70656 [GPS] New GPS pos@65e35b8d:3 lat=XXXXXX, lon=XXXXXX, alt=111, pdop=2.36, track=227.87, speed=0.03, sats=6
DEBUG | 17:02:03 70656 [GPS] onGPSChanged() pos@65e35b8d, time=1709398923, lat=XXXXXX, lon=XXXXXX, alt=111
INFO  | 17:02:03 70656 [GPS] updatePosition LOCAL pos@65e35b8d, time=1709398923, latI=XXXXXX, lonI=XXXXXX, alt=111
DEBUG | 17:02:03 70656 [GPS] Setting local position: latitude=XXXXXX, longitude=XXXXXX, time=1709398923
DEBUG | 17:02:03 70656 [GPS] Node status update: 1 online, 3 total
DEBUG | 17:02:03 70656 [Power] Battery: usbPower=0, isCharging=0, batMv=4069, batPct=91
DEBUG | 17:02:13 70666 [GPS] WANT GPS=1
INFO  | 17:02:13 70666 [GPS] Setting GPS power=1
DEBUG | 17:02:14 70667 [GPS] WANT GPS=0
DEBUG | 17:02:14 70667 [GPS] GPS Lock took 0, average 0
DEBUG | 17:02:14 70667 [GPS] publishing pos@65e35b98:2, hasVal=1, Sats=6, GPSlock=1
DEBUG | 17:02:14 70667 [GPS] New GPS pos@65e35b98:3 lat=XXXXXX, lon=XXXXXX, alt=118, pdop=4.54, track=227.87, speed=0.02, sats=6
DEBUG | 17:02:14 70667 [GPS] onGPSChanged() pos@65e35b98, time=1709398934, lat=XXXXXX, lon=XXXXXX, alt=118
INFO  | 17:02:14 70667 [GPS] updatePosition LOCAL pos@65e35b98, time=1709398934, latI=XXXXXX, lonI=XXXXXX, alt=118
DEBUG | 17:02:14 70667 [GPS] Setting local position: latitude=XXXXXX, longitude=XXXXXX, time=1709398934
DEBUG | 17:02:14 70667 [GPS] Node status update: 1 online, 3 total
DEBUG | 17:02:23 70676 [Power] Battery: usbPower=0, isCharging=0, batMv=4070, batPct=91
DEBUG | 17:02:24 70677 [GPS] WANT GPS=1
INFO  | 17:02:24 70677 [GPS] Setting GPS power=1
DEBUG | 17:02:25 70678 [GPS] WANT GPS=0
DEBUG | 17:02:25 70678 [GPS] GPS Lock took 0, average 0
DEBUG | 17:02:25 70678 [GPS] publishing pos@65e35ba3:2, hasVal=1, Sats=6, GPSlock=1
DEBUG | 17:02:25 70678 [GPS] New GPS pos@65e35ba3:3 lat=XXXXXX, lon=XXXXXX, alt=118, pdop=4.55, track=227.87, speed=0.02, sats=6
DEBUG | 17:02:25 70678 [GPS] onGPSChanged() pos@65e35ba3, time=1709398945, lat=XXXXXX, lon=XXXXXX, alt=118
INFO  | 17:02:25 70678 [GPS] updatePosition LOCAL pos@65e35ba3, time=1709398945, latI=XXXXXX, lonI=XXXXXX, alt=118
DEBUG | 17:02:25 70678 [GPS] Setting local position: latitude=XXXXXX, longitude=XXXXXX, time=1709398945
DEBUG | 17:02:25 70678 [GPS] Node status update: 1 online, 3 total
DEBUG | 17:02:35 70688 [GPS] WANT GPS=1
INFO  | 17:02:35 70688 [GPS] Setting GPS power=1
DEBUG | 17:02:36 70689 [GPS] WANT GPS=0
DEBUG | 17:02:36 70689 [GPS] GPS Lock took 0, average 0
DEBUG | 17:02:36 70689 [GPS] publishing pos@65e35bae:2, hasVal=1, Sats=6, GPSlock=1
DEBUG | 17:02:36 70689 [GPS] New GPS pos@65e35bae:3 lat=XXXXXX, lon=XXXXXX, alt=127, pdop=4.55, track=227.87, speed=0.01, sats=6
DEBUG | 17:02:36 70689 [GPS] onGPSChanged() pos@65e35bae, time=1709398956, lat=XXXXXX, lon=XXXXXX, alt=127
INFO  | 17:02:36 70689 [GPS] updatePosition LOCAL pos@65e35bae, time=1709398956, latI=XXXXXX, lonI=XXXXXX, alt=127
DEBUG | 17:02:36 70689 [GPS] Setting local position: latitude=XXXXXX, longitude=XXXXXX, time=1709398956
DEBUG | 17:02:36 70689 [GPS] Node status update: 1 online, 3 total

Logs after the config change look very similar to the initial post.

from firmware.

That almost seems to indicate that either:
There is no backup battery/supercap.
It is defective.
Some other random wierdness.
What I would do is connect the GPS module up to a PC with a usb-serial adapter and run u-center (u-blox's config program). Set up the config to something other than the defaults and save it to the RAM & BBR layers, then turn off the VCC, wait a bit turn VCC back on and see it it reverted to the default config.
That will tell us if there is a backup or not.
use u-center 23.08, 2.0 is terrible IMHO.
https://www.u-blox.com/en/product/u-center
It takes a little learning to use it effectively...

from firmware.

Alright, I think I've been able to confirm that the GPS has some kind of battery-backed RAM:

I don't have access to a Windows-based system, but was able to get u-center 23.08 working under Linux with Wine. I could see the position messages received from the GPS coming over over the Packet Console, as well as configuration messages being sent back and forth when modifying things in the Configuration View. I made a change to the device's baud rate (38400 -> 115200), first sending the change the device and then (after reconnecting at the new baudrate) saving the configuration to the BBR/Flash devices (the default selection).

The GPS would reliably start up at 115200 when power was applied. I then removed power, and then reconnected after waiting 10 minutes. Once again, it connected at 115200.

Just in case having "Flash" selected when saving the config was the cause, I reset back to 38400, saved the config to BBR/Flash, removed/reapplied power, verified the device was operating at 38400, and then once again changed the config to 115200 (except this time being sure to only select "BBR" before saving). After removing power for another 10 minutes and re-connecting, the device came up at 115200 again.

from firmware.

Good to hear that someone other than me runs u-center under wine! IMHO it runs better under wine than on Windoze.
I was able to duplicate your results and have traced the issue back to my forgetting to save the baud rate settings for the M10. there are several flavors of M10 based modules about. The one I have with an SMA has a default baud of 9600 baud. The one I have with 38400 has no external connector so I had to switch back and forth to do testing on all the VALSET commands I was developing.
I have a PR ready to go that fixes this issue.

from firmware.

Just FYI, I applied the fix on top of 2.2.23.5672e68 and the updated firmware is not experiencing the endless checksum failures anymore. The GPS seems to now always start up at 9600 baud instead of the 38400 that was its default. I assume that was intentional.

from firmware.

Yes, Meshtastic really wants the GPS to run at 9600 baud, although only the u-blox modules are commanded to change from what they start out as to 9600. The UC6580 on the Heltec Wireless Tracker defaults to 115200, which is the lowest it can go, so Meshtastic leaves it alone.

from firmware.