dcc-lab / projetneige Goto Github PK

Local arduino clock

Runs 10h everyday from 6h56 to 16h56 before time change. 7h56 to 17h56 after time change (seen from outside).

• Found a code where it's written ((tm.Hour>=7) && (tm.Hour<=17)) which implies 7am to 6pm. This is WRONG. Looking thoroughly at the data AND the server backprop correction algorithm which considers 10h uptime, the Arduino code was probably updated locally before deployment to ((tm.Hour>=7) && (tm.Hour<=16)), which means 7am to 5pm.
• Clean logs were obtained as early as 6h56. This means there was an offset with the clock used, which is fine and understandable.
• This clock is offline, so no changes were made during the time change (2nd sunday of march).

Time backtrack algorithm

• Runs on server (connected to internet, true EST time)
• When it receives a data packet of N samples at time T, it places each sample i at timestamp t_i = T - (57s * (N-i))
• If the resulting timestamp t_i falls back earlier than 7am (up to 6pm* ), the timestamp is shifted back by 14 hours.
• This means 6h30am is moved to 16h30
• !! This condition was not updated after time change !! This means that when the system was actually opening at 8am, if a large data packet was backtracked to 6h30am, only the portion from 6h30 to 7am was moved to the previous day (and "wrongly" at 16h30 to 17h instead of 17h30 to 18h BUT as you can see this "cancels out".) instead of moving 6h30 to 8am back to the previous day (from 16h30 to 18h). These two errors (after time change) partly cancels out: The data from 6h30 to 7h is correctly moved to 16h30 and 17h, but the data from 7am to 8am is NOT moved to the previous day.

* this 6pm condition is weird I know but should almost never happen anyway. But that's already a potential error for large data packets.

Server-side code backtrack algorithm

def backTrackTime(filePath, index=0, delta=1):
    timeObject = datetime.fromtimestamp(pathlib.Path(filePath).stat().st_ctime)
    correctTime = timeObject - timedelta(seconds=57 * delta * index)
    shiftTime = correctTime + timedelta(minutes=4)  # opens at 6h56, not 7
    if shiftTime.hour < 7 or shiftTime.hour > 17:
        correctTime = correctTime - timedelta(hours=14)
    return correctTime

Time jumps

Because data packets are sent with no timestamps, if the data packet is large (covers lets say 8 hours of data), there is a chance that the hardware did NOT actually worked at all for lets say 30 minutes during that 8.5 hours of "uptime". The backtracking algorithm has no idea when the system skipped measurements, so this 8 hours of data is uniformly distributed in the past 8 hours / 8.5 hours. So if this data packet is received at 17h, it will backtrack to 9am only and there will be NO logged data between 8h30 to 9h.

Other 2 interesting cases when data packet has to be backtracked to the previous day:

1. Before time change (correct) [measurements are done from 7am to 5pm]:
   If the packet is received at noon, it will first evenly backtrack up to 4am. The data from 4am to 7am will be shifted back by 14 hours and land the previous day from to 14h to 17h.

2. After time change (incorrect) [meaasurements are done from 8am to 6pm] :
   Same thing initially: "If the packet is received at noon, it will first evenly backtrack up to 4am. The data from 4am to 7am will be shifted back by 14 hours and land the previous day from to 14h to 17h. "
   However, the data from 7am to 8am will be left there (BAD) instead of being moved to the previous day from 17h to 18h. This requires attention / correction.

Another intersecting case (that probably never happened though) related to the 6pm condition above.
If the data packet contains 20 hours of data and is received at noon. The data is first evenly backtracked back up to 16h the previous day. The overnight data from 6pm to 7am is shifted back by 14 hours, landing between 4am and 5pm the previous day. The data that landed between 4am and 7am would unfortunately stay there instead of being shifted back again by 14 hours. And the initial backtracked data from 4pm to 6pm will also stay there instead of being shifted back by 2*14 hours ...
Im not so sure about that since I think I remember backtracking multiple days in a single packet correctly... Hard to tell, but lets forget about this rare case for now.