Giter VIP home page Giter VIP logo

tagtracker's Introduction

TagTracker by Julias Hocking

This is a simple tag tracking/data gathering system for bike parking operations for use at Victoria, BC's downtown bike parking service.

Some important assumptions the program makes:

  • Opening hours don't cross midnight - the program automatically decides whether to start a new .dat based on the computer's date
  • Each tag has a unique identifier that starts with at least 1 letter that relates consistently to the tag's colour
  • Tag names don't match any of the command keywords listed below
  • Each tag separates into two pieces, one for the bike and one for the customer, like a coat check
  • Tags are reused day-to-day but not within a single day

The "intended" workflow using the tracker is:

  1. bike checks in - bike receives half tag, owner receives half tag
  2. log check-in with tracker
  3. bike checks out - half tags are reunited
  4. log check-out with tracker
  5. tag is placed in some kind of return basket and not reused that day

LICENSING

This program is largely covered by the GNU Affero General Public License v 3, with this important caveat: Notwithstanding the rest of its licensing provisions, this code may not be used in a commercial (for-profit, non-profit or government) setting without the copyright-holder's written consent.

SETUP

See installation instructions in wiki

USAGE - basic commands / short versions

Usage is described when you run the help command in TagTracker

tagtracker's People

Contributors

tevpg avatar ironwoodcall avatar frantarkenton avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar

Forkers

frantarkenton

tagtracker's Issues

Implementation: "Stay" or "Tag" (or "Bike") class

Consider 'Stay' (or 'Visit') as a class. Attributes time_in, time_out, tag, biketype [regular/oversize]; methods length, ....

Because there would be a 1:1 between instances and unique tags, this could as readily be a Tag class, which is also a real-world object, in which case could include static methods for things like: canonical_tag

logfile: make backup before overwriting

When saving logfile:

  1. delete any {filename}.bak;
  2. rename {filename} to {filename}.bak;
  3. write current data to {filename}.

This in case of power zot or program crash or other weirdness when saving

use fix_hhmm

fix_hhmm can replace valid_time plus some of the time parsing that happens around it. Can use to check times when reading log, checking input, etc.

Right now valid_time() is stubbed to simply call fix_hhmm; once things that call it are updated it can be removed.

fix_hhmm is implemented in branch

    def fix_hhmm(inp:str) -> str:
    """Convert a string that might be a time to HH:MM (or to "")

    Return is either "" (doesn't look like a valid time) or
    will be HH:MM, always length 5 (i.e. 09:00 not 9:00)
    """

Audit and Stats formatting

I think that formatting for the Audit and Stats screens can be improved. I will play with this & then give you a pull request

sample/test data

If we had a dir of sample data it could help people who want to use tagtracker themselves (or can help us when we want top test it). I picture something like

$ python tagtracker.py sample

which copies the sample data dir into a temp dir, naming a logfile appropriately for today, and starts up tagtracker with that temp dir as its data dir.

The sample dir could have records up until something like 13:15 for that day

Part of this would logically be to separate the program dir (.py files. cheat sheet, etc) from the data dir location (normal_tags.cfg, logs file etc)

Audit & other reports for City Hall as static text in PUBLISH folder

This would make current audit info available on the ipad. This is especially useful when a person is walking around the valet reconciling system data to what's on the floor.

Effort: surprisingly little! (Except for the complications arising from use of iprint() -- see comment below)

Proposal:

  • when TT saves, it also saves an audit report in text format on the shared google drive.
  • ipad browser (or a person's phone for that matter) reads the file.

Comments

  • opening a link to a google drive text file puts it inside a google drive context, which may not be desirable (because it uses screen real estate)
  • opening as a file:// is superior, but this requires that the remote viewer has the shared google drive content presented as part of its filesystem. Don't know how fussy this is for ipad, it would certainly not be part of valets' phones.
    • if it were accessed in the filesystem, then could write as html, which means ablity to include colours etc
  • embedding the google drive text file in a google sites page makes a good presentation, and the google site seems to (re-)read the content from the embedded text file with each page refresh. So this is a good solution, though it does require that the Valet google account set up a google sites page with the info on it.

Of course, the great way to do this is to have a server available, and a script that processes the TT datafiles on the fly. And while we're doing that, then of course allow the ability to update through a web interface (contention dealt with through opportunistic concurrency control). Mmmmm, nice.

Ties to (Issue #53)

Config: Missing parentheses

TrackerConfig line 66 -- header string split between lines needs parentheses to be actually processed as one

convert_time (replaces minutes_to_time_str and time_str_to_minutes)

I want to replace both of these with a single function

def convert_time(time_in:Union[int,str], as_number:bool=False) -> time_out:Union[str,int]:

which will detect whether its input is minutes (int|float) or HHMM (str) by type, and return either minuutes or HH:MM depending on the as_number flag.

I think this could also replace fix_hhmm

lint check for main data structures

A routine that scans the main data structures looking for:
check-outs without check-ins
impossible times
invalid or unavailable tags

I only suggest it because it's cheap and easy, and buys us some comfort

separate script for running stats etc

A separate script that could run (e.g.) summary stats. This script would take a fileglob or list of logfiles on command line and be able to process multi-days stats.

This might or might not also provide the stats function that (main) uses -- i.e. (main) would import and use this for its reporting

presumably this could also do the other kinds of reporting:

  • log reporting style
  • extra stats (high-water, etc)
  • diary style (everything, in order) <-- this one seems likely unnecessary

Improve parser/dispatcher

I think we could make a data structure for actions (commands) plus a parser that uses regular expressions to make the system not require separate globals for every action, and to make the parsing more straightforward and robust. Would also want to separate the parsing from the dispatching (i.e., executing the action).

So, something like:

  • config: actions keywords and variants are defined in a dictionary
  • main: a command line is parsed into action, action_tag, action_decorator [dunno if need that last one], where
    • action is a canonical action (not a variant)
    • action_tag is canonical representation of the tag (if tag needed)
    • action_decorator is a token that might follow the tag (like an exclamation point, to make it make the change wthout asking for confirmation. Don't know if we have that)
  • main: dispatch becomes a simple match/case statement calling various functions based on action

option to print tags uppercase not lowercase

Consider having the system list all tags uppercase not lowercase. This does not necessarily have to change the way they are stored in datafiles or processed internally.

Why?

  • I think uppercase tags stand out more.
  • They're bigger, which helps
  • When the system repeats a tag back to the user (who will typically work in lowercase) it's like a confirmation that the system 'got it'
  • I was using tags BI and BL yesterday and it wsa visually ambiguous. BL1 and BI1 just seem more clear to me than than bl1 and bi1

optional program arg: logfile_to_use (for testing)

make an optional arg to the program proper which would be the name of a non-default logfile to use. Seems helpful for testing.
e.g. python tagtracker.py test.log

Also, make error statements during logfile read more informative. E.g. poorly formed tag, unrecognized tag,... this will help with troubleshooting.

crashes for some tags

Found during meeting with JP

Bike tag or command >>> bi4
Traceback (most recent call last):
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 1007, in <module>
    main()
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 962, in main
    tag_check(cmd)
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 864, in tag_check
    query_tag(tag)
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 454, in query_tag
    target = (args + [None])[0]
TypeError: can only concatenate str (not "list") to str

C:\Users\todd\Documents\wk\capital_bike\TagTracker>python tagtracker.py
TagTracker 0.7.2 by Julias Hocking
Previous log for today successfully loaded

Bike tag or command >>> ba1
  Unrecognized tag or command.
  Enter 'h' for help.

Bike tag or command >>> pa1
  pa1 checked IN

Bike tag or command >>> be0
Traceback (most recent call last):
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 1007, in <module>
    main()
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 962, in main
    tag_check(cmd)
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 864, in tag_check
    query_tag(tag)
  File "C:\Users\todd\Documents\wk\capital_bike\TagTracker\tagtracker.py", line 454, in query_tag
    target = (args + [None])[0]
TypeError: can only concatenate str (not "list") to str

cityhall_2023-05-11.log

minor cleanup tasks

Cleanup Tasks

simplify get_date() and get_time()

import datetime
...

def get_date() -> str:
    """Return current date as string: YYYY-MM-DD."""
    return datetime.datetime.today().strftime("%Y-%m-%d")

def get_time() -> str:
    """Return current time as string: HH:MM."""
    return datetime.datetime.today().strftime("%H:%M")

Loop iterations to use items(). E.g., in write_tags():

    for tag in check_ins: # for each bike checked in
        lines.append(f'{tag},{check_ins[tag]}') # add a line "tag,time"

could be

    for tag, time in check_ins.items(): # for each bike checked in
        lines.append(f'{tag},{time}') # add a line "tag,time"

Places to change:

2 s for loops in write_tags()
for loop in calc_stays()
for loop in show_stats()
2 x for loop in count_colours()

Other

  1. specify an encoding when open( files
  2. make minutes_to_time_str() always return HH:MM (never H:MM)

implementation and training plan

Implementation & training plan would address:

  • updates to procedures manual
  • training staff
  • procedure for updating to latest version
  • procedures for city staff to access data
  • how & when do we decommission old spreadsheets etc?

database structure

  • include a separate table of DAILY_SUMMARY (or something like that). Opening and closing times can now be read from the logfile. Other stuff is TBD regarding how it gets in here. (Day end form? Ask in TT? Other?)
    • DATE: PK text ISO8601 date
    • OPENED: text HHMM of when valet opened that day
    • CLOSED: HHMM of when valet closed that day
    • DAY: int 0-6 day of week (for convenience; calculate at load from DATE). Or text day of week (sigh)
    • PRECIP: number, to be bulk-populated from Environment Canada records [can be null]
    • TEMP10AM: same [can be null]
    • SUNSET: HHMM - calculated or bulk-populated [can be null]
    • EVENT: free text - NULL if none, else statement, eg "orange shirt day" "election" "Vic Day parade". Make it short - eg 50 chr max
    • EVENT_PROXIMITY: num -- estimated distance to EVENT in km
    • REGISTRATIONS: # of 529 registrations

And probably create some views that join this with the main table to include things like:

  • bikes_leftover *
  • max_full_regular *
  • max_full_oversize *
  • max_full_total *
  • parked_regular *
  • parked_oversize *
  • parked_total *
  • etc?

* these ones might best be calculated and inserted into the DAILY_SUMMARY table as they would be nasty to calculate in a view. If we were real DB-heads we would recalculate them using stored procedures as a trigger on commits to the details database.

Also: avoid mixed case in table names

stats not showing up

"s" command does nothing when using the attached logfile

cityhall_2023-05-08.seems-to-trigger-bug.log

A few timestamps didn't contain ":" -- that seems to have confused it. Something to perhaps address in the reading of the logfile. When I made them into HH:MM it seemed happy again. ALSO not clear why it got confused by the oddball timestamps.

Goal: when reading files, timestamps should be able to be HMM or HHMM or HH:MM or H:MM

File formats

I would like to change the logfile format:

  • each file start with a # line saying its time/date created.
  • each line would be HH:MM,{tag},IN|OUT (more robust format)
  • filenames would be more like 'tagtracker_YYYY-MM-DD.log' than just date [the root should prob be a config item]
  • maybe only rewrite the file when the data changes (checkin/out/edit/delete functions would mark the data as 'dirty' - but this seems unneccessary)
  • ignore blank lines and any lines that start with '#'. Not sure I can even justify that, I just always want to be able to put comment lines into files.
  • maybe each file end with "# EOF" marker (edited)

I would like to change the tag config file format:

  • ignore blank lines and any lines that start with '#', and any other lines would have to match a tag pattern, e.g. "^ *[a-zA-Z][a-zA-Z]0*[1-9][0-9]* *$"
  • allow multiple tags on a line, whitespace (or comma) separated

For all config files:

  • remove embedded whitespace from the filenames, just use snake case (e.g. "normal_tags.cfg") (e

blur the difference between numeric times and HHMM times

Increasingly it would be nice to work with numeric times. To help get ready for a possible transition, we would adjust functions that either accept or return time (e.g. check-in, check-out or current time) values:

  1. accept times as either HHMM or minutes-since-midnight. This is readily accomplished by having the function type-test the input to see whether a str or int|float
  2. return times as either HHMM or minutes-since-midnight. This is accomplished by adding a "as_minutes" flag to the function header with an appropriate default value (typically as_minutes=False)

This cascades down into some simplification of blocks that call stuff like time_str_to_minutes(), etc.

This relates to #16

consider making all commands only single-line

Consider whether to make TT commands single-line only. E.g., if you enter "edit", right now it sub-prompts, like this:

7:42  Bike tag or command >>> edit
  Which bike's record do you want to edit? (tag ID) >>> wa0
  Do you want to change this bike's check-(i)n or check-(o)ut time? (i/o) >>> in
  What is the correct time for this event? (HHMM or 'now') >>> now
  Check-IN time for wa0 set to 07:42

However, in use, this often seems to happen:

7:42  Bike tag or command >>> wa0
  This bike checked in at 07:42 (2 mins ago)
  Do you want to check it out? (Y/n) >>> wa3
  Cancelled return bike out

IRL people pop commands in and don't necessarily stare carefully at the screen to check the result. In the example above, a person (re)entered wa0, the system asked for confirmation, the person didn't notice, a person entered the next tag (wa3), which was of course not recorded.

The result is that the operator has to scratch their heads to make sure they're then catching up in the right way.

Proposed: Incomplete commands result in id of any errors, plus syntax reminder. E.g.:

7:42  Bike tag or command >>> edit
  syntax: edit <tag> <in|out> <time>

7:42  Bike tag or command >>> edit swa2 700
  Unrecognized tag: 'swa2'
  syntax: edit <tag> <in|out> <time>

7:42  Bike tag or command >>> edit wa2 700
  Must say whether edit check-in or check-out
  syntax: edit <tag> <in|out> <time>

7:52  Bike tag or command >>> edit wa0 in 700
  Check-IN time for wa0 set to 07:00

License

Good to put a file in the project to say what sort of license you are sharing this under. (Presuming you are sharing.) There are a variety of open source licenses. Here's some guidance

Edit when only check-IN recorded

Crashes because when editing a check-in, the Tracker checks whether the proposed new event time is later than an existing check out to prevent physically unfeasible negative-duration stays. Predictably, this throws a KeyError when no check-out event exists to find.

colours

Use colours to increase clarity & visual appeal

What library?

Python supports text colour on the terminal. Based on the JH injunction against installing additional py packages, the best option appears to be colorama which is part of base Python

Usage is pretty straightforward:

import colorama
print( colorama.Fore.YELLOW, colorama.Back.BLUE, "Here is some text", colorama.Style.RESET_ALL )

Style guide

Yah, this is important! If we use colours inconsistently, then we create an effect that's unappealing at best and nausea-inducing at worst. Style guide started here in the wiki.

convert spreadsheet records to compatible format

Convert the existing google sheets daily tracking data to textfile format compatible with to-be bulk analysis tools (issue #7 )

Proposed Method

  • old records are in google sheets. downloaded to excel, unhide all the sheets, then (one by one ugh) save as csv (nb: ^C the UL cell, then paste that as filename; allows the whole thing to be fairly brainless keyboard-only task, at least in libre office)
  • script would grab date from line 1; then for lines that match pattern, create tags records.
  • Ideas for what time within the half-hour block to use as tag time:
    • make them all at the beginning of the time block
    • spread them evenly within the time block (depend how many) --> though need to handle the edge case of > 30 tags [has this ever happened?]
    • place them at 1-minute intervals starting at beginning of time block
    • possibly write some info at top of file in comments like: unmatched tags, this converted from csv, etc

canonical_tag definition regex

Tags may not necessarily have just one letter as a colour abbreviation. The abbreviation the tag spreadsheet uses for "purple" is "pu" to distinguish it from "p" for "pink".

add sounds (beep boop)

Audio confirmation of actions.

  • successful check-in sound
  • successful check-out sound
  • "I need your attention" sound

Chrome OS shell (crosh) seems to support a "sound" command. In the probable event that there isn't a Python library to play sounds on chrome os, then would make os calls at appropriate times.

But of course first place to look would be something like Python's cross-platform playsound library

Considerations: want this to not break if running on Windows or Linux:

  • could call subprocess.run and ignore errors
  • could check OS and not make calls
  • could see if playsound is cross-platform (would need to be installed - maybe a try: on the import? and then gracefully skip sounds if not)

extra protection from data loss

Am I too belt-and-braces? But if we want to genuinely use this and discard paper backup, then I think it's not a bad idea to make it as data-protective as we reasonably can.

  1. new possibly best idea: save to google drive portion of the chromebook's filesystem. This assumes of course that the running system is chrome or windows (with Google Drive on it) but that works for now.

  2. This thing can email a copy of its logfile to every hour or so. This would leverage something super-simple like linux package s-nail and an app pw for a gmail account. Obv would only do this if flagged to do so in config. Here fwiw are my notes on how I enabled it on my server:

Summary of what took WAY too long to figure out:
get an “app password” for the gmail account (requires turning on 2FA)
apt install s-nail
(had installed msmtp but turns out did not need it)
create ~root/.mailrc with permissions 600 (or 640):
  # This seems super minimalist but it seems to work for my purposes.
  # The pw is a gmail "App Password"
  # todd 2022-12-12
  set v15-compat
  set mta=smtp://{sender-gmail-account}:{key}@smtp.gmail.com:587 smtp-use-starttls
  set from="{Name} <[email protected]>"
that’s it!
Example:
# echo this is body text | s-nail -a an_attachment.jpg -s “subject” [email protected]
  1. This could check to see if there's a (particular?) USB key inserted when it saves, and if so then would save a copy onto that. Workflow thus would be: occasionally a staff member inserts CB USB key, presses enter (which does saves as we know), then removes the key

Multi-user (on laptop and ipad)

In this scenario, multiple instances of TT can run simultaneously, sharing datafiles that are on google drive. This is a bit clunky (the boy scout method would be to use an app server) but it could work.

Requirements:

  • run python terminal on ipad (ideas from stackexchange)
  • ipad and laptop both have access to datafiles on Drive
  • TT frequently reads and writes datafiles (e.g. re-reads datafile before performing any command) and can deal appropriately with race conditions resulting in datafile changing between read and write.

A simplified way of doing this is to have the ipad one run in read-only mode

⭐ A statistically reasonable approach would be to have the program:

  • get a command
  • re-read the datafile
  • execute the command
  • write the datafile (if data has changed) <-- (this could use a TrackerDay compare)
  • helpful in this scenario is if re-prompts (like edit --> what tag? --> in or out?, etc) did all their prompting and then separately called a 'make it so' function, so that the refreshed read could happen in between times:
    • it could be a very-not-good thing if the system idled for ten minutes at a sub-prompt after thinking it had done its refresh.
    • another, simpler approach is to encapsulate the changes to check_ins and check_outs, so that that method could do the refresh and re-write.
    • possibility for a race condition still exists but (a) unlikely and (b) would get caught & corrected in an audit

report for blocks (data-entry sheets) and for lookback (recent activity)

print out activities to current time in a format that is helpful for comparing to or filling out the paper spreadsheet &/or the excel spreadsheet. see screen mockups. Likely to use same summary data structure as #9 and #8, possibly also #11

Would need to know the length of time (duration) of the blocks; presumably in config. Therefore simply count blocksize starting at midnight and let the chips fall where they may.

Improve Audit report

Improve audit report. Based loosely on this:

Audit (YYYY-MM-DD HH:MM)

Summary             Regular Oversize  Total
Bikes checked in:       12      10       22
Bikes returned out:      7       9       16
Bikes still at valet:    5       1 	  6

Bikes returned out:
WA       02    04    06 07 08 09 10       13
WB 00 01 02 03 04 05 06 07 08 09 10 11 12 13
WC          03
BE                         08 09       12
BF    01 02 03

Bikes still at valet:
WA 00 01    03    05                11 12
WB    01       04       07 08    10 11 12 13
WC 00 01 02    04 05 06 07 08 09 10
BE 00 01 02 03 04 05 06 07       10 11    13
BF 00          04 05 06 07 08

Consider also:

  • possibly drop lead zero's in tag matrix
  • possibly use a '-' instead of blanks in tag matrix
  • whether to separate (or otherwise indicate) oversize/regular tag rows in matrix

Write bike type info to logfile

Need to write info to the logfile to indicate what tags are for Regular, what for Oversize bikes. This would be read by logfile aggregator; TagTracker will not need to read it (since it gets that info from its config files).

Tricky part: Do we identify bike type on a tag by tag basis (which is the precedent set by TT), which means we're sticking a few hundred extra lines at the bottom of each logfile [but: so what]; or do we change TT config so that regular/oversize are identified by colour prefix not by individual tag?

I lean toward the colour-prefix approach since I think it more closely matches our business. (I'm having a hard time picturing a situation where we would intentionally use the same colour for both R & O bikes, and it would simplify the work of log aggregator.)

In which case, the R/O could simply become part of tag_colour_abbreviations.cfg (which might then be better named tag_colours.cfg; and the normal_tags.cfg and oversize_tags.cfg combined into one file.

This is closely connected to #51

feature: new report: lookback (log of recent activity)

print most recent (e.g. last hour's) activities in chronological order. Likely to use same summary data structure as #10 and #8, possibly also #11

Proposed keywords: ['log', 'lookback', 'l','lo','loo','look']

  • parameters: lookback_start, lookback_end as HHMM
    • if lookback_start is missing, uses rightnow - 1 hr
    • if lookback_end is missing, uses rightnow
Activity log from 13:36 to 14:36

Time    Bike In  Bike Out
13:40    we9
13:41    we10
13:41            ba15
13:45    pa6
13:47    we11   
13:47            we11
14:03            wa0
14:30    bd7

One tag per line. Events at same time, list ins before outs.

feature: stats for high-water-mark, busiest-time, and queue-like vs. stack-like

more stats! Fun ones! Possible keyword "z" [zany stats?]

see mockups in screen mockups text file.

reports on:

  • daily high water mark (most bikes on hand at any given time)
  • busiest times of day
  • both of above as histograms
  • degree to which bike activity was queue-like vs stack-like (Todd is probably the only person who cares about this one)

was unable to delete tag wa7 which was checked in but not out

Delete seems to have inconsistent syntax relative to other commands (well, notably 'edit')

edit:
edit <tag> <in|out> <time>
delete:
delete <tag> <both|out> <y|n> if check in and check-out
delete <tag> ...? if only check-in
delete preferred:
delete <tag> <in|out|both> <y|n|!> <-- (though of course hard to see why a person would say "n")

Below shows the undesired behaviour:

TagTracker 0.7.4 by Julias Hocking

  No datafile for today found. Will create new datafile logs/cityhall_2023-05-18.log.

  Today's valet opening time >>> 730
  Opening time now set to 07:30
  Enter today's valet closing time >>> 2000
  Closing time now set to 20:00

13:28  Bike tag or command >>> wa0
  13:28   Bike wa0 checked IN    (bike #1)

13:28  Bike tag or command >>> d wa0 in
  wa0 has only a check-in (13:28) recorded; can't delete a nonexistent check-out

13:29  Bike tag or command >>> e wa0 in 700
  Check-IN time for wa0 set to 07:00

13:29  Bike tag or command >>> wa0
  13:29   Bike wa0 checked OUT   (at valet for 6:29h)

13:29  Bike tag or command >>> d wa0 out
  Delete wa0 check-out? (y/N) >>>
  Delete cancelled

13:29  Bike tag or command >>> d wa0 both
  Delete wa0 check-in and check-out (y/N) >>>
  Delete cancelled

13:29  Bike tag or command >>> d wa0 in
  Delete cancelled

13:29  Bike tag or command >>> d wa0 in
  Delete cancelled

13:29  Bike tag or command >>> d wa0
  Bike wa0 checked in at 7:00 and out at 13:29.
  Delete (b)oth events, or just the check-(o)ut?  (b/o) >>> in
  Delete cancelled

13:30  Bike tag or command >>> d wa0 both
  Delete wa0 check-in and check-out (y/N) >>>
  Delete cancelled

13:31  Bike tag or command >>> d wa0 out
  Delete wa0 check-out? (y/N) >>> y
  Deleted wa0 check-out

13:31  Bike tag or command >>> d wa0
  Bike wa0 checked in at 7:00. Delete check-in? (y/N) >>> n
  Delete cancelled

13:31  Bike tag or command >>> d wa0 in
  wa0 has only a check-in (07:00) recorded; can't delete a nonexistent check-out

13:31  Bike tag or command >>>

stats for multi-day

Update: combine multiple logfiles into single archive (csv or sqlite3).

Sample dialog:

$ this_script *.log cityhall2023.sqlite3
'Cityhall2023.sqlite3' does not exist.  Create new? (y/N): y
Creating archive
Adding data for:
  2023-05-01
  2023-05-02
  2023-05-03
Done.

$  this_script *.log cityhall2023.sqlite3
Adding data for:
   2023-05-04
   2023-05-06
This will REPLACE data for:
   2023-05-03
Proceed? (y/N): y
Done

$

Proposed data structure (if in database):
DATE, TIME, TAG, INOUT, TIMESTAMP, BATCH
Where:

  • DATE and TIME are separated ISO8601 date & time
  • INOUT is the text "IN" or "OUT"
  • TIMESTAMP is ISO8601 text (e.g. "2023-05-03 14:00") for easy database manipulation, set by trigger from DATE and TIME
  • BATCH uniquely identifies the records that were updated at a particular use of the script; suggested YYYYMMDDHHMMSS when the script was invoked
    With integrity constraint
  • UNIQUE DATE+TAG+INOUT (Can't have more than one in or out record for a given tag on a given day)

Original:
A separate script that would process multiple logs to

  • produce multi-day summary stats &/or
  • produce csv files of data in excel-friendly structure for analysis & charting

missing f-string f's

When splitting lines up to fit within the character limit recently I neglected to add two necessary f's before the new string literals in error messages in read_tags

use pendulum module for datetime

IMO a big miss for python is how it implements date time type operations... datetime objects, time objects, time tuples, unix epoch (seconds from some date in 1970!!!), then add in timezones and pytz module... It feels messy. Consider using this module:

https://pendulum.eustace.io/

its the closest thing to a clean datetime interface.

Save data to "publish" folder

What

Periodically save data to a separate folder in the filesystem

Why

  • makes audit info available on the ipad (or phones) (Issue #85)
  • provides some data protection in case of massive system failure (Issue #34)
  • makes data easily available to City or other downstream users
  • allows things like logfile aggregator to run in an environment that is locked down against change (presuming this folder is made read-only)

How

  • Config would identify a "publish" folder.
  • if present, then TT would periodically
    • re-save logfile to the publish folder
    • save an audit report (as text or html) to "publish" folder
  • Chromebook would have to be set up such that the "publish" folder is in its Google Drive sync area

tweaks based on usage May 15

Tweaks &/or possible bugs based on use at City Hall 2023-05-15

Arranged in approx order of priority

PRIORITY 1 group

DONE - Allow delete a tag that has only a check-in

  • DONE (Is it possible to delete a check-in when there is a check-out? This would be an integrity error.) <-- is ok now
  • DONE - DO NOT ALLOW a CHECK_OUT EARLIER THAN CHECK-IN: this leaves logfile in an error state. <-- Seems ok
  • DONE - busy-ness does not appear to count check-outs, just check-ins
  • DONE change use of var time (name of a py module) -- changed to "atime"
  • DONE add "more stats" to help screen
  • DONE all prompts need to be in PROMPT not SUBPROMPT colour (too dim to see otherwise). In general colours on the linux on chromebook are very different. See colour styles in the wiki

PRIORITY 2 group

  • DONE Longest stay with list of tags -- the list of tags can be very long. Too long. Truncate if too long
    • "wa3, be2, wb4 & 12 others", or
    • "14 tags"
  • DONE Form command - list wraps onto a second line when lots ... prettify it?
  • DONE - confirm that program does something appropriate when reading log file about duplicate check-ins (or outs)
  • DONE when check in, put something at end in normal type like (bike # 14 - 12 bikes in valet)
  • DONE when check out, put something at end in NORMAL like (stayed at valet 3:14h)
  • DONE maybe L (log) is the diary thing and R (recent) is the recent ins/outs
  • DONE - add valet opening hours to commands, datafile read and datafile write

PRIORITY 3 group

  • periodically save a copy of the file to shared_files_path (e.g. /mnt/chromeos/GoogleDrive/MyDrive/tagtracker_logs/... etc.) Alternatively, do this with rsync in cron
  • csv2tracker -- make sure looking for, reporting same bike checked in 2x
  • csv2tracker -- look for check-out before check-in
  • for edit, delete, give a Q-style synopsis before further prompts. Maybe extra-indented
  • use pretty_time consistently in output
  • for anything that takes start and end, would it be better if it were
    • command [as_of] | [start as_of]
  • form command might benefit from start and end times. Maybe recent also.
  • for Q show extra info (like: duration of stay [so far])

PRIORITY 4 group

  • rework help screen to show the optional args
  • "undo" command reverses the most recent check-in, check-out, delete or edit
  • Allow edit a tag that is not yet checked in. Or consider optional [time] arg to checkin/out
  • form and audit reports: consider putting # of tags (on that line) somewhere on each line of tags
  • Recent - maybe default is half an hour instead of an hour? (so things don't run off the screen)
  • In all the various "answers" probably consistently put time first then other info

PRIORITY SOME DAY MAYBE

  • Consider what a GUI would look like

Summary stats: new items, reformat, some refactor

calc_stays passes a list back but also a bunch of info through side-effect globals that are set here and then read in the caller.
This seems messy. Stats calculations should be refactored.

Also I think stats should include all bikes, not just those that have been returned to owners. Consider this as an example:

  • We have two bikes on hand. Both arrived at 9am. One returned out at 10am, the other is still here (e.g. at 13:30).
  • I contend that the longest stay is 4.5 hours (the bike still here), not 1 hr.
  • I would want the stats report though to say something like "As at 13:30, with x bikes still on hand."

enhancements to `help` info

Add more detailed online help

In addition to help command, add detailed info for specific commands.
E.g.

Bike tag or command >>> help audit
  audit [time]
  Prints an audit report showing what bikes are currently at valet, what tags have been used,
  and what bikes.  Optionally provide a [time], in which case the report shows the conditions
  at that time.  If [time] is not given, uses the current time.

Reduce redundancy

Right now the help/usage info is in three places:

  1. config file
  2. readme.md
  3. printable cheat sheet

I suggest putting it in one place - probably a standalone file - which config reads, and readme.md references by URL. The PDF will simply have to get recreated occasionally.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.