Database does not retain data about garmindb HOT 9 CLOSED

You changed the date, but not the number of days it's going to fetch. If you want o override to get a shorter period, change this line:

        days = 365 * 2

to be something less than two years (365 * 2). I assume you are overriding this because you had problems specifying a starting date on the command line. I'll look into that.

from garmindb.

As to your DB being empty, how do you know that? Is it because you using a DB browser program and it shows no data? If so, try reloading the DB in the browser program. With the DB browser I'm using, if I change the DB, I have to reload the DB (disconnect and reconnect) to see the changes. Using the refresh feature in the DB browser does not show the changes.

from garmindb.

Yes I was overriding because I could not get the starting date via commandline to work as you saw.

I used the DB Browser program that you had recommended in the readme file SQLiteStudio - I tried a few times to open the database and close it again in the program (not just refresh as you said) and it still shows as empty. I even deleted the file, removed the DB from the program and re-downloaded the files using the python script and then opened the new DB file with no luck. The tables are all there - just no data that I can see inside of them.

Not sure if this is relevant or not but the program makes the DB folder and then the actual DB file - but no other folders/files in the HealthData folder

from garmindb.

By the way, scrape_garmin.py directly enters the weight data it scrapes into the DB, but for monitoring data it downloads monitoring files that need to be processed by import_garmin_fit.py.

from garmindb.

I'm not sure which issue to put this on, but seems like there is a challenge with initializing the system using os year.

I did this:

#MONITORING_FIT_FILES_DIR=$(FIT_FILE_DIR)/$(YEAR)_Monitoring
MEW_MONITORING_FIT_FILES_DIR=$(FIT_FILE_DIR)/Incoming
MONITORING_FIT_FILES_DIR=$(FIT_FILE_DIR)/Incoming

and then import_monitoring. Now my DB is starting to be populated.

I think the logic is clear for how to fix this long term, but short term after the DB is populated, I will delete all my fit files in Incoming. Then, the undo my switch-a-roo with the directories in Makefile.

from garmindb.

Thanks for this tip, i also had to comment out this line around 125:

#python import_garmin_fit.py -e --input_dir "$(OLD_MONITORING_FIT_FILES_DIR)" --sqlite $(DB_DIR)

Or else it would error out because the old_monitoring_fit_files_dir doesn't exist

from garmindb.

It seems to work!

I switched the Makefile back to master, wiped out incoming, and did this which looks good to me. I have 3 databases all populated with what seems to be ~1 year of history.

[bpb@Benjamins-MacBook-Pro-4 ~ 07:36 AM] ~/repos/garmin_activity/GarminDB (master) > make GC_USER=USERNAME GC_PASSWORD=PASSWORD
python scrape_garmin.py -l --sqlite /Users/bpb/HealthData/DBs -u USERNAME -p PASSWORD  -m "/Users/bpb/HealthData/FitFiles/Incoming"
INFO:scrape_garmin.py:Automatically downloading monitoring data from: 2018-02-13 16:09:59
INFO:scrape_garmin.py:Date range to update according to DB: 2018-02-14 (0)
if ls /Users/bpb/HealthData/FitFiles/Incoming/*.fit 1> /dev/null 2>&1; then \
		python import_garmin_fit.py -e --input_dir "/Users/bpb/HealthData/FitFiles/Incoming" --sqlite /Users/bpb/HealthData/DBs && \
		mv /Users/bpb/HealthData/FitFiles/Incoming/*.fit /Users/bpb/HealthData/FitFiles/2018_Monitoring/.; \
	fi
python scrape_garmin.py -l --sqlite /Users/bpb/HealthData/DBs -u USERNAME -p PASSWORD  -w
INFO:scrape_garmin.py:Automatically downloading weight data from: None
INFO:scrape_garmin.py:Automatic date not found, using: 2017-02-14
INFO:scrape_garmin.py:Date range to update according to DB: 2017-02-14 (365)
INFO:scrape_garmin.py:Creating profile: temp_dir= /var/folders/z3/4dckg1_s68n6dhdlnsggxs6w0000gn/T/tmpJ9SDrQ
INFO:scrape_garmin.py:Creating driver
INFO:scrape_garmin.py:login: USERNAME PASSWORD
INFO:scrape_garmin.py:load_page: https://connect.garmin.com/en-US/signin
INFO:scrape_garmin.py:switch_frame_by_id: gauth-widget-frame-gauth-widget
INFO:scrape_garmin.py:fill_field_by_id: username = USERNAME
INFO:scrape_garmin.py:fill_field_by_id: password = PASSWORD
INFO:scrape_garmin.py:click_by_id: login-btn-signin
INFO:scrape_garmin.py:get_weight: 2017-02-14 : 365
INFO:scrape_garmin.py:load_page: https://connect.garmin.com/modern/weight
INFO:scrape_garmin.py:wait_for_pagecontainer: 
INFO:scrape_garmin.py:Waiting for: //div[@id='pageContainer']
INFO:scrape_garmin.py:click_by_id: lastYearLinkId
INFO:scrape_garmin.py:wait_for_pagecontainer: 
INFO:scrape_garmin.py:Waiting for: //div[@id='pageContainer']
INFO:scrape_garmin.py:click_by_xpath: //button[@class='icon-arrow-left']
INFO:scrape_garmin.py:wait_for_pagecontainer: 
INFO:scrape_garmin.py:Waiting for: //div[@id='pageContainer']
INFO:scrape_garmin.py:click_by_id: lastYearLinkId
INFO:scrape_garmin.py:wait_for_pagecontainer: 
INFO:scrape_garmin.py:Waiting for: //div[@id='pageContainer']
INFO:scrape_garmin.py:DB updated with weight data for 2017-02-14 (365)

from garmindb.

Ok. Closing this bug. Open a new one for any new issues. I will be pushing a cleanup of the Makefile tonight after work.

from garmindb.

Can also confirm adjusting the makefile to get initial data into the DB works, i imported 5 days originally where my data started, then reverted my makefile back to master and removed data from the incoming folder and then ran

make GC_USER=USR GC_PASSWORD=PW

and it was able to see the last day that was imported and started there. It's currently importing my data from the past year or so

from garmindb.