felipeam86 / garpy Goto Github PK

Make your garmin data yours again!

License: Apache License 2.0

Python 100.00%

garmin-data garmin-connect sports-data garmin backup sports

garpy's Introduction

Garpy: Make your garmin data yours!

garpy is a simple app used to backup your data from Garmin Connect. It can be used to do incremental backups of your data from Garmin Connect or to download one specific activity.

Incremental backup of activities

The first time you use it, all the activities found on your Garmin Connect account will be downloaded to the directory that you specify. Afterwards, each time you run the command, only the newly available activities will be downloaded.

The command is used as follows:

garpy download {backup-dir}

Behind the scenes, this is what will happen:

garpy will prompt you for your password and will then authenticate against Garmin Connect.
It will first fetch the list of all your activities from garmin.
It will check which activities have already been backed up on the given backup-dir
It will proceed to download all the missing activities.

Downloading one activity from its ID

If you wish to download only one activity or simple you want to refresh an already downloaded activity, use the '-a/--activity' flag as follows:

garpy download --activity 1674567326 {backup-dir}

This will download the activity in all existing formats to the given backup_dir

Full CLI options

For more detailed usage, invoke the '--help' command:

$ garpy download --help
Usage: garpy download [OPTIONS] [BACKUP_DIR]

  Download activities from Garmin Connect

  Entry point for downloading activities from Garmin Connect. By default, it
  downloads all newly created activities since the last time you did a backup.

  If you specify an activity ID with the "-a/--activity" flag, only that
  activity will be downloaded, even if it has already been downloaded before.

  If no format is specified, the app will download all possible formats.
  Otherwise you can specify the formats you wish to download with the
  "-f/--formats" flag. The flag can be used several  times if you wish to
  specify several formats, e.g., 'garpy download [OPTIONS] -f original -f gpx
  [BACKUP_DIR]' will download .fit and .gpx files

  Options:
  -f, --formats [original|gpx|fit|tcx|kml|summary|details]
                                  Which formats to download. The flag can be
                                  used several times, e.g. '-f original -f
                                  gpx'
  -u, --username {username}       Username of your Garmin account
  -p, --password {password}       Password of your Garmin account
  -a, --activity {ID}             Activity ID. If indicated, download only
                                  that activity, even if it has already been
                                  downloaded. Otherwise, do incremental update
                                  of backup
  --user-agent {user_agent}       User agent to be used by requests
  --help                          Show this message and exit.

Installation

garpy requires Python 3.7 or higher on your system. For those who know your way around with Python, install garpy with pip as follows:

pip install -U garpy

If you are new to Python or have Python 2 installed on your computer, I recommend you install Miniconda. To my knowledge, it is the simplest way of installing a robust and lightweight Python environment.

Acknowledgements

The library is based on garminexport. I borrowed the GarminClient, refactored it to my taste and created a package from it.

garpy's People

Contributors

Stargazers

Watchers

Forkers

8ch9azbsfifz 0r jamesmstone lgtm-migrator martinca vincentdavis

garpy's Issues

Credential issues again

Hi
Looks like the end of the road..

Access denied

This website is using a security service to protect itself from online attacks.

Ray ID: 6e6bbe79ec1f4f87
Timestamp: 2022-03-04 15:46:19 UTC
Your IP address: 66.251.179.200
Requested URL: sso.garmin.com/sso/signin?service=https%3A%2F%2Fconnect.garmin.com%2Fmodern
Error reference number: 1020
Server ID: FL_45F48
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36

ConnectionError: authentication failure: did you enter valid credentials?

2023-05-25 15:16:53 INFO - Authenticating GarminClient(username='[email protected]', user_agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36')
Traceback (most recent call last):
  File "/usr/local/bin/garpy", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/garpy/cli.py", line 88, in download
    username=username, password=password, user_agent=user_agent
  File "/usr/local/lib/python3.7/dist-packages/garpy/client.py", line 97, in __enter__
    self.connect()
  File "/usr/local/lib/python3.7/dist-packages/garpy/client.py", line 117, in connect
    self._authenticate()
  File "/usr/local/lib/python3.7/dist-packages/garpy/client.py", line 142, in _authenticate
    "authentication failure: did you enter valid credentials?"
ConnectionError: authentication failure: did you enter valid credentials?

Error stating backup directory path is invalid

Hi @felipeam86

Thanks for creating this and sharing it. I was able to install easily but am running into an error with actually trying to download data.

OSError: [Errno 22] Invalid argument: 'c:\users\selene\downloads\2021-04-28T12:52:38+00:00_6687511285.fit'

It states this error no matter what I specify as the backup directory or what other options I do or do not include. Any thoughts?

Thanks,
Selene

"mkl" == "kml"?

Hi,

Garpy supports formats 'original', 'mkl', 'details', 'gpx', 'summary', 'tcx'. However, mkl is a KML file and surely should be named as such as well?

Also, README.md could be updated with the available formats.

Currently "mkl" is missing.

Options:
  -f, --formats [tcx|gpx|original|summary|fit|details]
                                  Which formats to download. The flag can be
                                  used several times, e.g. '-f original -f
                                  gpx'

garpy download --help reports

Options:
  -f, --formats [tcx|gpx|mkl|original|fit|summary|details]
                                  Which formats to download. The flag can be
                                  used several times, e.g. '-f original -f
                                  gpx'

Also, is there a difference between original and fit?

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Getting 400 error or syntax error when I run this

Hi @felipeam86,

I am new to python but worked with a friend of mine who isn't a garmin user but is very well versed in python. He helped me install and run this brilliant code you put together but i am getting some odd error messages that neither him nor I can figure out. He thinks the 400 response is probably because of garmin's recent hack or because maybe your program needs to be updated (maybe because they changed something after their recent outtage). I've attached a picture for your review and consideration.

Great job buddy! Hope you can help me get all my fit files so I can run some great analysis in Golden Cheetah.

Download fails with error code 402 (payment required)

Hello,

Thank you for making such a useful package. I use it daily to summarize and analyze my workouts, and it has helped enormously in my fitness journey. The garmin site falls far, far short of what I needed, so your package really fills a gap.

Today the package stopped working, and I saw the following error output. I believe the HTTP code 402 means "payment required".

2021-02-26 12:07:52 INFO - Downloading the following formats: ('summary',) to this folder: <snip>
2021-02-26 12:07:52 INFO - Querying list of activities
2021-02-26 12:07:52 ERROR - Failed to fetch activities 0 to 99.
Response code: 402

Traceback (most recent call last):
  File "venv/bin/garpy", line 8, in <module>
    sys.exit(main())
  File "<projectdir>/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "<projectdir>/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "<projectdir>/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "<projectdir>/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "<projectdir>/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "<projectdir>/venv/lib/python3.8/site-packages/garpy/cli.py", line 80, in download
    downloader(formats=formats, activity_id=activity_id)
  File "<projectdir>/venv/lib/python3.8/site-packages/garpy/download.py", line 134, in __call__
    activities = Activities.list(self.client)
  File "<projectdir>/venv/lib/python3.8/site-packages/garpy/activity.py", line 155, in list
    for activity in client.list_activities()
  File "<projectdir>/venv/lib/python3.8/site-packages/garpy/client.py", line 222, in list_activities
    response = self.get(
  File "<projectdir>/venv/lib/python3.8/site-packages/garpy/client.py", line 184, in get
    raise ConnectionError(err_message)
ConnectionError: Failed to fetch activities 0 to 99.
Response code: 402

garpy was invoked as follows, where GARMIN_USER and GARMIN_PASS correspond to (working) credentials for the garmin site.

venv/bin/garpy download -f summary -u $GARMIN_USER -p $GARMIN_PASS

Thanks again for your hard work maintaining this package.

Alex

Provide a Dockerfile?

In case you want to provide a Dockerfile:
https://github.com/8cH9azbsFifZ/garpy

Garmin breaks Garpy? now always getting a HTTP: 402

Hello,

firstly, thanks for a super useful tool.
This was working well for me up until ~ 1 week ago https://github.com/jamesmstone/garmin/actions?page=3

I am now however getting the following error: when running:

garpy run garpy download --username *** --password '***' data/garmin

2022-12-21 08:15:29 INFO - Authenticating GarminClient(username='***', user_agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36')
2022-12-21 08:15:36 INFO - Claiming auth ticket
2022-12-21 08:15:47 INFO - Downloading the following formats: ('summary', 'mkl', 'original', 'details', 'tcx', 'gpx') to this folder: /data/garmin
2022-12-21 08:15:47 INFO - Querying list of activities
2022-12-21 08:15:48 ERROR - Failed to fetch activities 0 to 99.
Response code: 402

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/root/.cache/pypoetry/virtualenvs/garpy-il7asoJj-py3.10/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/root/.cache/pypoetry/virtualenvs/garpy-il7asoJj-py3.10/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/root/.cache/pypoetry/virtualenvs/garpy-il7asoJj-py3.10/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/root/.cache/pypoetry/virtualenvs/garpy-il7asoJj-py3.10/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/.cache/pypoetry/virtualenvs/garpy-il7asoJj-py3.10/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/garpy/cli.py", line 113, in download
    downloader(formats=formats, activity_id=activity_id)
  File "/garpy/download.py", line 134, in __call__
    activities = Activities.list(self.client)
  File "/garpy/activity.py", line 159, in list
    for activity in client.list_activities()
  File "/garpy/client.py", line 229, in list_activities
    response = self.get(
  File "/garpy/client.py", line 192, in get
    raise ConnectionError(err_message)
ConnectionError: Failed to fetch activities 0 to 99.
Response code: 402

Please let me know if I can provide any more detailed information.

Aunthentication failure

I randomly get this error, if I repeat the process a few times it will then work fine eventually.
I download 2 sets my self and my wife sometime 1 will fail sometimes the other..
I run both in a bash script. Which i now fail on Error, but would ideally like to retry!

Could it be a timing issue? Maybe my internet is slow or poor latency ?
Traceback (most recent call last):
File "/home/pi/.local/bin/garpy", line 8, in
sys.exit(main())
File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/pi/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/garpy/cli.py", line 88, in download
username=username, password=password, user_agent=user_agent
File "/home/pi/.local/lib/python3.7/site-packages/garpy/client.py", line 101, in enter
self.connect()
File "/home/pi/.local/lib/python3.7/site-packages/garpy/client.py", line 114, in connect
self._authenticate()
File "/home/pi/.local/lib/python3.7/site-packages/garpy/client.py", line 139, in _authenticate
"authentication failure: did you enter valid credentials?"
ConnectionError: authentication failure: did you enter valid credentials?

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

https://sso.garmin.com

I believe it is offline or was changed.

{"message":"HTTP 403 Forbidden","error":"ForbiddenException"}

sigh.... not working

➜ ~ garpy download --formats fit -u [email protected] -p password -a 12xxxxxxx
2023-09-26 14:31:42 INFO - Authenticating GarminClient(username='[email protected]', user_agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36')
2023-09-26 14:31:43 DEBUG - POST request sent to https://sso.garmin.com/sso/signin?service=https%3A%2F%2Fconnect.garmin.com%2Fmodern
2023-09-26 14:31:43 DEBUG - Request headers: {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8', 'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate, br', 'origin': 'https://sso.garmin.com', 'Content-Length': '66', 'Content-Type': 'application/x-www-form-urlencoded'}
2023-09-26 14:31:43 DEBUG - Request data: username= username%40gmail.com&password=password&embed=false
2023-09-26 14:31:43 DEBUG - auth ticket url: 'https://connect.garmin.com/modern?ticket=ST-0362034-LUiLPhKH6UdSgt6nXuYZ-cas'
2023-09-26 14:31:43 INFO - Claiming auth ticket
2023-09-26 14:31:45 INFO - Downloading the following formats: ('original',) to this folder: /Users/username/activities
2023-09-26 14:31:45 INFO - Fetching summary information for activity: '12xxxxxx'
2023-09-26 14:31:45 DEBUG - -->GET request sent to URL: https://connect.garmin.com/proxy/activity-service/activity/12xxxxxxx with params: None
2023-09-26 14:31:46 ERROR - Failed to fetch 'summary' for activity id 12xxxxxx.
Response code: 403
{"message":"HTTP 403 Forbidden","error":"ForbiddenException"}
Traceback (most recent call last):
File "/Users/username/.pyenv/versions/3.9.1/bin/garpy", line 8, in
sys.exit(main())
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/garpy/cli.py", line 91, in download
downloader(formats=formats, activity_id=activity_id)
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/garpy/download.py", line 141, in call
activity = Activity.from_garmin_connect(activity_id, self.client)
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/garpy/activity.py", line 108, in from_garmin_connect
response = client.get_activity(activity_id, "summary")
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/garpy/client.py", line 228, in get_activity
response = self.get(
File "/Users/username/.pyenv/versions/3.9.1/lib/python3.9/site-packages/garpy/client.py", line 203, in get
raise ConnectionError(err_message)
ConnectionError: Failed to fetch 'summary' for activity id 12091532009.
Response code: 403
{"message":"HTTP 403 Forbidden","error":"ForbiddenException"}