Giter VIP home page Giter VIP logo

strava-uploader's Introduction

Runkeeper to Strava Uploader

Uses the Strava v3 API (documented here) to upload GPX and CSV activities exported from RunKeeper.

Borrows liberally from @anthonywu's Strava API Experiment and @marthinsen's Strava Upload projects. Uses @hozn's stravalib to interact with the Strava API. Thanks to all.

Usage:

  1. Register Strava application
    First, you need to register an application with the Strava API service. Go to the Strava API Management Page, and create a new application. Note the Client ID and Client Secret - you will need them later.
  2. Get data from Runkeeper
    Next, you need to get your data from Runkeeper. Go to the Settings page, and look for "Export Data" near the bottom. Define your time range, wait a minute or two, and then click download. Unzip the file - the directory should have .gpx files for all of your GPS-tracked runs, and two spreadsheets - "measurements.csv" and "cardio_activities.csv".
  3. Copy or move the Runkeeper data folder to the script directory
    The directory should be something like "runkeeper-data-export-1234567"). Copy or move it to <script_root_dir>/runkeeper-data.
  4. Install requirements
    Install the requirements - from any shell run pip install -r requirements.txt
  5. Get authorization from Strava
    Next, we need to get an Authorization Token from Strava for your Athlete account.
    Make sure to add CLIENT_ID=<your_client_id> and CLIENT_SECRET=<your_client_secret> to the file .env, with the codes you pulled from the Strava API Management Page
    Run the command python strava_local_client.py get_write_token. It should open a browser and ask you to log in to Strava. You should then be shown a code - copy this, and add STRAVA_UPLOADER_TOKEN=<your_code> to the .env file with the code you received.
  6. Upload to Strava
    Now we're ready to upload. Run the command python uploader.py and let it run!

A few notes on how this works:

  • The script will crawl through the cardio activities csv file line by line, uploading each event.
  • Right now it handles runs, rides, walks, swims, hikes and elliptical exercises. You can add more - be sure to grab the RunKeeper definition and the Strava definition and add to the activity_translator function.
  • If there is a GPX file listed in the last column, it will look for that file in the directory. If there is no GPX file, it will manually upload using the distance and duration data listed in the spreadsheet.
  • Strava's API rate-limits you to 200 requests every 15 minutes, and 2000 daily requests. The uploader.py script will automatically wait for 15 minutes when the upload count hits 199. This is probably too conservative - feel free to adjust.
  • It will move successfully uploaded GPX files to a sub-folder called archive.
  • It will try to catch various errors, and ignore duplicate files.
  • It will log everything in a file strava-uploader.log.

Misc other notes:

  • Do NOT modify or even save (without modification) the CSV from Excel. Even if you just open it and save it with no modification, Excel changes the date formatting which will break this script. If you do need to modify the CSV for some reason (e.g., mine had a run with a missing distance, not clear why), do it in Sublime or another text editor.
  • I personally ran into a few errors of "malformed GPX files". You can try opening the file in a text editor and looking for issues - look for missing closure tags (e.g., </trkseg>) - that was the issue with one of my files. You could also try to use other solutions - some ideas that solved other issues here.

Updates specific to this branch

You can use this script to upload a non-Runkeeper file in CSV format. The current Runkeeper CSV file format includes the following columns: Activity Id, Date,Type, Route Name, Distance (mi), Duration, Average Pace, Average Speed (mph), Calories Burned, Climb (ft), Average Heart Rate (bpm), Friend's Tagged, Notes, GPX File. If you wish to upload a non-Runkeeper file you have to create a cardioActivities.csv in this folder containing at least the following columns: Activity Id, Date, Type, Distance (mi), Duration. The non-Runkeeper file must have matching column names to the Runkeeper original! The GPX file if included should be a filename located in the same folder.

Some specific information about formatting requirements:

  • The Activity Id is just an internal identifier that must be unique per activity. You can use numbers, letters, whatever.
  • Date format must be YYYY-MM-DD HH:MM:SS.
  • Distance should be decimal formatted in miles. This is converted to meters for Strava.
  • Duration must be formatted as MM:SS even for times over 1 hour! So 1 hour 5 minutes 3 seconds = 65:03. This is converted to total duration in seconds in the duration_calc function if you want to use a different format.
  • Some attribute errors are returned when running this script which seem to be related to missing pieces in the create_activity API call; however, the activity is still successfully uploaded if these errors are received.
  • Pip install requirements only works with versions of pip < 9.0.3. I did not update the strava_local_client.py file to work with the updated pip as it was very simple to downgrade pip to a workable version.
  • When manually creating an activity (no GPX file), only the following information is saved: Date, Type, Distance (mi), and Duration. The rest of the file row contents are ignored.

The primary changes from the original branch are updating the CSV file to be read as a dictionary, allowing Runkeeper to change their file format all they want as long as they keep the important column headers the same. I did this because they added some new columns since the original script was written and it was difficult to figure out what the old file format was, and what updates needed to be made to accomodate the new format.

Running tests

cd tests
PYTHONPATH=..:$PYTHONPATH python -m unittest test_get_date_range

strava-uploader's People

Contributors

aunicornwithnolife avatar barrald avatar ccolgrove avatar dannytranlx avatar dependabot[bot] avatar donroyco avatar heatherbmayer avatar kendaleiv avatar redthor avatar stefanfoulis avatar szilard-nemeth avatar timokoole avatar versipellis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

strava-uploader's Issues

Rate limit issue.

I have a new api key with 100 requests every 15 minutes, 1000 daily. I have successfully used the application to upload 20 gpx files every 20 minutes by changing the line counter >= 20. For each file the request counter on the website goes up to an average of 4.5 requests per file. I looked through the code and only see one request per file. Any clue what is going on here?

'err' not referenced before assignment

gives error:

Traceback (most recent call last): File "uploader.py", line 327, in <module> main() File "uploader.py", line 297, in main if upload_gpx(client, gpxfile, strava_activity_type, row['Notes']): File "uploader.py", line 178, in upload_gpx logger.error("Problem raised: {}\nExiting...".format(err)) UnboundLocalError: local variable 'err' referenced before assignment

Uploading failing for activities manually added in Runkeeper

Everything ran smoothly until the uploader got to the first row in my CVS without a GPX file because it was manually added in Runkeeper.

Traceback (most recent call last):
  File "../uploader.py", line 364, in <module>
    main()
  File "../uploader.py", line 352, in main
    if create_activity(client, activity_id, duration, distance, start_time, strava_activity_type, notes):
  File "../uploader.py", line 267, in create_activity
    upload = client.create_activity(
  File "/home/jhebert/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/stravalib/client.py", line 585, in create_activity
    return model.Activity.deserialize(raw_activity, bind_client=self)
  File "/home/jhebert/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/stravalib/model.py", line 130, in deserialize
    o.from_dict(v)
  File "/home/jhebert/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/stravalib/model.py", line 63, in from_dict
    setattr(self, k, v)
  File "/home/jhebert/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/stravalib/attributes.py", line 57, in __set__
    self.data[obj] = self.unmarshal(val)
  File "/home/jhebert/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/stravalib/attributes.py", line 184, in unmarshal
    v = LatLon(lat=v[0], lon=v[1])
IndexError: list index out of range

Unable to connect to www.strava.com

Hello All,
Thanks for the scripts, I did try to use it, but I always have this issue about SSL certificate, I did try both Python 2.7 & 3.7, w/ no Luck. here is the dump of the exception. Any help / fix is very much appreciated.

`C:\Data\Drv_E\Download\strava-uploader>C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\python.exe uploader.py
[2019-10-04 10:43:25,921] [INFO]:Found access token
[2019-10-04 10:43:25,922] [DEBUG]:Connecting to Strava
Traceback (most recent call last):
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
chunked=chunked)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 839, in validate_conn
conn.connect()
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\connection.py", line 344, in connect
ssl_context=context)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\util\ssl
.py", line 345, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 423, in wrap_socket
session=session
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 870, in _create
self.do_handshake()
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\urllib3\util\retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.strava.com', port=443): Max retries exceeded with url: /api/v3/athlete?access_token=bfa38b3261462057d03d8dba1bec2145a67193d5 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "uploader.py", line 336, in
main()
File "uploader.py", line 280, in main
athlete = client.get_athlete()
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\stravalib\client.py", line 235, in get_athlete
raw = self.protocol.get('/athlete')
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\stravalib\protocol.py", line 299, in get
return self._request(url, params=params, check_for_errors=check_for_errors, use_webhook_server=use_webhook_server)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\stravalib\protocol.py", line 215, in _request
raw = requester(url, params=params)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\requests\sessions.py", line 537, in get
return self.request('GET', url, **kwargs)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\requests\sessions.py", line 524, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\requests\sessions.py", line 637, in send
r = adapter.send(request, **kwargs)
File "C:\Users\UserAB\AppData\Local\Programs\Python\Python37-32\lib\site-packages\requests\adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.strava.com', port=443): Max retries exceeded with url: /api/v3/athlete?access_token=bfa38b3261462057xxxxxxxxxxxxxxxxxxx5 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)')))`

Getting a 401 error from the internal server after clicking 'authorize' on Strava

Hello!

I'm getting an error after clicking 'authorize'.

I enabled debugging by inserting "app.debug = True" on line 21, and got this:

python strava_local_client.py get_write_ten 8c3d2fafb8b857d699e7411a2ff8d7a3c4364b38 e8a0d4153cdc683e90322818d938cc9df5ee79bc
On OS X - launching https://www.strava.com/oauth/authorize?state=from_cli&redirect_uri=http%3A%2F%2F127.0.0.1%3A8000%2Fauth&response_type=code&client_id=8c3d2fafb8b857d699e7411a2ff8d7a3c4364b38&scope=view_private%2Cwrite&approval_prompt=auto at default browser
 * Running on http://127.0.0.1:8000/ (Press CTRL+C to quit)
 * Restarting with stat
On OS X - launching https://www.strava.com/oauth/authorize?state=from_cli&redirect_uri=http%3A%2F%2F127.0.0.1%3A8000%2Fauth&response_type=code&client_id=8c3d2fafb8b857d699e7411a2ff8d7a3c4364b38&scope=view_private%2Cwrite&approval_prompt=auto at default browser
 * Debugger is active!
 * Debugger pin code: 236-568-594
127.0.0.1 - - [20/Aug/2016 19:54:26] "GET /auth?state=from_cli&code=cfec5c3b0e6404d12fe7d616c0934c32446b5ea9 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/melt/strava-uploader/strava_local_client.py", line 35, in auth_callback
    code=code
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/stravalib/client.py", line 134, in exchange_code_for_token
    code=code)
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/stravalib/protocol.py", line 106, in exchange_code_for_token
    method='POST')
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/stravalib/protocol.py", line 137, in _request
    self._handle_protocol_error(raw)
  File "/Users/melt/strava-uploader/venv/lib/python2.7/site-packages/stravalib/protocol.py", line 182, in _handle_protocol_error
    raise x
HTTPError: 401 Client Error: Unauthorized [Authorization Error: [{u'field': u'', u'code': u'invalid', u'resource': u'Application'}]]

I am running on Mac OSX, using virtualenv. Any ideas?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.