I like my GoPros and have been using GoPro cloud since around the end of 2019. I wrote a blog post about it in more detail, but in summary, GoPro Plus isn't sufficiently useful to me without some additional tools. This package provides those tools.
Prerequisites:
In order to process streams out of video files, you'll need ffmpeg (most likely available in your system package manager) somewhere on the path.
You'll also need stack to build and install the software.
Install the tool (which requires stack):
git clone https://github.com/dustin/gopro
cd gopro
stack install
Now you've got the gopro tool available. Everything the tool does
requires a database, which is gopro.db in the directory you run it
(i.e., you can have multiple if you just run it from different
locations).
Pick a spot and authenticate:
mkdir ~/gopro
cd ~/gopro
gopro auth
This will prompt you for your GoPro Plus username and password. Once that's done, you're ready to go!
At this point, you can either use the commandline tools, e.g. gopro sync to pull down all the GoPro metadata into your local database, or
you can run gopro serve and do it all via the web interface
(assuming you tell it where its static content is).
All tools take the following arguments:
--db ARG db path (default: "gopro.db")
--static ARG static asset path (default: "static")
-v,--verbose enable debug logging
-u,--upload-concurrency ARG
Upload concurrency (default: 3)
-d,--download-concurrency ARG
Download concurrency (default: 11)
Most of them should be obvious, but --static is the location of the
web media, which you may need to provide if you're running it
somewhere in particular.
Several parameters may be configured with configuration files. The following paths are searched:
- ~/.config/gopro/config.toml
- .gopro.toml (in the current directory)
See the example config to see how this works.
The auth command, as mentioned in the "Getting Started" section
authenticates you with the service. This authentication token lasts
around a day or less.
You should generally only ever need to do this once per gopro.db.
If you have multiple installations, each one will need to be
authenticated.
The reauth command will use a stored token to refresh your
credentials. You may need to do this periodically, but the software
will automatically refresh known stale credentials.. You do not need
to supply your password again (i.e., you can just run this on a timer
or something if you plan to mostly interact with the web service).
This feature is available via the web interface by clicking on "🔒".
The sync command finds all of your recently uploaded media that the
local database does not have yet, and grabs that data.
It also extracts all the metadata and gives you some nice rich local data features.
This feature is available via the web interface by clicking on "🔃".
The refresh command updates the local metadata for specific
MediumIDs. e.g., if you've made changes to highlights or similar on
the GoPro site, you'd use this to retrieve the latest changes.
This feature is available via the web interface when looking at specific item details.
The upload command uploads media from your computer. This is
similar functionality to what your camera might do, but doing it from
your computer gives you a bit more control.
upload in particular does two distinct things:
- It creates GoPro-side media (tells them there's something coming and gets IDs) and associates those media with local files.
- It gets all the bits of those files uploaded and marks things done.
These two things are technically decoupled, but from a UX point of view, you can start an upload, interrupt it, and then start it again later and it will continue from where you left off.
Technically, if all you want to do is resume an upload, you can just
type gopro upload and it will finish any that it knows about, but
are not done.
See also: createupload, createmulti
The download command copies media from GoPro cloud to a local
directory.
The serve command runs a web server and lets you browse and search
your media quickly and easily.
The createupload command only executes the first half of the
upload sequence. This is useful when you want to define a bunch of
uploads for local files and then execute them later. It's also useful
when you have multi-part uploads you want to mix in with the upload
batch and want to just prepare everything first.
The createmulti command creates uploads that span multiple files.
There are two main (and a few minor) use cases for this:
- Video recordings that exceed individual file length and were split into multiple files.
- Timelapse photos (as opposed to video) where a single button press spat out a large number of image files.
While this is less commonly used (at least for me), it's been extremely important functionality that isn't available anywhere other than the camera itself, so it's helped me with ~thousand file time lapses I took with my Hero 4 back in the day, for example.
Usage is a bit tedious, but it's not too bad. First, you have to consider the type of media you're intending to upload. It must be one of the following (exact, case sensitive):
- Photo
- Video
- TimeLapse
- TimeLapseVideo
- Burst
Then, you just run the command with the media you want to upload, in
order (usually alphabetical, so * tends to work). e.g.:
gopro createmulti TimeLapse *.JPG
This will define an upload that includes all matching JPGs that will
be considered a TimeLapse.
After you define your multipart uploads, you can use the regular gopro upload command to complete them.
The fetchall command is essentially sync, but without stopping
when it sees something it's seen before. It will take longer than
sync but should generally do the same thing unless you've rampage
deleted some data from gopro.db.
The cleanup command cleans unprocessed data both on the GoPro side
as well as the local state regarding what is being uploaded.
This only includes files that aren't expected to be fully uploaded.
i.e., items in registered, uploading, or failed states. It does
not include items in transcoding or similar states where GoPro has
the item in full and is processing it.
If you've ever tried to upload things from the web UI and had it tell
you the media's already been upload (when it hasn't), or if you
started an upload you don't intend to finish and want to get rid of
in-progress stuff, the cleanup command will delete all of these
in-progress things.
The config command lets you view and update configuration
parameters. It's got three modes of execution.
To list all configuration parameters and their values:
gopro config
To display the value of the configuration parameter bucket:
gopro config bucket
To set the value of the configuration parameter to
some.bucket.name:
gopro config bucket some.bucket.name
The wait command just waits for in-progress uploads and (GoPro-side)
transcoding to finish. sync does this automatically, but if you
want to wait for some other reason without syncing, this will do it.
The reprocess command tells the GoPro cloud service to reprocess any
uploads that fell into a failed state (as seen by the wait command.
The fixup command allows you to write a SQL query against your local metadata database that will update fields in GoPro's service for any matching rows.
This is a fairly advanced and potentially terribly destructive thing
to do. Everything possible to do here is well beyond the scope of a
README, but at a high level, you write a query that returns rows whose
column names must include media_id for the item you wish to update,
and then additional columns whose names match metadata field names in
GoPro metadata. By the time you get to the point where you're doing
this, you probably know all this.
As an example, when uploading from some source (maybe my phone?) or in some fashion I don't remember, GoPro seemed to not know which camera various media camera from. But the local metadata knew because it read it directly from the GPMF data. I wanted GoPro's metadata to be consistent, so I wrote the following query:
select m.media_id, g.camera_model
from media m join meta g on (m.media_id = g.media_id)
where m.camera_model is null
and g.camera_model is not nullThis query spat out every row where GoPro didn't know the camera
model, but I was able to derive it from metadata. I fed that to the
fixup command and all my metadata on their end was happily updated.
(I'd still need to use refresh to update the local copies).
The backup command will orchestrate a move of all original media
stored in GoPro to your own S3 bucket. Lots of stuff is involved in
setup here including that S3 bucket, an AWS Lambda copy function and
an SQS queue.
This does work, and should be able to move a huge amount of data in a short amount of time. The tl;dr on how to use is:
- Create an S3 bucket in
us-west-2(Oregon) to store all your stuff. Note that this is used for both metadata cache and backups. - Create a lambda function (I called it
download-to-s3, but it's configurable) inus-west-2as a node.js runtime withindex.jscontaining the contents from the file lambda/download-to-s3.js - Set up an SQS queue, also in
us-west-2to capture the results. - Configure destinations for both success and failure to this new SQS queue.
- Configure (using
gopro config)s3copySQSQueueto point to your new SQS queue,bucketto point to your S3 bucket, ands3copyfuncconfig variable is pointed to the correct function name (i.e., change it if it's notdownload-to-s3).
With this AWS-side infrastructure in place, gopro backup should copy
all the things.
This primarily exists as a proof of concept that I hope I won't ever need, as I don't mind using GoPro's storage (it's S3) and paying for the whole thing to be reproduced in my own storage kind of makes the service less useful.
When I do, I suspect I should be able to move my >TB storage from GoPro's buckets to my own with a tiny amount of bandwidth to my house.
The backupall command is similar to the backup command, but grabs
all known media stored on your behalf, including derivatives.
The backuplocal command is similar to the backup command, except
it copies data locally instead of to an S3 bucket (and runs entirely
locally).
Given an argument for the destination path, it will attempt to download all original artifacts for a given medium and once complete, will move the destination directory into place. Once a directory for a given medium is in place, it will not attempt to download the same medium again (i.e., if you delete the directory for a medium, it will be redownloaded).
The backuplocalall command is similar to the backuplocal commnand,
except it fetches all known media instead of just originals.
The processSQS command is automatically run by the backup command
to catch results of asynchronous lambda function calls, but if the
process is interrupted or you just want to make sure you've picked up
everything, this command can be invoked separately without potentially
issuing more copy requests.
The clearmeta command removes any local metadata storage for any
medium whose metadata is backed up on S3. This can save a lot of
space if you have a lot of video assets.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent