bikedataproject / go-file-processing-daemon Goto Github PK
View Code? Open in Web Editor NEWDaemon service to decode Garming .FIT files and store the data as contributions for our service.
License: MIT License
Daemon service to decode Garming .FIT files and store the data as contributions for our service.
License: MIT License
No it's not.
As talked about during the Demo Day and during meetings, it would be really nice to implement Google Location History as a way for contributors to open-source their own Google data, so that it could be used with their permission to have real-world use.
I have done some research, and the Google Location History files are very useful for the bikedataproject. The data dump comes in a highly compressed .zip
-file (22.6MB for 453MB of JSON
) which would be very nice for users to be able to upload large amounts of data in a very short amount of time.
Playing around with Go a little further, I discovered that the main JSON
file is just a large file (split up per 1GB according to Google) containing location, timestamp, type of activities that are predicted & their confidence as a percentage. Using a loop over this data, we can extract all points where the activity ON_BICYCLE
has a confidence of at least 50% (can be tweaked though) and create trips based on these points.
Some days contain wrongly classified activities, but this is often just 1-5 points and can be filtered out if there are not enough points in this trip. The Google Location service (dump) contains datapoints between every 1 second to 5 minutes, depending on the movement etc. This makes the trips not super accurate, but more a rough estimate of the trajectory.
We could make another file upload option for users. There could be 2 ways forwards:
.zip
-file. This would be beneficial for the user, as the upload will go very fast. A file between 1-100MB can be uploaded, and the bikedataproject service can then unzip the file in the backend and process the delivered files. The downside of this method is that we will need a lot of validation for malicious or fake files.Locationhistory.json
file, and make them upload just this. The upside to this is that it get's much easier for us. Less validation for malicious files, no unzipping procedure, etc. The downsides are though that this method will take much longer for the user: manual extraction + uploading times will increase drastically (500MB vs 22MB upload as an example).My preference goes out to option 1, though can be discussed.
Not applicable.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.