Giter VIP home page Giter VIP logo

opensensemap-mass-data's Introduction

OpenSenseMap-Mass-Data

A few programs and python scripts to download and analyze data from opensensemap

Current progress:
python_get_boxes
python_process_boxes
python_master_reader

python get boxes:
A simple way to get boxIDs from sensemap using the sensemapi. Choose your city, add km around it ( innacurate, needs fixing ) add phenomenen and itll save a list of boxIDs to the current folder in a csv file.

python process boxes:
A simple way to process the data from many boxes at the same time. It reads the boxIDs stored in the csv made by python_get_boxes and then it will ask you a few questions about data and then download it.
Itll ask how many days of data you want and what kind of data.
Once its downloaded you should see a folder called "dumpingground", in there is a master.csv file that contains the boxIDs and their associated csv files.
The download progress works by reading the boxIDs created earlier, splitting it up into 4 batches and then downloading them together at once.

python master reader:
Not really supposed to be used, but it was more testing if it was possible to collect and read the data downloaded earlier. It just goes through the master.csv and extracts the values.

What i want to add:
C# based collection, I am more experienced in C# than anything so if I can create a C# solution the speed and quality should go up.
C# api, related to topic above.
More efficient downloading in python_process_boxes.py.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.