Giter VIP home page Giter VIP logo

manila-hackathon-jupyter's Introduction

Open Traffic Hackathon Data

This project contains information about Open Traffic data sets, plus example analysis techniques using Python and Jupyter Notebook.

Data

As part of the hackathon, OpenTraffic is making avaialble data from 2016 for the Cebu and Manila metro regions. The data is availabe as CSV files with average traffic speeds by day and hour for each roadway segment. Additionally, a Shapefile contains the geometry for each roadway segment. The CSV and Shapefiles can be linked via the "edge_id" found in both files.

All data is avaiablle for download from Amazon AWS S3. Please refer to the "data_list.txt" file for a complete list of data files and corresponding URLs.

The example code in this project demonstrate methods for downloading and processing data files.

Installation of Python / Jupyter environemnt

The examples in this project use Python 2.7 and the Jupyter Notebook. If you already have Python installed on your computer you can install Jupyter using these instructions. Example code in this project makes use of the following python modules: numpy, pyshp, shapely and ipyleaflet.

With Jupyter installed copy the contents of this project into your notebook working directory.

Alternatively you can use a pre-built Docker image with Jupyter and this project pre-configured. With Docker installed, run the following command from the terminal:

docker pull opentraffic/hackathon

Then start the docker image using:

docker run -p 8888:8888 opentraffic/hackathon

Once running you'll see the message:

Copy/paste this URL into your browser when you connect for the first time, to login with a token: http://localhost:8888/?token=4ddef1....

Copy the complete URL into your browser to get started!

Examples

Example 1: Download and process data. This example downloads a single weekly data file for Cebu, and demonstrates techniques for calculating average roadway speeds by day of week and hour of day.

Example 2: Combine CSV data with Shapefile. This example uses a GeoJSON bounding box to select a subset of street segments in Cebu, and calculates the average travel speed using this filtered subset of streets. This demonstates data loading and analysis, and geospatial analysis techniques.

manila-hackathon-jupyter's People

Contributors

kpwebb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

manila-hackathon-jupyter's Issues

Repetitive Data

I'm not sure if its in the way I extracted it - I extracted the .csv.gz files using a zip application and got the .csv files which I could open in Excel - but the data seems to be repetitive, except that the dates in the week change.

Is this an error? Or is the data really this way?

21903477_10155859479394474_71149652_n
21903828_10155859479664474_469397969_n

Missing links for dates in October to early November 2016

I'm not so sure if this is really an issue, but in the data_list.txt, there are no links for some dates in October leading up to early November 2016.

I have yet to check if content of the files before or after the gap has the data for those dates though.

For Cebu:

...
/cebu/week_2016-09-05.csv.gz
/cebu/week_2016-09-12.csv.gz
/cebu/week_2016-09-19.csv.gz
/cebu/week_2016-09-26.csv.gz

-- There's a gap here --

/cebu/week_2016-11-14.csv.gz
/cebu/week_2016-11-21.csv.gz
/cebu/week_2016-11-28.csv.gz
/cebu/week_2016-12-05.csv.gz
/cebu/week_2016-12-12.csv.gz
/cebu/week_2016-12-19.csv.gz
/cebu/week_2016-12-26.csv.gz
...

For Manila:

...
/manila/week_2016-09-05.csv.gz
/manila/week_2016-09-12.csv.gz
/manila/week_2016-09-19.csv.gz
/manila/week_2016-09-26.csv.gz

-- Gap here, too. --

/manila/week_2016-11-14.csv.gz
/manila/week_2016-11-21.csv.gz
/manila/week_2016-11-28.csv.gz
/manila/week_2016-12-05.csv.gz
/manila/week_2016-12-12.csv.gz
/manila/week_2016-12-19.csv.gz
...

I also tried accessing them by link (assuming all dates are Monday dates) to no avail.

P.S. Nice data by the way. I've been looking for one for the longest time.

Thank you so much for sharing this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.