Giter VIP home page Giter VIP logo

paris-bikes's Introduction

"Where to build new bicycle parking spots in Paris? Supporting data-driven decision-making with open data"

TL;DR This is the repository of the CorreAid project paris-bikes in collaboration with the City of Paris. The main tangible output is a prototype of the Paris Parking Demand Index. It is a map that shows bike parking supply and indicators for parking demand (using open data) at the scale of IRIS (the smallest statistical spatial units used in France). Click here to run the prototype (it might take up to a minute to load).

Table of Contents

Context

Metropolitan areas like the city of Paris are facing important challenges transitioning to green mobility. Space to build new infrastructures for alternative transport, like bicycle lanes or parking spots, is limited. Often, existing infrastructure has to be repurposed, and there are lots of local restrictions. In the last years, the City of Paris has implemented ambitious goals in terms of mobility transition, which will continue in the years to come.

Paris, and France on a regional and national level, have also started publishing lots of open data on a multitude of platforms, like data.gouv.fr or parisdata.opendatasoft.com.

This raises the question, if (or rather how) all of this data could be used to support decision-making, e.g. on where to build new bike parking spots.

And this is where CorrelAid comes in! (You don't know CorrelAid? Have a look here)

After a first contact for Open Data Day 2022 the mobility agency "Mission Vélo" (Mission Bike) and the innovation office of the City of Paris teamed up with an international group of CorrelAid volunteers to work on this question.

From July to December 2022, the team explored open data repositories, researched approaches of other cities, brainstormed, discussed, designed, and coded. The results can be found in this repository :)

Paris Parking Demand Index

The Paris Parking Demand Index is a web app that visualizes the number of existing bicycle parking spaces in the City of Paris in relation to metrics that might indicate demand for bicycle parking, such as the number of stores or people entering the metro. Aggregated at the IRIS level, the smallest unit of municipal infrastructure in France, this index helps determine how adequately areas are served in terms of parking facilities, while leaving flexibility as to which exact location they should be built. All data used in this project is from French open data portals.

Click here to access a hosted version of the Parking Demand Index (it might take up to a minute to load).

You can also clone the repository and work with the code yourself. See section Project setup.

Screenshot

Blog article

Would you like to learn more about the project, and how we worked together as a group of volunteers to address the challenge of supporting decision-making in the City of Paris with open data?

Check out the blog article about this project! [Link will follow soon]

Contributors and supporters

Meet the project team! (listed in alphabetical order)

Special thanks go to @friep from CorrelAid for helping us setting up the project, to the CorrelAid organization, to Mélanie, Benoît, and Mourad from the City of Paris for taking the time to collaborate with us and thus giving us the chance to work on this interesting project, and to Gaëll from the Learning Planet Institute for setting up the contact, and to the Learning Planet Institute for having CorrelAid as a student club and enabling us to use rooms and infrastructure.

We thank

        

Project setup

Dash application

The is the web app of the Parking Demand Index. It is an interactive map, which allows you to select and combine different metrics to calculate and visualize parking demand for IRIS sectors in Paris.

To start the Dash application server locally, from the root of this repo, execute:

python paris_bikes/dash_application.py

and navigate to http://localhost:5000 in your browser.

Development environment

We use python>=3.10 and poetry to manage our development environment. poetry manages dependencies and makes it easy to create a virtual environment that is compatible with our package. You can install poetry using conda or pip.

To setup a development environment:

poetry install

If you are on Apple Silicon, you might have to install gdal, a dependency of fiona/geopandas. You can do this with homebrew: brew install gdal

To activate the virtual environment:

poetry shell

Data management

We use DVC for data management and version control. DVC is setup to store our data repository on our project's Google Drive (folder name dvcstore).

Important: We stored the files for dvc in a private cloud from CorrelAid, so you won't be able to access the raw data unless you have access to that cloud. However, the feature.geojson file, which you need to run the dash application, is in this GitHub repository. Since we used open data, all the raw data can be found online, see metadata.md.

DVC is easy to use and follows a syntax similar to git.

To install DVC:

DVC is already installed if you followed the Environment setup. If necessary, check here for other installation methods.

If you are a member of the project, copy the config.local file that can be found in the shared Google Drive folder into the .dvc folder.

To get DVC-managed files from the remote repository:

dvc pull

To track a new data file with DVC:

dvc add data/new_file.csv
git commit -m "Add new data file"

To push new DVC-managed files to the remote repository:

dvc push

To sync DVC-managed files with the current git branch/commit:

dvc checkout

A quick starting guide can be found here.

Deploying the application to Google Cloud Run

The application has been deployed on Google Cloud Run and can be found here.

To deploy a new version:

  1. Make sure you have access to the project (contact @operte)
  2. Make sure you are familiar with the Cloud Run Python guide, that you have the Google Cloud CLI installed and initialized
  3. Make sure that .dockerignore and Dockerfile are up-to-date (and ideally try them out locally using e.g. Docker)
  4. Deploy using gcloud run deploy

Project management

Useful links

Definition of Done

How do we know that we're really done with a task and have not forgotten anything important? In this project, we use the following Definition of Done:

  • Functionality: the code runs / has no defects
  • Code documentation: the code itself is properly documented
  • Data documentation: metadata of raw files should be documented in data/metadata.md
  • Project documentation such as README/wiki is updated where necessary
    • scope/features of project: what does this project do? How can you run it?
    • setup of project
      • data requirements (files, folders)
      • software requirements (packages, other tooling)
    • developer documentation: things like the definition of done, agreed upon standards, ...
  • Peer Review: someone else has had a look at the code and tested it on their machine
  • Git: the code is ready to be merged to the main branch or is already on the main branch
  • Clean Repository: old, outdated code and files are deleted
  • Consistent code style: code is styled consistently and linted using black

See here for more details.

paris-bikes's People

Contributors

akashrajkn avatar dietrichsimon avatar friep avatar katoss avatar liyubov avatar operte avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

liyubov

paris-bikes's Issues

Blog article(s) for CorrelAid blog [Project outcomes for community]

Hey,

Frie suggested we could write for the CorrelAid blog, if there are any learnings we'd like to share with the community :)

Especially for our project, that involved a lot of research, this could be interesting.

So, if there is something you think would be interesting to share on the blog, and/or if you'd like to give writing a blog article a try, please comment here!
This is still in brainstorming phase, feel free to share any thoughts, and then we'll see if and how we implement this. And this could be a great chance also for anyone who has never written a blog article before and would like to give it a try (you'll get support of course). You can also just share topic ideas, maybe someone else would like to write and is looking for ideas :)

Possible topics (non exhaustive):

  • learnings from the systematic review -- interesting articles, approaches?
  • something on the project context (approaching the bike parking topic in a big city like paris)
  • something more specific on one of our deliverables
  • ...

Host app

We should host the app somewhere. Either in AWS (@akashrajkn ) or GCP.

GCP seems to have a free tier which includes App Engine (which would run the app) and GCS/BigQuery (which would store the data) could probably handle our app quite well: https://cloud.google.com/free/docs/free-cloud-features#free-tier

I'm not familiar with AWS. I see it also has a free tier but i am not sure which services we need. I guess S3 for data storage, but which service would run the app?

Scheduled data pipeline

Melanie asked if the maps could be updated as new data is available. I guess this would be possible to achieve by scheduling the data pipelines to connect to the various data APIs and recomputing the feature table.

Do we wanna do that? Looks like a bit of work, but it could be technically interesting. It would be a lot of playing around with whatever hosting environment we choose and also developing the connections to the raw data APIs.

Normalize metrics by area

Benoit suggested we normalize metrics by area, since the area of each IRIS varies considerably.

What do you think?

App issues to resolve until presentation

Hi, I paste this comment from slack, as it is easier to discuss open issues here:)

I think design-wise the app looks great now!
(the slightly moving map because of the labels is not an urgent problem i'd say) :stern2:

In terms of description/intro and title I'd say I can do that in the end once the functionality is finalized.

with regards to the functionality I see a few issues that impact the usability and that we should try to solve before the presentation:

  • the parking spots metric should be moved to a place apart from the other metrics (maybe in the footer, above the normalize slider?)
  • the normalize functionality should be disabled for parking spots
  • with regards to bucketing - i feel that now that the color scheme is different it looks already less confusing. maybe there is another way of cutting off outliers, too? i would say something like this would be nice to have, but I don't know how quick this would be to implement, that's maybe not priority 1
  • I think it would be useful to have the tooltips display not only the iris and selected metric, but also the number of parking spots on default, and when normalized, also the absolute number of the non-normalized metric
  • There is a normalization issue when the number of parking spots is 0. e.g. in iris saint germain auxerrois 4 (see image) the number of shops is 58, parking spots 0, and normalized it is 58
  • nice to have: a way to add and weigh metrics

image

Calibrate variables for IRIS model

The IRIS level model will rely on an aggregation of (potential) determinants of demand for bike parking. One big question is how the different variables should be weighted. The bike flow model could provide useful input as it estimates trips for each flow layer / activity type.

decide how to store data

During the exploration team meeting we discussed where we could store data.
@dietrichsimon already has some preliminary exploration files that would be good to share with others but we need a solution to store the data that these files use.

One option is to use git-lfs (@operte and @Liyubov have experience with it).
I'll do a first experiment with a single dataset. If it works out fine we can then write up some instructions on how to use it (if needed) and push Simon's files into the repo.

Summary Citizen Initiatives

Going through the citizen initiatives Mélanie sent and summarize them in a document on the google drive.

The citizen intiatives Mélanie sent are:

Setup requirements management

As we start writing code it's important to use the same development environments, e.g. by using a package requirements manager.

For python we could just simply have a requirements.txt or something similar that would be easily installable with pip install -r requirements.txt.

@akashrajkn proposed doing this with conda's environment.yml, which would also support R packages.

Agreeing on a deliverable (web app?) for parking demand index

Hey,

so in yesterday's meeting we talked about in which form we could create a deliverable from the parking demand index for the city.

Probably a small interactive web-app would be nice, if there are interested/experienced team members, and if it doesn't take too much time. Otherwise we could of course also do notebooks as deliverable.

We talked about implementing it in shiny (R-based, which @dietrichsimon has experience in), or with FastAPI (python based, @akashrajkn has experience).

Anyone who's interested, feel free to join the discussion :)

Using insights from recent article about improving bike infrastructure in Paris

Hey, in one of our last meetings @ryanrakusin introduced a recently published article about a new method to assess bike infrastructure quality using open data + Paris as a case study. I think it would be great to summarize insights from this paper, and to see if and in which ways this could be interesting for us (especially because they use open, spatial data).

Here is the link:
Where to improve cycling infrastructure? Assessing bicycle suitability and bikeability with open data in the city of Paris.

I assigned @ryanrakusin because he introduced the paper, but of course everyone interested can participate:)

I have read the article and have some thoughts about it (e.g. the introduction of 30km/h limits has big impact on the results of the model. They still use data with the 50km limit). Will try to take notes in the next days. Would be interesting to discuss it further!

Collecting topics for next meeting with City of Paris

If there is something you think we should share with City of Paris during our next meeting, please add it to this issue :)

Some points to start with:

  • Idea about collecting citizen data, possibly using infrastructure of existing app "dans ma rue" (Issue)
  • WIP of our notebooks / analyses
  • Interesting insights from research articles / city planning (e.g. best practices from Amsterdam, bikeability analysis for Paris, comparative table / summary of variables used by others)

Documentation

This is rather for the final stage of the project (but also a good practice during development) -- documenting the code, so that other people understand how to use it :)

Could also be a good issue for beginners (for example, identify if instructions are missing or unclear in the READMEs).

Something like direkt.bahn.guru but for vélib

I discovered https://direkt.bahn.guru/ today, which is an interactive map that shows which direct train connections exist from many train stations in Europe.

I thought this could be an interesting way to visualize which ways people take on vélibs (as we don't have start - end data from other bikes). I.e. for each start station, which end stations are targeted? And then instead of color coding the duration of the ride, like in bahn.guru, we could color code how many rides to respective end stations are taken. This could visualize common ride patterns, and maybe highlight popular destination areas.

This is another map with a similar approach: https://chronotrains-eu.vercel.app/

Might be a bit overkill, maybe just visualizing the most important start and destination stations is enough..?

What do you think?

Streaming data from velib stations

Streaming data from single bicycles and their routes at certain days/time slots may help us to estimate the demand for bicycle parking
Assumption: high density of velib bikes implies that there would also be more "own" bikes

Collect citizen data on missing bike parking

I think directly asking cyclists to report frustrating bike parking situations could be a promising direction to follow.

There is an app called "Dans Ma Rue" ("In my street"), developed by City of Paris, where people can report anomalies with a geotag, a category (like "trash on the street" etc), photo, and description.

I think something similar (or even exactly that, but with new categories for cycling issues) could work well for our purpose.

We could ask Mélanie about this idea, to see if something like this already exists, or if we could get in touch with the developers of Dans Ma Rue etc.

If anyone else is interested in this direction, feel free to comment!

DVC remote isn't working

We can't use the gdrive DVC remote anymore.

When doing dvc pull we get:

ERROR: unexpected error - <HttpError 403 when requesting https://www.googleapis.com/drive/v2/files?q=%28%2713kP6MXL5QBkYS8-aeuJ860kZf4cCS4NF%27+in+parents+or+%271wKK0w2ajY7Ik-5wvEWy5rGJXnoYMNjFz%27+in+parents%29+and+trashed%3Dfalse&maxResults=1000&corpora=drive&driveId=0APzUn7ywXlbhUk9PVA&supportsAllDrives=true&includeItemsFromAllDrives=true&alt=json returned "The attempted action requires shared drive membership.". Details: "[{'domain': 'global', 'reason': 'teamDriveMembershipRequired', 'message': 'The attempted action requires shared drive membership.'}]">

Following that link we get:

{
 "error": {
  "errors": [
   {
    "domain": "usageLimits",
    "reason": "dailyLimitExceededUnreg",
    "message": "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup.",
    "extendedHelp": "https://code.google.com/apis/console"
   }
  ],
  "code": 403,
  "message": "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup."
 }
}

This seems to be some kind of usage limit of google drive.
We can either ask Frie to check if these limitations can be increased, or move to NextCloud.

I'll pick this up after the Paris meeting.

Open Online Data Meetup [Project outcomes for community]

Hey,

CorrelAid has a format called Open Online Data Meetup, where we are invited to present aspects of our project, if we'd like to share something.

What is the Open Online Data Meetup?

  • an online-based meetup series which provides space to share interesting and entertaining stories from the field of data science with other people from the community
  • open format: from short lightning talks to longer workshops, from technical over use-case oriented topics
  • main target group are the correlaid netword and other volunteers, but can be open for NPOs etc

See their youtube channel for past events (I just saw that there is a recording of a meetup on mobility where @mpadge presented :))

So, if you think this might be interesting, if you have any ideas for content, or would like to contribute in any way, please comment here! This could be a great opportunity if you're interested in practicing how to prepare and present data science-related content to a target audience (and of course anyone will get support, especially if this is new for you).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.