Giter VIP home page Giter VIP logo

warsawgtfs's Introduction

WarsawGTFS

Description

Creates GTFS data feed for Warsaw. Static data comes from ZTM Warszawa FTP server and optionally mkuran.pl website. Realtime feeds incorporate data from https://api.um.warszawa.pl.

Features

  1. Line colors
  2. Calendar exceptions
  3. Trip headsigns and On-Request stops
  4. Fares
  5. Adding town names to stop_name
  6. Proper handling of virtual stakes
  7. Getting railway platforms from manually curated data
  8. Getting missing stop positions from manually curated data
  9. Inserting metro schedules from mkuran.pl
  10. Realtime data
  11. Shapes generator: Buses based on OSM Data, Rail/Tram based on my own graph.

Static GTFS script

Requirements

First of all you need Python3 and several modules, included in requirements.txt, so run pip3 install -r requirements.txt.

Then start the script with python3 warsawgtfs.py

Configuration

Run python3 warsawgtfs.py -h to see all possible options with their descriptions. All of those are optional.

Creating GTFS

Run python3 warsawgtfs.py with desired command line options. After some time (up to 1 min, or 15 mins with the --shapes option turned on) the gtfs.zip file should be created.

Produced GTFS feed has additional columns not included in standard GTFS specification:

  • is_data_source in attributions.txt - to indicate that this attribution represents entity that provides data,
  • platform_code in stops.txt - A platform identifier for (most) railway stops (from Google Transit extensions),
  • stop_IBNR and stop_PKPPLK in stops.txt - railway station ids shared by rail operators in Poland.
  • exceptional in trips.txt - Value 1 indicates an unusual trip which does not follow common line's route (e.g. trips to depot) (from Google Transit extensions).

Realtime GTFS script

These scripts are written in go and as such require the go command to be available. You can run the main warsawgtfs_realtime.go script with go run warsawgtfs_realtime.go or by compiling it first with go build warsawgtfs_realtime.go.

There are several dependencies required by this project, all listed in the go.mod file. AFAIK they should be downloaded automatically when running/compiling the project. If they are not, go mod download explicitly downloads the dependencies.

Brigades and positions feeds are mostly based on data from https://api.um.warszawa.pl.

Alerts

Creates the GTFS-Realtime feed with a list of all known alerts.

This mode is enabled by the -a command line flag. Here are all available options:

  • -json: Apart from the GTFS-RT file, write the parsed alerts into a custom JSON format
  • -readable: Use a human-readable GTFS-RT format instead of the binary one
  • -strict: Failing to get more data about an alert from the wtp.waw.pl causes an error, instead of being ignored
  • -gtfs-file SOME_FILE_OR_URL: from which file should available routes be loaded? defaults to https://mkuran.pl/gtfs/warsaw.zip
  • -target SOME_FOLDER: where to put the created files? defaults to data_rt
  • -loop DURATION: if positive (e.g. 1m30s), updates the files every DURATION. defaults to 0s, loop mode disabled.
  • -checkdata DURATION: when in loop-mode, decides how often should the -gtfs-file be checked for changes. defaults to 30m.

Brigades

Creates a file joining a brigade number to a list of trips (for today only), required for positions.

This mode is enabled by the -b command line flag. Here are all available options:

  • -k (required): apikey to api.um.warszawa.pl
  • -strict: any mismatches between api.um.warszawa.pl and gtfs data will become fatal, instead of being ignored
  • -gtfs-file SOME_FILE_OR_URL: from which file should the trips be loaded? defaults to https://mkuran.pl/gtfs/warsaw.zip
  • -target SOME_FOLDER: where to put the brigades.json file? defaults to data_rt

Positions

Creates the GTFS-Realtime feed with a vehicle positions and their active trips.

This mode is enabled by the -p command line flag. Here are all available options:

  • -k (required): apikey to api.um.warszawa.pl
  • -json: Apart from the GTFS-RT file, write the parsed positions into a custom JSON format
  • -readable: Use a human-readable GTFS-RT format instead of the binary one
  • -brigades-file SOME_FILE_OR_URL: path/url to the brigades.json file created by the -b mode. defaults to https://mkuran.pl/gtfs/warsaw/brigades.json
  • -target SOME_FOLDER: where to put the created files? defaults to data_rt
  • -loop DURATION: if positive (e.g. 30s), updates the files every DURATION. defaults to 0s, loop mode disabled.
  • -checkdata DURATION: when in loop-mode, decides how often should the -brigades-file be checked for changes. defaults to 30m.

License

WarsawGTFS is provided under the MIT license. Please take a look at the license.md file.

warsawgtfs's People

Contributors

danieljuschus avatar dependabot[bot] avatar domeqdev avatar marcinbittel avatar michalpazur avatar mkuranowski avatar starsep avatar van1757 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

warsawgtfs's Issues

Route L33 has missing stops

A strange situation, the locations of the stops and the route are correctly specified in the Pliki Rozkładowe, but GTFS is not generated correctly.

image

Brigades - reoccuring issue

The script sometimes crashes with the following:

Failed to parse API response:     {"result":"B\u0142\u0119dna metoda lub parametry wywo\u0142ania"}
json: cannot unmarshal string into Go struct field .Result of type []struct { Values []struct { Key string; Value string } }

Maybe we could also implement backoff in such a case?

Seperate SKM GTFS feed

It would be great if there was a separate GTFS feed for SKM, just like there is for the metro. Currently, developers who want to create an app that only shows train schedules have to download and process the entire GTFS feed for Warsaw, which is a lot of data that they don't need.

Inconsistency in GTFS Data for Warszawa Główna

I've encountered a specific issue in the GTFS data for Warszawa Główna (stop_id: 5909pUnknown). The parent_station field references stop_id 5909, but this parent_station is not present in the dataset.

This absence is causing errors and inconsistencies in data processing. Can you please confirm whether this is an oversight or if it's intended?

Brigades not working

Column order is changed in GTFS, so the brigades creator doesn't work

Solution: use csv.DcitReader to read GTFS when creating brigades table

L lines zoning

# Separate zone for Otwock is required, as L20 & L22 tickets

There is separate zone for Otwock since different price tickets.
Thing is all L lines have a special tarriff just for their trips. Instead of zone "2/O" a "2/LT-S" should be used to indicate that it is zone 2 bus but only long term and special tickets (sold onboard) are accepted. (long term meaning 24h, 72h, monthly etc.)

Then there is fact that some L lines work for free for residents of given area, but no structured data about this exists (only some pdf iirc)

unparsable attributions.txt

In the current attributions.txt, there is an extra ",pl" between the fields for organiziation_name and is_producer in the line for ZTM which should be removed or be part of the organization_name.

organization_name,is_producer,is_operator,is_authority,is_data_source,attribution_url
"Data provided by: ZTM Warszawa (retrieved 2022-05-15 01:01:00)",pl,0,0,1,1,"https://www.ztm.waw.pl/pliki-do-pobrania/dane-rozkladowe/"
"Bus shapes based on data by: © OpenStreetMap contributors (retrieved 2022-05-15 01:01:00, under ODbL license)",0,0,1,1,"https://www.openstreetmap.org/copyright/"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.