Animating scheduled transit trips using the Transitland API and Processing
License: MIT License
We use this GitHub repository to have public discussions about the Transitland platform, including:
Hi there! Just discovered this project — so awesome, thanks! However I'm finding that it fails on larger datasets. The issue appears to be with how the transitland_api.py handles pagination.
For example, the metro rail of Wellington, New Zealand:
If I run
transitflow sam$ python3 transitflow.py --name=wellington_rail --operator=o-rbs-metlinkrail --date=2019-09-09
I can see that there are 10 routes, 62 stops and 3806 schedule stop pairs, and I get no issue.
Now I go to line 17 in transitflow.py and change PER_PAGE from 10000 to 3805, rerun the above command and get a download error:
transitflow sam$ python3 transitflow.py --name=wellington_rail --operator=o-rbs-metlinkrail --date=2019-09-09
INPUTS:
date: 2019-09-09
name: wellington_rail
API key: None
1 operators to be downloaded.
b'o-rbs-metlinkrail' 1 / 1
http://transit.land/api/v1/routes?operated_by=o-rbs-metlinkrail&per_page=3805
10 routes found.
http://transit.land/api/v1/stops?served_by=o-rbs-metlinkrail&per_page=3805
62 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-rbs-metlinkrail&date=2019-09-09&per_page=3805&sort_min_id=0
https://api.transit.land/api/v1/schedule_stop_pairs?date=2019-09-09&operator_onestop_id=o-rbs-metlinkrail&per_page=3805&sort_key=id&sort_min_id=4923869135&sort_order=asc
failed:
0 operators successfully downloaded.
1 operators failed.
failed operators: [b'o-rbs-metlinkrail']
But if I change PER_PAGE to 3806 then there's no issue either. So it appears when the amount of schedule/stop pairs exceeds per_page limit then the download breaks... :(
Also, both those URLS for getting the schedule/stop data are fine. Will have a crack at fixing myself but just putting this out there is case anyone has any ideas!
Hi,
unfortunately it seems the project is broken due to the v1 API being deprecated and the schedule stop pair endpoint is no longer working. (transitland/transitland#396 (reply in thread)). It would be great if it could be updated to use the v2 APIs.
Hi,
I'm trying to work with Budapest transport company (BKK). The data downloads but it fails to create the sketch file and gives me this:
1 operators successfully downloaded.
0 operators failed.
Concatenating individual operator outputs.
Calculating trip segment bearings.
Traceback (most recent call last):
File "transitflow.py", line 376, in
df['bearing'] = df.apply(lambda row: calc_bearing_between_points(row['start_lat'], row['start_lon'], row['end_lat'], row['end_lon']), axis=1)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 2331, in setitem
self._set_item(key, value)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 2396, in _set_item
self._ensure_valid_index(value)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 2378, in _ensure_valid_index
raise ValueError('Cannot set a frame with no defined index '
ValueError: Cannot set a frame with no defined index and a value that cannot be converted to a Series
best
Hi,
I haven't been able to download for operator Champaign-Urbana Mass Transit District . Changing date didn't help ("service_start_date":"2017-08-13","service_end_date":"2017-12-23").
This is a really nice tool. Would appreciate if someone could take a look!
Thanks!
Shuake
failed:
u's-dp1k179mzs-floridaandjamescherry<fljascher~3'
0 operators successfully downloaded.
1 operators failed.
failed operators: ['o-dp1h-champaignurbanamasstransitdistrict']
Concatenating individual operator outputs.
Traceback (most recent call last):
File "transitflow.py", line 379, in
df = concatenate_csvs("sketches/{}/{}/data/indiv_operators".format(OUTPUT_NA
ME, DATE))
File "transitflow.py", line 158, in concatenate_csvs
concatenated_df = pd.concat(df_from_each_file, ignore_index=True)
File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\core\reshape\concat.py
", line 206, in concat
copy=copy)
File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\core\reshape\concat.py
", line 239, in init
raise ValueError('No objects to concatenate')
ValueError: No objects to concatenate
Hello Will,
Thank you for the API it is very nice. I was testing it with different cities in the World, it worked very well in Portland, Or, but I found two issues in toher cities. I tested in Lisbon, Portugal using:
python transitflow.py --name=lisbon --bbox=-9.276933,38.592729,-8.940477,38.803201 --clip_to_bbox
When downloading the transit operators, the API found routes and stops but finds 0 schedule stop pairs. An example of one of the largest operators.
o-eyck-carris 7 / 8
http://transit.land/api/v1/routes?per_page=10000&operated_by=o-eyck-carris
217 routes found.
http://transit.land/api/v1/stops?per_page=10000&served_by=o-eyck-carris
2093 stops found.
http://transit.land/api/v1/schedule_stop_pairs?date=2018-04-26&per_page=10000&sort_min_id=0&operator_onestop_id=o-eyck-carris
0 schedule stop pairs found.
Another test I did was in Santiago, Chile. Here the API has problems downloading the stops data.
python transitflow.py --name=Santiago --bbox=-70.673777,-33.460993,-70.595499,-33.394518 --clip_to_bbox
And it seems it cannot connect:
o-66jc-transantiago 2 / 2
http://transit.land/api/v1/routes?per_page=10000&operated_by=o-66jc-transantiago
383 routes found.
http://transit.land/api/v1/stops?per_page=10000&served_by=o-66jc-transantiago
retry 1 / 5: HTTP Error 504: Gateway Time-out
retry 2 / 5: HTTP Error 504: Gateway Time-out
retry 3 / 5: HTTP Error 504: Gateway Time-out
retry 4 / 5: HTTP Error 504: Gateway Time-out
retry 5 / 5: HTTP Error 504: Gateway Time-out
failed:
HTTP Error 504: Gateway Time-out
1 operators successfully downloaded.
1 operators failed.
I thinking in Lisbon case, it may be a problem with the structure of the GTFS data, and in Santiago maybe the file is too large?
Do you have any clues?
Thanks!
Hi,
I have been playing around with this transit software and have tried many different ways to download the data from my local service operator, Translink and convert it into a map. The examples work fine, however, the operator in my area operates two feeds which ends up being a couple of trillion entries. I assume that it is a server error but sometimes the downloading just stops and I get this error message:
failed operators: [b'o-r-translink']
Concatenating individual operator outputs.
Traceback (most recent call last):
File "transitflow.py", line 404, in <module>
df = concatenate_csvs("sketches/{}/{}/data/indiv_operators".format(OUTPUT_NAME, DATE))
File "transitflow.py", line 184, in concatenate_csvs
concatenated_df = pd.concat(df_from_each_file, ignore_index=True)
File "C:\Program Files\Python37\lib\site-packages\pandas\core\reshape\concat.py", line 255, in concat
sort=sort,
File "C:\Program Files\Python37\lib\site-packages\pandas\core\reshape\concat.py", line 304, in __init__
raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate
I have also tried to change the amount of entries per page, and even setting it to 1 and leaving it on the whole night still returned a similar error message.
You can see the problem with e.g.
python transitflow.py --name="CrossCountry" --operator o-gc-crosscountry
The first train is at 04:44, last train ends at 00:07.
The start/end frames of the trips are calculated by treating minDate-maxDate as mapping to all the frames. But the counts are treated as one count per frame, starting at midnight. This means that the displayed lines/counts do not match the display of the trips.
I have worked around this by editing the csv files to match up the counts to the right times.
Hi,
I was running following example code,
python transitflow.py --name=san_francisco --bbox=-122.515411,37.710714,-122.349243,37.853983 --clip_to_bbox --date=2020-01-01
But I see following error:
I try to fix it, but don't know where is the problem. Would really appreciate if anyone could help me. @bcamper @drewda @irees @meghanhade @Danie
error
`
INPUTS:
date: 2020-01-01
name: san_francisco
API key: None
bbox: -122.515411 37.710714 -122.349243 37.853983
http://transit.land/api/v1/operators?bbox=-122.515411%2C37.710714%2C-122.349243%2C37.853983&per_page=10000
18 operators in bounding box.
18 operators to be downloaded.
b'o-9q8zm-blue~goldfleet' 1 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8zm-blue~goldfleet&per_page=10000
6 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8zm-blue~goldfleet&per_page=10000
6 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8zm-blue~goldfleet&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q9j-commuteorgshuttles' 2 / 18
http://transit.land/api/v1/routes?operated_by=o-9q9j-commuteorgshuttles&per_page=10000
0 routes found.
http://transit.land/api/v1/stops?served_by=o-9q9j-commuteorgshuttles&per_page=10000
0 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q9j-commuteorgshuttles&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q9-caltrain' 3 / 18
http://transit.land/api/v1/routes?operated_by=o-9q9-caltrain&per_page=10000
10 routes found.
http://transit.land/api/v1/stops?served_by=o-9q9-caltrain&per_page=10000
66 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q9-caltrain&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8zn-alcatrazhornblowerferry' 4 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8zn-alcatrazhornblowerferry&per_page=10000
3 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8zn-alcatrazhornblowerferry&per_page=10000
2 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8zn-alcatrazhornblowerferry&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8zr-tidelinewatertaxi' 5 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8zr-tidelinewatertaxi&per_page=10000
3 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8zr-tidelinewatertaxi&per_page=10000
4 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8zr-tidelinewatertaxi&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9qbb-marintransit' 6 / 18
http://transit.land/api/v1/routes?operated_by=o-9qbb-marintransit&per_page=10000
31 routes found.
http://transit.land/api/v1/stops?served_by=o-9qbb-marintransit&per_page=10000
923 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9qbb-marintransit&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q9-actransit' 7 / 18
http://transit.land/api/v1/routes?operated_by=o-9q9-actransit&per_page=10000
174 routes found.
http://transit.land/api/v1/stops?served_by=o-9q9-actransit&per_page=10000
5631 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q9-actransit&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8y-sfmta' 8 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8y-sfmta&per_page=10000
110 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8y-sfmta&per_page=10000
3712 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8y-sfmta&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8z-goldengateferry' 9 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8z-goldengateferry&per_page=10000
8 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8z-goldengateferry&per_page=10000
4 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8z-goldengateferry&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9qc-westcatwesterncontracosta' 10 / 18
http://transit.land/api/v1/routes?operated_by=o-9qc-westcatwesterncontracosta&per_page=10000
16 routes found.
http://transit.land/api/v1/stops?served_by=o-9qc-westcatwesterncontracosta&per_page=10000
227 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9qc-westcatwesterncontracosta&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8-samtrans' 11 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8-samtrans&per_page=10000
98 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8-samtrans&per_page=10000
2259 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8-samtrans&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q9p-sanfranciscobayferry' 12 / 18
http://transit.land/api/v1/routes?operated_by=o-9q9p-sanfranciscobayferry&per_page=10000
15 routes found.
http://transit.land/api/v1/stops?served_by=o-9q9p-sanfranciscobayferry&per_page=10000
15 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q9p-sanfranciscobayferry&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8v-commuteorgshuttle' 13 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8v-commuteorgshuttle&per_page=10000
26 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8v-commuteorgshuttle&per_page=10000
146 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8v-commuteorgshuttle&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q8yy-missionbaytma' 14 / 18
http://transit.land/api/v1/routes?operated_by=o-9q8yy-missionbaytma&per_page=10000
0 routes found.
http://transit.land/api/v1/stops?served_by=o-9q8yy-missionbaytma&per_page=10000
0 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q8yy-missionbaytma&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9qb-goldengatetransit' 15 / 18
http://transit.land/api/v1/routes?operated_by=o-9qb-goldengatetransit&per_page=10000
44 routes found.
http://transit.land/api/v1/stops?served_by=o-9qb-goldengatetransit&per_page=10000
794 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9qb-goldengatetransit&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9qc-capitolcorridor' 16 / 18
http://transit.land/api/v1/routes?operated_by=o-9qc-capitolcorridor&per_page=10000
4 routes found.
http://transit.land/api/v1/stops?served_by=o-9qc-capitolcorridor&per_page=10000
23 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9qc-capitolcorridor&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9q9-bart' 17 / 18
http://transit.land/api/v1/routes?operated_by=o-9q9-bart&per_page=10000
19 routes found.
http://transit.land/api/v1/stops?served_by=o-9q9-bart&per_page=10000
98 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9q9-bart&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
b'o-9qb-sonomacountyairportexpress' 18 / 18
http://transit.land/api/v1/routes?operated_by=o-9qb-sonomacountyairportexpress&per_page=10000
0 routes found.
http://transit.land/api/v1/stops?served_by=o-9qb-sonomacountyairportexpress&per_page=10000
0 stops found.
http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-9qb-sonomacountyairportexpress&date=2020-01-01&per_page=10000&sort_min_id=0
0 schedule stop pairs found.
success!
18 operators successfully downloaded.
0 operators failed.
Concatenating individual operator outputs.
Calculating trip segment bearings.
Traceback (most recent call last):
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2889, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 97, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'bearing'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/generic.py", line 3565, in _set_item
loc = self._info_axis.get_loc(key)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2891, in get_loc
raise KeyError(key) from err
KeyError: 'bearing'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "transitflow.py", line 406, in <module>
df['bearing'] = df.apply(lambda row: calc_bearing_between_points(row['start_lat'], row['start_lon'], row['end_lat'], row['end_lon']), axis=1)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/frame.py", line 3040, in __setitem__
self._set_item(key, value)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/frame.py", line 3117, in _set_item
NDFrame._set_item(self, key, value)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/generic.py", line 3568, in _set_item
self._mgr.insert(len(self._info_axis), key, value)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/internals/managers.py", line 1189, in insert
block = make_block(values=value, ndim=self.ndim, placement=slice(loc, loc + 1))
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 2714, in make_block
return klass(values, ndim=ndim, placement=placement)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 2370, in __init__
super().__init__(values, ndim=ndim, placement=placement)
File "/Users/home/Documents/transitland-processing-animation/venv_transit/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 130, in __init__
f"Wrong number of items passed {len(self.values)}, "
ValueError: Wrong number of items passed 8, placement implies 1
`
Hi,
Thank you very much for this wonderful tool. I've been trying it with San Francisco BART Onestop ID and Chicago bounding box and it worked very well.
However, I am having issue when I try to create an animation of Cairo transit flows. While Cairo Metro has its own Onestop ID on Transitland Feed registry, it seems that it has never been sketched. Whether I use the Onestop ID or a bounding box, as shown below:
(Carte1) MacBook-Pro-de-moi-2:transitflow moi$ python transitflow.py --name=cai --operator=o-stq4-nationalauthorityfortunnels
INPUTS:
date: 2019-11-23
name: cai
API key: None
1 operators to be downloaded.
b'o-stq4-nationalauthorityfortunnels' 1 / 1
http://transit.land/api/v1/routes?operated_by=o-stq4-nationalauthorityfortunnels&per_page=10000
3 routes found.
(Carte1) MacBook-Pro-de-moi-2:transitflow moi$ python transitflow.py --name=Cairo2 --bbox=30.799484,29.819200,31.678391,30.205674 --clip_to_bbox
INPUTS:date: 2019-11-23
name: Cairo2
API key: None
bbox: 30.799484 29.819200 31.678391 30.205674
http://transit.land/api/v1/operators?bbox=30.799484%2C29.819200%2C31.678391%2C30.205674&per_page=10000
8 operators in bounding box.
8 operators to be downloaded.
b'o-stq4-nationalauthorityfortunnels' 1 / 8
http://transit.land/api/v1/routes?operated_by=o-stq4-nationalauthorityfortunnels&per_page=10000
3 routes found.
...it always gives me the same three Value errors, as shown here:
success!
8 operators successfully downloaded.
0 operators failed.
Concatenating individual operator outputs.
Calculating trip segment bearings.
Traceback (most recent call last):
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/common.py", line 377, in _asarray_tuplesafe
result[:] = values
ValueError: could not broadcast input array from shape (0,8) into shape (0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/frame.py", line 2376, in _ensure_valid_index
value = Series(value)
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/series.py", line 248, in __init__
raise_cast_failure=True)
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/series.py", line 3029, in _sanitize_array
subarr = _asarray_tuplesafe(data, dtype=dtype)
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/common.py", line 380, in _asarray_tuplesafe
result[:] = [tuple(x) for x in values]
ValueError: cannot copy sequence with size 8 to array axis with dimension 0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "transitflow.py", line 406, in <module>
df['bearing'] = df.apply(lambda row: calc_bearing_between_points(row['start_lat'], row['start_lon'], row['end_lat'], row['end_lon']), axis=1)
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/frame.py", line 2331, in __setitem__
self._set_item(key, value)
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/frame.py", line 2396, in _set_item
self._ensure_valid_index(value)
File "/Users/moi/opt/anaconda3/envs/Carte1/lib/python3.6/site-packages/pandas/core/frame.py", line 2378, in _ensure_valid_index
raise ValueError('Cannot set a frame with no defined index '
ValueError: Cannot set a frame with no defined index and a value that cannot be converted to a Series
Thank you very much for your help and, again, for this tool.
Cheers!
Hello, I have the following error when I run the tool as shown below - I am using Python 3.6 not sure if this is the issue? Thanks for any help, I am fairly new but very keen to try and make this work!
C:\Users\Anthony\Desktop\transitland-processing-animation-master\transitflow>python transitflow.py --name=GO --operator=o-dpz-gotransit
File "transitflow.py", line 37
print len(lookup_vehicle_types.keys()), "routes found.\n"
^
SyntaxError: invalid syntax
Running transitflow for the operator Cumfybus, i.e. python transitflow.py --name=cumfybus --operator=o-gcw0-cumfybus --recording, currently fails after downloading the data.
Digging into it, it seems to be because the origin_stops from the dataset includes a stop (s-gcw0vhqpk4-blackrodchurchst~redlionstopc) that isn't in the lats/lons, which causes python to throw an exception.
I've worked round it (because claiming the stop is in the mid-Atlantic, while hopefully quite obvious to see, doesn't seem the correct fix :-) with the following mod to transitflow.py:
def generate_output(operator_onestop_id, origin_datetimes, destination_datetimes, durations, origin_stops, destination_stops, route_ids, lookup_stop_lats, lookup_stop_lons, lookup_vehicle_types):
"""This function generates the output table, to be saved later as a csv."""
- origin_stop_lats = [lookup_stop_lats[i] for i in origin_stops]
- origin_stop_lons = [lookup_stop_lons[i] for i in origin_stops]
- destination_stop_lats = [lookup_stop_lats[i] for i in destination_stops]
- destination_stop_lons = [lookup_stop_lons[i] for i in destination_stops]
+ origin_stop_lats = []
+ origin_stop_lons = []
+ for i in origin_stops:
+ try:
+ origin_stop_lats.append(lookup_stop_lats[i])
+ origin_stop_lons.append(lookup_stop_lons[i])
+ except:
+ origin_stop_lats.append(-3.094025)
+ origin_stop_lons.append(53.460255)
+ destination_stop_lats = []
+ destination_stop_lons = []
+ for i in destination_stops:
+ try:
+ destination_stop_lats.append(lookup_stop_lats[i])
+ destination_stop_lons.append(lookup_stop_lons[i])
+ except:
+ destination_stop_lats.append(0)
+ destination_stop_lons.append(0)
Happy to submit it as a pull request, if that is a valid fix, mind you.
Hi, Thank you so much for this repository. It's really useful.
I was trying to install it earlier and was not sure about certain steps following your instructions.
I'm using Linux Ubuntu. The following is the steps to install Processing based on your read me file.
1: Download Processing 3.
2: Download Unfolding Maps version 0.9.9 for Processing 3.
3: Navigate to ~/Documents/Processing/libraries on your machine.
4: Drag and drop the unzipped Unfolding Maps folder into ~/Documents/Processing/libraries.
5: Open Processing, navigate to Sketch > Import Library > Add Libary. Search for "Video Export" and click Install.
6: Quit and re-open Processing.
I'm not exactly sure about the step 3 and step 4. More explicitly, how to find the ~/Documents/Processing/libraries folder? I have Processing 3 installed in my home folder. So, I can find ~/Processing folder, and inside of this folder, I have
devel@hpz4:~/processing$ ls
core install.sh java lib modes processing processing-java revisions.txt tools uninstall.sh
devel@hpz4:~/processing$
So, what shall I do then?
I finally choose to use the lib folder. I mean, I drag and drop the unzipped Unfolding Maps into
~/Processing/lib.
Eventually, when I tried the first example following the read me file, the output is as the following.
devel@hpz4:~/Desktop/python_begin_again/transitland/transitflow$ python2 transitflow.py --name=bart --operator=o-9q9-bart
INPUTS:
date: 2019-03-02
name: bart
API key: None
1 operators to be downloaded.
o-9q9-bart 1 / 1
http://transit.land/api/v1/routes?per_page=10000&operated_by=o-9q9-bart
12 routes found.
http://transit.land/api/v1/stops?per_page=10000&served_by=o-9q9-bart
50 stops found.
http://transit.land/api/v1/schedule_stop_pairs?date=2019-03-02&per_page=10000&sort_min_id=0&operator_onestop_id=o-9q9-bart
9301 schedule stop pairs found.
success!
1 operators successfully downloaded.
0 operators failed.
Concatenating individual operator outputs.
Calculating trip segment bearings.
Total rows: 9301
Counting number of vehicles in transit.
Time elapsed: 0:00:24.503688
and
devel@hpz4:~/Desktop/python_begin_again/transitland/transitflow$ python2 transitflow.py --name=chicago --bbox=-87.992249,41.605175,-87.302856,42.126747 --clip_to_bbox --exclude=o-9-amtrak,o-9-amtrakcharteredvehicle
INPUTS:
date: 2019-03-02
name: chicago
API key: None
exclude: ['o-9-amtrakcharteredvehicle', 'o-9-amtrak']
bbox: -87.992249 41.605175 -87.302856 42.126747
http://transit.land/api/v1/operators?per_page=10000&bbox=-87.992249%2C41.605175%2C-87.302856%2C42.126747
5 operators in bounding box.
3 operators to be downloaded.
o-dp3-pace 1 / 3
http://transit.land/api/v1/routes?per_page=10000&operated_by=o-dp3-pace
228 routes found.
http://transit.land/api/v1/stops?per_page=10000&served_by=o-dp3-pace
https://api.transit.land/api/v1/stops?offset=10000&per_page=10000&served_by=o-dp3-pace&sort_key=id&sort_order=asc
https://api.transit.land/api/v1/stops?offset=20000&per_page=10000&served_by=o-dp3-pace&sort_key=id&sort_order=asc
25556 stops found.
http://transit.land/api/v1/schedule_stop_pairs?date=2019-03-02&per_page=10000&sort_min_id=0&operator_onestop_id=o-dp3-pace
10000
https://api.transit.land/api/v1/schedule_stop_pairs?date=2019-03-02&operator_onestop_id=o-dp3-pace&per_page=10000&sort_key=id&sort_min_id=4707804518&sort_order=asc
20000
https://api.transit.land/api/v1/schedule_stop_pairs?date=2019-03-02&operator_onestop_id=o-dp3-pace&per_page=10000&sort_key=id&sort_min_id=4707819294&sort_order=asc
30000
https://api.transit.land/api/v1/schedule_stop_pairs?date=2019-03-02&operator_onestop_id=o-dp3-pace&per_page=10000&sort_key=id&sort_min_id=4707864774&sort_order=asc
40000
https://api.transit.land/api/v1/schedule_stop_pairs?date=2019-03-02&operator_onestop_id=o-dp3-pace&per_page=10000&sort_key=id&sort_min_id=4707904035&sort_order=asc
retry 1 / 5: HTTP Error 502: Bad Gateway
retry 2 / 5: HTTP Error 502: Bad Gateway
retry 3 / 5: HTTP Error 502: Bad Gateway
retry 4 / 5: HTTP Error 502: Bad Gateway
retry 5 / 5: HTTP Error 502: Bad Gateway
failed:
HTTP Error 502: Bad Gateway
o-dp3-chicagotransitauthority 2 / 3
http://transit.land/api/v1/routes?per_page=10000&operated_by=o-dp3-chicagotransitauthority
retry 1 / 5: HTTP Error 502: Bad Gateway
retry 2 / 5: HTTP Error 502: Bad Gateway
retry 3 / 5: HTTP Error 502: Bad Gateway
137 routes found.
http://transit.land/api/v1/stops?per_page=10000&served_by=o-dp3-chicagotransitauthority
https://api.transit.land/api/v1/stops?offset=10000&per_page=10000&served_by=o-dp3-chicagotransitauthority&sort_key=id&sort_order=asc
retry 1 / 5: HTTP Error 502: Bad Gateway
retry 2 / 5: HTTP Error 502: Bad Gateway
retry 3 / 5: HTTP Error 502: Bad Gateway
retry 4 / 5: HTTP Error 502: Bad Gateway
retry 5 / 5: HTTP Error 502: Bad Gateway
failed:
HTTP Error 502: Bad Gateway
o-dp3-metra 3 / 3
http://transit.land/api/v1/routes?per_page=10000&operated_by=o-dp3-metra
retry 1 / 5: HTTP Error 502: Bad Gateway
retry 2 / 5: HTTP Error 502: Bad Gateway
retry 3 / 5: HTTP Error 502: Bad Gateway
retry 4 / 5: HTTP Error 502: Bad Gateway
retry 5 / 5: HTTP Error 502: Bad Gateway
failed:
HTTP Error 502: Bad Gateway
0 operators successfully downloaded.
3 operators failed.
failed operators: ['o-dp3-pace', 'o-dp3-chicagotransitauthority', 'o-dp3-metra']
Concatenating individual operator outputs.
Traceback (most recent call last):
File "transitflow.py", line 404, in <module>
df = concatenate_csvs("sketches/{}/{}/data/indiv_operators".format(OUTPUT_NAME, DATE))
File "transitflow.py", line 184, in concatenate_csvs
concatenated_df = pd.concat(df_from_each_file, ignore_index=True)
File "/usr/lib/python2.7/dist-packages/pandas/core/reshape/concat.py", line 212, in concat
copy=copy)
File "/usr/lib/python2.7/dist-packages/pandas/core/reshape/concat.py", line 245, in __init__
raise ValueError('No objects to concatenate')
ValueError: No objects to concatenate
Any comments are greatly appreciated.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.