This can happen if the feed is formatted poorly.

The current implementation assumes that the GTFS is already unpacked into a directory. However, most of the GTFS that can be downloaded from sources such as TransitFeeds come as a ZIP file. Hence, I would suggest to allow for both, directories and ZIP files to be supported by the CLI.

Change pandera validator to optional for this field.

GTFS Time is not defined relative to midnight, but relative to noon - 12h. While that makes "writing" GTFS feeds easier, it makes processing a lot harder.

Expected functionality

As explained in my note about GTFS Time values, with the Europe/Berlin time zone (+1h standard time to +2 DST shift occurs at 2021-03-28T02:00+01:00), I expect

• the departure_time of 00:30 of a trip running on 2021-03-28 to happen at 1616884200/2021-03-28T00:30+02:00, not at 1616887800/2021-03-28T00:30+01:00;
• the departure_time of 06:30 of a trip running on 2021-03-28 to happen at 1616905800/2021-03-28T06:30+02:00, not at 1616909400/2021-03-28T06:30+01:00.

Describe the bug

I'm not familiar with this code base (just a random stranger dropping by to poke around 🙈), but it seems that combine_gtfs_feeds is affected by this problem on those days that the DST <-> standard time switch occurs on.

I'm not entirely sure how that actually manifests in combine_gtfs_feeds' output, but I assume that wrong headway-based stop_times will be calculated.

I tried to find some places in the code base:

• combine_gtfs_feeds/combine_gtfs_feeds/cli/run.py

  Lines 106 to 112 in cb78603

  	def convert_to_seconds(value):
  	"""
  	Converts hh:mm:ss format to number
  	of seconds after midnight.
  	"""
  	h, m, s = value.split(':')
  	return int(h) * 3600 + int(m) * 60 + int(s)

• combine_gtfs_feeds/combine_gtfs_feeds/cli/run.py

  Lines 150 to 168 in cb78603

  	stop_times_update['arrival_time_secs'] = stop_times_update[
  	'arrival_time'].apply(convert_to_seconds)
  	stop_times_update['departure_time_secs'] = stop_times_update[
  	'departure_time'].apply(convert_to_seconds)
  	stop_times_update = stop_times_update.loc[
  	stop_times_update.index.repeat(
  	stop_times_update['total_trips'])].reset_index(drop=True)
  	stop_times_update['counter'] = stop_times_update.groupby(
  	['trip_id', 'stop_id']).cumcount()
  	stop_times_update['departure_time_secs'] = stop_times_update[
  	'departure_time_secs'] + (stop_times_update[
  	'counter'] * stop_times_update['headway_secs'])
  	stop_times_update['arrival_time_secs'] = stop_times_update[
  	'arrival_time_secs'] + (stop_times_update[
  	'counter'] * stop_times_update['headway_secs'])
  	stop_times_update['departure_time'] = stop_times_update[
  	'departure_time_secs'].apply(to_hhmmss)
  	stop_times_update['arrival_time'] = stop_times_update[
  	'arrival_time_secs'].apply(to_hhmmss)

related: google/transit#15

The same stops/stations may appear in more than one feed. Perhaps create an option to only represent these stops once- right now in the output they would appear once for each feed.

Use the Pandera library to enforce data types in exported GTFS.

Currently, frequencies are converted to trips and a frequencies.txt file is not written out. Make this an optional argument.

Some feeds might be missing calendar_dates.txt or leaving it empty, even though this violates the recommended approaches. In such cases, assume that calendar.txt is adding service for all service_ids.

Neglected to include this during recent code changes that allow in-memory access to combined GTFS files.

This will help with poorly formatted feeds.

[note: this issue is likely not worth fixing as the issue does not prevent import into transit-service-analyst but I'm logging the issue with the relevant documentation briefly just in case I encounter it again and want to submit a corresponding PR later on.]

combine_gtfs_feeds run -g C:\Users\craigth\pythonwork\gtfs\subgroup -s 20220815 -o C:\Users\craigth\pythonwork\gtfs_stops

Two input files
-ICT.zip
-RCT.zip

Output file:
gtfs_stops.zip

In routes.txt and other files, integers in columns like route_sort_order have been transformed into decimals by .0 being appended. File fails validation per MobilityData validator "invalid_integer".

Some feeds may not include calendar.txt and just provide calendar_dates.txt.

Follow these instructions to run python black as a pre-commit.

Hi,

I'm trying to follow the example shown for cli.run.combine with three Chicago GTFS feeds using an input directory, date, and output directory as the arguments, but I keep running into this error:

"combine() missing 1 required positional argument: 'logger'"

I've tried setting the logger argument to null etc. and I can't figure out what data type it takes. Any help would be much appreciated!

Isaac

Convert trip_id to string when concatenating with trip counter variable.

Code is not written to account for same trip_id being used multiple times.

Deal with things like:

• input/output dirs do not exist
• no GTFS feeds/folders in input dir

I am running into an issue where every single call I make to merge GTFS it results in a KeyError start_date. There's no logging to tell me which file this is affecting. I have tried this with combinations of four different agencies in Canada as well as Pierce Transit and Intercity Transit. Any thoughts about what I might be doing wrong?

The calendar.txt file is present in each of the subfolders.

Directory structure is as follows:
/GTFS/19/
agency.txt fare_attributes.txt routes.txt stops.txt transfers.txt
calendar_dates.txt feed_info.txt shapes.txt stop_times.txt trips.txt
/GTFS/3/
agency.txt calendar.txt feed_info.txt shapes.txt stop_times.txt
calendar_dates.txt fare_attributes.txt routes.txt stops.txt trips.txt
/output/

combine_gtfs_feeds run -g "GTFS" -s "20230505" -o "output"

Traceback (most recent call last):
File "/opt/conda/bin/combine_gtfs_feeds", line 8, in
sys.exit(main())
File "/opt/conda/lib/python3.8/site-packages/combine_gtfs_feeds/cli/main.py", line 19, in main
sys.exit(combine.execute())
File "/opt/conda/lib/python3.8/site-packages/combine_gtfs_feeds/cli/cli.py", line 28, in execute
args.func(args)
File "/opt/conda/lib/python3.8/site-packages/combine_gtfs_feeds/cli/run.py", line 509, in run
feeds = combine(args.gtfs_dir, args.service_date, args.output_dir, logger)
File "/opt/conda/lib/python3.8/site-packages/combine_gtfs_feeds/cli/run.py", line 578, in combine
service_id_list = get_service_ids(
File "/opt/conda/lib/python3.8/site-packages/combine_gtfs_feeds/cli/run.py", line 125, in get_service_ids
(calendar["start_date"] <= service_date)
File "/home/datascience/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3807, in getitem
indexer = self.columns.get_loc(key)
File "/home/datascience/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3804, in get_loc
raise KeyError(key) from err
KeyError: 'start_date'

For example, missing calendar_dates.txt does not cause a crash, but the user should be warned.

We can create shapes.txt using the unique stop sequences for each route/trip permutation. Each stop location, in order, for each unique stop sequence by route, becomes a record in shapes (shape_pt_lon, shape_pt_lat). Create a shape_id for each one and update this field in trips.txt.