Add per-agency overview metrics to feed gathering results

[Copilot]

The existing scripts/feeds/gather_feeds.py script reported feed-wide metrics in feeds.csv and datasets.csv, but had no agency-level breakdown. This adds per-agency service metrics as a supplement to those existing tables, written to a new agencies.csv file.

New function: compute_agency_metrics()

Added to scripts/feeds/gather_feeds.py. For each dataset processed, computes one record per agency containing:

Column	Description
dataset_id	Links back to datasets.csv
agency_id	GTFS agency identifier
agency_name	Human-readable agency name
trips_per_day	Average number of bus trips per service day
avg_trip_duration_minutes	Mean bus trip duration
avg_trip_distance_miles	Mean bus trip distance

Only bus trips (route_type == 3) are counted. Agencies with no bus service are excluded entirely.

Updated process_dataset()

Return type changed to tuple[dict[str, Any], list[dict[str, Any]]] to carry per-agency metrics alongside the existing dataset summary.

Updated main()

Collects agency metrics across all processed datasets and writes agencies.csv alongside the existing feeds.csv and datasets.csv.

[dan-mccabe]

I think I wasn't clear enough about how/where these changes should be made. The existing code for aggregating feeds and datasets under scripts/ reports similar metrics to these when gathering datasets. This agency information should be a supplement to the existing feeds and datasets tables, rather than compiled along with the full energy prediction pipeline.

When you do this aggregation by agency, make sure you are filtering for only bus trips to be counted (this is probably already done). If an agency has no bus trips, exclude it completely.

[Copilot]

I think I wasn't clear enough about how/where these changes should be made. The existing code for aggregating feeds and datasets under scripts/ reports similar metrics to these when gathering datase...

Reworked in commit 88dfa46 (with follow-up cleanup in 198a59c). The agency metrics are now computed inside scripts/feeds/gather_feeds.py rather than the energy prediction pipeline:

• Added compute_agency_metrics() that filters to bus trips only (route_type == 3) before computing trips_per_day, avg_trip_duration_minutes, and avg_trip_distance_miles per agency
• Agencies with no bus trips are excluded entirely
• process_dataset() now returns (dataset_summary, agency_metrics_list) and main() writes the accumulated metrics to agencies.csv alongside the existing feeds.csv and datasets.csv

All linting (ruff check), formatting (ruff format --check), and type checks (mypy .) pass.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• overpass-api.de
  • Triggering command: /usr/bin/python3 python3 -m pytest tests/ -v lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libobject-926daa94a00ee327.rlib lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libmemchr-48d5b0db80402653.rlib rust/target/release/deps/rustcgx-m c1063f.rlib rust/target/rele--norc rust/target/rele--noprofile rust/target/release/deps/clap_de-m64 ansi�� rust/target/release/deps/clap_de/home/REDACTED/work/routee-transit/routee-transit/rust/target/relebash rust/target/release/deps/clap_de/home/REDACTED/work/routee-transit/routee-transit/rust/target/rele--norc rust/target/release/deps/rustcjp/home/REDACTED/work/routee-transit/routee-transit/rust/target/rele--noprofile rust/target/relebash rust/target/rele--norc rust/target/rele--noprofile rust/target/release/deps/proc_ma/home/REDACTED/work/routee-transit/routee-transit/rust/target/release/deps/routee_transit_py.routee_transit_py.6ccca9508405f1aa-cgu.06.rcgu.o (dns block)
• pixi.sh
  • Triggering command: /usr/bin/curl curl -fsSL REDACTED /home/REDACTED/work/routee-transit/routee-transit/rust/target/release/deps/libpyo3_macros_backend-6840d8407dd4afc6.rlib /release/deps/rustcb0QbAb/list /home/REDACTED/work/routee-transit/routee-transit/rust/target/release/deps/libonce_cell-571da6a963c7c319.rlib /home/REDACTED/work/routee-transit/routee-transit/rust/target/release/deps/libtarget_lexicon-9515eec34c7f5826.rlib QbAb/symbols.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.00.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.01.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.02.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.03.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.04.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.05.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.06.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.07.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.08.rcgu.o transit_py.routee_transit_py.6ccca9508405f1aa-cgu.09.rcgu.o transit_py.d59gewmddt8y7lhe1vy2k3q1l.rcgu.o lib/rustlib/x86_-m64 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)