Add Normalised Dataset for the UK Railway Network

[iiAnderson]

Summary

Adding a normalised, analysis-ready dataset of every train service on the UK rail network, derived from the National Rail Darwin Push Port feed.

• ~30,000 services per day, one row per service, updated daily
• Pre-aggregated delay metrics, cancellation status, passenger loading, and a nested per-station stops array
• Hive-partitioned Apache Parquet (Snappy-compressed), partitioned by service date
• Data available from January 2025 onwards
• Stored in S3 (eu-west-1, requester pays)

Schema highlights

Field	Description
rid, toc, train_id	Service identifiers
origin_tpl, destination_tpl	Route endpoints (TIPLOC codes)
cancellation_status	ran, cancelled, or partially_cancelled
avg_delay_mins, max_delay_mins	Aggregate delay metrics
avg_loading	Passenger loading percentage
stops	Nested array of per-station scheduled/actual times and delays

Why this dataset is useful

UK rail performance data is widely discussed but hard to access in a structured, queryable format. This dataset enables researchers, journalists, and transport analysts to run queries like:

• Which train operating companies have the worst cancellation rates?
• How do delays propagate across the network by time of day?
• What are the busiest and most delayed stations?

The data is directly queryable via AWS Athena or any Parquet-compatible tool (DuckDB, pandas, Spark).

Documentation & usage

• Documentation: https://blog.robbiea.co.uk/posts/the-darwin-normalised-dataset/
• Source code & pipeline: https://github.com/iiAnderson/darwin-connect
• License: CC-BY-NC-4.0 (upstream data subject to National Rail T&Cs)

Note: This replaces my earlier PR #2474, which offered the raw (un-normalised) tables. The normalised dataset is significantly more useful for consumers — single-row-per-service with pre-computed metrics eliminates the need for complex multi-table joins.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.