Skip to content

Commit 13a3bc9

Browse files
rachelekmrachelekm
authored
Add transit routing architecture ADR (#804)
* Add transit routing architecture ADR * Update after reco updates in 806 Updates the "Networks Use In Application" section following recommendation logic updates in PR 806. "Routability" no longer constraint in recommendation logic so updates selector sections accordingly. * Update CHANGELOG
1 parent b12ce9e commit 13a3bc9

2 files changed

Lines changed: 262 additions & 0 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
5151
- Add Favorites page [#765](https://github.com/azavea/echo-locator/pull/765)
5252
- Add conditional school choice link [#775](https://github.com/azavea/echo-locator/pull/775)
5353
- Add client feedback copy updates [#793](https://github.com/azavea/echo-locator/pull/793)
54+
- Add transit routing architecture ADR [#804](https://github.com/azavea/echo-locator/pull/804)
5455

5556
### Changed
5657

‎doc/arch/adr-004-transit-network-architecture.md‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
# Transit Routing Architecture
2+
3+
ECHO Locator uses transit network analysis data generated by
4+
[Conveyal](https://www.conveyal.com/) to calculate travel times and routes by
5+
transportation and traffic type. The travel times and routes are then used, in
6+
combination with user-defined commute settings, to create accessibility scores
7+
for recommendations and to display commute time and transit route paths for each
8+
neighborhood.
9+
10+
This document outlines the frontend transit routing structure following the
11+
migration from the initial Taui implementation, planned in the
12+
[state management planning ADR](https://github.com/azavea/echo-locator/blob/develop/doc/arch/adr-003-state-management-migration.md).
13+
A bulk of the core networks migration work and discussions can be found in the
14+
migration PR, [PR #693](https://github.com/azavea/echo-locator/pull/693) as well
15+
as [PR #699](https://github.com/azavea/echo-locator/pull/699). As a post-hoc ADR
16+
documenting transit routing decisions, this document is organized by the core
17+
steps that enable the frontend to work, bottom-up: Transit network analysis
18+
source data --> transit network state management --> component use of transit
19+
network state.
20+
21+
## Network analysis files
22+
23+
The network analysis files are pre-computed transit accessibility datasets that
24+
enable the application to rank neighborhoods based on travel accessibility
25+
without API calls. The application supports the five below network modes, each
26+
representing different travel scenarios that may affect commute estimates (and
27+
therefore neighborhood rankings) differently. Note that “peak” can be defined as
28+
times with high commute demand and traffic congestion, e.g. afternoons
29+
4:30-6:30pm.
30+
31+
| **Mode Key** | **Label** | **Includes Commuter Rail** | **Description** |
32+
| ------------------ | ------------------- | -------------------------- | --------------------------------------------------------- |
33+
| `peak` | Peak | Yes | Peak hours with commuter rail and express bus service |
34+
| `offPeak` | Off Peak | Yes | Off-peak hours with commuter rail and express bus service |
35+
| `peakNoExpress` | Peak No Express | No | Peak hours without commuter rail or express buses |
36+
| `offPeakNoExpress` | Off Peak No Express | No | Off-peak hours without commuter rail or express buses |
37+
| `car` | Car | N/A | Driving times (no routing path) |
38+
39+
**NOTE** Route path results are not available for the car network mode. However,
40+
basic travel time results are available and we can continue to use that data for
41+
commute times and to generate a neighborhood accessibility score. By default,
42+
driving speeds will not reflect congestion and will therefore generally
43+
overstate access.
44+
45+
The network analysis data is static and hosted remotely on S3 in the
46+
`echo-locator-analysis-files` bucket, organized by year of analysis generation
47+
and then by network type. Each network type directory contains the following:
48+
49+
1. `request.json`: Network configuration and spatial metadata
50+
2. `transitive.json`: Transit network topology, route segment and stop
51+
information
52+
3. `{index}_times.dat`: Time surface binary data object for a given index.
53+
Indices are dynamically created on the frontend via the `coordinateToIndex()`
54+
function.
55+
4. `{index}_paths.dat`: Paths grid binary data object for a given index. Not
56+
present for the car network mode. Indices are dynamically created on the
57+
frontend via the `coordinateToIndex()` function.
58+
59+
ECHO does not have a set cadence for updating transit networks data. The
60+
neighborhood recommendations rely on accessibility analysis vs real-time
61+
routing, so we are not reliant on updating frequently. However, after a few
62+
years, we may need to have Conveyal re-generate new files on the latest street
63+
network and transit data to capture broader accessibility changes. The steps to
64+
update the network analysis include:
65+
66+
1. Ask our Conveyal point-of-contact to re-generate the analysis results with
67+
recent MBTA schedules for peak, off-peak, peak (no commuter rail/express
68+
bus), and off-peak (no commuter rail/express bus) network types as well as
69+
for driving.
70+
2. Create a new directory in the `echo-locator-analysis-files` S3 bucket for the
71+
current year
72+
3. Within the current year directory, create five sub-directories for each
73+
network mode following the naming conventions from the other year directories
74+
4. Ensure CloudFront CORS headers allow frontend read-only access to updated
75+
year path
76+
5. Once analysis files are generated from Conveyal they will be available in
77+
their own public S3 buckets for us to copy over. Use `aws cli` or `s5cmd` to
78+
sync buckets.
79+
6. Update the `NETWORK_URL_ROOT` env variable path to point to the latest year
80+
bucket directory in the `dotenv` file for both staging and development, in
81+
the `echo-locator-stgdjango-config-us-east-1` and
82+
`echo-locator-devdjango-config-us-east-1` buckets respectively.
83+
7. When ready to release with new transit network data, trigger the deployments
84+
on staging and production so that the new frontend bundle includes the
85+
updated `NETWORK_URL_ROOT` env var
86+
87+
**POTENTIAL ISSUES**
88+
89+
- The frontend processes the path and times data via parser functions migrated
90+
from Taui. The functions use a `DataView` object to read the binary data
91+
sequentially. A `byteOffset` variable tracks the current position in the
92+
buffer, advancing after each read operation by the number of bytes consumed.
93+
If there are changes to the incoming `*_paths.dat` and `*_times.dat` that
94+
affect the `byteOffset`, it could cause a misalignment that returns corrupted
95+
paths and stops IDs which hangs the application. This happened in the 2025
96+
analysis file update and a fix and troubleshooting notes are available
97+
[here](https://github.com/azavea/echo-locator/issues/628#issuecomment-3184615811).
98+
- The `.dat` files are `application/octet-stream` files, which CloudFront does
99+
not compress by default. These files are already gzipped at source and
100+
downloading will decompress the files, which causes performance issues once
101+
the application is fetching times and paths data. To avoid this, we must
102+
always copy/sync directly from the source bucket when updating analysis files.
103+
See [Issue #629](https://github.com/azavea/echo-locator/issues/629) for more
104+
details.
105+
106+
## Networks State Management
107+
108+
Every network type’s `request.json` and `transitive.json` files are fetched on
109+
login and stored in `networks` redux state by network mode. Once a user profile
110+
is fetched and user destinations are known, the `getAllTimesAndPathsData` thunk
111+
function is dispatched. This async thunk fetches and processes route paths and
112+
travel times for every destination for each network and stores the results in
113+
redux state (`networks.timesAndRoutesData`). This process is illustrated by the
114+
below sequence diagram:
115+
116+
```mermaid
117+
sequenceDiagram
118+
participant User
119+
participant React
120+
participant Redux
121+
participant Thunks
122+
participant S3
123+
participant Selectors
124+
125+
126+
User->>React: Login / Load app
127+
React->>Thunks: getNeighborhoods()
128+
React->>Thunks: getNetworks()
129+
130+
131+
Thunks->>S3: GET request.json (x5 modes)
132+
Thunks->>S3: GET transitive.json (x5 modes)
133+
S3-->>Thunks: Network metadata
134+
Thunks->>Redux: Store networks state
135+
136+
137+
React->>Thunks: getUserProfile()
138+
Thunks->>Redux: Store destinations
139+
140+
141+
React->>Thunks: getAllTimesAndPathsData(destinations)
142+
143+
144+
loop For each destination
145+
loop For each network mode
146+
Thunks->>Thunks: coordinateToIndex()
147+
Thunks->>S3: GET {index}_times.dat
148+
Thunks->>S3: GET {index}_paths.dat (if not car)
149+
S3-->>Thunks: Binary data
150+
Thunks->>Thunks: parseTimesData()
151+
Thunks->>Thunks: parsePathsData()
152+
Thunks->>Thunks: createNetworkNeighborhoodRoutes()
153+
Thunks->>Thunks: createNetworkNeighborhoodTravelTimes()
154+
end
155+
end
156+
157+
158+
Thunks->>Redux: Store timesAndRoutesData
159+
Redux->>Selectors: Trigger reselect memoization
160+
Selectors->>React: Ranked neighborhoods
161+
React->>User: Display results
162+
```
163+
164+
## Networks Use In Application
165+
166+
The `timesAndRoutesData` state provides two critical features: travel time and
167+
route path segments. This data is used to create:
168+
169+
- The accessibility weight in the neighborhood recommendation algorithm
170+
- Commute ranges for each neighborhood
171+
- Transit route paths rendered on the neighborhood details map.
172+
173+
The commutes times, accessibility weight, and route paths will re-render if a
174+
user changes the destination or travel settings. Changes to travel settings will
175+
trigger a new active network mode conditionally dependent on the following
176+
travel setting values: `userProfile.hasVehicle`, `userProfile.useCommuterRail`,
177+
`networks.trafficConditions`.
178+
179+
| **Mode** | **hasVehicle** | **useCommuterRail** | **trafficConditions** |
180+
| ------------------ | -------------- | ------------------- | --------------------- |
181+
| `peak` | False | True | “peak” |
182+
| `offPeak` | False | True | "offPeak" |
183+
| `peakNoExpress` | False | False | “peak” |
184+
| `offPeakNoExpress` | False | False | "offPeak" |
185+
| `car` | True | False | N/A |
186+
187+
The active network mode can be used (with the `userProfile.activeDestination`
188+
state value and a neighborhood zipcode) to index times and routes data from
189+
`networks.timesAndRoutesData` state.
190+
191+
For example, to access the commute time to a given neighborhood in state:
192+
193+
```ts
194+
networks.timesAndRoutesData[destination][networkMode].travelTimesByNeighborhood[
195+
neighborhoodIndex
196+
];
197+
```
198+
199+
The time and route state is structured in this way to support easily creating
200+
commute ranges (the min and max travel time values across network modes) for
201+
each destination as well as for rendering route paths to multiple destinations
202+
side-by-side in the neighborhood details modal.
203+
204+
### Neighborhood recommendation algorithm
205+
206+
Selector reference: `neighborhoodsSortedWithRoutes`
207+
208+
The algorithm measures neighborhood accessibility by scaling and evaluating a
209+
neighborhood’s travel time. If a neighborhood's travel time for a given network
210+
mode is under the maximum commute time (120 minutes), its time value is scaled
211+
to create the `accessibilityWeight` which is then used in the final neighborhood
212+
score calculation. If a neighborhood's travel time is equal to or greater than
213+
the maximum commute time, the `accessibilityWeight` for that neighborhood is set
214+
to the default weight value, `0` ie "worst". All neighborhoods are then added to
215+
the "recommended" list to be sorted by final score.
216+
217+
### Transit route map display
218+
219+
Selector reference: `drawNeighborhoodRoutes`
220+
221+
A neighborhood is routable if:
222+
223+
- The active network mode is not “car”
224+
- There are transit route path segments OR the travel time is under the maximum
225+
(120 minutes)
226+
227+
Routability does not affect the recommendation logic
228+
([see decision here for more details](https://github.com/azavea/echo-locator/issues/788#issuecomment-4014316420)),
229+
however it does affect the ability to display a route path on the map. Routes
230+
are generated by neighborhood for each network type (with the exception of
231+
“car”) as part of processing the analysis files and setting networks state, in
232+
`createNetworkNeighborhoodRoutes`. The correct route is indexed by active
233+
neighborhood for a given network mode in the `drawNeighborhoodRoutes` selector
234+
which then creates a GeoJSON FeatureCollection to display on the neighborhood
235+
details map from its respective stops and segments.
236+
237+
- If a neighborhood → destination is not routable (ie, there are no route path
238+
segments): the selector will return a FeatureCollection of two Points (the
239+
origin and destination) and one LineString to between the two points (styled
240+
as the default, “walking”).
241+
- If a neighborhood → destination is routable (ie, there are route path
242+
segments): the selector will return a FeatureCollection of two Points (the
243+
origin and destination), one LineString to the first stop (styled as the
244+
default, “walking”), a Point for each transit stop, and a Polyline of styled
245+
transit path segments between each stop.
246+
247+
The transit LineStrings are populated with properties, like `routeColor` and
248+
`name`, in the selector logic which is then used to dynamically render styled
249+
routes and transit line directions over the map. These feature properties are
250+
derived from the segments themselves, which are enriched during the data
251+
processing pipeline in `createNetworkNeighborhoodRoutes()`:
252+
253+
- `routeColor` and `name`:
254+
- Populated by looking up the `patternId` in the networks state for that
255+
network type (attributes set from a network’s `transitive.json` file)
256+
- Pattern → Route ID → Route details (color, name)
257+
- `fromStop` and `toStop`:
258+
- Populated by indexing by Stop IDs in the networks state for that network
259+
type (attributes set from a network’s `transitive.json` file)
260+
- Uses `boardStopId` and `alightStopId` from parsed paths data
261+
- Retrieves full stop details including name, coordinates, and ID

0 commit comments

Comments
 (0)