Skip to content

Commit 6e3133c

Browse files
committed
ctbk update: rebuild rides pyramids (v1/v2/v3) each new month
`update.sh` (unused by ci.yml — legacy) already had normalize/consolidate/ aggregate/spj/sm steps. `ctbk update` (the CLI wired into `.github/workflows/ci.yml`) mirrored those but omitted the rides-pyramid rebuild — so /api/rides-{v1,v2,v3} kept serving up-through-prev-month data while the normalized parquets and aggregates advanced. Adds `ctbk rides-v1-build -v {v1,v2,v3}` invocations after `spj`, covering `[prev_month, current]` for all tiers (1h base + 10 derived). Range includes prev month because start-anchored /1h shards read `normalized[ym] + normalized[ym+1]` — when prev month was originally processed, the current month's normalized parquet didn't exist, so rides that started in prev but ended in current were dropped; rebuilding the prev-month shard captures them now that the current normalized parquet exists. Derived tiers span multi-month (1mo-shard cadences for /3h,/6h,/12h in v1; broader coverage in v2/v3) or all-time — always overwrite so they fold in the new data. Verified via `ctbk update -n -S -W 202606`: emits 3 variants × 11 builds = 33 invocations targeting `[2026-05, 2026-06]`. Manual backfill for 202605+202606 already run before this commit.
1 parent 6eb324f commit 6e3133c

1 file changed

Lines changed: 18 additions & 0 deletions

File tree

ctbk/update.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,24 @@ def ctbk_run(*args):
6060
err(f"--- Station pair JSONs ---")
6161
ctbk_run('spj', 'create', ym)
6262

63+
# Rebuild rides pyramids (v1/v2/v3) that back /api/rides-{v1,v2,v3}.
64+
# Range = prev-month → current: rebuilding prev month's /1h start-anchored
65+
# shard picks up rides that started in prev but ended in current (dropped
66+
# previously because the current month's normalized parquet didn't exist).
67+
# Derived tiers span multi-month or all-time, so `-O` (overwrite) is
68+
# required to fold in the new /1h data.
69+
err(f"--- Rides pyramids (v1/v2/v3) ---")
70+
yyyy, mm = ym[:4], ym[4:]
71+
ym_new = f"{yyyy}-{mm}"
72+
if mm == '01':
73+
ym_prev = f"{int(yyyy) - 1}-12"
74+
else:
75+
ym_prev = f"{yyyy}-{int(mm) - 1:02d}"
76+
for v in ('v1', 'v2', 'v3'):
77+
ctbk_run('rides-v1-build', '-v', v, '-f', ym_prev, '-T', ym_new)
78+
for t in ('3h', '6h', '12h', '1d', '3d', '7d', '14d', '1mo', '3mo', '1y'):
79+
ctbk_run('rides-v1-build', '-v', v, '-f', ym_prev, '-T', ym_new, '-t', t, '-O')
80+
6381
if not no_www:
6482
err(f"--- WWW assets ---")
6583
run('node', 'www/scripts/gen-station-urls.js', dry_run=dry_run)

0 commit comments

Comments
 (0)