Skip to content

Commit 7ddb710

Browse files
committed
avail-geo: thread binCol + range into fetchSegmentRows
So hyparquet can prune row groups by `dt` column stats. Currently a no-op for v2 shards (`avail_v2.py` writes them as a single 956K-row RG, and rows are `(cell, dt)`-sorted) — sub-day queries against `/api/avail-v2[/cells]` still OOM the CFW. The full read-side optimization needs the v2 build to write smaller, `dt`-first-sorted RGs; once that lands this call site is already wired for it.
1 parent 46cb517 commit 7ddb710

1 file changed

Lines changed: 11 additions & 1 deletion

File tree

gbfs/api/src/avail_geo.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -256,8 +256,18 @@ async function serveGeoReduced(
256256
const reducer = reducerRaw as Reducer;
257257

258258
const plan = planGeoQuery(pyramid, { range: { from, to }, binBudget, bbox, cellBudget });
259+
// Thread `binCol` + per-segment range so hyparquet prunes row groups by
260+
// `dt` column stats. Currently a no-op for v2 shards (`avail_v2.py` writes
261+
// them as a single 956K-row RG, and rows are `(cell, dt)`-sorted so a
262+
// hypothetical multi-RG file would still have each RG spanning the full
263+
// month). The full read-side optimization needs the v2 build to write
264+
// smaller, `dt`-first-sorted RGs; until then v2 endpoints OOM on sub-day
265+
// queries and only PoC `/api/avail-geo` reliably serves.
259266
const shardRows = await Promise.all(
260-
plan.segments.map((seg) => fetchSegmentRows(pyramid.storage, seg.keys)),
267+
plan.segments.map((seg) => fetchSegmentRows(pyramid.storage, seg.keys, {
268+
binCol: pyramid.binCol,
269+
range: { from: seg.from, to: seg.to },
270+
})),
261271
);
262272
// Filter to chosen output resolution + bbox-covering cells.
263273
const filtered = shardRows.map((rows) =>

0 commit comments

Comments
 (0)