Skip to content

Commit 205e432

Browse files
authored
pyramid: two-phase iterative build (finest from source, then rollups) (#138)
* spec: two-phase iterative hex pyramid build to bound build-time memory Replace the single 7-branch UNION ALL COPY with one finest-from-source COPY followed by iterative parent rollups from the previous resolution. Bounds working memory by output cardinality per step instead of by full source size, so global high-cardinality datasets (GBIF, irrecoverable carbon) build without OOM. Also adjusts .gitignore to track committed specs under docs/superpowers/specs/ while continuing to ignore the rest of docs/superpowers/ (matches the precedent set by the existing 2026-04 specs). * plan: implementation steps for two-phase iterative hex pyramid build * tiles: bump layout version to v3-iterative for two-phase pyramid * pyramid: two-phase iterative build (finest from source, then rollups) * pyramid: rename AVG internal weight column; fix docstring; cover multi-col SUM - Rename AVG-mode internal column count → __pyramid_weight to avoid collision when a user names their value column "count" - Fix register_hex_tiles docstring: "Other aggs" bullet now correctly states aggregation happens at every resolution including finest (not passed-through raw) - Add test_sum_mode_with_multiple_value_columns covering multi-value-column SUM * test: end-to-end mathematical invariants for two-phase pyramid * test: AVG weighted-mean fixture must span two h5 cells Original fixture placed all five points within ~100m, so they collapsed to a single h5 cell at the finest level. Phase 2 then saw one child per parent, where SUM(v*w)/SUM(w) and an incorrect AVG(v) both yield the same answer. The test would have passed even with the unweighted bug it was meant to catch. Spread the two clusters to (37.0, -122.0) and (37.2, -122.0) — distinct h5 cells, same h4 parent — and add a sanity assertion that the finest level really has two distinct h5 children before checking parent AVG.
1 parent 9cc4462 commit 205e432

7 files changed

Lines changed: 1273 additions & 167 deletions

File tree

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@
88
# Claude Code / superpowers tooling
99
.claude/
1010
memory/
11-
docs/superpowers/
11+
docs/superpowers/*
12+
!docs/superpowers/specs/
13+
!docs/superpowers/plans/
1214

1315
# Byte-compiled / optimized / DLL files
1416
__pycache__/

0 commit comments

Comments
 (0)