Skip to content

Commit 06cd195

Browse files
ryan-williamsclaude
andcommitted
Parse MOD-IV Hudson-wide + aggregate JSON; spec homepage rework
Phases 1-2 of `specs/homepage-rework.md` (data layer; runs on the box where the source zips live, so the frontend session needs no large pull): - `jct modiv parse`: default `-m/--mun` now `all` (every Hudson district), retains `county_district`; supports a single code or comma list. Re-tracked `data/modiv/treasury/{year}.parquet` (all 12 muns, ~150k rows/yr). - `crossval`/`blq-history`: add `-m/--mun` (default `0906`) so JC outputs are byte-identical to before despite the now-Hudson-wide parquets. - `jct modiv aggregate`: emit tiny committed rollups for the homepage charts → `www/public/data/modiv_{tax_base,class_composition,exempt_share}.json` (60/420/60 rows; JC tax base $57.6B→$64.1B, exempt 26.6% = $17.0B). - Tests for `class_group`/`_muni_display` (9 pass). - Add `specs/homepage-rework.md` (frontend Phases 3-4 for a www session). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent c85998c commit 06cd195

11 files changed

Lines changed: 4722 additions & 18 deletions

File tree

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
outs:
2-
- md5: 2c7bba2a95d645a9a10ff63f37890046
3-
size: 2969797
2+
- md5: 5482353ea12afc83068e881ab7806d2b
3+
size: 6599793
44
hash: md5
55
path: 2021.parquet
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
outs:
2-
- md5: c9739afe5e2bb0401b509f9c2082b48b
3-
size: 2983230
2+
- md5: 77660bdd9b14aa73a3e197cc018d5359
3+
size: 6646890
44
hash: md5
55
path: 2022.parquet
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
outs:
2-
- md5: 61419fbb67a166e1d041ebd6a0b91716
3-
size: 2997957
2+
- md5: 651235d352ffd136ccf29d38228fb173
3+
size: 6684955
44
hash: md5
55
path: 2023.parquet
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
outs:
2-
- md5: e4f88fb336c9dda50de97ee7b75e00f7
3-
size: 3018282
2+
- md5: b48141b7ee6af1f8c5a349464857523d
3+
size: 6713152
44
hash: md5
55
path: 2024.parquet
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
outs:
2-
- md5: 4f93a797dec672992851ab8a43acdbd9
3-
size: 3035511
2+
- md5: 500d6109f2ab3a50dcf61e17dc49a3ca
3+
size: 6733783
44
hash: md5
55
path: 2025.parquet

specs/homepage-rework.md

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
# Homepage rework: landing page with copy + charts + map CTA
2+
3+
> Written by an EC2 session for a `www`-focused session to implement.
4+
> Turns the site from "the whole thing is a fullscreen map" into a regular
5+
> landing page (copy + aggregate charts) that links into the fullscreen map.
6+
7+
> **Status (2026-06-08):** Phases **1–2 are done** (on the EC2 box, where the
8+
> ~1.4GB of source zips live). The all-Hudson parquets are re-DVC-tracked +
9+
> pushed to R2, and the three aggregate JSON files are committed at
10+
> `www/public/data/modiv_{tax_base,class_composition,exempt_share}.json`.
11+
> A laptop session can start at **Phase 3** (frontend) with **no large data
12+
> pull** — the committed JSON is all the charts need.
13+
14+
## Goal
15+
16+
Today the entire site ([jct.rbw.sh]) is a single fullscreen 3D map (`www/src/App.tsx`),
17+
showing **taxes *paid*** per parcel (from the HLS `data/payments.parquet` pipeline).
18+
19+
We've since parsed **NJ Treasury MOD-IV** data, which gives a different axis the
20+
map doesn't surface and which is *better as charts than as a 3D map*:
21+
**assessed value (the tax base) + property classification**, for all of Hudson
22+
County, 2021–2025.
23+
24+
Rework `/` into a scrollable landing page:
25+
- Hero + intro copy
26+
- 3–4 aggregate charts (tax base over time, class composition, exempt share, JC-vs-Hudson)
27+
- A prominent **"Explore the map →"** CTA into the current fullscreen view (moved to `/map`)
28+
29+
## Data we have (ground truth — sanity-check against these)
30+
31+
Parsed Treasury MOD-IV, **JC only (MUN 0906)** today, at
32+
`data/modiv/treasury/{2021..2025}.parquet` (DVC-tracked). Schema includes
33+
`county_district`, `block/lot/qualifier`, `blq`, `property_class`,
34+
`land_value`/`improvement_value`/`net_value` (dollars, int), `last_year_tax`,
35+
`property_use_code`, `year_constructed`, `old_property_id`, etc. See
36+
[`src/jc_taxes/modiv.py`] (`MODIV_FIELDS`).
37+
38+
**JC net assessed value (tax base), $B:**
39+
40+
| Year | Parcels | Net | Land | Improvements |
41+
|------|--------:|----:|-----:|-------------:|
42+
| 2021 | 63,341 | 57.6 | 20.8 | 37.5 |
43+
| 2022 | 63,944 | 58.8 | 21.1 | 38.4 |
44+
| 2023 | 64,068 | 61.4 | 22.8 | 39.2 |
45+
| 2024 | 64,294 | 62.8 | 23.6 | 39.6 |
46+
| 2025 | 64,885 | 64.1 | 23.6 | 40.8 |
47+
48+
**+11.3%** over 4 years; ~1,500 net new parcels (JC's construction boom).
49+
50+
**JC 2025 net value by property class** (NJ codes: 1=vacant, 2=residential,
51+
4A=commercial, 4B=industrial, 4C=apartment, 15A–F=exempt):
52+
53+
| Class | Parcels | Net $B | Share |
54+
|-------|--------:|-------:|------:|
55+
| 2 (residential) | 42,138 | 20.3 | 31.7% |
56+
| 4A (commercial) | 4,617 | 15.1 | 23.6% |
57+
| 15F (exempt) | 7,299 | 8.0 | 12.5% |
58+
| 4C (apartment) | 1,771 | 6.6 | 10.4% |
59+
| 15C (exempt) | 973 | 6.3 | 9.8% |
60+
| 1 (vacant) | 6,812 | 2.6 | 4.1% |
61+
| 4B (industrial) | 446 | 2.3 | 3.6% |
62+
63+
**Exempt (class 15*) = 26.6% of all assessed value ($17.0B)** — a headline stat.
64+
Residential is 65% of parcels but only 32% of value.
65+
66+
**Hudson County is one filter away.** The Treasury extracts (`data/modiv/treasury/modiv-{year}.zip`,
67+
DVC-tracked; member `HudsonRE.txt`) contain **all 12 Hudson municipalities** — we just
68+
filter to 0906 today. 2025 record counts:
69+
70+
| Code | Municipality | Records |
71+
|------|--------------|--------:|
72+
| 0901 | Bayonne | 13,986 |
73+
| 0902 | East Newark | 421 |
74+
| 0903 | Guttenberg | 2,796 |
75+
| 0904 | Harrison | 2,797 |
76+
| 0905 | Hoboken | 19,668 |
77+
| 0906 | Jersey City | 64,885 |
78+
| 0907 | Kearny | 8,676 |
79+
| 0908 | North Bergen | 12,712 |
80+
| 0909 | Secaucus | 5,623 |
81+
| 0910 | Union City | 9,640 |
82+
| 0911 | Weehawken | 3,844 |
83+
| 0912 | West New York | 8,994 |
84+
| | **total** | **154,042** |
85+
86+
## Phase 1 — Data: parse Hudson-wide
87+
88+
`jct modiv parse` currently defaults `-m/--mun 0906` and writes JC-only parquets.
89+
Change it to parse **all districts present in the file** and keep `county_district`
90+
in the output (already a column); JC-only consumers filter `county_district == "0906"`.
91+
92+
- Make `-m/--mun` accept `all` (default) or a comma-separated list of codes.
93+
- Re-generate `data/modiv/treasury/{year}.parquet` with all 12 muns and re-`dvc add`
94+
(the `.dvc` sidecars + DVC remote update; ~150k rows/yr vs ~65k today).
95+
- Verify `jct modiv crossval` / `blq-history` still work (they should filter to 0906,
96+
or be parameterized by muni). Re-run and confirm JC numbers are unchanged.
97+
98+
**Constraints:** Do **not** commit `*.parquet` directly — DVC-track them
99+
(remote is now R2; see [`specs/done/modiv-etl-and-r2-migration.md`]). Do not commit credentials.
100+
101+
## Phase 2 — Data: tiny aggregate datasets for the charts
102+
103+
Add `jct modiv aggregate` (or extend `parse`) to emit small per-(year, muni) rollups
104+
that the frontend loads directly. All are tiny (≤ a few KB):
105+
106+
1. **`tax_base_by_year_muni`**`(year, mun_code, mun_name, parcels, net_value, land_value, improvement_value)`. 12 munis × 5 years = 60 rows.
107+
2. **`class_composition_by_year_muni`**`(year, mun_code, class_group, parcels, net_value)` where `class_group` ∈ {residential, commercial, apartment, industrial, vacant, exempt, other} (map raw `property_class` → group; document the mapping).
108+
3. **`exempt_share_by_year_muni`** — derived (exempt net_value / total), or compute in FE from #2.
109+
110+
**Output location/format — recommended:** write these as **committed JSON** to
111+
`www/public/data/*.json` (a few KB each, derived-not-source, FE-loaded). This keeps
112+
the homepage's data **independent of S3/R2** (see prod caveat below). The "don't commit
113+
parquet" rule is about the large per-parcel files; these tiny rollups are config-sized.
114+
*(Alternative: DVC-track + `vite-plugin-dvc`-resolve like the geojson — but that re-couples
115+
the homepage to the S3 bucket for no real benefit at this size.)* Pick one; recommend committed JSON.
116+
117+
## Phase 3 — Frontend: routing split + charts
118+
119+
`www/` is Vite + React + TS; today `App.tsx` (~66KB) is the only view. No router, no
120+
chart lib yet (only deck.gl + custom canvas in `DistributionChart.tsx`/`GradientEditor.tsx`).
121+
122+
1. **Routing.** Add `react-router`. `/` = new `Home` landing page; move current map to
123+
`/map` (rename `App.tsx``MapView.tsx`, mount under `/map`).
124+
- **Back-compat:** existing deep links carry map query params (the map uses `use-prms`).
125+
If `/` is hit *with* known map params, redirect to `/map` preserving the query string,
126+
so old `jct.rbw.sh/?...` links keep working. Confirm the param set with the current
127+
`use-prms` usage in `App.tsx`.
128+
2. **Chart library.** Recommend **Observable Plot** (`@observablehq/plot`) — small API,
129+
strong defaults, easy line/stacked-area/bar; wrap each chart in a React component that
130+
re-renders on theme change. *(Alt: Recharts if you want pure-React/JSX charts; visx if
131+
you need full control. Per global prefs, use [@floating-ui/react] for any tooltips, not
132+
native.)* Default to a unique dev port already set (3201) — no change.
133+
3. **Components** (each reads the Phase-2 JSON):
134+
- `TaxBaseChart` — stacked area (land vs improvements) over 2021–2025; muni selector (default JC).
135+
- `ClassCompositionChart` — stacked bar / 100%-stacked by `class_group` for a chosen year+muni.
136+
- `ExemptShareChart` — exempt share over time, JC vs Hudson average (or small-multiples).
137+
- `MuniCompareChart` — bar of 2025 net_value (or per-parcel avg) across the 12 munis, JC highlighted.
138+
- Stat callouts: "$17B (26.6%) of JC value is tax-exempt", "+11.3% tax base since 2021",
139+
"JC = 42% of all Hudson County assessed value".
140+
4. **Theming:** reuse the existing `ThemeContext` (LM/DM); charts must read theme colors.
141+
5. **CTA:** prominent "Explore the map →" linking to `/map` (and keep the map's SpeedDial /
142+
`use-kbd` shortcuts on the map view only).
143+
144+
## Phase 4 — Copy (draft; edit freely)
145+
146+
> **Jersey City pays the most property tax in Hudson County — by far.**
147+
> This site maps every parcel's tax bill in 3D. Below: where the city's
148+
> $64B assessed value actually sits, how it's grown, and how JC stacks up
149+
> against its 11 Hudson County neighbors.
150+
151+
Short sections framing each chart (tax base growth; residential vs commercial vs
152+
exempt; the exempt quarter; JC vs Hudson). Keep it tight; let the charts carry it.
153+
154+
## Prod / deploy caveat (important)
155+
156+
The live site fetches geojson from **public S3** (`https://jc-taxes.s3.amazonaws.com/...`).
157+
`vite-plugin-dvc`'s `s3ToHttps()` derives the URL from a remote's `url` field and
158+
**ignores `endpointurl`**, so the recent DVC S3→R2 default-remote flip does **not**
159+
change the site's URLs (both remote sections share `url = s3://jc-taxes/...`). Verified: 200 OK.
160+
161+
Two consequences for this work:
162+
- Keeping the Phase-2 aggregates as **committed JSON** avoids adding any new S3/R2
163+
dependency for the homepage.
164+
- The MOD-IV spec's step 7 ("retire S3 after a few weeks") would **break the live map**
165+
as-is, since the map still pulls from `jc-taxes.s3.amazonaws.com`. Before retiring S3,
166+
either teach `vite-plugin-dvc` to emit a public R2 URL (`baseUrl` option / public
167+
`r2.dev` or custom domain) or keep S3 as the public web origin. Out of scope here, but
168+
do not let an S3 retirement land without addressing it.
169+
170+
## Honest caveats
171+
172+
- MOD-IV data is **2021–2025** only; the map's paid-tax data covers 2015–2025.
173+
- `net_value` is **assessed value**, not market value or tax paid; the MOD-IV
174+
`last_year_tax` is **billed**, not collected (the ~15–20% Treasury-vs-HLS crossval
175+
gap is largely PILOT/abated properties — a JC-specific quirk worth a footnote).
176+
- Assessment practices differ per muni (revaluation timing), so cross-muni
177+
*level* comparisons deserve a caveat; *trends* and *composition* are safer.
178+
179+
## Done criteria
180+
181+
- `/` is a styled landing page with copy + the charts above; `/map` is the current
182+
fullscreen experience; old deep links still resolve.
183+
- Charts driven by committed `www/public/data/*.json` from `jct modiv aggregate`.
184+
- Hudson-wide parse landed (all-muni parquets re-DVC-tracked); JC numbers unchanged.
185+
- Playwright e2e (`www/e2e/`) updated for the new route split; `pnpm build` green.
186+
- On completion, move this spec to `specs/done/` with an implementation report
187+
(final chart list, chart lib chosen, any data surprises), per the usual workflow.
188+
189+
[jct.rbw.sh]: https://jct.rbw.sh
190+
[`src/jc_taxes/modiv.py`]: ../src/jc_taxes/modiv.py
191+
[`specs/done/modiv-etl-and-r2-migration.md`]: done/modiv-etl-and-r2-migration.md
192+
[@floating-ui/react]: https://floating-ui.com/docs/react

0 commit comments

Comments
 (0)