Mapping the Physical Footprint of Quantitative Finance

Working repo for the essay + interactive map. Three-tier inventory of publicly documented quant/HFT infrastructure, with an evidence grade on every record.

Structure

• data/sites_seed.csv — the site inventory (72 records; target ~80-120)
• data/paths.csv — Tier-2 arcs (origin/dest site_ids, operator, medium), rendered as a kepler arc layer
• data/compute_sites.csv — the megawatts sheet: research-tier compute centres only, with GPU counts, power (MW), and build status (generated by pipeline/build_compute_sheet.py)
• data/evidence.csv — normalized multi-source table: one row per (record, distinct source URL), so each pin can show how many independent sources back it (generated by pipeline/build_evidence.py)
• build_map.py — CSVs → standalone kepler.gl HTML (python build_map.py --out docs/index.html); joins evidence.csv to show a source count per pin
• pipeline/geocode.py — Nominatim geocoder (rate-limited, cached) for verifying coordinates against documented addresses
• docs/index.html — the full three-layer kepler map (tracked; auto-deployed to GitHub Pages via .github/workflows/pages.yml); embeddable via iframe
• data/firm_datacenters.csv — the deep firm→data-centre mapping (167 sites across 88 firms; role, operator, location, GPU/MW where public, evidence), from a triple-checked research pass over the FIRMS.md universe (pipeline/assemble_datacenters.py, raw in data/raw/)
• docs/compute.html — the compute-layer view ("The megawatts of quant AI"): a deck.gl page driven by firm_datacenters.csv where each site is a 3D extruded column, height and orange-red colour = disclosed power (MW, √ scale); sites with no public MW show as dots. Built by pipeline/build_compute_view.py. HQ→data-centre arcs planned
• ENRICHMENT.md — candidate records awaiting research/verification

Regenerate derived files after editing sites_seed.csv: python pipeline/build_evidence.py && python pipeline/build_compute_sheet.py && python build_map.py --out docs/index.html

Schema

column	meaning
tier	execution (exchange colo) / network (microwave, shortwave) / research (ML compute)
coord_precision	exact (verified address/geocode) / approximate (needs verification) / city_level / symbolic (no single site exists, e.g. cloud usage)
confidence	confirmed (primary source: license, filing, corporate announcement) / reported (credible press) / inferred (triangulated from job ads, LinkedIn, network data)
evidence_type	spectrum_license / planning_docs / exchange_docs / corporate_announcement / operator_docs / press / regulatory_docs / academic / job_posting
status	active / historical (e.g. Basildon, Moscow) / under_construction
power_mw	numeric IT/campus power where a public source states it; blank otherwise
evidence_url	working public URL for the strongest source backing the record
verify_url_hint	legacy pointer for pre-enrichment records (being replaced by evidence_url)

data/paths.csv (Tier-2 arcs): path_id, origin_site_id, dest_site_id, operator, medium (microwave/mm-wave/laser/shortwave/fiber), status, confidence, evidence_note, evidence_url.

Publication rule: every pin must carry a public primary or credible secondary source. Nothing gets mapped that isn't already in the public record.

Evidence pipeline (in order of strength)

1. Spectrum licenses — FCC ULS (Part 101 + experimental), Ofcom fixed-links register, ANFR Cartoradio (FR), BIPT (BE). Exact tower coordinates + licensee shells traceable via Companies House / state registries.
2. Planning applications — councils/municipalities for towers and DC builds.
3. Corporate + contractor disclosure — press releases; contractor project pages.
4. Job postings / LinkedIn — location signals for unannounced presence.
5. Network forensics — PeeringDB, ASN registrations, exchange connectivity docs.
6. Industry databases — DataCenterMap, DCD archives, colo operator case studies.

Key references

• Alexandre Laumonier, Sniper in Mahwah (blog + books "6"/"5") — the Channel towers, shortwave series (with Bob Van Valzah)
• Donald MacKenzie, Trading at the Speed of Light (2021)
• Shkilko & Sokolov, "Every Cloud Has a Silver Lining" (microwave outages as natural experiment)
• Exchange colocation service documents (NYSE, Nasdaq, CME, DB, HKEX, JPX)

TODO

• Verify/refine all approximate coordinates (geocode against addresses)
• Replace verify_url_hint with real evidence URLs
• FCC ULS pull: script the Part 101 licensee extraction for known shells
• Add arc layer: tower-pair paths for Tier 2 (kepler arc layer, needs origin/destination pairs in a second CSV)
• Ofcom fixed-links + ANFR Cartoradio extraction for the London-Frankfurt corridor
• Expand Tier 3: Jane Street, HRT, Two Sigma, DE Shaw compute footprints (job-ad triangulation)
• Test kepler tooltip URL behavior in target blog; fall back to MapLibre popup page if links aren't clickable