Skip to content

Commit 5336311

Browse files
whiteskylondonclaude
whiteskylondon
and
claude
committed
Add compute-layer view: 3D extruded MW columns (docs/compute.html)
pipeline/build_compute_view.py renders a deck.gl + MapLibre page focused on Layer 3 (compute): every data centre is a 3D column whose height and orange-red colour encode disclosed power (MW); sites with no public MW show as dots so absence stays visible. Tooltips, MW legend, 3D pitch. Currently 3 MW columns (Verne 140, XTX Kajaani 22.5, Fire-Flyer 3) + 17 sites total; column height auto-normalizes, so it scales as the deep data-center pass fills in more sites. HQ->data-centre arcs to follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 9dc31a6 commit 5336311

3 files changed

Lines changed: 273 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,12 @@ documented quant/HFT infrastructure, with an evidence grade on every record.
1919
source count per pin
2020
- `pipeline/geocode.py` — Nominatim geocoder (rate-limited, cached) for
2121
verifying coordinates against documented addresses
22-
- `docs/index.html` — output (tracked; auto-deployed to GitHub Pages via
23-
`.github/workflows/pages.yml`); embeddable in Ghost via iframe
22+
- `docs/index.html` — the full three-layer kepler map (tracked; auto-deployed
23+
to GitHub Pages via `.github/workflows/pages.yml`); embeddable via iframe
24+
- `docs/compute.html` — the compute-layer view ("The megawatts of quant AI"):
25+
a deck.gl page where each data centre is a 3D extruded column, height and
26+
orange-red colour = disclosed power (MW); undisclosed sites show as dots.
27+
Built by `pipeline/build_compute_view.py`. HQ→data-centre arcs planned
2428
- `ENRICHMENT.md` — candidate records awaiting research/verification
2529

2630
Regenerate derived files after editing `sites_seed.csv`:

‎docs/compute.html‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8" />
5+
<meta name="viewport" content="width=device-width, initial-scale=1" />
6+
<title>Quant AI compute — megawatts</title>
7+
<link href="https://cdn.jsdelivr.net/npm/maplibre-gl@4.5.0/dist/maplibre-gl.css" rel="stylesheet" />
8+
<script src="https://cdn.jsdelivr.net/npm/maplibre-gl@4.5.0/dist/maplibre-gl.js"></script>
9+
<script src="https://cdn.jsdelivr.net/npm/deck.gl@9.0.32/dist.min.js"></script>
10+
<style>
11+
html, body { margin: 0; height: 100%; background: #0a0a0f; }
12+
#map { position: absolute; inset: 0; }
13+
.panel { position: absolute; z-index: 2; color: #e8e6e3;
14+
font: 13px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
15+
#title { top: 14px; left: 16px; max-width: 340px; }
16+
#title h1 { font-size: 16px; font-weight: 600; margin: 0 0 4px; }
17+
#title p { margin: 0; color: #9a978f; font-size: 12px; }
18+
#legend { bottom: 16px; left: 16px; background: rgba(16,16,22,0.78);
19+
border: 1px solid rgba(255,255,255,0.14); border-radius: 8px; padding: 10px 12px; }
20+
#legend .bar { height: 9px; width: 180px; border-radius: 3px;
21+
background: linear-gradient(90deg, #fac775, #ef7f2a, #b2182b); margin: 5px 0 3px; }
22+
#legend .row { display: flex; justify-content: space-between; font-size: 11px; color: #9a978f; }
23+
#legend .note { margin-top: 7px; font-size: 11px; color: #9a978f; max-width: 210px; }
24+
#legend .dot { display: inline-block; width: 8px; height: 8px; border-radius: 50%;
25+
background: #7f8a99; border: 1px solid rgba(255,255,255,0.5); vertical-align: 1px; margin-right: 3px; }
26+
.deck-tooltip { font: 12px/1.45 -apple-system, sans-serif !important;
27+
background: rgba(16,16,22,0.92) !important; border: 1px solid rgba(255,255,255,0.16) !important;
28+
border-radius: 6px !important; color: #e8e6e3 !important; }
29+
</style>
30+
</head>
31+
<body>
32+
<div id="map"></div>
33+
<div id="title" class="panel">
34+
<h1>The megawatts of quant AI</h1>
35+
<p>Each column is a firm's compute site — height &amp; colour = disclosed power (MW).
36+
Dots are sites with no public MW figure.</p>
37+
</div>
38+
<div id="legend" class="panel">
39+
<div style="font-weight:600; font-size:12px;">Disclosed power (MW)</div>
40+
<div class="bar"></div>
41+
<div class="row"><span id="mwmin">0</span><span id="mwmax">MW</span></div>
42+
<div class="note"><span class="dot"></span>site with no public MW figure (undisclosed)</div>
43+
</div>
44+
<script>
45+
const DATA = {"columns": [{"lon": 120.1551, "lat": 30.2741, "firm": "High-Flyer Quant (幻方量化)", "site": "High-Flyer Fire-Flyer 2 (萤火二号) AI cluster — undisclosed location, symbolic pin at Hangzhou HQ", "city": "Hangzhou", "status": "active", "mw": 3.0, "gpu": 10000}, {"lon": -22.5758, "lat": 63.9778, "firm": "XTX Markets (tenant)", "site": "Verne Global Keflavik campus (XTX Markets + HPC tenants)", "city": "Keflavik", "status": "active", "mw": 140.0, "gpu": null}, {"lon": 27.68174, "lat": 64.22312, "firm": "XTX Markets", "site": "XTX Markets Kajaani campus", "city": "Kajaani", "status": "under_construction", "mw": 22.5, "gpu": 25000}], "points": [{"lon": -80.19243, "lat": 25.766, "firm": "Citadel Securities", "site": "Citadel Securities on Google Cloud", "city": "Miami FL", "status": "active", "mw": null, "gpu": null}, {"lon": -0.1301, "lat": 51.5162, "firm": "G-Research", "site": "G-Research ML compute (cluster locations undisclosed; London research lab)", "city": "London", "status": "active", "mw": null, "gpu": null}, {"lon": 120.1551, "lat": 30.2741, "firm": "High-Flyer Quant (幻方量化)", "site": "High-Flyer Fire-Flyer 2 (萤火二号) AI cluster — undisclosed location, symbolic pin at Hangzhou HQ", "city": "Hangzhou", "status": "active", "mw": 3.0, "gpu": 10000}, {"lon": 5.5062, "lat": 61.932, "firm": "Hudson River Trading", "site": "Hudson River Trading AI research data center — Lefdal Mine Datacenter (Norway)", "city": "Lefdal, Stad municipality (near Måløy)", "status": "active", "mw": null, "gpu": null}, {"lon": -74.006, "lat": 40.7128, "firm": "Hudson River Trading", "site": "Hudson River Trading cloud research compute — Google Cloud + Lambda (symbolic pin, New York)", "city": "New York", "status": "active", "mw": null, "gpu": null}, {"lon": -96.797, "lat": 32.7767, "firm": "Jane Street", "site": "Jane Street AI training data center (Dallas, TX — city-level)", "city": "Dallas", "status": "active", "mw": null, "gpu": 4032}, {"lon": -74.0154, "lat": 40.7147, "firm": "Jane Street", "site": "Jane Street on CoreWeave AI cloud (symbolic pin, NYC HQ)", "city": "New York", "status": "active", "mw": null, "gpu": null}, {"lon": -87.644, "lat": 41.8965, "firm": "Jump Trading", "site": "Jump Trading AI research cluster (Vera Rubin NVL72 — symbolic pin, Chicago HQ)", "city": "Chicago", "status": "active", "mw": null, "gpu": null}, {"lon": -96.89, "lat": 32.9756, "firm": "Jump Trading", "site": "Jump Trading HPC data center (Carrollton, TX — city-level, from job listings)", "city": "Carrollton", "status": "active", "mw": null, "gpu": null}, {"lon": 121.4737, "lat": 31.2304, "firm": "Minghong Investment (明汯投资)", "site": "Minghong Investment HPC cluster (明汯投资) — symbolic pin at Shanghai HQ", "city": "Shanghai (HQ city; cluster location undisclosed)", "status": "active", "mw": null, "gpu": 1500}, {"lon": -73.1106, "lat": 40.93068, "firm": "Renaissance Technologies", "site": "Renaissance Technologies East Setauket campus", "city": "East Setauket NY", "status": "active", "mw": null, "gpu": null}, {"lon": -74.0048, "lat": 40.7247, "firm": "Two Sigma", "site": "Two Sigma private datacenters + Google Cloud (symbolic pin, New York HQ)", "city": "New York", "status": "active", "mw": null, "gpu": null}, {"lon": 116.4074, "lat": 39.9042, "firm": "Ubiquant (九坤投资)", "site": "Ubiquant 'Bei Ming' AI supercomputing cluster (北溟超算集群) — symbolic pin at Beijing HQ", "city": "Beijing (HQ city; cluster location undisclosed)", "status": "active", "mw": null, "gpu": null}, {"lon": -22.5758, "lat": 63.9778, "firm": "XTX Markets (tenant)", "site": "Verne Global Keflavik campus (XTX Markets + HPC tenants)", "city": "Keflavik", "status": "active", "mw": 140.0, "gpu": null}, {"lon": 113.11, "lat": 41.02, "firm": "DeepSeek (High-Flyer lineage)", "site": "DeepSeek Ulanqab data centre (Inner Mongolia, signalled by job postings)", "city": "Ulanqab", "status": "under_construction", "mw": null, "gpu": null}, {"lon": 27.68174, "lat": 64.22312, "firm": "XTX Markets", "site": "XTX Markets Kajaani campus", "city": "Kajaani", "status": "under_construction", "mw": 22.5, "gpu": 25000}, {"lon": 27.68174, "lat": 64.22312, "firm": "XTX Markets", "site": "XTX Markets Kajaani — Building 2 (second data center, Sokajärventie)", "city": "Kajaani", "status": "under_construction", "mw": null, "gpu": null}]};
46+
const {MapboxOverlay, ColumnLayer, ScatterplotLayer} = deck;
47+
48+
const mws = DATA.columns.map(d => d.mw);
49+
const MAXMW = Math.max(80, ...mws);
50+
document.getElementById('mwmax').textContent = Math.round(MAXMW) + ' MW';
51+
52+
// orange -> red ramp on sqrt(MW) so small sites stay distinguishable
53+
function mwColor(mw) {
54+
const t = Math.sqrt(mw) / Math.sqrt(MAXMW);
55+
const stops = [[250,199,117],[239,127,42],[178,24,43]];
56+
const seg = t <= 0.5 ? 0 : 1, f = t <= 0.5 ? t/0.5 : (t-0.5)/0.5;
57+
const a = stops[seg], b = stops[seg+1];
58+
return [0,1,2].map(i => Math.round(a[i] + (b[i]-a[i])*f)).concat(255);
59+
}
60+
const STATUS = {operational:[90,170,160], under_construction:[232,163,61],
61+
planned:[130,138,153], historical:[110,110,120]};
62+
63+
function layers() {
64+
return [
65+
new ScatterplotLayer({
66+
id: 'sites', data: DATA.points, radiusUnits: 'pixels',
67+
getPosition: d => [d.lon, d.lat], getRadius: 6, radiusMinPixels: 4,
68+
getFillColor: d => (STATUS[d.status] || [130,138,153]).concat(220),
69+
stroked: true, lineWidthMinPixels: 1, getLineColor: [255,255,255,120],
70+
pickable: true,
71+
}),
72+
new ColumnLayer({
73+
id: 'mw', data: DATA.columns, diskResolution: 24, radius: 30000,
74+
extruded: true, radiusUnits: 'meters', elevationScale: 560000 / MAXMW,
75+
getPosition: d => [d.lon, d.lat], getElevation: d => d.mw,
76+
getFillColor: d => mwColor(d.mw), pickable: true,
77+
material: {ambient: 0.64, diffuse: 0.6, shininess: 28, specularColor: [60,60,60]},
78+
}),
79+
];
80+
}
81+
82+
const map = new maplibregl.Map({
83+
container: 'map', style: 'https://basemaps.cartocdn.com/gl/dark-matter-gl-style/style.json',
84+
center: [2, 34], zoom: 1.15, pitch: 42, bearing: 0, antialias: true,
85+
maxPitch: 70,
86+
});
87+
map.addControl(new maplibregl.NavigationControl({visualizePitch: true}), 'top-right');
88+
map.on('load', () => {
89+
map.addControl(new MapboxOverlay({
90+
interleaved: true, layers: layers(),
91+
getTooltip: ({object}) => object && {html:
92+
`<b>${object.firm}</b><br>${object.site}<br>${object.city} · ${object.status}`
93+
+ `<br>MW: ${object.mw ?? '—'} · GPUs: ${object.gpu ? object.gpu.toLocaleString() : '—'}`},
94+
}));
95+
});
96+
</script>
97+
</body>
98+
</html>

‎pipeline/build_compute_view.py‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
"""
2+
Build docs/compute.html — the compute-layer view.
3+
4+
A focused deck.gl page for Layer 3 (research/ML compute): every data centre is
5+
a 3D extruded column whose HEIGHT and orange-red COLOR encode disclosed power
6+
(MW). Sites with no public MW still appear as dots, so absence stays visible.
7+
HQ->data-centre arcs will layer on later (ArcLayer, same page).
8+
9+
Renderer: MapLibre GL basemap (CARTO Dark Matter, no token) + a deck.gl
10+
MapboxOverlay, both from CDN — fine for a GitHub Pages page (no artifact CSP).
11+
12+
Reads data/compute_sites.csv (later: the deep firm_datacenters.csv). Usage:
13+
python pipeline/build_compute_view.py [--csv data/compute_sites.csv]
14+
[--out docs/compute.html]
15+
"""
16+
17+
from __future__ import annotations
18+
19+
import argparse
20+
import json
21+
from pathlib import Path
22+
23+
import pandas as pd
24+
25+
ROOT = Path(__file__).resolve().parent.parent
26+
DECK = "https://cdn.jsdelivr.net/npm/deck.gl@9.0.32/dist.min.js"
27+
MAPLIBRE_JS = "https://cdn.jsdelivr.net/npm/maplibre-gl@4.5.0/dist/maplibre-gl.js"
28+
MAPLIBRE_CSS = "https://cdn.jsdelivr.net/npm/maplibre-gl@4.5.0/dist/maplibre-gl.css"
29+
BASEMAP = "https://basemaps.cartocdn.com/gl/dark-matter-gl-style/style.json"
30+
31+
32+
def load(csv_path: Path) -> tuple[list, list]:
33+
df = pd.read_csv(csv_path, dtype=str).fillna("")
34+
df = df[(df.lat != "") & (df.lon != "")].copy()
35+
df["lat"] = df.lat.astype(float)
36+
df["lon"] = df.lon.astype(float)
37+
df["mw"] = pd.to_numeric(df.power_mw, errors="coerce")
38+
df["gpu"] = pd.to_numeric(df.get("gpu_count", ""), errors="coerce")
39+
40+
def rec(r):
41+
return {
42+
"lon": r.lon, "lat": r.lat,
43+
"firm": r.firm, "site": r.site_name, "city": r.city,
44+
"status": r.status,
45+
"mw": None if pd.isna(r.mw) else float(r.mw),
46+
"gpu": None if pd.isna(r.gpu) else int(r.gpu),
47+
}
48+
49+
points = [rec(r) for r in df.itertuples()]
50+
columns = [p for p in points if p["mw"]]
51+
return columns, points
52+
53+
54+
def html(columns: list, points: list) -> str:
55+
data = json.dumps({"columns": columns, "points": points}, ensure_ascii=False)
56+
return f"""<!DOCTYPE html>
57+
<html lang="en">
58+
<head>
59+
<meta charset="utf-8" />
60+
<meta name="viewport" content="width=device-width, initial-scale=1" />
61+
<title>Quant AI compute — megawatts</title>
62+
<link href="{MAPLIBRE_CSS}" rel="stylesheet" />
63+
<script src="{MAPLIBRE_JS}"></script>
64+
<script src="{DECK}"></script>
65+
<style>
66+
html, body {{ margin: 0; height: 100%; background: #0a0a0f; }}
67+
#map {{ position: absolute; inset: 0; }}
68+
.panel {{ position: absolute; z-index: 2; color: #e8e6e3;
69+
font: 13px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }}
70+
#title {{ top: 14px; left: 16px; max-width: 340px; }}
71+
#title h1 {{ font-size: 16px; font-weight: 600; margin: 0 0 4px; }}
72+
#title p {{ margin: 0; color: #9a978f; font-size: 12px; }}
73+
#legend {{ bottom: 16px; left: 16px; background: rgba(16,16,22,0.78);
74+
border: 1px solid rgba(255,255,255,0.14); border-radius: 8px; padding: 10px 12px; }}
75+
#legend .bar {{ height: 9px; width: 180px; border-radius: 3px;
76+
background: linear-gradient(90deg, #fac775, #ef7f2a, #b2182b); margin: 5px 0 3px; }}
77+
#legend .row {{ display: flex; justify-content: space-between; font-size: 11px; color: #9a978f; }}
78+
#legend .note {{ margin-top: 7px; font-size: 11px; color: #9a978f; max-width: 210px; }}
79+
#legend .dot {{ display: inline-block; width: 8px; height: 8px; border-radius: 50%;
80+
background: #7f8a99; border: 1px solid rgba(255,255,255,0.5); vertical-align: 1px; margin-right: 3px; }}
81+
.deck-tooltip {{ font: 12px/1.45 -apple-system, sans-serif !important;
82+
background: rgba(16,16,22,0.92) !important; border: 1px solid rgba(255,255,255,0.16) !important;
83+
border-radius: 6px !important; color: #e8e6e3 !important; }}
84+
</style>
85+
</head>
86+
<body>
87+
<div id="map"></div>
88+
<div id="title" class="panel">
89+
<h1>The megawatts of quant AI</h1>
90+
<p>Each column is a firm's compute site — height &amp; colour = disclosed power (MW).
91+
Dots are sites with no public MW figure.</p>
92+
</div>
93+
<div id="legend" class="panel">
94+
<div style="font-weight:600; font-size:12px;">Disclosed power (MW)</div>
95+
<div class="bar"></div>
96+
<div class="row"><span id="mwmin">0</span><span id="mwmax">MW</span></div>
97+
<div class="note"><span class="dot"></span>site with no public MW figure (undisclosed)</div>
98+
</div>
99+
<script>
100+
const DATA = {data};
101+
const {{MapboxOverlay, ColumnLayer, ScatterplotLayer}} = deck;
102+
103+
const mws = DATA.columns.map(d => d.mw);
104+
const MAXMW = Math.max(80, ...mws);
105+
document.getElementById('mwmax').textContent = Math.round(MAXMW) + ' MW';
106+
107+
// orange -> red ramp on sqrt(MW) so small sites stay distinguishable
108+
function mwColor(mw) {{
109+
const t = Math.sqrt(mw) / Math.sqrt(MAXMW);
110+
const stops = [[250,199,117],[239,127,42],[178,24,43]];
111+
const seg = t <= 0.5 ? 0 : 1, f = t <= 0.5 ? t/0.5 : (t-0.5)/0.5;
112+
const a = stops[seg], b = stops[seg+1];
113+
return [0,1,2].map(i => Math.round(a[i] + (b[i]-a[i])*f)).concat(255);
114+
}}
115+
const STATUS = {{operational:[90,170,160], under_construction:[232,163,61],
116+
planned:[130,138,153], historical:[110,110,120]}};
117+
118+
function layers() {{
119+
return [
120+
new ScatterplotLayer({{
121+
id: 'sites', data: DATA.points, radiusUnits: 'pixels',
122+
getPosition: d => [d.lon, d.lat], getRadius: 6, radiusMinPixels: 4,
123+
getFillColor: d => (STATUS[d.status] || [130,138,153]).concat(220),
124+
stroked: true, lineWidthMinPixels: 1, getLineColor: [255,255,255,120],
125+
pickable: true,
126+
}}),
127+
new ColumnLayer({{
128+
id: 'mw', data: DATA.columns, diskResolution: 24, radius: 30000,
129+
extruded: true, radiusUnits: 'meters', elevationScale: 560000 / MAXMW,
130+
getPosition: d => [d.lon, d.lat], getElevation: d => d.mw,
131+
getFillColor: d => mwColor(d.mw), pickable: true,
132+
material: {{ambient: 0.64, diffuse: 0.6, shininess: 28, specularColor: [60,60,60]}},
133+
}}),
134+
];
135+
}}
136+
137+
const map = new maplibregl.Map({{
138+
container: 'map', style: '{BASEMAP}',
139+
center: [2, 34], zoom: 1.15, pitch: 42, bearing: 0, antialias: true,
140+
maxPitch: 70,
141+
}});
142+
map.addControl(new maplibregl.NavigationControl({{visualizePitch: true}}), 'top-right');
143+
map.on('load', () => {{
144+
map.addControl(new MapboxOverlay({{
145+
interleaved: true, layers: layers(),
146+
getTooltip: ({{object}}) => object && {{html:
147+
`<b>${{object.firm}}</b><br>${{object.site}}<br>${{object.city}} · ${{object.status}}`
148+
+ `<br>MW: ${{object.mw ?? '—'}} · GPUs: ${{object.gpu ? object.gpu.toLocaleString() : '—'}}`}},
149+
}}));
150+
}});
151+
</script>
152+
</body>
153+
</html>
154+
"""
155+
156+
157+
def main() -> None:
158+
p = argparse.ArgumentParser()
159+
p.add_argument("--csv", default=ROOT / "data" / "compute_sites.csv", type=Path)
160+
p.add_argument("--out", default=ROOT / "docs" / "compute.html", type=Path)
161+
args = p.parse_args()
162+
columns, points = load(args.csv)
163+
args.out.write_text(html(columns, points), encoding="utf-8")
164+
print(f"Wrote {args.out} — {len(points)} compute sites, "
165+
f"{len(columns)} with disclosed MW (columns)")
166+
167+
168+
if __name__ == "__main__":
169+
main()

0 commit comments

Comments
 (0)