3D scope support: --dimensions for UMAP, voxel indices, 3D-aware metadata - #154
Merged
Conversation
Thread an n_components choice (default 2) through the CPU, cuML, and AlignedUMAP reducer sites and both process_umap_embeddings paths. When dimensions=3 the parquet gains a normalized z column ([-1,1] like x/y), min/max_values extend to 3 entries automatically, and the umap json records "dimensions". The 2D matplotlib preview still plots x,y. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When the chosen umap has a z column, compute voxel_index_32/64 using the frozen row-major x-fastest cell-index convention (idx = (z_bin*n + y_bin)*n + x_bin), and record dimensions in the scope json. tile_index_64/128 stay 2D on x,y so existing consumers are unchanged; z and the voxel columns flow into scopes-NNN.parquet / -input.parquet via the umap join. make_tiles and make_voxels are lifted to importable module-level helpers. Cluster-quality metrics include z when present. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lock the §2.4 voxel index constants, assert the 3D umap/scope pipeline emits x,y,z + voxel_index_32/64 matching the frozen formula, and guard the 2D default path against z/voxel leakage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 956d567e9f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+18
to
+19
| parser.add_argument('--dimensions', type=int, help='Number of UMAP output dimensions (2 or 3)', | ||
| default=2) |
There was a problem hiding this comment.
When a user runs the new ls-umap --dimensions 3 and then follows the normal ls-cluster ... --method hdbscan/kmeans/gmm path, clusterer still builds the UMAP clustering input from only x and y and passes that for cluster_on='umap' (latentscope/scripts/cluster.py:238-258). The resulting scope contains z and voxel indices, but its cluster labels were computed in 2D, so 3D viewers and any 3D-aware downstream analysis can show labels that do not match the 3D geometry. Please either include all UMAP axes when clustering a 3D projection or reject/mark 3D UMAPs until that path is updated.
Useful? React with 👍 / 👎.
Owner
Author
There was a problem hiding this comment.
Fixed in b4e0d0f. cluster_on='umap' now builds the clustering input from all UMAP axes present in the projection: a 3D umap (with a z column) is clustered on (x, y, z) so its labels match the 3D geometry the voxel/3D-scatter views render (higher-dim d3, d4, … fold in too). A 2D umap still passes exactly the (n, 2) x, y array — byte-identical to the previous behavior. The separate 2D umap_embeddings array used for plotting, hulls, and noise reassignment is unchanged. Added two tests in tests/test_cluster_methods.py that spy on the clustering runner and assert the input is (n, 3) for a 3D umap and (n, 2) for a 2D umap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The umap step gains a Dimensions control (2D default / 3D). The web form passes `dimensions` to the /jobs/umap route, which appends --dimensions to the ls-umap command. The experiment gallery badges 3D umaps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- lib/clusterColor: cluster id -> hue (Tableau10/rainbow), mirroring useColorBy, so 3D scatter and voxels match the 2D map's cluster colors. - hooks/useCellMembers: client-side cell->members map (memoized) with an instant count+dominant-cluster summary and debounced, cached, cancel-on-move snippet fetching (150ms dwell) via a new apiService.getSnippets batch query. - Explore/MembersTooltip: floating 'N datapoints - dominant label + snippets' card, reused by the voxel view and the 2D heatmap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
VisualizationPane offers a [2D - 3D - Voxels] toggle when scope.dimensions === 3 (2D scopes keep the untouched ScatterGL path). New three.js components: - Scatter3D: soft-splat billboard points (FOV-correct size attenuation with min/max px clamps, ported from latent-renders pointset), cluster-colored, camera-controls orbit/pan/cursor-dolly, GPU color-ID picking wired into the existing hover/select flow (react-tooltip + PointDetail). Double-click flies to a point; selecting a cluster dims others (alpha 0.15) and fitToBox-frames it. - VoxelView: InstancedMesh cubes aggregated from voxel_index_64 (32 via a resolution toggle), colored by dominant cluster with a density luminance factor and lit for depth. A view-aligned slice plane (slider + shift-wheel, re-derived on controlend) keeps in-plane voxels opaque and fades the rest to 0.06; raycast hover on in-plane cells shows the members tooltip. - The 2D heatmap (TilePlot) now drives the same members tooltip from the hovered point's tile_index_64. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
For cluster_on='umap', the clustering input was built from only x,y even for 3D umaps (--dimensions 3), so a 3D scope's labels were computed in 2D and did not match its 3D geometry (voxel views, 3D scatter). Build the input from all umap axes present (x,y and z when the projection is 3D; d3,d4,... fold in too). 2D umaps keep exactly the x,y input array — byte-identical to before. The 2D projection used for plotting/hulls/noise reassignment is unchanged. Addresses the Codex P2 review on PR #154. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gallery preview PNG for a 3D umap was a flat x,y scatter. Render it as a matplotlib 3D scatter with subtle depth cueing (back-to-front draw order, size/ opacity/lightness falloff with camera distance) so the thumbnail reads as a volume. 2D umaps are unchanged. The PNG generation is refactored into a shared _save_umap_preview() helper used by both the dense and sparse umappers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…le, 3D preview Revisions to the 3D explore views (PR #154) from user feedback: - Color By: Scatter3D and VoxelView now honor the useColorBy state exactly like the 2D map — live categorical/continuous palettes on dropdown change, cluster palette as the default. Fixes "the 3d points dont seem to respect colorby". - Voxel slice: the plane orientation is derived ONCE from the initial view and held FIXED while orbiting (the slice becomes a static world-space slab); the slider / shift+wheel move depth along that fixed normal. Added an "Align to view" button to re-derive on demand; removed the controlend re-derivation. - Cell detail drawer: clicking a heatmap tile (2D) or voxel (3D) opens a drawer listing every datapoint in the cell (count + dominant cluster header, collapsible ~120-char entries, windowed pagination, deep-link to PointDetail). - Mobile: touch camera mapping (one-finger orbit, pinch zoom, two-finger pan), tap-to-select routed to the detail drawer, DPR cap + larger min point size, and top-anchored voxel controls that clear the bottom sheet. - Setup: 3D umaps get an interactive lightweight Scatter3D preview (orbit only, no picking/tooltip) via a new `lightweight` mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Python: umapper() supports both dimensions (3D umaps) and register_to (incremental/aligned umaps) — registration stays 2D-only by construction and raises clearly on a 3D target; transform_umap inherits the source umap's dimensionality. Job routes carry both sides' argument sets. Cluster tests from both branches kept. Web: the 3D rendering paths (Scatter3D, VoxelView, cell drawer, members tooltip) now live inside the restyled Amber Console Explore pane, and the new 3d components adopt the design system: token-driven canvas chrome via useColorMode, CellDetail drawer on PointDetail's drawer treatment with shared Pagination, MembersTooltip on floating panel tokens, VoxelView HUD as .ls-panel--floating with mono SLICE/RES readouts and chip toggles, view-mode switch as .ls-tab segmented tabs, Umap's 2D/3D select styled like its sibling fields with a mono 3D badge on gallery cards. Verified: ruff clean, pytest 295 passed, eslint 0 errors, vitest 103 passed, production build clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
enjalot
added a commit
that referenced
this pull request
Jul 11, 2026
Amber Console restyle of 3d components (follow-up to #154)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds 3D data production to latent-scope (no renderer changes — 3D viewing happens in external tools like latent-renders):
umapper.py: new--dimensionsoption (default 2). With--dimensions 3, the umap parquet gains azcolumn normalized to [-1,1] like x/y,min_values/max_valuesextend to 3 entries, and the json records"dimensions": 3. Threaded through CPU, cuML, and AlignedUMAP paths; 2D preview PNG unchanged.scope.py: when the chosen umap hasz, scopes also computevoxel_index_32/voxel_index_64columns using the frozen conventionidx = (z_bin*n + y_bin)*n + x_bin,bin(c,n) = clip(floor((c+1)/2*n), 0, n-1).tile_index_64/128remain 2D so existing consumers are untouched. Scope json records top-level"dimensions".server/datasets.py: cluster-quality metrics include z when present.sprite_atlas.py/export_plot.pyverified safe (they select x,y explicitly).Testing
tests/test_pipeline_e2e.py; full suite 222 passed, 2 skipped (baseline main: 219/2 — no regressions), ruff clean.dataispluraldataset: 3D umap → scope with z + voxel columns → lancedb export, all verified including an independent numpy recompute of the voxel formula."dimensions": 2metadata field.How external consumers detect 3D
Read
scopes-NNN.json→ top-level"dimensions": 3(or feature-detectzincolumns).voxel_index_*mod (n·n) equals the 2D tile index at that resolution.🤖 Generated with Claude Code
Frontend
The web UI now produces AND explores 3D scopes end-to-end. Added deps:
three,camera-controls.Setup — Dimensions control. The UMAP step gains a
Dimensionsdropdown (2D default / 3D). The web form threadsdimensionsto the/jobs/umaproute, which appends--dimensionsto thels-umapcommand; the experiment gallery badges 3D umaps and the scope builder shows them. Verified against the live server: running a 3D umap from this path produced az-bearing umap (x,y,zall in [-1,1]) with"dimensions": 3and the 3D badge in the gallery.Explore — [2D · 3D · Voxels] toggle (shown only when
scope.dimensions === 3; 2D scopes keep the untouchedScatterGLpath → zero regression).Scatter3D.jsx, three.js): soft round splats with FOV-correct size attenuation + min/max pixel clamps (ported from the latent-renderspointsetsplat shader), colored by cluster with the same palette as the 2D map (lib/clusterColor, mirroringuseColorBy); deleted points hidden.camera-controlsorbit / two-finger-and-right-drag pan / cursor-centered dolly with inertia; double-click flies to a point. GPU color-ID picking (full-buffer render + 1px readback — the ANGLE/Metal-safe path) feeds the existing hover flow (immediate highlight + debouncedgetHoverText→ the existingreact-tooltip) and click → the existingPointDetaildrawer. Selecting a cluster (existingclusterFilterstate, e.g.?cluster=8) dims non-members to alpha ~0.15 and animatesfitToBoxto frame it.VoxelView.jsx): client-side aggregation ofvoxel_index_64(fallbackvoxel_index_32via a resolution toggle) into anInstancedMeshof lit cubes, colored by dominant cluster with a density-luminance factor. A slice plane driven by a slider and shift+wheel; in-plane voxels render opaque, others fade to ~0.06 via a signed-distance uniform. Raycast hover on in-plane cells shows the members tooltip. (See revision round below: the plane is now static, Color By is respected, and clicking a cell opens the detail drawer.)useCellMembershook +MembersTooltip): one reusable implementation. MemoizedMapcell→member indices fromscopeRows; instant "N datapoints · dominant cluster label" summary; up to 6 sampled snippets fetched via a new batchedapiService.getSnippets(POST/api/query), 150ms dwell, cached per cell, cancel-on-move. Wired into both the voxel hover and the 2DTilePlotheatmap hover (works on 2D scopes too).Testing. Frontend suite green (103 tests, 14 files); Vite production build green; Python suite still 222 passed / 2 skipped. Driven through the real UI with Playwright (headless:false for Metal WebGL) against the live server — screenshots captured and reviewed for: 3D scatter default, hover tooltip, cluster fit-to-view + dim, voxel view sliced mid-cloud, members tooltip on a voxel, members tooltip on the 2D heatmap, and the Setup Dimensions control + 3D gallery badge, on both
dataisplural(small) and a newfineweb-edu-100k3D scope (100k points).Try it (live build served at the test server):
/datasets/fineweb-edu-100k/explore/scopes-002/datasets/fineweb-edu-100k/explore/scopes-002?cluster=3/datasets/dataisplural/explore/scopes-001→ ⚙ → Show Heat Map → hoverFrontend — revision round (feedback)
Six revisions from review of the first pass:
_save_umap_preview()inumapper.py) and (b) an interactive mini 3D view in the selected-umap preview panel —Scatter3Dgains alightweightmode (points + orbit only, no picking/tooltip) mounted inSetup/Preview.jsx.Scatter3DandVoxelViewnow consume the existinguseColorBystate exactly like the 2D map: live categorical/continuous palettes update on dropdown change (in-place instance-color rewrite, no rebuild), and the shared cluster palette is the default. Voxels average their members' color-by hues per cell.controlendre-derivation.CellDetail.jsx). Clicking a heatmap tile (2D) or a voxel (3D) opens a right drawer listing every datapoint in the cell: header with count + dominant cluster label, scrollable body of collapsible entries (~120-char preview → expand to full text), windowed pagination for large cells, and a per-entry deep-link into the fullPointDetaildrawer. ReusesuseCellMembers; follows thePointDetailvisual patterns.MobileExplorenow wiresPointDetail); DPR cap + larger minimum point size for legibility at 100k on a phone; voxel slice controls anchor to the top on small screens so they clear the data-table sheet. Verified with Playwright iPhone emulation (390×844,hasTouch, DPR 3).cluster.py, Codex P2).cluster_on='umap'clusters on all UMAP axes (x,y,z for 3D) instead of the 2D shadow, so 3D labels match the 3D geometry; 2D is byte-identical. +2 tests.Testing (revision round). Python suite 224 passed / 2 skipped (was 222/2 — +2 new cluster tests); frontend 103 tests green; Vite production build green; ruff + eslint clean. Reviewed with Playwright (desktop + iPhone emulation,
headless:falsefor Metal WebGL): Color By on the 3D scatter (Viridis) and voxels, the static slice slab after orbiting, the cell drawer (desktop + mobile), the mobile 3D scatter and voxel views with tap-to-drawer, and the interactive Setup 3D preview.