feat(stage6): static-JSON-shard UI rewrite on GitHub Pages — US1–US8#9
Merged
Conversation
…der site
First Stage 6 PR per the Session-2026-05-17 sequencing decision: ship the
deploy workflows + minimal placeholder so US1–US7 can be reviewed via live
PR previews surfaced in the PR's Deployments box (not bot comments).
What's in it:
- src/ohbm2026/ui_data/ — Python data-package builders (manifest, abstracts,
authors, cells, topics, state_key, builder). Every shard is an object
envelope with a byte-identical build_info block; raw-array shards are
forbidden (FR-019 + CA-008). The builder discovers cells/inputs/facet
catalogs from the Stage 4 rollup at build time (CA-007).
- scripts/build_ui_data.py + scripts/fetch_ui_inputs.sh.
- site/ — SvelteKit 2 + Vite 6 + TypeScript with adapter-static and
BASE_PATH support for per-PR subdirectory deploys. The placeholder route
renders manifest.build_info; the short committish appears in the page
<title> suffix + the persistent footer affordance so reviewers verify
which commit a PR-preview deploy reflects without DevTools (FR-022 +
SC-011).
- .github/workflows/{deploy-ui,pr-preview,pr-preview-cleanup}.yml. The
pr-preview workflow declares environment: name=pr-preview-<N> so the
URL surfaces in the PR's Deployments box; cleanup uses the Deployments
API to mark each pr-preview-<N> deployment inactive on PR close.
- tests/test_ui_data_*.py — 23 unit tests (incl. AST scan that fails on
hardcoded facet literals + the "every shard carries build_info"
invariant T020a).
Smoke-tested against the real 3,244-abstract corpus: 15 cell shards + 33
topic shards + manifest + abstracts + authors at 4.3 MB gzipped (vs
SC-006's 11 MB budget). pnpm build produces a working static site.
Deferred to follow-up PRs:
- Invariant 4 (author raw→synthetic id remap) — WARN-only; lands in US1.
- T026 (real-PR Deployments-box verification) — requires a live PR run.
- US1–US7 user-story content.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request establishes the foundational infrastructure for the Stage 6 UI rewrite, introducing a Python-based data-package builder and a SvelteKit site scaffold. Key additions include the ui_data package for generating JSON shards from Stage 1–4 artifacts and GitHub Action workflows for automated production and PR-preview deployments. Feedback from the review identifies a potential OSError during atomic directory renames across different filesystems, contradictory documentation regarding state-key discovery logic, and a request to move redundant inline imports to the module level for better maintainability.
Comment on lines
+136
to
+137
| output = Path(output_dir) | ||
| with tempfile.TemporaryDirectory(prefix=".ui-data-", dir=output.parent if output.parent.exists() else None) as tmp_root: |
There was a problem hiding this comment.
Renaming a directory across different filesystems (e.g., from a system temp directory to the project directory) will fail with an OSError (Invalid cross-device link). The current implementation only uses output.parent as the base for the temporary directory if it already exists. If the parent directory needs to be created (as handled on line 172), the temporary directory will be created in the default system location, making the subsequent rename on line 173 fragile. Ensuring the parent exists before creating the temp dir solves this and makes the move atomic.
Suggested change
| output = Path(output_dir) | |
| with tempfile.TemporaryDirectory(prefix=".ui-data-", dir=output.parent if output.parent.exists() else None) as tmp_root: | |
| output = Path(output_dir) | |
| output.parent.mkdir(parents=True, exist_ok=True) | |
| with tempfile.TemporaryDirectory(prefix=".ui-data-", dir=output.parent) as tmp_root: |
Comment on lines
+50
to
+57
| def discover_rollup_state_key(analysis_root: Path) -> str: | ||
| """Return the active Stage 4 rollup state-key under *analysis_root*. | ||
|
|
||
| Globs ``annotations__*.sqlite`` and picks the most recent by mtime. | ||
| Raises :class:`Stage6BuildError` when zero matches are found OR when | ||
| multiple matches exist and the operator hasn't disambiguated (forcing | ||
| explicit selection in production builds). | ||
| """ |
There was a problem hiding this comment.
The docstring for discover_rollup_state_key is contradictory. It states that the function "picks the most recent by mtime," but the implementation explicitly raises a Stage6BuildError if more than one candidate is found (lines 75-79). If the intention is to fail on ambiguity (consistent with Principle VI), the docstring should be updated to remove the claim about picking the most recent. If picking the most recent was intended, the logic from discover_minilm_bundle (sorting by mtime) should be applied here.
Comment on lines
+165
to
+169
| import shutil | ||
| shutil.rmtree(backup) | ||
| output.rename(backup) | ||
| staging.rename(output) | ||
| import shutil |
There was a problem hiding this comment.
Avoid inline imports and redundant imports. shutil should be imported at the top of the module per PEP 8. The current placement is messy and results in multiple import attempts within the same function. While technically functional, it hinders readability and maintainability.
…nifest.json The first US8 deploy reached the live URL but the placeholder rendered "build info unavailable" — the workflow's data-package build was skipped because Stage 1–4 inputs aren't materialized in CI yet, so manifest.json was absent, so the page had no SHA to display. That defeats the FR-022 verification this whole sequencing decision was designed for. Fix: inject the build's git SHA at SITE build time via Vite env vars (VITE_BUILD_SHA + VITE_BUILD_SHA_SHORT + VITE_BUILD_AT). The placeholder falls back to the env-injected SHA when manifest.json is absent, so the short committish always appears in the page <title> and the persistent footer affordance — regardless of whether the data package was built. Also enable prerender on the route so the SHA bakes into the static HTML (visible via view-source, not only after JS hydration). Removed the stub favicon link that was breaking the prerender pass with a 404, and renamed the adapter-static fallback to 404.html so it doesn't overwrite the prerendered index.html. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real US1 home page on top of the US8 placeholder + workflows. The site now loads the abstracts/authors shards, supports full-text search across title / poster_id / topics / methods / authors / facets (diacritic-insensitive for author search per FR-010), and opens a detail panel with poster_id as the header (not submission_id, FR-002), ordered author list with affiliations (FR-003), all four abstract sections, Topics + Methods extra-questions only (FR-011), and references opening in new tabs (FR-012). Bonus fix: closed cross-shard invariant 4 — `build_authors` now returns the raw→synthetic id remap and `build_abstracts` uses it, so every `author_id` in `abstracts.json` resolves to a record in `authors.json`. 20,513 author refs across the real 3,244-abstract corpus all resolve. The WARN that was deferred in the prior PR is gone. Also drops the 1 accepted record without a poster_id (FR-002 — without a program-assigned id it can't be linked to or displayed). Test surface: - Vitest unit tests (20 green): shards loaders + cart store + filter math. Required a setup file to polyfill localStorage around Node 25's native shim that lacks `removeItem`. - Playwright e2e (6 green): browse → detail flow, mobile-viewport no-horizontal-scroll at 360 × 640 (SC-004), accepted-only invariant scanned via `window.__abstracts`, FR-011 negative (no forbidden extra-question testids in the panel), footer short-SHA, title short-SHA. - Workflows now run `pnpm test:unit --run` always and `pnpm exec playwright test --project=chromium` gated on the data package being built (`UI_DATA_AVAILABLE=1`). Playwright config switched to re-using the existing `build/` directory instead of rebuilding in a subshell that loses VITE_BUILD_* env vars. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the user's decision, CI no longer materializes Stage 1–4 inputs. The
maintainer builds the data package locally and hosts the tarball at
~/dbm/shares/ohbm2026/ → a stable Dropbox URL. The deploy workflow reads
the URL from the OHBM2026_UI_DATA_PACKAGE_URL repo variable and downloads
+ sha256-verifies the tarball before each build.
What changed:
- scripts/fetch_ui_inputs.sh: three modes now. (1) Download + extract a
pre-built tarball when OHBM2026_UI_DATA_PACKAGE_URL is set; sha256-
verify via OHBM2026_UI_DATA_PACKAGE_SHA256 if also set; rewrite
Dropbox `?dl=0` to `?dl=1` transparently; drop a `.fetched-from-package`
marker. (2) If raw inputs are present locally, signal "build via
build_ui_data.py". (3) Otherwise exit 2 with clear messaging.
- .github/workflows/{deploy-ui,pr-preview}.yml: pass the URL + sha256
repo variables through to the fetch step; skip build_ui_data.py when
the package was pre-fetched (gated on the marker file).
- src/ohbm2026/ui_data/state_key.py: drop the Stage6Error parent to
avoid the exceptions.py → fetch.graphql_api → fetch.stage circular
import that breaks `python -m ohbm2026.ui_data.state_key`. Tests still
green; the bare RuntimeError parent is a no-op for callers that just
except on Stage6BuildError.
- README.md: refresh recipe — operator rebuilds locally, drops the
tarball in ~/dbm/shares/ohbm2026/, updates the sha256 repo var.
The repo variables OHBM2026_UI_DATA_PACKAGE_URL +
OHBM2026_UI_DATA_PACKAGE_SHA256 are now set; the next PR-preview run
should pick up the real 3,243-abstract data package and the deployed
preview will show the full US1 UI instead of the "data missing"
placeholder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eploy/data SHAs; full-width layout
Three fixes the live preview surfaced:
1. Empty section text — `_section()` was reading `enriched.<section>_markdown`
fields that don't exist on the current Stage 2 enriched.sqlite. Section
bodies live in the raw corpus's `responses[].value` (HTML from the
rich-text editor) under fixed question names (Introduction, Methods,
Results, Conclusion, References/Citations). New `_html_to_text` helper
strips tags + decodes entities + preserves paragraph boundaries so the
UI can render with `white-space: pre-wrap`. Result: abstracts now
render the real ~1k–4k char section bodies.
2. Empty references — `_load_references_by_id` was reading `references`
off the enriched record (incomplete) instead of the curated Stage 2.1
`data/cache/reference_metadata/openalex_resolved.json`. Switched. Now
pulls matched refs with `{doi, openalex_id, title_guess}`. DetailPanel
renders them as an ordered list with the citation text as the link
label + the DOI in a muted monospace next to it. CLI default for
`--references` now points at the canonical curated path.
3. Deploy SHA vs data SHA confusion — BuildInfo footer takes two props:
`deployBuildInfo` (from VITE_BUILD_SHA env, always = the live deploy
commit) and `dataBuildInfo` (from manifest.json's build_info, = when
the tarball was rebuilt). When they differ the footer shows both
("build X · data Y · corpus Z · …"). The page <title> prefers the
deploy SHA so reviewers can verify each PR-preview reflects the
latest pushed commit at-a-glance.
Also: dropped the `max-width: 56rem` constraint on the layout shell so
the UI fills any viewport — list pane uses the full available width on
ultra-wide displays, detail pane scales from 22rem → 38rem at >1600px.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ds FR-023 AI indicator US2 user-facing slice on top of US1: - ModelSelector.svelte: two dropdowns (model × input) bound to the selectedCell store; values come from manifest.models/inputs (CA-007 discovery). - UmapPanel.svelte: tabbed 2D + 3D, lazy-loads `plotly.js-basic-dist-min` only when the user opens the map (SC-006 bundle-size hygiene). Colors by community_id. 2D scatter ships with lasso modebar (desktop) or pan + tap-to-filter-by-community (mobile, viewport < 1024px). 3D scatter supports rotate/pan/zoom (no lasso per FR-006). - Plotly events → stores: `plotly_selected` populates `lassoSelection`, `plotly_deselect` clears it, `plotly_click` sets `focusedAbstract` (or, on mobile 2D, also fills `lassoSelection` with the tapped community's members per Edge Case). - Home page intersects `searchAbstracts` ∩ `lassoSelection` for the ResultList filter; a "Show map" toggle + clear-selection button keep the map dismissable. - Model-switch persistence: `selectedCell` change re-fetches the new cell shard but `lassoSelection` survives — same abstract_ids stay selected; only coordinates move (US2 acceptance scenario 2). Tests: - Vitest: 3 new `loadCell` tests covering fetch/404/per-key cache (23 Vitest total). - Playwright: 2 new umap.spec.ts tests covering map open + Plotly lazy-load + synthetic lasso + model-switch persistence + clear- selection (8 Playwright total). Spec (/speckit-clarify Session-2026-05-17 settled the AI-attribution indicator): - FR-023 (new): `✨ AI` pill on AI-authored content surfaces only (figure interpretations, extracted claims, LLM-grouped topic-cluster titles/descriptions). Header-level only. Tooltip names the specific model + deep-links to the About-page section explaining the pipeline. Not on references / sections / authors / picklists. Implementation lands when the first AI-surfaced US ships (claims + figures = follow-up US). - SC-012: Playwright assertion every AI section header carries the pill + reveals the model name on hover. - FR-022 tightened to spec the deploy-vs-data SHA split that landed in 92b7ded. - FR-017 (About page) expanded: per-AI-surface methodology sections for the FR-023 tooltips to deep-link into. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…to-rotate; theme; topic-titled hover Five user-surfaced fixes: 1. **Per-cell UMAP layout** (the big one) — the wide annotations table stores UMAP coords per model only; the per-(model, input) UMAPs live in `data/outputs/analysis/<cell_key>/projections__<state-key>/*.npy`. `cells.py` now reads those `.npy` files when `--analysis-root` is passed, so switching input within a model now actually moves points. The annotations-table coords stay as a fallback. Verified locally: `voyage_abstract` row 0 at (9.76, 2.82), `voyage_claims` at (-0.98, 9.01), `neuroscape_abstract` at (7.71, 4.82) — all distinct. `umap_missing` flag emitted for abstracts without a projection in that cell (voyage_claims drops 2 abstracts that lacked claim embeddings). 2. **3D not actually 3D** — `plotly.js-basic-dist-min` doesn't include `scatter3d`. Switched to `plotly.js-gl3d-dist-min` which carries both `scatter` (2D) and `scatter3d` at ~900 KB gz total (vs 700 KB). 3. **Side-by-side 2D + 3D + auto-rotate** — the panel renders both charts in a 2-column grid (stacks on < 880 px). The 3D pane auto- rotates by default; a ⏸/▶ button lets the reader pause / resume. 4. **Topic-titled hover** — `loadTopics(cell_key, 'communities')` joined into the trace's customdata. Hover now shows the community title (or top-3 keywords as fallback) instead of just "community N". 5. **Lasso double-listener bug** — `node.on(...)` was stacking handlers on every render. Track per-chart `handlersAttached` flags; attach exactly once. Also full theme support: CSS variables in `app.css`, light/dark/auto ThemeToggle in the header, FOUC-free boot via inline app.html script. Plotly bg + font colors track the theme. All component CSS migrated from hardcoded hex to var(--name). The detail-pane column widened on desktop (clamp 24rem / 28vw / 42rem). Empty-state placeholders use the warning palette. Tests: - 23/23 Vitest still green. - 9/9 Playwright e2e green (added rotate-toggle + replaced legacy `umap-chart` testid with `umap-chart-2d` / `umap-chart-3d`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled changes so the data-package refresh stops invalidating the Dropbox share URL on every rebuild: 1. `_write_json` opens the target file directly (`O_WRONLY|O_CREAT|O_TRUNC`) instead of writing to `*.tmp` and calling `Path.replace()`. The rename pattern changed the inode of every shard on every build; in-place truncate preserves the inode. Also pins mtime to a fixed timestamp (2026-01-01) so subsequent `tar -czf` produces byte-identical archives when inputs are unchanged. 2. The output-directory swap (rename data → data.prev → swap → rmtree) is gone. The builder now writes into the existing `data/` dir directly, then prunes any stale .json shards from prior runs. Refresh recipe in README updated: `tar -czf` straight to the canonical Dropbox path (`~/dbm/shares/ohbm2026/ui-data.tar.gz`); no `cp` alias step. Verified end-to-end: rebuilt the tarball with a content change; inode preserved (217294146 before and after); Dropbox share link still serves the new content with no need to re-share. Repo variables refreshed: OHBM2026_UI_DATA_PACKAGE_URL points at the new canonical share link; OHBM2026_UI_DATA_PACKAGE_SHA256 bumped to 0f93f470… (the new bytes). Future refreshes will only need to bump the sha256. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ger carries data
Per the user's decision, the deployed site no longer bundles any data.
Instead the client downloads the canonical tarball from the Dropbox URL
(`OHBM2026_UI_DATA_PACKAGE_URL` repo var → `VITE_DATA_PACKAGE_URL` at
build time), decompresses via the browser's native
`DecompressionStream('gzip')`, parses the tar in memory, and indexes
the JSON shards in a `Map<path, json>`. Every existing loader
(`loadManifest`, `loadAbstracts`, `loadAuthors`, `loadCell`,
`loadTopics`) reads from that map.
Why:
- gh-pages branch was getting ~50 data files committed on every PR
deploy (peaceiris `keep_files: true` retained them, then each new
deploy added new ones). User wants iteration not to touch data.
- The data-package refresh cycle (rebuild → re-tar → upload to
Dropbox → bump sha repo var) was thrashing across UI commits even
though the data didn't change. Now the only thing that re-triggers
a data refresh is the user re-running the Python builder + re-tar.
- The runtime fetch + decompress is ~14 MB on first paint (gzip on
the wire, decompressed in browser) — still below the spec
SC-006 budget at this scale.
Key bits:
- `site/src/lib/data_package.ts`: ~100-line runtime loader. Uses
`DecompressionStream` (no pako/fflate dep). Handcrafted tar parser
(~50 lines) that walks 512-byte headers. Rewrites
`www.dropbox.com` → `dl.dropboxusercontent.com` because the
former returns a CORS-headerless 302 that Chrome refuses; the
latter sends `Access-Control-Allow-Origin: *` directly.
- `site/src/lib/shards.ts`: simplified to in-memory map lookups.
- Workflows: dropped `fetch_ui_inputs.sh` + `build_ui_data.py` +
Playwright e2e steps from CI. Vitest unit tests still run (now
with `vi.spyOn(loadDataPackage)` mocks). pr-preview switched to
`keep_files: false` on its `pr-<N>/` so each push fully replaces
that subdir — no stale data files survive.
- Cleaned gh-pages branch: deleted all `pr-9/data/` shards in a
separate commit on that branch (53 files removed).
Verified end-to-end with a headless browser pointed at a local
preview build: page fetches Dropbox at runtime, 3243 abstracts hydrate,
zero console errors, zero data files in `build/`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…etting camera
Two real bugs surfaced once we tested with a real mouse-drag (the earlier
synthetic emit test passed because it only fired one `plotly_selected`):
1. **Lasso wasn't filtering the list.** Plotly fires two `plotly_selected`
events per lasso — first with the real points, then a follow-up empty
`{ points: [] }` after its internal `plotly_relayout`. My handler
treated the empty event as "clear the selection," so the lasso filter
appeared to never engage. Fix: ignore empty `plotly_selected` (real
deselect comes through `plotly_deselect` on double-click).
2. **3D camera reset every time the lasso changed.** Setting
`scene.camera` in the layout passed to `react()` overrides the
current camera state — `uirevision` should have preserved it but in
this Plotly bundle doesn't. Fix: before each react(), read the
current camera from `el._fullLayout.scene.camera.eye` and pass it
back explicitly. The rotation animation continues to update camera
via `relayout` between react() calls; whatever camera is current at
render time is what we carry forward.
Bonus: 3D now also reflects the lasso selection. scatter3d ignores
`selectedpoints`, so I encode it via `marker.opacity` as an array —
selected points stay fully opaque, unselected drop to 0.08. Verified
locally: 2717 selected (opacity 1) + 526 dimmed (opacity 0.08) after a
~60% lasso on the 2D pane, with camera unchanged from
(2.13, 0.55, 0.9) before and after.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… works via two-trace split
Two coupled changes:
1. **Per-cell neighbors store** — new `scripts/compute_neighbors.py`
walks every `data/outputs/analysis/<cell_key>/projections__*/`,
loads `reference_matrix.npy` (the per-cell embeddings, dim varies
from 64 NeuroScape → 1536 OpenAI), and computes the k=10 nearest +
k=10 farthest cosine-distance neighbors per abstract. Output lands
at `data/outputs/analysis/<cell_key>/neighbors__<state-key>/{ids,
nearest_ids, nearest_distances, farthest_ids, farthest_distances}.npy`.
All 15 cells in ~3s total (BLAS does most of the work via
`M @ M.T`).
New `src/ohbm2026/ui_data/neighbors.py` reads those `.npy` files and
emits one `data/neighbors/<cell_key>.json` shard per cell — array-
form for compact JSON (~250 KB raw, ~30 KB gz × 15 cells = ~0.4 MB
gz). Joined positionally with the cells shard. The
`build_ui_data_package` orchestrator emits them after cells.
2. **3D dim on lasso actually shows** — `scatter3d` silently ignores
per-point `marker.opacity` arrays (treats opacity as scalar only),
which is why the earlier per-point opacity array read back correctly
but didn't render. Fix: split into two traces (`selected` @
opacity 1 + `unselected` @ opacity 0.05) drawn in that order so
selected layers on top. Both traces share `cmin`/`cmax` so the
Viridis colorscale stays consistent.
The 3D click handler now reads `pt.customdata[0]` (the poster_id)
directly instead of mapping `pointIndex` against a single trace —
robust to whether the chart has 1 or 2 traces at click time.
Verified locally with a real-mouse lasso: trace count goes 1 → 2,
2717 selected + 526 unselected, camera preserved across the lasso.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…acts" Wires the per-(model, input) k=10 nearest / farthest shards committed in the prior commit into the abstract detail panel. The panel: - Loads `data/neighbors/<cell_key>.json` from the runtime data package whenever the focused abstract or the selected cell changes. - Renders two blocks under "Related abstracts — from <cell_key>": "Most similar" (top 5 of the 10 nearest) and "Most different" (top 5 of the 10 farthest), each row showing rank · poster_id · title · cosine distance. - Each entry is a button that switches focus to that abstract — a one- click way to walk the embedding neighborhood. Model-switch behavior: the related list re-fetches when the user changes the (model, input) cell, so the "neighbors according to voyage_abstract" become "neighbors according to neuroscape_claims" on switch. `abstractsById` map plumbed through from `+page.svelte` and the permalink route so the panel can resolve neighbor records to title + poster_id without a second round trip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the substring-based search with Damerau-Levenshtein typo tolerance over a per-corpus inverted index. The implementation: - `lib/filter.ts:lexicalSearch(...)`: lazily builds an in-memory `token → Set<abstract_id>` index over the corpus's title, poster_id, topics, methods checklist, author names, and facet values. For each query token, walks the ~10k unique corpus tokens and matches any within Damerau-Levenshtein distance ≤ 2 (≤ 1 for tokens < 4 chars, per FR-008). Multi-token queries intersect across tokens. - `damerauLevenshtein` is a 30-line DP implementation with adjacent- transposition support and an early-exit when the running row min exceeds the threshold. Length-difference pre-filter shaves the hot path further. - Diacritic-insensitive via NFD-normalize-+-fold (already in `normalize()` — query 'Garcia' matches author 'García' as a 0-distance match after folding). At the 3243-abstract scale brute force over the unique-token list is fast enough (sub-millisecond per query token in the typical case); the trigram-bucket pre-filter from the original spec design isn't needed yet — revisit if SC-002 regresses. Live smoke against the deployed corpus: - 'connectivity' → 736 (exact) - 'connectvity' (1 typo)→ 734 (matches the typo case) - 'defautl mode netwrk' (3 typos) → 16 - 'default mode network' (exact) → 16 — same set - 'Smtih' (transposition) → 56 (Smith-named authors) - 'Garcia' (diacritic fold)→ 166 (García authors) - 'xyzpdqzzz' (gibberish) → 0 Tests: 15 new Vitest cases covering damerauLevenshtein, tokenization, exact + typo + multi-word intersect + spec examples. Total Vitest suite now 37/37 green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…, FR-009)
Always-on semantic search merged with the lexical typo-tolerant results.
No toggle — the worker boots in the background on page load and joins
the existing lexical pipeline as soon as it's ready.
Data pipeline:
- `src/ohbm2026/ui_data/vectors.py`: composes the per-abstract semantic
vector as the mean of {introduction, methods, results, conclusion}
MiniLM-L6 embeddings, L2-renormalized. Quantized to int8 with a single
global scale (max-abs → 127). Cosine-recovery MAE measured on a
held-out 100-pair sample = 0.00057 (well under the 0.5% threshold in
data-model.md §7).
- The orchestrator emits `data/search/minilm_vectors.bin` (1.2 MB raw
for 3243 × 384 int8) + a sidecar `minilm_vectors.build_info.json`
carrying scale, shape, components used, missing_abstract_ids, and the
cosine-recovery MAE.
- The runtime tar parser in `data_package.ts` was extended to surface
`.bin` entries as `Uint8Array`s alongside the JSON shards (with an
explicit copy so the underlying 50 MB decompressed package can be
GC'd once parsed).
Browser:
- `lib/workers/semantic.worker.ts`: loads `Xenova/all-MiniLM-L6-v2`
via transformers.js from the Hugging Face CDN (~23 MB one-time,
browser-cached thereafter), embeds query text with mean-pooling +
L2-normalize, then dot-products against every int8 corpus vector for
cosine similarity. Returns top-K positional indices + scores.
- `lib/search/semantic.ts`: main-thread facade. Lazy-initializes the
worker; transfers the corpus `ArrayBuffer` to the worker (zero-copy);
reports init status via a Svelte store; serializes queries with a
monotonic id so out-of-order completions can't overwrite a newer
result.
- `routes/+page.svelte`: `onMount` kicks off `warmSemantic()` in the
background (non-blocking). On every `$searchQuery` change, runs both
`lexicalSearch` (synchronous) and `semanticSearch` (async, behind
the worker). Results union-merged into `searchIds`; if the worker
isn't ready or errors, the union degrades gracefully to lexical-only.
Bundle impact: transformers.js + ONNX runtime add a 1.6 MB chunk that
Vite code-splits via the `new Worker(new URL(...))` pattern; that chunk
loads only when `warmSemantic()` fires (which is `onMount`, so de
facto first paint, but off the critical render path).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-(model, input) cell-independent filter sidebar on the home page:
- `lib/facets.ts`: 13 facet keys discovered from each abstract's
`facets` block + the implicit `accepted_for` / `primary_topic` /
`secondary_topic`. `recomputeFacets(abstracts, filters, preFilterIds)`
returns `Map<facet_key, [{ value, count }]>` honoring FR-013:
counts for facet F use the intersection of (search ∩ lasso ∩ OTHER
facets) excluding F itself, so selecting "Methods=fMRI" doesn't
zero out the other Methods options. `filterByFacets` intersects
all per-facet sets across abstracts and returns the matching ids.
- `components/FacetSidebar.svelte`: collapsible-section list with a
checkbox + count per option. Most-used facets (topics, methods,
study type, etc.) expanded by default; verbose ones (keywords,
processing packages, brain regions/networks, accepted-for)
collapsed. Per-facet section caps at top-30 options + "+ N more"
hint to keep the DOM bounded for facets like keywords.
- `+page.svelte` wiring: `facetIds = filterByFacets(...)` is now in
the intersect chain alongside `searchIds` + `lassoSelection`, so
facet + lasso + search are all multiplicative filters on the
result list. Counts displayed in the sidebar use a `pre-filter`
derived from (search ∩ lasso) — they show "what's reachable from
here" rather than corpus-absolute totals.
Layout: desktop ≥ 1024px is now 3-col (facets · results · detail);
the facet column is always visible + sticky. Below 1024px the facet
pane hides by default + a `🔍 Filters` toggle button reveals it
inline (full-width).
Verified locally: 3243 abstracts → click Methods=Functional MRI →
1615 → Clear → 3243.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DetailPanel: the zones used CSS Grid with `data-zone`-driven column
placement, which kept the two columns yoked at the row level even
with `grid-auto-flow: dense` — tall sections in one column padded
the matching row in the other, producing the user-reported white-
space "linking". Refactored using Svelte 5 snippets:
* Each section block (body / topics / methods / refs on the left;
cluster / related / claims / figures on the right) is now a
`{#snippet xxxBlock()}` definition.
* Non-compact mode wraps the page in two sibling `<div class="zone">`
flex-column containers, each calling its own set of snippets.
* Compact mode renders the three home-pane snippets (claims,
cluster, related) in a linear flow with no zones.
CSS: `.zone` is `display: contents` on mobile/compact (so it doesn't
introduce extra boxes), and `display: flex` (column) inside the grid
on landscape ≥ 980 px. The two zones are TRULY independent — each
sets its own internal row heights with no cross-column alignment.
ResultList card: the lead-author name is now a clickable button.
Restructured the card body from `<button>` → `<div role="button">`
so a real `<button class="lead-author-link">` can nest inside legally.
Clicking it adds the author to the chip filter via `authorChips`,
non-destructive — coexists with whatever else is selected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
About page: each stage now opens with the lay paragraph FIRST. A `Technical details ▸` toggle below it expands a collapsible TL;DR that's much more code-grounded: Stage 1 — fetch: names the three GraphQL queries, the tiered HARD/SOFT/INFORMATIONAL schema-diff classifier, the per-record state-machine validator, the figure-asset reuse hash, the checkpoint path / state-key scheme, and the schema-snapshot for the next run's diff base. Stage 2 — enrichment: spells out the three function tools the agentic claims call exposes (verify_source_quote, lookup_eco_code, dedupe_check), the figure JPEG-q85@1024 + four-field Pillow probe, the flex-tier retry pattern (1 flex + 1 standard, 120s/180s timeouts), cache-key hashes per component, and the references pipeline that uses the LLM only to SPLIT (canonical metadata comes from OpenAlex/Semantic Scholar — that's why references carry no ✨ AI pill). Stage 3 — embeddings: per-component bundle dir layout, per-abstract cache key, the `compose_recipe` composition contract, BlingFire sentence chunking, the exact model identifiers per family, and the int8 SPA-vector wire format (scale = 127/max_abs, cosine-recovery MAE 0.00057 vs the ≤ 0.005 invariant). Stage 4 — analysis: FAISS IndexFlatIP kNN at k=15, Leiden CPM resolution sweep [0.001, 0.1] × 20 points, UMAP (n_neighbors=15, min_dist=0.1, metric=cosine, random_state=42), HDBSCAN topic clusters reusing the UMAP layout with min_cluster_size=10, hybrid spaCy + c-TF-IDF + LLM cluster naming, and the neighbour pre-compute at k=10. Stage 6 — UI: in-memory inverted index + Damerau-Levenshtein thresholds, Web Worker semantic search with zero-copy buffer transfer + clamp logic, the dominant-filter Saved-only rule, the 3D camera plotly_relayout tracking, the gh-pages 404 SPA-redirect handshake (incl. why we pass the FULL stash to goto rather than base-stripped), the deploy / preview workflows, and the LinkML schema entrypoint. UI: `showMap` now defaults to ON for new users — the UMAP is the most distinctive piece of the site, no reason to hide it on first paint. localStorage `'0'` still wins so a chosen-OFF state survives reloads. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In the permalink (non-compact) view, swap the order of the computed zone so AI-derived sections (Claims, Figure interpretations) sit at the top and the algorithmic neighbourhood (Cluster membership, Related abstracts) follows below. The AI surfaces carry the most distinctive value on the detail page and were getting buried under the cluster grid + related rails. Compact mode (home pane) keeps its existing claims-first order so it's unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The earlier Python-driven refactor extracted the FR-011 comment block but truncated the opening `<!--` marker, so the explanatory text + the trailing `-->` rendered as visible text inside the submitter zone column. Wrapped the comment cleanly so it disappears from the DOM again. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Playwright probe at iPhone SE (320 × 568) revealed `.controls`
overflowing to 657 px wide because the row had no `flex-wrap`. The
five toggle buttons (Semantic / Filters / Map / Saved-only / Cart)
plus the ModelSelector pair pushed past the right edge, requiring
horizontal scroll.
* `.controls` is now `flex-wrap: wrap` with tighter mobile padding;
a `min-width: 720px` media query restores the original "single
line" layout for tablets and up.
* `.control-toggle` gains `white-space: nowrap` so labels don't
word-break inside the chip when squeezed, and `padding`/
`font-size` tighten on viewports ≤ 480 px.
* Model + Input `<select>` min-width drops from 8 rem → 6 rem
globally, → 4.5 rem under 480 px, so the model-selector pair
fits in the controls row even when the toggle buttons are also
on the same wrap.
* UMAP `.chart` div now has `overflow: hidden` to clip Plotly's
occasional sub-pixel-wide SVG render (was poking 7 px past on
the 320 px viewport).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six-viewport Playwright probe now passes with zero horizontal overflow: iSE-1st-portrait (320 × 568) docW=320 overflow=0 iSE-2nd-portrait (375 × 667) docW=375 overflow=0 iSE-1st-landscape (568 × 320) docW=568 overflow=0 iSE-2nd-landscape (667 × 375) docW=667 overflow=0 tablet-portrait (768 × 1024) docW=768 overflow=0 desktop-landscape (1440 × 900) docW=1440 overflow=0 UMAP chart height: `clamp(220px, 50vh, 480px)` for normal portrait, plus an `@media (max-height: 480px) and (orientation: landscape)` override pinning it at 60 vh. Without that override the chart's 280 px minimum dominated a 320-px-tall phone-landscape viewport, hiding the search + result list below the fold. The probe spec is committed so future iterations can re-run `pnpm exec playwright test mobile-check` against the live preview. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
US6 — Guided tour (T077, T078, T079, T080, T082):
* `site/src/lib/stores/tour.ts` — state-machine store
(`idle | running | dismissed`) + two persistent flags
(`ctaDismissed`, `completedOrSkipped`) under
`ohbm2026.ui.tour.v1`.
* `site/src/lib/components/Tour.svelte` — shepherd.js-backed
six-step walkthrough (search → model selector → map → lasso
hint (desktop) → facets → cart). Lazy-loads `shepherd.js` +
CSS only when the tour actually starts. Tooltips switch to
bottom-attached on mobile.
* Header `Tour` button + first-visit CTA banner in
`+layout.svelte` (banner self-dismisses once the user runs
or skips the tour at least once).
* 8 unit tests for the state machine — all green.
US7 — References + link-check (T083-T086, T088, T089):
* `specs/008-ui-rewrite/contracts/references.yaml` — 19-entry
source-of-truth registry covering Stages 1-6, each with
`{section, title, authors, year, url, doi?}`.
* `src/ohbm2026/ui_data/link_check.py` — HEADs every URL with
a 10 s timeout; falls back to GET on 403/405/501 hosts;
returns exit 3 per `contracts/data-package.md` on any
4xx/5xx/timeout/connection failure. CLI usable directly via
`python -m ohbm2026.ui_data.link_check`.
* `tests/test_ui_data_link_check.py` — 8 hermetic tests using
the `responses` library (HEAD 200, 405-falls-back-to-GET,
4xx, 5xx, ConnectionError, missing YAML, empty refs, clean
pass). All green.
* `.github/workflows/deploy-ui.yml` wires the link check
between the JS test step and the site build — non-zero exit
blocks the deploy (FR-017).
Docs reconcile (T092, T093, T094):
* `CLAUDE.md` — drop the legacy `ui.py / export-ui / build-ui`
surface; the Stage 6 SvelteKit site + `ui_data/` package
are the canonical UI track now.
* `README.md` — Stage 6 section reflects the shipped capabilities
(typo-tolerant search, semantic search via Web Worker, 2D + 3D
UMAP, facets, cart + email-my-list, guided tour, About page
with link-checked references).
* `docs/reproducibility-vision.md` — Reproduction Ladder Level 2
now ends with the canonical `scripts/build_ui_data.py` →
`scripts/validate_ui_data.sh` → `link_check.py` → Dropbox
retar recipe.
Other:
* Constitution check `--full` exits 0 (T095).
* User-memory entry saved at `memory/stage6_atlas.md` (T098).
* Marked 16 closed tasks in `specs/008-ui-rewrite/tasks.md`
with [X].
Tests / build / schema:
Vitest: 55/55 pass (added 8 tour tests)
link_check unittest: 8/8 pass
vite build: clean
scripts/validate_ui_data.sh: 68/68 shards validate
constitution-check --full: exit 0
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous single-step-list tour pointed at home-page elements
regardless of which route the user was on, so navigating to a detail
or about page mid-tour left shepherd trying to attach to selectors
that didn't exist. Refactored:
* `detectKind(pathname)` picks one of three configs:
* `home` — `/` (or `/pr-N/`)
* `detail` — anything matching `/abstract/<id>/`
* `about` — anything matching `/about/`
* `onPhaseOrPathChange()` reactive trigger cancels the active
shepherd when the URL changes mid-tour rather than leaving it
bound to stale targets.
Home tour expanded:
search → model → map → lasso (desktop) → facets
→ click-a-card (auto-focuses the first visible card so the
related-works pane is rendered for the next step)
→ the per-card 🛒 icon (testid `card-cart-add` / `card-cart-remove`)
— explicitly the per-card save action, not the header cart
→ the "full details ↗" permalink link
→ the header 🛒 toggle to open the saved-list drawer
Detail tour (new): permalink callout → submitter zone →
computed zone → claims (✨ AI) → related abstracts → cluster
membership → detail-pane cart action.
About tour (new): intro → per-stage deep-dive blocks → external
references (mentioning the build-time link checker).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…malinks DetailPanel * Cart action moved from the footer to the header — now sits as a proper outlined/filled cart icon (with green ✓ pip when saved) right next to the poster id, matching the per-card pattern in the result list. The footer's text `+ add to list` / `Remove from list` button is gone. * New `← back to the atlas` link at the bottom of the panel, shown on the permalink page (compact=false) so the last thing a reader sees is a clear route back to the search + map. Tour * Drop the redundant "Permalink view — you're looking at the full-detail page" step (the user already knows they're on the detail page). * Cart step now points at the header icon position. * New final step on the detail tour: "Back to the atlas" — attaches to the new back link so users learn the navigation arrow. Email body (cart_email) * Each saved abstract now renders as a four-line block: numbered title, lead author, and an explicit `→ Open: <permalink>` line. The arrow + "Open:" label makes the URL unambiguously click-to-view inside any email client. * Body opens with a one-line note that each entry carries an Open link, and closes with `— Browse the rest at <site root>` so the recipient has a way back to the atlas even without a specific saved id. * Unit-test the new format (numbered block, `→ Open:` prefix, footer link). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two augmented steps on the home tour: * The "Full-detail (permalink) page" step now describes the two-zone layout (submitter on the left, AI + algorithmic on the right) and the back-to-the-atlas link, so the user knows what to expect when they click through. * New "About + methodology" step (final step on the home tour) attaches to the header "About" link and explains the per-stage layout (lay paragraph + Technical-details toggle + HEAD-checked citations). The user can still re-run the matching detail / about tour by hitting the header "Tour" button after navigating to those pages. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
satra
changed the title
satra
added a commit
that referenced
this pull request
May 17, 2026
Addresses the gemini-code-assist review comments on PRs #9 and #10 plus the axe-core a11y audit results from the live production site. PR #9 review fixes: * state_key.discover_rollup_state_key — docstring previously said "picks the most recent by mtime" but the implementation raises on >1 match; corrected the docstring (intentional design: no implicit newest-wins, the operator must disambiguate). * PR #9 also flagged a cross-device rename OSError in builder.py and redundant inline `shutil` imports. Both already addressed by the earlier in-place-write refactor — no shutil/tempfile/rename calls remain in builder.py. Verified. PR #10 review fixes: * references.yaml authors fields: `ccnmaastricht` → `Mario Senden` (the canonical author per the Aperture Neuro paper). Year corrected to 2026. * Both entries' titles now follow the same `<Author> (<Year>) — <title>` format as the other references in the registry. a11y (axe-core against production): * `scrollable-region-focusable` was the one serious WCAG 2.1 violation surfacing on every route. Added `tabindex="0"` (+ `role="region"` + `aria-label` where useful) to every `overflow-y:auto` container: cluster-grid, related-list scroll, facet options scroll, cart items. * New `site/src/tests/e2e/a11y.spec.ts` Playwright spec that runs the axe audit against the home / about / permalink routes. Reads `TARGET_BASE` so the same probe can hit production OR a PR preview. * `@axe-core/playwright` added as a dev dep. (beta) tag: * Page <title> + the H1 in the header now read "OHBM 2026 Atlas (beta)" so users see clearly the site is pre-launch. Empty-state tip: * Dropped the `Models × inputs` and `Cells` lines from the right-pane "Tap an abstract to see its details here" affordance — they were more developer-facing than reader-facing. Just `Accepted abstracts` stays. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
satra
added a commit
that referenced
this pull request
May 17, 2026
…ate (#11) Addresses the gemini-code-assist review comments on PRs #9 and #10 plus the axe-core a11y audit results from the live production site. PR #9 review fixes: * state_key.discover_rollup_state_key — docstring previously said "picks the most recent by mtime" but the implementation raises on >1 match; corrected the docstring (intentional design: no implicit newest-wins, the operator must disambiguate). * PR #9 also flagged a cross-device rename OSError in builder.py and redundant inline `shutil` imports. Both already addressed by the earlier in-place-write refactor — no shutil/tempfile/rename calls remain in builder.py. Verified. PR #10 review fixes: * references.yaml authors fields: `ccnmaastricht` → `Mario Senden` (the canonical author per the Aperture Neuro paper). Year corrected to 2026. * Both entries' titles now follow the same `<Author> (<Year>) — <title>` format as the other references in the registry. a11y (axe-core against production): * `scrollable-region-focusable` was the one serious WCAG 2.1 violation surfacing on every route. Added `tabindex="0"` (+ `role="region"` + `aria-label` where useful) to every `overflow-y:auto` container: cluster-grid, related-list scroll, facet options scroll, cart items. * New `site/src/tests/e2e/a11y.spec.ts` Playwright spec that runs the axe audit against the home / about / permalink routes. Reads `TARGET_BASE` so the same probe can hit production OR a PR preview. * `@axe-core/playwright` added as a dev dep. (beta) tag: * Page <title> + the H1 in the header now read "OHBM 2026 Atlas (beta)" so users see clearly the site is pre-launch. Empty-state tip: * Dropped the `Models × inputs` and `Cells` lines from the right-pane "Tap an abstract to see its details here" affordance — they were more developer-facing than reader-facing. Just `Accepted abstracts` stays. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
satra
added a commit
that referenced
this pull request
May 18, 2026
… typo-recall eval (#18) * chore(stage6): wrap-up — Playwright per-US specs, facets unit test, typo-recall eval Closes out the remaining Stage 6 todos that were deferred to "Phase 11 polish" pending a live preview / built data package. Now that the data package is local and production is up, they're all genuinely runnable. Tests added: - `site/src/tests/e2e/search.spec.ts` (T062) — FR-007 narrowing, FR-008 typo recovery on ≥ 7-char words, PR#17 operator grammar (`"phrase"` ≤ bare-AND set, `-word` subtracts, `OR` unions), and the ✨ semantic-only badge tooltip contract. - `site/src/tests/e2e/facets.spec.ts` (T068) — FR-005/FR-013: facet click narrows results, sibling-facet counts recompute against the narrowed set, `Clear` restores the corpus-wide count. - `site/src/tests/e2e/cart.spec.ts` (T071) — FR-006/SC-009: card-icon add + reload (localStorage persistence), `Clear` empties, `cart-email` produces a `mailto:` URL with the poster_id in the body. - `site/src/tests/e2e/tour.spec.ts` (T081) — US6 acceptance: CTA banner on first visit, "Start tour" opens `.shepherd-element`, dismissal hides the CTA while the header `Tour` button remains. - `site/src/tests/unit/facets.test.ts` (T064) — FR-013 nuance: a facet's own selections don't zero its sibling options' counts. 4 cases against a 4-abstract fixture. 75/75 unit tests pass. Eval script: - `scripts/eval_typo_recall.py` (T058a, SC-010) — ports the lexical- search core to Python and replays it against the data-package shards. Generates full-title probes (capped at 8 tokens) with one recoverable single-typo on a ≥ 4-char content word. Recall against the live shards: **0.9685 over 762 probes** (≥ 0.90 target). The threshold scheme (<4 → exact, 4–6 → DL ≤ 1, ≥ 7 → DL ≤ 2) confirmed sound. Exits 0 on pass / 3 on fail (so the script is usable as a CI gate later). Bookkeeping (T099 + T100 + several already-shipped tasks reconciled in `specs/008-ui-rewrite/tasks.md`): - T026 (US8 draft-PR verification) — DONE via PRs #9–#17 Deployments- box surface. - T090 (Lighthouse-CI) — DONE in PR #15. - T091 (a11y axe-core audit) — DONE in PRs #12 + #14. - T096 (SC sweep) — DONE; SC-002 / SC-003 / SC-004 / SC-005 / SC-011 measured against production via `sc-sweep.spec.ts`. - T097 (FR-021 / FR-022 acceptance) — DONE in spirit across PRs #9–#17. - T099 (reconciliation pass) — DONE in this commit. - T100 — marked OBSOLETE (Stage 6 shipped across 10+ PRs, not as one final consolidating PR). After this PR lands, the only Stage 6 work that remains is the operator-syntax UX shipped in PR #17 (post-spec enhancement) — the spec's 105 numbered tasks are complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(stage6-wrap): address gemini review — stable hash + web-first asserts Addresses 5 medium-priority gemini-code-assist comments on PR #18. scripts/eval_typo_recall.py — `hash(word) & 0xFFFFFFFF` was non- deterministic across Python processes (`PYTHONHASHSEED` randomisation), contradicting the docstring's "deterministic across runs" promise. Switched to `zlib.adler32(word.encode('utf-8'))` — a stable 32-bit hash that's plenty for seeding a per-word PRNG. Eval still passes: **recall@10 = 0.9894 over 378 probes** (was 0.9685 with the old randomised seed, well above the 0.90 target either way). site/src/tests/unit/facets.test.ts — replaced `JSON.stringify(...)` Map equality with Vitest's native `expect(...).toEqual(...)`. Walks the Map structurally and gives a useful diff on failure. site/src/tests/e2e/cart.spec.ts — converted `expect(await count())` one-shots to web-first `await expect(locator.first()).toBeVisible()` and `await expect(locator).toHaveCount(0)`. Dropped every `waitForTimeout` in this spec; Playwright auto-retries. site/src/tests/e2e/search.spec.ts — replaced the three `waitForTimeout` calls (250 ms between fills, 100 ms after clearing, 800 ms after a semantic query) with `expect.poll` based two-tick stability checks. The semantic-settle wait now has a 2 s budget and fails fast if the worker hangs, rather than always burning the hard-coded 800 ms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
satra
added a commit
that referenced
this pull request
May 18, 2026
Wraps the conference-subpath rework (spec 009) per the polish-phase
tasks T024–T031 in tasks.md.
T024 — README sweep
- Rename "Stage 6: UI" section to "Atlas UI" — the SvelteKit site is
one continuous deliverable spanning 008-ui-rewrite (US1–US8) and
009-conference-subpath (URL shape), so per-stage section names had
started to mislead readers.
- "Current Latest Step" item 9 + the section body both now name the
production URL as `abstractatlas.brainkb.org/ohbm2026/` (not the
bare root); PR previews mention `/pr-<N>/ohbm2026/`; the bare root's
static meta-refresh + JS `location.replace` is named honestly
(gh-pages cannot serve a true HTTP 301).
- Section 10 ("Build The Static UI") gains a paragraph about the
deploy workflows' staging step (`site/publish/ohbm2026/`) and the
redirect island at `site/conference-root-redirect/`.
T026 — constitution check
- `.specify/scripts/bash/constitution-check.sh --full` returns exit 0
against this branch. No principle violations.
T031 — tasks-list reconciliation
- T001–T026, T029, T030 marked `[X]` after their verification landed.
- T027 / T028 / T028a marked `[~] SKIPPED` with rationale. All three
are data-package regression checks (typo-recall, LinkML schema,
build_info byte-identical diff). They're irrelevant to a URL-only
rework: the staged commit's `git diff --stat` shows zero changes to
`src/ohbm2026/ui_data/`, `site/static/data/`, `scripts/build_ui_data.py`,
or the data-package URL vars. SC-105's promise holds by construction.
T028a's diff-vs-`main` design is also impossible to author cleanly
because `site/static/data/` is gitignored — there is no `main`
version to diff against.
T100 (from spec 008) remains OBSOLETE — Stage 6 + Stage 9 shipped
across multiple PRs (#9–#19, with this branch being the latest), not
one consolidating PR.
Memory file `stage6_atlas.md` updated outside the repo to reflect
the Stage-9 URL move + the FR-109 "no `conference` field in shards"
decision so future sessions know not to look for one.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
satra
added a commit
that referenced
this pull request
May 18, 2026
…19) * [Spec Kit] Add specification for conference subpath rework * [Spec Kit] Add plan + research + data-model + contracts + tasks for spec 009 (with /speckit-analyze remediations folded in) * feat(stage9): conference subpath rework — OHBM 2026 under /ohbm2026/ Implements spec 009-conference-subpath. Every OHBM 2026 surface (home, About, abstract permalink) now lives under `/ohbm2026/`; the root URL bounces there; legacy URLs (`<cname>/abstract/<id>`, `<cname>/about`) are NOT preserved per Q2; PR previews mirror production at `<cname>/pr-<N>/ohbm2026/`. Phase 2 — Foundational (T003–T014): - `site/svelte.config.js` (T003): `kit.paths.base` default flipped from `''` to `/ohbm2026`. PR previews still override via the existing `BASE_PATH` env var (now widened to `/pr-<N>/ohbm2026`). - `site/playwright.config.ts` (T004): `webServer.command` switched from `pnpm preview` (Vite serves only the SvelteKit build under `/ohbm2026/`) to `pnpm preview:gh-pages` which stages the gh-pages- shape tree (root redirect island + SvelteKit under `/ohbm2026/`) and serves it on the same port. baseURL stays root-bound so existing `goto('/')` tests bounce through the redirect island; subpath tests spell out `/ohbm2026/...` explicitly. - `.github/workflows/deploy-ui.yml` (T005): `BASE_PATH=/ohbm2026` for the build; new "Stage publish tree" step copies the build into `site/publish/ohbm2026/` and the redirect island into `site/publish/`; publish step uploads `site/publish/` to the gh-pages root. - `.github/workflows/pr-preview.yml` (T006): `BASE_PATH=/pr-<N>/ohbm2026`; same staging step; `environment.url` widens to `…/pr-<N>/ohbm2026/` so the Deployments box surfaces the right URL. - `.github/workflows/lighthouse.yml` (T007): `target_url` widens to `…/pr-<N>/ohbm2026/` so the audit runs against the conference shell. - `site/conference-root-redirect/index.html` (T008): static redirect island for the bare root. Meta-refresh + JS `location.replace`, preserves search + hash, detects PR-preview prefix at runtime. - `site/conference-root-redirect/404.html` (T009): smart root 404. gh-pages serves a SINGLE root 404 for ALL unknown paths; this file detects whether the path is inside `/ohbm2026/` (or `/pr-<N>/ohbm2026/`) and either hands the deep-link to the SPA via the existing `?spa=` query handoff in `+layout.svelte` (FR-107), or bounces to the conference home (FR-105). Lives outside `site/static/` so adapter-static doesn't bundle it into the SvelteKit build. - `site/src/lib/components/CartDrawer.svelte` (T010): imports `base` from `$app/paths` and composes `siteUrl = origin + base` so cart-email permalinks land under `/ohbm2026/abstract/<id>`. Fixes a pre-existing bug where `siteUrl` was constructed from `pathname` with a regex strip — already broken on the About page. - `site/src/lib/components/Tour.svelte` (T011): `detectKind` regex was already base-path-agnostic by accident (substring + `$` end-anchor); added a comment naming the invariant so a future refactor can't break it. - `site/src/routes/+layout.svelte` (T012): extended the existing SPA- redirect-handoff comment to call out the conference-subpath case: pass the FULL path with base to `goto`, never strip it. - `site/package.json` + `site/scripts/stage-and-serve.mjs`: new `pnpm preview:gh-pages` script + a small Node harness that reproduces the deploy workflow's file-tree shape locally. Stages the build into `publish/ohbm2026/`, copies the redirect island to `publish/{index,404}.html`, and serves the tree at `http://127.0.0.1:4173/`. Optionally re-uses an existing tarball via `OHBM2026_LOCAL_TARBALL` env var (e.g. the maintainer's local Dropbox sync) — saves a re-tar of `site/static/data/`. - `site/.gitignore`: adds `publish/` (build artifact, never committed). Phase 3 — US1 / SC-106 (T015–T017): - `site/src/tests/e2e/subpath.spec.ts` (T015 + T017 + T019 + T021): 10 e2e cases. US1 (home + About + permalink render under `/ohbm2026/`), SC-106 (build-info short SHA visible + consistent across home / about / permalink), US2 (incognito direct-load, refresh keeps URL, unknown poster_id renders inside the conference shell), US3 (root + query + hash redirects). 10/10 pass against the local harness with the local data-package tarball. - `site/src/tests/e2e/cart.spec.ts` (T016): one new assertion in the email-my-list test — the decoded `mailto:` body MUST contain `/ohbm2026/abstract/`, guards against a regression in the permalink composer. Phase 4–5 — US2 / US3 verification absorbed into `subpath.spec.ts` (T019 + T021), no additional spec files. Phase 6 — Polish (deferred to a follow-up commit on this branch): README sweep (T024), memory update (T025), constitution check (T026), typo-recall regression (T027), LinkML schema check (T028), byte- identical `build_info` diff (T028a), PR description + screenshots (T029 + T030 + T031). Holding for CI validation on the PR-preview deploy first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(stage9): extend stage-and-serve MIME map with font formats Addresses gemini-code-assist medium-priority review on PR #19. Some browsers refuse to load font files served as `application/octet-stream`; adding explicit MIME types for `.woff`, `.woff2`, `.ttf`, and `.otf` keeps the local preview faithful to what gh-pages serves. Gemini's high-priority finding about a "double-nested `publish/ohbm2026/ohbm2026/` directory" is a false positive: SvelteKit's `adapter-static` with `paths.base` does NOT create a subdirectory matching the base path; `site/build/` is flat (`index.html`, `_app/`, `about/`, etc.), and the base path appears ONLY as URL references inside the emitted files. Verified locally: $ ls site/build _app 404.html about data index.html $ ls site/publish/ohbm2026 _app 404.html about data index.html # ← single level, correct No change to the staging logic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(stage9): polish — README sweep + tasks-list reconciliation Wraps the conference-subpath rework (spec 009) per the polish-phase tasks T024–T031 in tasks.md. T024 — README sweep - Rename "Stage 6: UI" section to "Atlas UI" — the SvelteKit site is one continuous deliverable spanning 008-ui-rewrite (US1–US8) and 009-conference-subpath (URL shape), so per-stage section names had started to mislead readers. - "Current Latest Step" item 9 + the section body both now name the production URL as `abstractatlas.brainkb.org/ohbm2026/` (not the bare root); PR previews mention `/pr-<N>/ohbm2026/`; the bare root's static meta-refresh + JS `location.replace` is named honestly (gh-pages cannot serve a true HTTP 301). - Section 10 ("Build The Static UI") gains a paragraph about the deploy workflows' staging step (`site/publish/ohbm2026/`) and the redirect island at `site/conference-root-redirect/`. T026 — constitution check - `.specify/scripts/bash/constitution-check.sh --full` returns exit 0 against this branch. No principle violations. T031 — tasks-list reconciliation - T001–T026, T029, T030 marked `[X]` after their verification landed. - T027 / T028 / T028a marked `[~] SKIPPED` with rationale. All three are data-package regression checks (typo-recall, LinkML schema, build_info byte-identical diff). They're irrelevant to a URL-only rework: the staged commit's `git diff --stat` shows zero changes to `src/ohbm2026/ui_data/`, `site/static/data/`, `scripts/build_ui_data.py`, or the data-package URL vars. SC-105's promise holds by construction. T028a's diff-vs-`main` design is also impossible to author cleanly because `site/static/data/` is gitignored — there is no `main` version to diff against. T100 (from spec 008) remains OBSOLETE — Stage 6 + Stage 9 shipped across multiple PRs (#9–#19, with this branch being the latest), not one consolidating PR. Memory file `stage6_atlas.md` updated outside the repo to reflect the Stage-9 URL move + the FR-109 "no `conference` field in shards" decision so future sessions know not to look for one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 6 — full UI rewrite. Started as the US8 (deploy workflows + placeholder) slice; the rest of the user stories landed iteratively on the same branch via the per-PR-preview deploy chain. Net result: 48 commits delivering US1–US8 + polish.
What's in this PR
Architecture
site/,@sveltejs/adapter-static→ GitHub Pages.vars.OHBM2026_UI_DATA_PACKAGE_URL), in-browser decoded viaDecompressionStream('gzip')+ a ~50-line tar parser into aMap<path, JsonValue|Uint8Array>. No server, no per-query backend.src/ohbm2026/ui_data/(manifest,abstracts,authors,cells,topics,neighbors,enrichment,vectors,link_check,state_key,builder). CLI:scripts/build_ui_data.py. Validation: every JSON shard validates againstspecs/008-ui-rewrite/contracts/ui_data.linkml.yaml(scripts/validate_ui_data.sh— 68/68 shards pass).User stories shipped
plotly_relayout.mailto:body with per-posterOpen: <permalink>links +← Browse the rest at <site>footer.specs/008-ui-rewrite/contracts/references.yaml;link_check.pyHEAD-checks them in CI and blocks the deploy on any failure (FR-017).deploy-ui,pr-preview,pr-preview-cleanup) with PR-preview URLs surfaced in the PR's Deployments box (not bot comments) viaenvironment:declaration. Production deploy on merge to main → root of gh-pages; CNAMEabstractatlas.brainkb.org.Provenance + reproducibility
build_infoenvelope (corpus_state_key + stage4_rollup_state_key + full + short SHA + ISO built_at + builder_version). Footer + page title render the deploy SHA so the deploy can be visually verified (FR-022)./404.htmlthat stashes the original path in?spa=…+ sessionStorage and replaces location with the SPA shell root for the detected base path.Test sweep
scripts/validate_ui_data.sh: 68/68 shards validate.specify/scripts/bash/constitution-check.sh --full: exit 0Deferred to a polish PR (need a live production deploy to measure against)
🤖 Generated with Claude Code