Skip to content

Commit 49418e9

Browse files
jeswrclaude
andauthored
fix: small misc fixes (substrate comment, pred_stats heap accounting, docs nav, virtuoso isql port) (#6664)
* fix: small misc fixes (substrate comment, pred_stats accounting, docs nav, virtuoso isql port) - sparq-substrate/Cargo.toml: the scaffold comment claimed nothing depends on the crate; sparq-engine and sparq-reason do, so it is in the feature-OFF wasm graph (#3806). - sparq-core: TripleStore::heap_bytes now counts the pred_stats planner map, with a direct unit test (#3115). - site: the Docs nav item points at the mdBook guide overlaid at /guide/ instead of the agent-skills router; e2e witness updated (#2549). - scripts/virtuoso-same-box.sh: docker exec isql always targets the container-internal port 1111; VIRTUOSO_ISQL_PORT is only the host mapping (#3361). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ScyGGohDhirnbLbrUSnA5n * bench: declare feature-off bundle change for heap_bytes accounting (#3115) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ScyGGohDhirnbLbrUSnA5n * pages: overlay the mdBook guide before the site link-check The Docs nav link now points at /guide/, but pages.yml ran check-site-links.sh on site/out before the guide was built and overlaid, so lychee could not resolve /guide and failed the deploy. Build and overlay the guide first, and have check-site-links.sh exclude guide/ as an input (its own gate is build-guide.sh) while still resolving the site's links into it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ScyGGohDhirnbLbrUSnA5n --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 85d5ede commit 49418e9

8 files changed

Lines changed: 125 additions & 89 deletions

File tree

‎.github/workflows/pages.yml‎

Lines changed: 73 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -241,14 +241,86 @@ jobs:
241241
working-directory: gui/app
242242
run: npm run build:web # presync check + prebuild sync-wasm (js/wasm → public/wasm), then next build → out/ (basePath /app)
243243

244+
# ---- Build the mdBook guide for the /guide/ mount ----
245+
# [OPUS-5] issue #5022 — the guide's publish half. docs.yml validates the same guide on
246+
# PRs but deliberately ships no deploy job (one Pages slot, owned here); this is where it
247+
# actually reaches the web. The mdbook version below MUST equal docs.yml's pin — the two
248+
# lanes are only equivalent if they build with the same renderer, and
249+
# scripts/check-guide-publish-wiring.py fails the build if they diverge.
250+
- name: Install mdbook (pinned)
251+
uses: taiki-e/install-action@18b1216eba7f8039b0f8d131d5473787f0edce68 # v2.85.3
252+
with:
253+
tool: mdbook@0.4.40
254+
# Shared with docs.yml so the validate lane and the publish lane cannot drift. The script
255+
# carries the load-bearing detail: mdBook exits 0 on a broken {{#include}}/anchor
256+
# (rust-lang/mdBook#1094), so it greps the build log for [ERROR] — without that, an anchor
257+
# rename in README.md/SKILL.md would PUBLISH a guide full of raw include directives, green.
258+
- name: Build the mdBook guide
259+
run: bash scripts/build-guide.sh book
260+
261+
# ---- Overlay the mdBook guide at /guide/ ----
262+
# [OPUS-5] issue #5022 — place the rendered guide (book/book) into out/guide/, mirroring
263+
# the dev/bench* and /app overlays: another sub-tree of the SAME single Pages artifact, so
264+
# there is still exactly ONE producer for the one Pages deploy slot.
265+
#
266+
# Runs BEFORE the lychee link-check so the site's own links INTO /guide/ (the app-shell
267+
# Docs link) resolve against the real render; check-site-links.sh excludes guide/ as an
268+
# INPUT, so the guide's own HTML is not scanned. The guide is a separate first-party render with its own link shape (mdBook resolves chapter links
269+
# relatively, and the `link-fixup` preprocessor rewrites the {{#include}}d repo-relative
270+
# links to absolute github.com URLs, which lychee --offline cannot resolve anyway); its
271+
# broken-include gate is the [ERROR] grep in scripts/build-guide.sh, not lychee.
272+
#
273+
# UNLIKE the /app overlay there is NO placeholder route to replace here — the site export
274+
# has no /guide route — so an out/guide that ALREADY exists means the site grew one and
275+
# this step would silently clobber it. That is a hard failure, not a merge.
276+
- name: Overlay mdBook guide (out/guide/)
277+
working-directory: site
278+
run: |
279+
set -euo pipefail
280+
guide_out="$GITHUB_WORKSPACE/book/book"
281+
if [ ! -f "$guide_out/index.html" ]; then
282+
echo "::error::mdBook render missing ($guide_out/index.html) — the guide build step did not produce book/book/"
283+
exit 1
284+
fi
285+
if [ -e out/guide ]; then
286+
echo "::error::out/guide already exists BEFORE the guide overlay — the site export now has its own /guide route, which this overlay would silently clobber. Pick a different mount or drop the site route (issue #5022)."
287+
exit 1
288+
fi
289+
echo "Overlaying mdBook render -> out/guide/"
290+
mkdir -p out/guide
291+
# Plain `cp -a` (not the /app overlay's `rsync -a --delete`): that overlay mirrors
292+
# ONTO a pre-existing placeholder tree and needs --delete to clear its orphans. Here
293+
# the guard above has already proven out/guide did not exist, so there is nothing to
294+
# delete and a copy is the whole operation.
295+
cp -a "$guide_out/." out/guide/
296+
# MOUNT ASSERTION: mdBook stamps book.toml's `site-url` into the guide's OWN 404.html
297+
# as <base href="...">. GitHub Pages serves that directory-level 404 for /guide/<typo>,
298+
# so if the baked base href ever stops matching the sub-path we overlay to, every asset
299+
# on that page resolves against the wrong root. Assert the two agree here, at the point
300+
# where the mount is actually chosen.
301+
if grep -q '<base href="/guide/">' out/guide/404.html; then
302+
echo "OK: guide 404.html carries <base href=\"/guide/\"> matching the overlay mount"
303+
else
304+
echo "::error::out/guide/404.html does not carry <base href=\"/guide/\"> — book.toml's [output.html] site-url has drifted from the /guide/ mount, so the guide's 404 page would 404 its own assets"
305+
exit 1
306+
fi
307+
# NOTE (measured): a second "no raw {{#include}} survived into the HTML" scan was tried
308+
# here and REMOVED — it false-positives on the guide's own legitimate prose, which
309+
# discusses the single-sourcing mechanism by name (book/src comments and the README
310+
# `features-core` anchor both write `{{#include}}` in running text). The broken-include
311+
# guard is the `[ERROR]` log grep in scripts/build-guide.sh, which runs before this step
312+
# and whose teeth are pinned in scripts/tests/test_build_guide.sh.
313+
echo "OK: mdBook guide overlaid at out/guide/" >> "$GITHUB_STEP_SUMMARY"
314+
244315
# [OPUS-4.8] sq-d8or — anti-drift: link-check the BUILT site HTML before publish.
245316
# The docs-quality `internal-links` job (sq-5fd1) lychee-checks the repo MARKDOWN;
246317
# this extends that scope to the static-export HTML the Pages site actually serves —
247318
# catching renamed routes / stale cross-links / dangling #anchors that markdown linting
248319
# cannot see. lychee runs --offline (deterministic, no network) over site/out, remapping
249320
# the /sparq GitHub-Pages base path to the export root. Runs BEFORE the dev/ overlay so
250321
# it checks only THIS site's source (the overlaid benchmark dashboard is excluded by the
251-
# script anyway). The remap + teeth live in a shared, self-tested script
322+
# script anyway). It runs AFTER the guide overlay so links into /guide/ resolve; the
323+
# script skips guide/ as an input. The remap + teeth live in a shared, self-tested script
252324
# (scripts/check-site-links.sh + scripts/tests/test_check_site_links.sh, run in
253325
# docs-quality's ci-scripts job) so the logic is pinned and reusable locally. A broken
254326
# internal link FAILS the build → no broken deploy. lychee is installed via the same
@@ -262,23 +334,6 @@ jobs:
262334
- name: Link-check the static-export HTML (lychee --offline)
263335
run: bash scripts/check-site-links.sh site/out
264336

265-
# ---- Build the mdBook guide for the /guide/ mount ----
266-
# [OPUS-5] issue #5022 — the guide's publish half. docs.yml validates the same guide on
267-
# PRs but deliberately ships no deploy job (one Pages slot, owned here); this is where it
268-
# actually reaches the web. The mdbook version below MUST equal docs.yml's pin — the two
269-
# lanes are only equivalent if they build with the same renderer, and
270-
# scripts/check-guide-publish-wiring.py fails the build if they diverge.
271-
- name: Install mdbook (pinned)
272-
uses: taiki-e/install-action@18b1216eba7f8039b0f8d131d5473787f0edce68 # v2.85.3
273-
with:
274-
tool: mdbook@0.4.40
275-
# Shared with docs.yml so the validate lane and the publish lane cannot drift. The script
276-
# carries the load-bearing detail: mdBook exits 0 on a broken {{#include}}/anchor
277-
# (rust-lang/mdBook#1094), so it greps the build log for [ERROR] — without that, an anchor
278-
# rename in README.md/SKILL.md would PUBLISH a guide full of raw include directives, green.
279-
- name: Build the mdBook guide
280-
run: bash scripts/build-guide.sh book
281-
282337
# ---- Preserve the existing benchmark dashboard(s) at /dev/ ----
283338
# [OPUS-4.8] sq-zngy — fold ALL `dev/bench*` subtrees (not just dev/bench).
284339
# bench.yml writes benchmark-data:dev/bench (benchmark-data-dir-path: dev/bench);
@@ -366,59 +421,6 @@ jobs:
366421
fi
367422
echo "OK: operational GUI overlaid at out/app/" >> "$GITHUB_STEP_SUMMARY"
368423
369-
# ---- Overlay the mdBook guide at /guide/ ----
370-
# [OPUS-5] issue #5022 — place the rendered guide (book/book) into out/guide/, mirroring
371-
# the dev/bench* and /app overlays: another sub-tree of the SAME single Pages artifact, so
372-
# there is still exactly ONE producer for the one Pages deploy slot.
373-
#
374-
# Runs AFTER the lychee link-check, which checks only the SITE's own export. The guide is a
375-
# separate first-party render with its own link shape (mdBook resolves chapter links
376-
# relatively, and the `link-fixup` preprocessor rewrites the {{#include}}d repo-relative
377-
# links to absolute github.com URLs, which lychee --offline cannot resolve anyway); its
378-
# broken-include gate is the [ERROR] grep in scripts/build-guide.sh, not lychee.
379-
#
380-
# UNLIKE the /app overlay there is NO placeholder route to replace here — the site export
381-
# has no /guide route — so an out/guide that ALREADY exists means the site grew one and
382-
# this step would silently clobber it. That is a hard failure, not a merge.
383-
- name: Overlay mdBook guide (out/guide/)
384-
working-directory: site
385-
run: |
386-
set -euo pipefail
387-
guide_out="$GITHUB_WORKSPACE/book/book"
388-
if [ ! -f "$guide_out/index.html" ]; then
389-
echo "::error::mdBook render missing ($guide_out/index.html) — the guide build step did not produce book/book/"
390-
exit 1
391-
fi
392-
if [ -e out/guide ]; then
393-
echo "::error::out/guide already exists BEFORE the guide overlay — the site export now has its own /guide route, which this overlay would silently clobber. Pick a different mount or drop the site route (issue #5022)."
394-
exit 1
395-
fi
396-
echo "Overlaying mdBook render -> out/guide/"
397-
mkdir -p out/guide
398-
# Plain `cp -a` (not the /app overlay's `rsync -a --delete`): that overlay mirrors
399-
# ONTO a pre-existing placeholder tree and needs --delete to clear its orphans. Here
400-
# the guard above has already proven out/guide did not exist, so there is nothing to
401-
# delete and a copy is the whole operation.
402-
cp -a "$guide_out/." out/guide/
403-
# MOUNT ASSERTION: mdBook stamps book.toml's `site-url` into the guide's OWN 404.html
404-
# as <base href="...">. GitHub Pages serves that directory-level 404 for /guide/<typo>,
405-
# so if the baked base href ever stops matching the sub-path we overlay to, every asset
406-
# on that page resolves against the wrong root. Assert the two agree here, at the point
407-
# where the mount is actually chosen.
408-
if grep -q '<base href="/guide/">' out/guide/404.html; then
409-
echo "OK: guide 404.html carries <base href=\"/guide/\"> matching the overlay mount"
410-
else
411-
echo "::error::out/guide/404.html does not carry <base href=\"/guide/\"> — book.toml's [output.html] site-url has drifted from the /guide/ mount, so the guide's 404 page would 404 its own assets"
412-
exit 1
413-
fi
414-
# NOTE (measured): a second "no raw {{#include}} survived into the HTML" scan was tried
415-
# here and REMOVED — it false-positives on the guide's own legitimate prose, which
416-
# discusses the single-sourcing mechanism by name (book/src comments and the README
417-
# `features-core` anchor both write `{{#include}}` in running text). The broken-include
418-
# guard is the `[ERROR]` log grep in scripts/build-guide.sh, which runs before this step
419-
# and whose teeth are pinned in scripts/tests/test_build_guide.sh.
420-
echo "OK: mdBook guide overlaid at out/guide/" >> "$GITHUB_STEP_SUMMARY"
421-
422424
# [OPUS-4.8] sq-zngy — ARTIFACT-LEVEL smoke check (NOT a live-URL check).
423425
# This step asserts the ASSEMBLED Pages output dir (site/out, the exact tree handed to
424426
# upload-pages-artifact below) rather than probing the live https://sparq.jeswr.org/dev/...
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"pr": 6664,
3+
"date": "2026-10-05",
4+
"reason": "TripleStore::heap_bytes (always-compiled sparq-core) now also counts the pred_stats map (#3115)"
5+
}

‎crates/sparq-core/src/store.rs‎

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1027,11 +1027,14 @@ impl TripleStore {
10271027
self.overlay.as_ref().map_or(0, |ov| ov.added.len() + ov.deleted.len())
10281028
}
10291029

1030-
/// Heap footprint of the permutation indexes in bytes (for benchmarking). Memory-
1031-
/// mapped permutations contribute 0 — their resident pages are OS page cache.
1030+
/// Heap footprint of the permutation indexes, the overlay and the planner's
1031+
/// per-predicate statistics map in bytes (for benchmarking). Memory-mapped
1032+
/// permutations contribute 0 — their resident pages are OS page cache. The stats map
1033+
/// is counted as capacity × (entry + 1 control byte), like `Dict`'s tables (#3115).
10321034
pub fn heap_bytes(&self) -> usize {
10331035
self.perms.iter().map(PermData::heap_bytes).sum::<usize>()
10341036
+ self.overlay.as_ref().map_or(0, |ov| ov.heap_bytes())
1037+
+ self.pred_stats.capacity() * (std::mem::size_of::<(Id, PredStat)>() + 1)
10351038
}
10361039

10371040
/// Chooses the permutation whose sort order places all bound pattern
@@ -2138,6 +2141,25 @@ mod tests {
21382141
}
21392142
}
21402143

2144+
/// `heap_bytes` must count the planner's per-predicate statistics map (`pred_stats`),
2145+
/// not only the permutations + overlay (#3115). Wide-predicate corpora (Wikidata has
2146+
/// ~100k properties) carry MBs here; it was silently uncounted.
2147+
#[test]
2148+
fn heap_bytes_counts_pred_stats() {
2149+
// 1_000 distinct predicates so the stats map is non-trivially sized.
2150+
let triples: Vec<[Id; 3]> = (0..1_000u64).map(|p| [1, p as Id, 2]).collect();
2151+
let store = TripleStore::from_triples(triples);
2152+
assert_eq!(store.pred_stats.len(), 1_000);
2153+
let perms: usize = store.perms.iter().map(PermData::heap_bytes).sum();
2154+
let stats = store.pred_stats.capacity() * (std::mem::size_of::<(Id, PredStat)>() + 1);
2155+
assert!(stats > 0);
2156+
assert_eq!(
2157+
store.heap_bytes(),
2158+
perms + stats,
2159+
"heap_bytes must include pred_stats"
2160+
);
2161+
}
2162+
21412163
/// [SONNET-4.6 sq-7d3dj.32.1] Each built raw-mode permutation Vec must carry zero
21422164
/// capacity slack after construction. heap_bytes() uses capacity(), so any excess
21432165
/// inflates the reported B/triple; shrink_to_fit() (called in both build_raw_perms

‎crates/sparq-substrate/Cargo.toml‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,13 @@ all-features = true
2424
# unblocks the substrate move-chain (research/shared-eval-substrate.md, Option C). It
2525
# depends ONLY on sparq-core (NOT sparq-engine) so BOTH the engine AND the reasoners
2626
# (sparq-reason / sparq-reason-el, which depend only on sparq-core) can later consume it
27-
# with NO dependency cycle. This crate is a SCAFFOLD: it defines / re-exports the shared
28-
# id-tuple vocabulary and the numeric value tower, behind DEFAULT-OFF features, with NO
29-
# evaluation logic moved yet (the join kernels + the engine `compare_values`/`Num`
30-
# arithmetic land in later beads of the epic). Nothing in the workspace depends on it, so
31-
# the lean default / wasm build does not compile any of it and the wasm_bundle_bytes floor
32-
# stays byte-identical (Phase-1 acceptance, research record §7.1).
27+
# with NO dependency cycle. It holds the shared id-tuple vocabulary, the numeric value
28+
# tower, the comparison helpers and the hash-join kernels, each behind a DEFAULT-OFF
29+
# feature. It is NO LONGER unconsumed: `sparq-engine` depends on it unconditionally
30+
# (features numeric + join + compare) and `sparq-reason` depends on it (numeric, plus
31+
# rows/join behind its own features), so it IS compiled into the lean default / feature-OFF
32+
# wasm bundle. A change to code behind those features moves `wasm_bundle_bytes` and may need
33+
# a feature-off declaration (bench/feature-off-declarations/) like any engine change (#3806).
3334

3435
[features]
3536
# DEFAULT-OFF by construction: the lean core/engine/wasm build must compile NONE of this

‎scripts/check-site-links.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,10 @@ echo "check-site-links: lychee --offline over ${OUT_ABS}/**/*.html (root-relativ
7070
# fragment. `/?` makes it match both `/#frag` and the slashless `#frag` normalisation.
7171
# `dev/` (the overlaid benchmark dashboard, a separate first-party artifact written by bench.yml
7272
# onto benchmark-data) is excluded: it is not part of THIS site's source and carries its own links.
73+
# `guide/` (the overlaid mdBook guide, issue #5022) is excluded as an INPUT for the same reason: its
74+
# broken-include gate is scripts/build-guide.sh, and its link-fixup output is not lychee-offline
75+
# shaped. It must still EXIST under the export when this runs (pages.yml overlays it first), so the
76+
# site's own links INTO /guide/ (the app-shell Docs link) are resolved and checked against it.
7377
lychee \
7478
--offline \
7579
--include-fragments \
@@ -78,4 +82,5 @@ lychee \
7882
--remap "file://${OUT_ABS}/((?:[^#]*/)?[^#/.]+)/?#(.+) file://${OUT_ABS}/\$1/index.html#\$2" \
7983
--remap "file://${OUT_ABS}/?#(.+) file://${OUT_ABS}/index.html#\$1" \
8084
--exclude-path "${OUT_ABS}/dev" \
85+
--exclude-path "${OUT_ABS}/guide" \
8186
"${OUT_ABS}/**/*.html"

‎scripts/virtuoso-same-box.sh‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
# TUNABLES (env; all have safe defaults):
3434
# VIRTUOSO_IMAGE Docker image (default docker.io/openlink/virtuoso-opensource-7:latest)
3535
# VIRTUOSO_HTTP_PORT SPARQL HTTP port (default 8890)
36-
# VIRTUOSO_ISQL_PORT isql port (default 1111)
36+
# VIRTUOSO_ISQL_PORT host isql port (default 1111)
3737
# VIRTUOSO_PASSWORD dba password (default dba)
3838
# VIRTUOSO_GRAPH target named graph IRI (default http://sparq.bench/graph)
3939
# VIRTUOSO_PULL_TIMEOUT image-pull hard cap, s (default 600)
@@ -128,7 +128,8 @@ docker run -d --name "$VIRTUOSO_NAME" \
128128
|| die "failed to start Virtuoso container"
129129

130130
# ---- 2. bounded readiness poll (isql answers once the server is up) --------------------
131-
isql() { docker exec "$VIRTUOSO_NAME" isql "$VIRTUOSO_ISQL_PORT" dba "$VIRTUOSO_PASSWORD" "$@"; }
131+
# Inside the container isql always listens on 1111; VIRTUOSO_ISQL_PORT is only the host-side mapping (#3361).
132+
isql() { docker exec "$VIRTUOSO_NAME" isql 1111 dba "$VIRTUOSO_PASSWORD" "$@"; }
132133
ready=0
133134
poll_n=$(( VIRTUOSO_READY_TIMEOUT / 3 )); [ "$poll_n" -ge 1 ] || poll_n=1
134135
for _ in $(seq 1 "$poll_n"); do
@@ -147,7 +148,7 @@ log "server ready"
147148
# checkpoint persists. Then verify the graph is non-empty before trusting any query.
148149
log "bulk-load (<= ${VIRTUOSO_LOAD_TIMEOUT}s) $CORPUS_FILE into <$VIRTUOSO_GRAPH>"
149150
LOAD_SQL="ld_dir('/data', '${CORPUS_FILE}', '${VIRTUOSO_GRAPH}'); rdf_loader_run(); checkpoint;"
150-
if ! timeout "$VIRTUOSO_LOAD_TIMEOUT" bash -c "docker exec '$VIRTUOSO_NAME' isql '$VIRTUOSO_ISQL_PORT' dba '$VIRTUOSO_PASSWORD' exec=\"$LOAD_SQL\"" >&2; then
151+
if ! timeout "$VIRTUOSO_LOAD_TIMEOUT" bash -c "docker exec '$VIRTUOSO_NAME' isql 1111 dba '$VIRTUOSO_PASSWORD' exec=\"$LOAD_SQL\"" >&2; then
151152
rc=$?
152153
[ "$rc" = 124 ] && die "bulk load hit the ${VIRTUOSO_LOAD_TIMEOUT}s timeout"
153154
die "bulk load failed (rc=$rc)"

‎site/e2e/site-nav.spec.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,11 +46,11 @@ test("the slim top bar shows the destinations (no Try item) and no full sidebar
4646
]) {
4747
await expect(primary.getByRole("link", { name: label, exact: true })).toBeVisible();
4848
}
49-
// [GPT-5.6] sq-f8ufg — mutation witness: removing Docs or pointing it at the nonexistent
50-
// /docs route makes this assertion fail. The Skills router is the canonical live docs index.
49+
// #2549 — mutation witness: Docs must point at the mdBook guide overlaid at /guide/ by
50+
// pages.yml (basePath-prefixed, trailing slash), not at the agent-skills router.
5151
await expect(primary.getByRole("link", { name: "Docs", exact: true })).toHaveAttribute(
5252
"href",
53-
"https://github.com/sparq-org/sparq/blob/main/skills/SKILL.md",
53+
/\/guide\/$/,
5454
);
5555
// [OPUS-4.8] sq-4hiqe — the "Try" nav item is REMOVED (the /try playground is gone). The bar
5656
// must expose NO "Try" destination now.

0 commit comments

Comments
 (0)