Commit b4258ee
install: ship what the docs promise + close the smoke-test hole that hid it (#3571)
* install: ship what the docs promise + close the smoke-test hole that hid it
Audited `cmake --install` coverage against the tree. Seven gaps broke workflows
the *shipped* docs tell users to follow; the worst is that `utils/mcp/setup.das`
has never been installed, while the installed `utils/mcp/README.md` and the
shipped `skills/mcp_tools.md` both document running it. The MCP bootstrap was
impossible from a bundle through 0.6.4.
Root cause of the class: ci/smoke_test_bundle.sh compiled 9 of the 14 installed
.das entry points, and setup.das sat in the gap, so nothing ever noticed. All 14
are covered now (12 checks -> 17).
Ship what was missing:
- utils/mcp/setup.das + mcp_supervisor.py (the documented bootstrap).
- utils/mcp/tests/ non-.das fixtures. `FILES_MATCHING PATTERN "*.das"` dropped
_fixture_cpp_outline.h and _pretend_root/.../.das_module, both required by the
installed test_tools.das, so the suite shipped unable to run those cases.
- utils/lint/tests/ (77 files), so a bundle can self-check the linter. Verified:
51 tests, 51 passed via dastest inside the install tree.
- examples/daStrudel/sfx_lab sounds/*.sfx + .das_package. The scripts read these
via get_das_root(); shipping the scripts alone left them broken.
- examples/graphics per-example .das_package + .gitignore. Each subdir is a
daspkg -project-root whose release_main points at the shared source one level
up, so the non-recursive glob dropped the load-bearing manifests.
- examples/benchmarks/, documented in the installed examples/README.md.
- aot_testing / llvm_tune / perf_lint / style_lint skills, cross-referenced by
already-shipped skills. tutorials.md deliberately stays repo-only and the one
reference to it (skills/xml.md) is reworded instead, since shipping it would
drag in documentation_rst.md and onward.
- tree-sitter-daslang/zed-daslang/, which install/README.md points users at.
- CHANGELIST.md: a versioned bundle had no release notes.
- integration_cpp_23 in the standalone tutorial recipe. The source shipped with
no way to build it; handle_registry.h was already installed.
Decouple tutorial data install from DAS_TUTORIAL_DISABLED. The install rules for
all 79 C/C++ integration tutorial sources lived inside the gated include(), so
"don't build the tutorial executables" silently also meant "don't ship the
tutorial sources". release.yml never sets the flag, so shipping was fine today.
Verified by flipping it: 50 integration-source refs survive in the install script.
Hygiene:
- Drop `FLEX(utils/dasFsormatter/ds_lexer.lpp)`: typo'd dir that does not exist,
and dasFormatter has no .lpp at all. Inert only because no target consumed the
output path.
- examples/daspkg used a hardcoded DESTINATION instead of DAS_INSTALL_EXAMPLESDIR,
and neither it nor examples/games excluded modules/ or build output, so a dev
tree would ship daspkg junk.
- file(STRINGS) on install/skills.list now passes ENCODING UTF-8. Without it,
non-ASCII bytes act as separators: an em-dash in a comment splits the line and
the tail is read as a skill filename, failing the guard with a nonsense path.
Also fix the dasHV tutorial artifact that got committed: the server saves
{upload_dir}/_testfile.txt but cleanup_upload_dir removed {dir}/testfile.txt (no
underscore), so the file survived, remove(dir) failed on a non-empty dir, and
_upload_test/_testfile.txt landed in git. Fix the name, untrack the file,
gitignore the run dirs. Correct the Zed extension path in both READMEs
(zed-daslang/, not editors/zed/).
Record the durable rules in skills/install_instructions.md: the shipped-skill
closure invariant with a check script, and the ASCII-only rule for skills.list.
Verified end to end: release-flag bundle installed to a scratch prefix and diffed
against the pre-fix bundle -- exactly 134 files added, nothing unintended;
smoke test 17/17; zero dangling skill-to-skill references across 33 shipped skills.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiXc1jvihNmEnU7DDQidvt
* skills: make the SDK self-contained — one source per doc, and mark what can't ship
Follow-up to the install audit in this PR. The shipped skills sent readers to 70
repo paths that a bundle never contains. Two problems were tangled together there,
and they need opposite fixes.
Where the skill relies on the content, the content now SHIPS, as markdown, with
exactly one source file. Teach Sphinx to read markdown (myst-parser==2.0.0, the
last line supporting sphinx 7.2.6) and each .rst becomes a ~13-line pointer that
includes the .md via `.. include:: ... :parser: myst_parser.sphinx_`. daslang.io
and the SDK then read the same bytes, so they cannot drift, and there is no
generated artifact to keep in sync. Converted this way, and added to
install/skills.list:
skills/design_philosophy.md (was reference/design_philosophy.rst)
skills/tune.md (was reference/tune.rst)
skills/profiler.md (was reference/utils/profiler.rst)
skills/linq_fold_patterns.md (was reference/linq_fold_patterns.rst)
skills/detect_dupe_reference.md (was reference/utils/detect_dupe.rst)
Two of those RST pages share a stem with an existing shipped skill, because they
are the same topic twice. They are NOT the same case. memory_leak_detection.md
said outright that its RST "covers the same material in docs style", so that
pointer is simply reworded and nothing new ships — four of its five references now
resolve to skills/profiler.md, inside the bundle. detect_dupe.rst is genuinely
fuller than its skill (541 lines, five flag tables), so it ships under a
non-colliding name. Breadcrumb pointers in json.md / regex.md / das_macros.md
("RST companion at", "generated stdlib reference at" — two of them naming
stdlib/generated files that do not exist in a fresh checkout) point at the
published page instead.
Where the path is engine internals, the answer is the reverse of shipping it.
src/, tests/, benchmarks/ and modules/*/src are never going into a bundle, so the
line now says so with a repo-only token, and the reader skips it instead of
hunting a path that is not there. 21 refs marked: two heading retitles cover 7
(cpp_integration's "C++ Codebase Notes", jobque_debugging's "Key files"), and 12
one-off prose mentions are marked inline so the sentence stays with the point it
supports.
Enforced per-PR by ci/check_shipped_skill_refs.awk, run from
ci/smoke_test_bundle.sh (build.yml's bundle_smoke job, which triggers on
pull_request) — no new CI wiring. Smoke test 17 -> 18 checks. Exemption is
SECTION-scoped and resets at each heading: a trailing-marker rule would leak the
exemption through the rest of the file, which is wrong for cpp_integration.md
where the internals section sits mid-file. Fenced blocks are exempt because they
quote tool output verbatim, including a compiler warning that literally prints a
doc/source path. tests/ is deliberately not in the pattern yet (53 refs, 24 of
them in aot_testing.md) — that class still needs sorting into inline-the-snippet
vs quarantine, and the awk says so.
Verified, not assumed:
* docs built with CI's exact command (sphinx-build -W --keep-going): ZERO new
warnings, six removed, against a 282-line baseline. Page labels had to move
into the .rst stubs, because a MyST (label)= target does NOT register in
Sphinx's std domain when the markdown arrives through an include with
:parser:; and two bare :ref: uses needed explicit link text once the title
moved into the markdown.
* the checker negative-tested: clean passes; an unmarked src/ path fails with
file:line; the same line marked repo-only passes; a fenced path is ignored.
* doc/source citations from shipped skills: 9 -> 1, and that one is a compiler
warning quoted inside a fence, which must stay verbatim.
* bundle reinstalled; 38 skills ship; zero dangling skill-to-skill references.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiXc1jvihNmEnU7DDQidvt
* skills: gate the mechanical failures, split writing_tests, document how to write a skill
Three pieces: a per-PR gate for what a script can check, a review skill for what it
can't, and a CLAUDE.md section so the decision gets made before the writing.
ci/check_shipped_skills.py replaces the awk from the previous commit. The link check
needs to tell a missing file from a directory, which awk cannot do portably, so this is
Python — invoked from ci/smoke_test_bundle.sh and skipped (not failed) when no
interpreter is present, so the script stays runnable on a bare box. Five classes, all in
a SHIPPED skill:
* src/, doc/source/, benchmarks/, modules/*/{src,tests,tutorial,harness}
* bin/Release|Debug/... -- the Windows repo layout; a bundle has bin/daslang
* daslang.exe <script> -- .exe is Windows-only
* C:\Users\..., /home/<name>/..., AppData -- someone's machine in a doc
* a relative link whose target is absent from the bundle
* skills/<other>.md where <other> is not shipped (the closure invariant, ported over)
It found 34 real hits, all now fixed. 14 were bin/Release/daslang.exe -- including the
first three lines of das_formatting.md, i.e. instructions that are wrong on two of three
platforms. 8 were dead links, 3 named generated .rst that exists nowhere, 9 were .exe
invocations.
Narrowness was the hard part, and two of my own patterns were wrong before they were
right. A naive absolute-path regex matched every http:// URL (77 false hits) and the
\\-UNC branch matched \\d and \\s in regex prose; that check is now scoped to
machine-local paths only, so the 17 deliberate C:/DummyRoot examples in
external_module_debugging.md stay quiet. The link resolver joined against the bundle root
while links are authored relative to skills/, which made 19 live links look dead. And the
.exe check flagged prose ("shared across `daslang.exe` and every module DLL") until it
required the next token to be a flag or a real file. A gate that cries wolf gets muted,
and then it is worse than no gate — so all 21 shapes are negative-tested: 9 must-fail, 10
false-positive shapes that must stay quiet, plus a check that a section-scoped repo-only
marker RESETS at the next heading instead of leaking down the file.
Also fixed while wiring it up: `command -v python3` finds the Windows Store shim, which
is on PATH but exits "Permission denied". The probe now executes the interpreter.
Split writing_tests.md, which was two audiences in one file. The universal dastest
guidance stays and ships; the repo's own harness — AOT registration for a new tests/ dir,
the tests/.das_test gating filter and its root-path caveat, per-folder sweep gating,
deep-engine model-test rules — moves to skills/tests_in_repo.md, deliberately NOT shipped.
The shipped half also now points at dastest/tests, which ships and doubles as a worked
example, instead of tests/linq, which does not. My first draft of the shipped file
referenced skills/tests_in_repo.md — the exact closure violation documented one commit
earlier — which is why the closure check is now enforced by the script rather than living
as a shell snippet in a doc.
New skills/writing_skills.md (repo-only) covers what no script can: pick the audience
BEFORE writing, because a mixed file serves neither and splitting it afterwards is a
reorganization rather than a cut (aot_testing.md is the standing example, still unsplit);
check whether an existing skill or doc/source page already owns the topic, since two
documents drifting apart beats one imperfect one; verify syntax claims by probe rather
than recall; and never silence the gate with a marker when the honest answer is a split.
CLAUDE.md gains a short "Writing a new skill" section pointing at it, plus a table row.
Verified: smoke test 18 checks green; gate exits 0 against the reinstalled bundle;
38 skills ship; tests_in_repo.md and writing_skills.md correctly absent from it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiXc1jvihNmEnU7DDQidvt
* skills: don't ship aot_testing.md, and gate tests/ now that nothing needs it
aot_testing.md is the daslang repo's AOT test harness -- the test_aot binary, tests/aot
registration, libDaScriptAot regeneration, CI wiring. An SDK user has no way to act on any
of it, so it is not their skill and it no longer ships. That is simpler and more honest
than the split I was heading for, and it reverts churn: the six edits I had made to that
file were all gate-appeasement, including turning bin/Release/daslang.exe into bin/daslang,
which is a REGRESSION for Windows repo devs whose layout really is bin/Release. The file is
now byte-identical to master again.
The only shipped skill that referenced it was cpp_integration.md. Rather than drop the
pointer, that section now carries the actionable part inline -- generate stubs with
`bin/daslang -aot in.das out.cpp`, add -aot-macros for macros, regenerate on
error[50101] -- and names aot_testing.md as repo-only for the harness itself. install/
CLAUDE.md loses its row; install/skills.list explains WHY the file is absent, so the next
person does not re-add it to satisfy the closure invariant.
With aot_testing.md out and writing_tests.md already split, tests/ can finally join the
gate pattern. That surfaced 10 more refs in shipped skills, all worked-example pointers
into the repo test corpus (das_macros, dashv, daslib_modules, llvm_tune, xml, plus two in
my own new prose) -- each now marked rather than deleted, since they are useful to a repo
reader and simply absent for an SDK one.
The marker is now PARAGRAPH-scoped, not line-scoped. Two of those last ten hits were text
I had just written, where the marker landed one line away from the path because the
sentence wrapped -- twice. Scoping per-line means an author has to reflow prose to satisfy
the gate, which is exactly how a gate earns being muted. A marker anywhere in a paragraph
now covers it; the exemption still does not leak across a blank line, and heading-scoped
and line-scoped markers are unchanged. Verified both directions.
Two process traps hit while doing this, both worth knowing:
* `cmake --install` is ADDITIVE. Un-shipping a file leaves the old copy in the prefix, so
the gate ran green against a bundle that still contained aot_testing.md. Removing the
prefix first is the only way to see the real shipped set.
* install/skills.list is read at CONFIGURE time via file(STRINGS). Editing it and running
only `cmake --install` installs the OLD list. Reconfigure first.
Verified: fresh install ships 37 skills with aot_testing.md absent; gate exits 0; smoke
test 18 checks green; the full negative-test suite still passes (9 must-fail classes, 10
false-positive shapes that must stay quiet, section-marker reset, and the two new
paragraph-scope cases).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiXc1jvihNmEnU7DDQidvt
Also adds ci/reinstall_bundle.sh: reconfigure + wipe prefix + install + verify in one
command, so the loop is a single reviewable invocation instead of an ad-hoc `rm -rf`. It
guards the delete -- refuses anything that is not absent or recognizably an installed
bundle, and refuses a source checkout outright -- and its comments encode the two traps
above so the next person does not rediscover them.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 91b9a69 commit b4258ee
60 files changed
Lines changed: 2991 additions & 2358 deletions
File tree
- ci
- doc
- source
- reference
- utils
- install
- modules/dasAudio
- skills
- tutorials
- dasHV
- _upload_test
- integration
- cpp
- c
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
129 | 136 | | |
130 | 137 | | |
131 | 138 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
| 59 | + | |
59 | 60 | | |
60 | 61 | | |
61 | 62 | | |
| |||
69 | 70 | | |
70 | 71 | | |
71 | 72 | | |
| 73 | + | |
72 | 74 | | |
73 | 75 | | |
74 | 76 | | |
| |||
116 | 118 | | |
117 | 119 | | |
118 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
119 | 129 | | |
120 | 130 | | |
121 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
821 | 821 | | |
822 | 822 | | |
823 | 823 | | |
824 | | - | |
825 | 824 | | |
826 | 825 | | |
827 | 826 | | |
| |||
1799 | 1798 | | |
1800 | 1799 | | |
1801 | 1800 | | |
| 1801 | + | |
| 1802 | + | |
1802 | 1803 | | |
1803 | 1804 | | |
1804 | 1805 | | |
1805 | 1806 | | |
1806 | 1807 | | |
1807 | | - | |
| 1808 | + | |
| 1809 | + | |
| 1810 | + | |
| 1811 | + | |
1808 | 1812 | | |
1809 | 1813 | | |
1810 | 1814 | | |
| |||
1848 | 1852 | | |
1849 | 1853 | | |
1850 | 1854 | | |
| 1855 | + | |
| 1856 | + | |
| 1857 | + | |
| 1858 | + | |
1851 | 1859 | | |
1852 | 1860 | | |
1853 | 1861 | | |
| |||
1856 | 1864 | | |
1857 | 1865 | | |
1858 | 1866 | | |
| 1867 | + | |
| 1868 | + | |
1859 | 1869 | | |
1860 | 1870 | | |
1861 | 1871 | | |
| |||
1868 | 1878 | | |
1869 | 1879 | | |
1870 | 1880 | | |
| 1881 | + | |
| 1882 | + | |
| 1883 | + | |
1871 | 1884 | | |
1872 | 1885 | | |
1873 | | - | |
| 1886 | + | |
| 1887 | + | |
| 1888 | + | |
| 1889 | + | |
1874 | 1890 | | |
1875 | 1891 | | |
1876 | 1892 | | |
| |||
1909 | 1925 | | |
1910 | 1926 | | |
1911 | 1927 | | |
1912 | | - | |
| 1928 | + | |
| 1929 | + | |
1913 | 1930 | | |
| 1931 | + | |
| 1932 | + | |
| 1933 | + | |
| 1934 | + | |
1914 | 1935 | | |
1915 | 1936 | | |
1916 | 1937 | | |
| |||
1974 | 1995 | | |
1975 | 1996 | | |
1976 | 1997 | | |
1977 | | - | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
1978 | 2001 | | |
1979 | | - | |
| 2002 | + | |
| 2003 | + | |
| 2004 | + | |
| 2005 | + | |
| 2006 | + | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
1980 | 2012 | | |
1981 | 2013 | | |
1982 | 2014 | | |
| |||
1994 | 2026 | | |
1995 | 2027 | | |
1996 | 2028 | | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
1997 | 2032 | | |
1998 | 2033 | | |
1999 | 2034 | | |
| |||
2062 | 2097 | | |
2063 | 2098 | | |
2064 | 2099 | | |
2065 | | - | |
2066 | | - | |
2067 | | - | |
2068 | | - | |
2069 | | - | |
2070 | | - | |
| 2100 | + | |
| 2101 | + | |
| 2102 | + | |
| 2103 | + | |
| 2104 | + | |
| 2105 | + | |
2071 | 2106 | | |
| 2107 | + | |
| 2108 | + | |
| 2109 | + | |
| 2110 | + | |
| 2111 | + | |
| 2112 | + | |
| 2113 | + | |
| 2114 | + | |
| 2115 | + | |
| 2116 | + | |
| 2117 | + | |
| 2118 | + | |
| 2119 | + | |
| 2120 | + | |
| 2121 | + | |
| 2122 | + | |
| 2123 | + | |
| 2124 | + | |
| 2125 | + | |
| 2126 | + | |
| 2127 | + | |
| 2128 | + | |
| 2129 | + | |
| 2130 | + | |
2072 | 2131 | | |
2073 | 2132 | | |
2074 | 2133 | | |
| |||
2279 | 2338 | | |
2280 | 2339 | | |
2281 | 2340 | | |
| 2341 | + | |
| 2342 | + | |
| 2343 | + | |
| 2344 | + | |
| 2345 | + | |
2282 | 2346 | | |
2283 | 2347 | | |
2284 | 2348 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
110 | | - | |
| 110 | + | |
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
| |||
0 commit comments