Skip to content

Commit 13268a4

Browse files
jedudenclaude
andauthored
Plans 165–167: portable export + schema-driven extraction (#292)
* Add plan 163: schema-driven data extraction (mdsmith extract) Introduces a `bind:` projection layer on schema scopes and content entries plus an `extract` subcommand that turns a kind-conformant Markdown file into JSON/YAML/Lua/msgpack. Sequenced on top of the schema-composition rework (plan 156) and content-schema work (plan 149). https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * Rework plan 163: default binding layer; split custom bindings to plan 164 - Default projection derives the data tree from the schema hierarchy; no annotations required. - Custom bindings move to follow-up plan 164, layered on a single keyFor() seam. - Address Copilot review: disambiguate the duplicate plan-156 id via explicit file link; name the concrete schema.MatchTree type returned by an extended schema.Validate. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 164: use *string for Bind to distinguish unset vs explicit-empty Addresses Copilot review: a plain string cannot represent "unset" vs `bind: ""` (hoist), so Bind is now *string. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 163: resolve open questions - Repeating-scope elements always retain each placeholder as a name:value field. - Root holds `frontmatter` plus projected sections beside it (no nesting under a single key). - Preamble projected under `preamble`; wildcard/unlisted skipped. - Defer Lua; ship json/yaml/msgpack. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 163: use current repeat:{min,max} terminology, not repeats:true The schema parser rejects the legacy `repeats` key; repeating cardinality is `repeat: {min, max}`. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * Add plan 165: portable Markdown export (mdsmith export) A source-to-source `export` subcommand that strips directive markers, keeps generated bodies, and inlines includes — distinct from schema extraction (163/164). https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 163: align with landed schema syntax — no-heading section, no preamble key Plan 156 entry-unification landed in main (PR #295): the standalone preamble: key is gone, replaced by heading: null. Project the no-heading section's content into the enclosing object instead of a preamble wrapper key. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * Renumber plans 163/164 -> 166/167; fix gensection API ref in 165 Rebasing onto main pulled in plan/163_public-markdown-library and plan/164_github-ui-releases-and-split-website, colliding with the new plans. Renumber schema-extraction 163->166 and custom-bindings 164->167 (export stays 165, no collision); update cross-references and depends-on. Plan 165: marker stripping is driven by gensection.FindMarkerPairs (MarkerPair.StartLine/EndLine), not lint.File.GeneratedRanges (which only records include/catalog body ranges). Tighten the acceptance criteria to distinguish engine-recognized markers from literal-content marker-like text. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 167: fix remaining stale "plan 163" refs -> 166; plan 166 depends-on Renumbering leftovers: three "plan 163" references in plan 167 (keyFor seam origin, collision diagnostic, acceptance criterion) now point to the renumbered extraction plan 166. Drop ambiguous id 156 from plan 166's depends-on (two plan files share id 156); the composition dependency stays expressed as a filename link in the Sequencing section. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 165: export checks staleness by default, never auto-fixes Auto-regenerating directive bodies on export is surprising and masks drift. Default mode now fails on a stale body (exit non-zero, no output); --fix opts into in-memory regeneration; --no-check skips the check. The two flags are mutually exclusive. Tasks, acceptance criteria, and decisions updated. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 165: align Export signature — ([]byte, []lint.Diagnostic) + Mode Tasks 1 and 4 disagreed on the return shape. Settle on Export(f, mode) ([]byte, []lint.Diagnostic), mirroring plan 166's Extract: a non-empty diagnostic slice means refusal (nil bytes); a stale body in Check mode appends a directive-positioned diagnostic; hard error is reserved for I/O only. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR * plan 165: clarify Export contract — no error return, exactly one value set Export operates on an in-memory *lint.File, so it does no I/O and returns no error; reads/writes are the CLI layer's job (real error there). Spell out the success vs refusal contract: success = non-nil bytes + nil diagnostics; refusal = nil bytes + non-empty diagnostics. https://claude.ai/code/session_01Ar54BuJr8fFB9KzJGvLvYR --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 024dad8 commit 13268a4

4 files changed

Lines changed: 503 additions & 0 deletions

File tree

PLAN.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,4 +91,7 @@ footer: |
9191
| 162 || sonnet | [Split the overloaded `meta` rule category](plan/162_rule-category-cleanup.md) |
9292
| 163 | 🔲 | | [Extract mdsmith Markdown parse/produce as a public Go library](plan/163_public-markdown-library.md) |
9393
| 164 || | [GitHub-UI-triggered releases and a split website deploy](plan/164_github-ui-releases-and-split-website.md) |
94+
| 165 | 🔲 | opus | [Portable Markdown export (mdsmith export)](plan/165_portable-markdown-export.md) |
95+
| 166 | 🔲 | opus | [Schema-driven data extraction (mdsmith extract)](plan/166_schema-driven-data-extraction.md) |
96+
| 167 | 🔲 | opus | [Custom binding overrides for mdsmith extract](plan/167_custom-binding-overrides.md) |
9497
<?/catalog?>
Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
---
2+
id: 165
3+
title: Portable Markdown export (mdsmith export)
4+
status: "🔲"
5+
model: opus
6+
depends-on: []
7+
summary: >-
8+
Add an `export` subcommand that writes a portable,
9+
directive-free copy of a Markdown file: markers
10+
removed, generated bodies kept, includes inlined.
11+
---
12+
# Portable Markdown export (mdsmith export)
13+
14+
## Goal
15+
16+
`mdsmith export <file>` writes a portable copy of a
17+
Markdown file with every `<?…?>` directive marker
18+
removed. Generated bodies stay as plain Markdown and
19+
`<?include?>` content is inlined. The result renders
20+
identically on any Markdown tool with no mdsmith
21+
knowledge.
22+
23+
## Why a separate command
24+
25+
This is not schema extraction. `extract` (plan 166)
26+
projects a kind's schema into a data tree. `export` is a
27+
source-to-source transform of the document itself. It
28+
needs no kind, schema, or conformance gate — only that
29+
the file parses and its directive bodies are fresh.
30+
31+
Mixing it into `extract --format markdown` would couple a
32+
plain-document transform onto the schema-projection
33+
command. A dedicated `export` keeps the two concerns
34+
apart and leaves room to grow (output path, later batch).
35+
36+
## Staleness: check by default, never auto-fix
37+
38+
`export` does **not** silently regenerate directive
39+
bodies. Auto-fixing on export is surprising and would
40+
mask drift between a directive and its rendered body.
41+
The default is to *check*, not to *fix*:
42+
43+
- **Default (check).** Before stripping, verify each
44+
directive body equals what the engine would generate.
45+
If any body is stale, export writes nothing and exits
46+
non-zero with a diagnostic naming the stale directive
47+
and advising `mdsmith fix` or `--fix`. The export is
48+
faithful — it never papers over drift.
49+
- **`--no-check`.** Skip the staleness check and export
50+
bodies exactly as they appear in the file. For callers
51+
who know the file is fresh or deliberately want the
52+
on-disk bytes.
53+
- **`--fix`.** Regenerate stale bodies in memory (same
54+
engine as `mdsmith fix`) before stripping. Opt-in
55+
convenience for a one-shot fresh export.
56+
57+
`--fix` and `--no-check` are mutually exclusive (one
58+
regenerates, the other trusts as-is); passing both is a
59+
usage error. In every mode the source file is never
60+
modified.
61+
62+
## Behavior
63+
64+
- Drop the opening and closing marker lines of every
65+
directive region; keep the body text between them
66+
verbatim (regenerated first only under `--fix`).
67+
- `<?include?>` bodies are already expanded by
68+
regeneration, so keeping the body inlines the included
69+
content (recursively).
70+
- Markerless directives with no body (for example
71+
`<?allow-empty-section?>`, `<?require?>`) are removed
72+
outright.
73+
- Only lines the engine's marker-pair detection
74+
recognizes as real directive start/end markers are
75+
removed. Marker-like text the engine treats as literal
76+
content (for example inner same-type markers nested in
77+
an outer directive) is left untouched.
78+
- After stripping, normalize blank lines so the output is
79+
stable and lint-clean. Front matter is kept as-is.
80+
- Exporting an already directive-free file is a no-op;
81+
`export` is idempotent.
82+
83+
## Tasks
84+
85+
1. **Export core (red/green).** Add `internal/export`
86+
with `Export(f *lint.File, mode Mode) ([]byte,
87+
[]lint.Diagnostic)` — mirroring plan 166's `Extract`
88+
signature. It operates purely on the already-parsed
89+
in-memory `*lint.File`, so it performs no I/O and
90+
returns no `error`; file reads and `-o` writes are the
91+
CLI layer's job (task 5) and surface as a real `error`
92+
there. Contract: exactly one of the two return values
93+
is populated. **Success** → the exported bytes (which
94+
are never `nil`, since a directive-free file still
95+
yields its own content) and a `nil` diagnostic slice.
96+
**Refusal** (stale body in `Check` mode, or any
97+
document-level problem) → `nil` bytes and a non-empty
98+
diagnostic slice; the caller exits non-zero. `Mode` is
99+
the staleness mode from task 4. Unit-test marker
100+
removal, body retention, include-body inlining, and
101+
the no-directive no-op.
102+
2. **Nested / literal-content markers.** Drive removal
103+
off the engine's own marker-pair detection —
104+
`gensection.FindMarkerPairs` in
105+
[internal/archetype/gensection](../internal/archetype/gensection/parse.go),
106+
whose `MarkerPair.StartLine`/`EndLine` give the exact
107+
start- and end-marker line for every directive (not
108+
just the include/catalog *body* ranges that
109+
`lint.File.GeneratedRanges` records for diagnostic
110+
suppression). Only lines the engine recognizes as real
111+
markers are removed, so inner same-type markers that
112+
the engine treats as literal content survive. Add a
113+
test.
114+
3. **Whitespace normalization.** Collapse the blank
115+
lines left by removed markers so output is stable and
116+
passes `mdsmith check`. Test idempotence: export of
117+
export equals export.
118+
4. **Staleness check and modes.** Add a checker that
119+
compares each directive's on-disk body to what the
120+
engine would generate, reusing the `mdsmith fix`
121+
directive engine. `Mode` is `Check` (default), `Fix`,
122+
or `NoCheck`. In `Check`, each stale body appends one
123+
`lint.Diagnostic` (naming the directive, positioned at
124+
its start marker) and `Export` returns `nil` bytes.
125+
`Fix` regenerates stale bodies in memory before
126+
stripping. `NoCheck` skips the comparison. The CLI
127+
maps `--fix`/`--no-check` to the mode and rejects the
128+
combination. Unit-test all three modes on a stale
129+
fixture.
130+
5. **`export` subcommand.** Register `export` in
131+
[main.go](../cmd/mdsmith/main.go); `mdsmith export
132+
<file>` writes to stdout, `-o/--output <path>` writes
133+
a file, `--fix` and `--no-check` select the staleness
134+
mode (rejecting the combination). Never mutate the
135+
source. Reuse the config and file-load helpers that
136+
back `fix` in [main.go](../cmd/mdsmith/main.go). Exit
137+
non-zero with a clear message on parse errors and on a
138+
stale body in the default mode.
139+
6. **Fixtures and integration test.** Add `testdata`
140+
inputs covering include, catalog, toc, and build
141+
directives with golden directive-free outputs. Add a
142+
stale-body fixture: assert default mode exits non-zero
143+
with no output, `--fix` produces the fresh golden, and
144+
`--no-check` exports the stale bytes as-is. Assert
145+
idempotence and that fresh output passes `mdsmith
146+
check`.
147+
7. **Docs.** Add `docs/reference/cli/export.md` (covering
148+
the default check, `--fix`, and `--no-check`) and link
149+
it from the CLI reference catalog. Run `mdsmith fix`
150+
so catalogs and PLAN.md regenerate.
151+
152+
## Acceptance Criteria
153+
154+
- [ ] `mdsmith export <file>` removes every line the
155+
engine recognizes as a real directive start/end
156+
marker, keeps generated bodies, and inlines
157+
`<?include?>` content. Marker-like text treated as
158+
literal content is left in place.
159+
- [ ] The source file is never modified in any mode.
160+
- [ ] Default mode: a stale directive body makes
161+
`export` exit non-zero with a diagnostic naming the
162+
directive and writes no output.
163+
- [ ] `--fix` regenerates stale bodies in memory before
164+
stripping; `--no-check` exports on-disk bytes as-is;
165+
passing both is a usage error.
166+
- [ ] Nested same-type literal-content markers are
167+
preserved.
168+
- [ ] Output is idempotent and (when fresh) passes
169+
`mdsmith check`.
170+
- [ ] `-o <path>` writes to a file; stdout is the
171+
default.
172+
- [ ] A parse error or missing file exits non-zero with
173+
a clear message.
174+
- [ ] All tests pass: `go test ./...`
175+
- [ ] `go tool golangci-lint run` reports no issues
176+
- [ ] `mdsmith check .` passes
177+
178+
## Decisions
179+
180+
- **Keep generated bodies.** Markers are stripped but
181+
TOC, catalog, and included content stay as plain
182+
Markdown; includes are inlined for a portable copy.
183+
- **New `export` subcommand.** Not a fourth `extract`
184+
format and not a `fix` flag; a dedicated command keeps
185+
the source-to-source transform separate from schema
186+
extraction.
187+
- **Check by default, never auto-fix.** A stale body
188+
fails the export rather than being silently
189+
regenerated, so the output faithfully reflects the
190+
file. `--fix` opts into regeneration; `--no-check`
191+
opts out of the check.
192+
- **Front matter retained.** It is not a directive;
193+
stripping it is out of scope.
194+
- **Single file first.** Directory or glob batch export
195+
is a possible follow-up, not in this plan.

0 commit comments

Comments
 (0)