Skip to content

Commit df4fcfa

Browse files
jedudenclaude
andauthored
Plan 185: Expose extended-syntax parsers and the flavor model in pkg/markdown (#409)
* Start plan 185: Expose extended-syntax parsers and the flavor model in pkg/markdown * Plan 185: expose extended-syntax parsers and flavor model in pkg/markdown Promotes every custom goldmark parser and the per-flavor Feature support model into a new public pkg/markdown/flavor sub-package. The MDS034 rule is reduced to a thin adapter; internal/schema's hand-rolled goldmark config and the rule's own dual parser both fold into the single goldmark.New call site in pkg/markdown/flavor/parser.go. - pkg/markdown/flavor: Flavor and Feature types, the support table, Detect(doc *markdown.Document, accept func(Feature) bool) []Finding, Finding/HeadingIDExtra shapes, four NewParser*/NewPooledParser* constructors, and small rewriter helpers (FindHeadingID, IsGitHubAlert, LineCol). - pkg/markdown/flavor/ext: the five custom extensions (Superscript, Subscript, MathBlock, MathInline, Abbreviation) and their AST node kinds, moved wholesale from internal/rules/markdownflavor/ext. - internal/convention: Flavor is now a type alias for pkg/markdown/flavor.Flavor; constants and ParseFlavor are re-exported so internal/config compiles unchanged. - internal/rules/markdownflavor: Check builds a *markdown.Document from *lint.File, calls flavor.Detect, and maps findings to diagnostics. Fix retains its byte-range edit pipeline but uses flavor.NewPooledParser instead of a private singleton. - internal/schema/validate_content.go: replaces the local goldmark.New(extension.Table) with flavor.NewPooledParserWith. - internal/integration/rules_test.go: drops the goldmark-frontmatter test helper in favour of lint.StripFrontMatter + yamlutil.UnmarshalSafe so no goldmark.New remains under internal/. - Contract test pins the public flavor API shape. - Docs (markdown-library.md, architecture/index.md, cross-system.md, go.md) updated to drop the "CommonMark only" wording and document the new sub-package. Verifies all acceptance criteria: pkg/markdown imports no internal/, no goldmark.New under internal/ or cmd/, no custom AST node types or parsers outside pkg/markdown, and MDS034 / schema diagnostics remain byte-identical (existing tests pass unchanged). https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Fix broken links to moved ext package and lift FindHeadingID coverage The pinned-version and source mdsmith-check jobs both failed: three plan/185 references and one architecture-audit reference still pointed at the now-empty internal/rules/markdownflavor/ext path. Locally the references resolved because git mv left an empty directory behind; CI's fresh checkout sees them as dead links. - plan/185_public-markdown-flavor-library.md: retarget every internal/rules/markdownflavor/ext link at pkg/markdown/flavor/ext (the new home). - docs/development/architecture-audit.md: rewrite the "markdownflavor/ext sub-package" finding as resolved by plan/185, drop the broken link, and trim to stay under the 300-line cap. - pkg/markdown/flavor/detect.go: drop the defensive nil-AST guards I added in detectBareURLs / detectGitHubAlerts; flavor.Detect already short-circuits on doc == nil and the original *lint.File variant carried no such check. - pkg/markdown/flavor/detect_edge_test.go: exercise both branches of the public FindHeadingID wrapper so codecov/patch sees full coverage on the new helper. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Fix duplicate link-ref transformer in newParserInternal Copilot review caught a real bug: goldmark.New() calls DefaultParser() which already installs DefaultParagraphTransformers(), so the previous goldmark.WithParserOptions(parser.WithParagraphTransformers(defaults...)) call appended a second link-reference transformer on top. The reset closure only touched the appended instance; the one inside the default parser kept pinning the last parsed document's bytes. Build the parser explicitly with parser.NewParser (one set of block, inline, and paragraph parsers including the lrp captured for reset) and install it via goldmark.WithParser, then let goldmark.New's extension Extend hooks register the additional block / inline parsers they need. After this change there is exactly one link-ref transformer in the resulting parser, and the closure returned to NewPooledParser callers resets that instance. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Pool the dual parser in flavor.Detect; inline onlyAccept Self-review found two issues from the plan-185 changes: 1. Detect built a fresh goldmark parser per call via NewPooledParser, running the full Extend hook chain on every Check. The previous singleton in internal/rules/markdownflavor avoided this; the move to a stateless public Detect regressed it. Add a sync.Pool inside the flavor package that hands each Detect goroutine its own parser-with-reset pair, mirroring internal/schema's contentParserPool. The pool resets the link-reference transformer before Put so idle slots do not pin document bytes. 2. fix.go used a one-line onlyAccept helper that was only ever called from one site. Inline the closure literal at the call site and drop the helper. A new BenchmarkDetectReusesPool exercises the dual-parser code path repeatedly; coverage in pkg/markdown/flavor stays at 100%. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Expose NearestBlockAncestor, add byte-identical pin test, fill pyramid Addresses the two remaining self-review concerns and aligns the package's tests with the test-pyramid rule that every production function ships its dedicated unit test by name. Concern #2 — drop the nearestBlockAncestor duplicate: - Expose NearestBlockAncestor from pkg/markdown/flavor so external rewriters (and the rule adapter) share the helper instead of duplicating it. Replace the rule's private copy in fix.go with flavor.NearestBlockAncestor. - Add it to the contract test, the markdown-library stable surface list, and a dedicated TestNearestBlockAncestorPublic test. Concern #3 — byte-identical pin test: - pkg/markdown/flavor/detect_pin_test.go adds a corpus-driven table test (pinCorpus) that maps each input to the exact Finding stream (feature + 1-based line + 1-based column, in document order). Plan 185 acceptance criterion "Table tests pin this" is now an explicit gate; any subtle reorder, drop, or shift in MDS034 diagnostics will break the test with a side-by-side diff. Test-pyramid alignment: - TestNearestBlockAncestor (subtests for the skip-non-block and orphan branches) plus TestNearestBlockAncestorPublic for the exported wrapper. - TestIsGitHubAlertPublic exercises both branches of IsGitHubAlert (alert blockquote / heading-first-child). - TestLineColPublic pins the documented 1-based semantics of the exported LineCol wrapper. - TestDualFindings covers the dualFindings helper I extracted from Detect in the previous commit, asserting both the keep-filter and the still-emits-other-features path. Coverage in pkg/markdown/flavor stays at 100%; mdsmith check and golangci-lint are clean. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Address recall-mode review findings (14 of 15) Acts on every concrete finding from the local max-effort review pass except #5 (linkRefResetter interface duplication), which I'm keeping local: exposing it from pkg/markdown would surface a goldmark fork detail on the public surface for less duplication value than it costs. Bugs - fix.go (#1): MDS034 Fix was calling flavor.NewPooledParser() per invocation — same regression I fixed in flavor.Detect last commit, mirrored at the parallel Fix call site. Move the pool to package level in flavor and expose a callback API, flavor.WithSharedParser, used by both Detect's dualFindings and the rule's fixByteRangeFeatures. - rule.go fixGitHubAlerts (#7): the type assertion bq.FirstChild().(*ast.Paragraph) plus lines.At(0) was relying on a cross-package contract with flavor.IsGitHubAlert. Re-check the shape locally so a future relax of IsGitHubAlert cannot turn the walk into a panic. - fix.go taskCheckBoxEdits (#9): nil-check the flavor.NearestBlockAncestor return and the block's Lines() before calling At(0). - detect.go (#11, #15): IsGitHubAlert nil-guards bq and the paragraph's Lines.Len(); findHeadingID nil-guards h. Both are public-API entry points now. - rule.go ApplySettings (#12): iterate settings keys in sorted order so the error for multiple unknown settings is deterministic across Go map randomisation. Simplifications - detect.go (#2): drop the taskCheckBoxFinding specialisation; the TaskCheckBox case in builtinFindingFor calls inlineExtFinding directly. - detect.go (#3): promote isGitHubAlert / lineCol / nearestBlockAncestor to the exported names. The previous private + one-line public-wrapper pair was duplication; the lowercase versions are now gone. - detect.go dualFindings (#4): return nil rather than an always-allocated empty slice on no findings (CLAUDE.md allocation-budget rule). - parser.go (#6): drop NewParser and NewParserWith — they were one-line wrappers around the pooled forms. The public surface is now NewPooledParser, NewPooledParserWith, and WithSharedParser. - contract_test.go (#13): move signature pins to package-scope `var _ = ...` declarations so the staticcheck "could omit type" rule does not fight the explicit-type contract. Reuse - pkg/markdown.Edit + Splice (#8): added an optional `Repl []byte` field to Edit; Splice now supports replacement in addition to deletion. The rule's bespoke `edit` struct and `applyEdits` are gone; fix.go composes a []markdown.Edit and feeds it through markdown.Splice. Adjacent-edits-with-Repl behaviour is now pinned in pkg/markdown's TestSplice. - lint.UnmarshalFrontMatter (#14): extracted the StripFrontMatter → trim `---\n` delimiters → yamlutil.UnmarshalSafe pipeline into one helper in internal/lint/frontmatter.go. internal/integration's rules_test.go switched to it, dropping its open-coded copy and the goldmark-frontmatter import. Test pyramid - Added unit tests for IsGitHubAlert's nil/empty-Lines branches, FindHeadingID's nil-heading branch, and TestWithSharedParser for the new pool callback. Coverage in pkg/markdown/flavor is 100%. - TestApplyEditsHandlesAdjacentEdits moved into pkg/markdown's TestSplice as a sub-test covering the new Repl behaviour. All tests pass, golangci-lint clean, mdsmith check clean. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Refresh bench comment after pool move The pool moved from inside Detect to package level in commit 08446eb (WithSharedParser). Update the bench's docstring to point at the new home so the next reader does not look in the wrong file. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Address second-round recall review (F1, F3, F4, F5, F6) Second focused review of the previous fix-up commit surfaced seven findings. Fixing five; F2 (atomic ApplySettings) is pre-existing and masked by clone-before-apply at every call site, F7 (recover boundary in WithSharedParser) is hypothetical with no current trigger. F1 + F4: lint.UnmarshalFrontMatter conflated "no front matter" with "front matter that decoded into a zero struct". A misspelled key (dropping a letter from "diagnostics", a schema-mismatched field) or an empty `---\n---\n` block left fm.Settings == nil && fm.Diagnostics == nil, so the integration fixture loader silently accepted malformed bad fixtures. - UnmarshalFrontMatter now returns (body, hadFrontMatter, err). Callers that want to enforce schema check hadFrontMatter rather than inspect v's zero state. - integration/rules_test.go's parseFixtureFrontMatter uses the new bool. A bad fixture with malformed FM now fails loudly with the correct "missing front matter" message. - Unit tests in lint/frontmatter_test.go pin all four cases (valid block, no block, empty fences, unrecognised keys, decode error). F3: markdown.Splice's docstring promised "ascending and non-overlapping" but the implementation enforced neither — a violating edit list crashed inside body[prev:e.Start] with an opaque "slice bounds out of range" panic. Added an entry-point precondition check that panics with a descriptive message naming the offending edit's index, Start, and End. Three new sub-tests in TestSpliceInvariantViolation pin the message text so any future change to the check surfaces here. F5: taskCheckBoxEdits's comment claimed "a malformed AST cannot panic the fix". Technically true after the nil/empty-Lines guards landed in commit 08446eb, but the guards do NOT close the silent-corruption case where NearestBlockAncestor skips an empty-Lines TextBlock and returns the enclosing ListItem — start+3 then deletes the bullet instead of the checkbox. Documented the goldmark TextBlock invariant the fix relies on and the failure mode when a hand-built AST violates it. F6: fixGitHubAlerts's local (Paragraph, non-empty Lines) re-check was dead code today and would silently skip a fix if flavor.IsGitHubAlert ever drifted — the worst failure mode for a fix path (diagnostic flagged but fix did nothing). Removed the local check; the comment documents why trusting IsGitHubAlert is the right call and which test pins the contract. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Close the F5 silent-corruption path in taskCheckBoxEdits The previous round's nil/empty-Lines guards did NOT prevent the silent-corruption case the F5 finding pointed at: NearestBlockAncestor SKIPS ancestors with empty Lines() and keeps walking up, so a TaskCheckBox under a Paragraph-with-empty-Lines under a ListItem-with-populated-Lines yields block=ListItem and start = bullet-position, not '['-position. The guards both pass; start+3 deletes three bytes from the wrong block. Add an explicit `f.Source[start] != '['` check that declines the edit when the byte at Lines.At(0).Start is not the bracket the task-list parser's invariant promises, plus a bounds check on start+3 against len(f.Source) for the truly-short-source case. Three new red/green tests pin the guard: - non-bracket start (paragraph with arbitrary Lines) - nil block ancestor (orphan TaskCheckBox) - bracket runs past EOF (2-byte source) https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Address round-3 review: Splice negative-Start, IsGitHubAlert contract pin Recall agent surfaced three findings after commit 17cca70; one was the F5 silent-corruption path already closed in commit 131a9f1. The remaining two land here. #2: markdown.Splice's precondition check fired for negative-Start edits via the generic "overlaps previous edit ending at 0" panic. A producer that subtracts past 0 and emits Start=-1 sent the debugger chasing a non-existent previous edit. Add a dedicated `Start < 0` guard that names the actual fault, plus a pin in TestSpliceInvariantViolation. #3: fixGitHubAlerts removed the local (Paragraph, non-empty Lines) re-check and trusts flavor.IsGitHubAlert's contract — but no behavior test pinned that contract. The four existing fix tests exercise inputs where IsGitHubAlert returns true via the real parser; they would not catch a future relaxation of IsGitHubAlert that returns true for a non-Paragraph first child. Add TestIsGitHubAlertContractPostcondition: walk a corpus of alert / non-alert / degenerate blockquotes, and on every IsGitHubAlert==true case assert FirstChild is *ast.Paragraph with non-empty Lines. Coverage in pkg/markdown and pkg/markdown/flavor stays at 100%; mdsmith check and golangci-lint clean. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti * Mark plan/185 done Every task and acceptance criterion is checked off and the implementation has passed three rounds of recall review with all follow-up fixes landed. Flip the status from 🔳 to ✅ and refresh the PLAN.md catalog. https://claude.ai/code/session_0144ZKUS2Zrg7xBft54qyoti --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent db3a705 commit df4fcfa

46 files changed

Lines changed: 1884 additions & 773 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

PLAN.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ footer: |
111111
| 182 || sonnet | [Code block convention rules](plan/182_code-block-conventions.md) |
112112
| 183 || sonnet | [Skip DedupeDiagnostics via an audited rule.RepoScoped marker](plan/183_dedupe-diagnostics-repo-scoped-skip.md) |
113113
| 184 || opus | [Automate the cross-tool benchmark on merge to main and publish numbers to the assets branch](plan/184_release-benchmark-automation.md) |
114-
| 185 | 🔲 | | [Expose extended-syntax parsers and the flavor model in pkg/markdown](plan/185_public-markdown-flavor-library.md) |
114+
| 185 | | | [Expose extended-syntax parsers and the flavor model in pkg/markdown](plan/185_public-markdown-flavor-library.md) |
115115
| 186 || | [Centralize UTF-16 column helpers in internal/mdtext](plan/186_arch-fix-utf16-centralize.md) |
116116
| 187 || opus | [Neutral-corpus engine lever — shared AST walk and Punkt cost](plan/187_neutral-corpus-engine-lever.md) |
117117
| 188 | 🔲 | opus | [Regex-over-source rules — inventory and AST-resident replacements](plan/188_regex-vs-ast-inventory.md) |

docs/development/architecture-audit.md

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -244,15 +244,14 @@ graduate.
244244

245245
Sub-package of a rule.
246246

247-
The
248-
[`markdownflavor/ext` package](../../internal/rules/markdownflavor/ext/)
249-
is used only within the parent rule
247+
The `markdownflavor/ext` package is
248+
used only within the parent rule
250249
(`fix.go`, `parser.go`, `detect.go`).
251-
This is fine as an internal split.
252-
Worth a one-sentence package comment
253-
explaining why it is separate so
254-
future readers do not read it as a
255-
separate rule package.
250+
Fine as an internal split, but worth
251+
a package comment explaining why it is
252+
separate. Resolved by
253+
[plan/185](../../plan/185_public-markdown-flavor-library.md):
254+
moved to `pkg/markdown/flavor/ext`.
256255

257256
## Audit 2026-05-17 (range: 7464d273..b5a6d72)
258257

docs/development/architecture/cross-system.md

Lines changed: 26 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -29,22 +29,23 @@ the contract. The surface's own spec
2929
steps) lives where the table's "Spec
3030
doc" column says.
3131

32-
| Boundary | Owner in repo | Spec doc | Consumers |
33-
| ------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------- |
34-
| LSP wire protocol | `internal/lsp` | [CLI reference: `lsp`](../../reference/cli/lsp.md) | VS Code extension, other editors |
35-
| CLI flags + exit codes | `cmd/mdsmith` | [CLI reference](../../reference/cli.md) | shell scripts, CI, git hooks |
36-
| `.mdsmith.yml` schema | `internal/config` | [Conventions](../../reference/conventions.md) | every project using mdsmith |
37-
| `.mdsmith/kinds/` directory | `internal/config` | [Kind files](../../reference/kind-files.md) | every project using mdsmith |
38-
| Generated section markers | `internal/archetype/gensection` | [Generated sections](../../background/concepts/generated-section.md) | every project's Markdown files |
39-
| Claude plugin manifest (published) | `editors/claude-code/.claude-plugin/plugin.json` | [Install: Claude plugin](../../guides/install.md) | end users via Claude Code marketplace |
40-
| Claude plugin manifest (contributors) | `editors/claude-code-dev/.claude-plugin/plugin.json` | [editors/claude-code-dev/README.md](../../../editors/claude-code-dev/README.md) | mdsmith contributors |
41-
| Claude marketplace listing | `.claude-plugin/marketplace.json` | [Install: Claude plugin](../../guides/install.md) | Claude Code marketplace |
42-
| Claude skill definitions | `.claude/skills/*/SKILL.md` | [proto](../../../.claude/skills/proto.md) | Claude Code sessions |
43-
| npm package shim | `npm/mdsmith/` | [Install: npm](../../guides/install.md) | Node users |
44-
| PyPI wheel shim | `python/` | [Install: PyPI](../../guides/install.md) | Python users |
45-
| asdf / mise plugin | external repos | [Install: asdf / mise](../../guides/install.md) | language-tool users |
46-
| VS Code `contributes` | `editors/vscode/package.json` | [VS Code integration](../../guides/editors/vscode.md) | the extension host |
47-
| Public Markdown library | `pkg/markdown` | [Public Markdown Library](../markdown-library.md) | `internal/lint`, `internal/release`, external Go importers |
32+
| Boundary | Owner in repo | Spec doc | Consumers |
33+
| ------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
34+
| LSP wire protocol | `internal/lsp` | [CLI reference: `lsp`](../../reference/cli/lsp.md) | VS Code extension, other editors |
35+
| CLI flags + exit codes | `cmd/mdsmith` | [CLI reference](../../reference/cli.md) | shell scripts, CI, git hooks |
36+
| `.mdsmith.yml` schema | `internal/config` | [Conventions](../../reference/conventions.md) | every project using mdsmith |
37+
| `.mdsmith/kinds/` directory | `internal/config` | [Kind files](../../reference/kind-files.md) | every project using mdsmith |
38+
| Generated section markers | `internal/archetype/gensection` | [Generated sections](../../background/concepts/generated-section.md) | every project's Markdown files |
39+
| Claude plugin manifest (published) | `editors/claude-code/.claude-plugin/plugin.json` | [Install: Claude plugin](../../guides/install.md) | end users via Claude Code marketplace |
40+
| Claude plugin manifest (contributors) | `editors/claude-code-dev/.claude-plugin/plugin.json` | [editors/claude-code-dev/README.md](../../../editors/claude-code-dev/README.md) | mdsmith contributors |
41+
| Claude marketplace listing | `.claude-plugin/marketplace.json` | [Install: Claude plugin](../../guides/install.md) | Claude Code marketplace |
42+
| Claude skill definitions | `.claude/skills/*/SKILL.md` | [proto](../../../.claude/skills/proto.md) | Claude Code sessions |
43+
| npm package shim | `npm/mdsmith/` | [Install: npm](../../guides/install.md) | Node users |
44+
| PyPI wheel shim | `python/` | [Install: PyPI](../../guides/install.md) | Python users |
45+
| asdf / mise plugin | external repos | [Install: asdf / mise](../../guides/install.md) | language-tool users |
46+
| VS Code `contributes` | `editors/vscode/package.json` | [VS Code integration](../../guides/editors/vscode.md) | the extension host |
47+
| Public Markdown library | `pkg/markdown` | [Public Markdown Library](../markdown-library.md) | `internal/lint`, `internal/release`, external Go importers |
48+
| Public flavor library | `pkg/markdown/flavor` | [Public Markdown Library](../markdown-library.md) | `internal/rules/markdownflavor`, `internal/schema`, external Go importers |
4849

4950
Treat each surface as a public API.
5051
mdsmith is at major 0 today, so strict
@@ -166,7 +167,15 @@ doc says how the rule applies to it.
166167
shape `Parse` returns is a break for
167168
this surface. `Splice` output is
168169
byte-exact and pinned by the sync-docs
169-
golden corpus. Full policy in
170+
golden corpus. The
171+
**`pkg/markdown/flavor` sub-package**
172+
follows the same policy and ships the
173+
extended-syntax parsers (tables,
174+
strikethrough, task lists, footnotes,
175+
definition lists, heading IDs, plus
176+
five custom extensions) and the
177+
Flavor / Feature support model.
178+
Full policy in
170179
[Public Markdown Library](../markdown-library.md).
171180

172181
## Common violations to flag

docs/development/architecture/go.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,9 @@ question. The current production set:
5252
- `internal/lsp` — speak the Language
5353
Server Protocol; consumes the engine.
5454
- `pkg/markdown` — the one goldmark
55-
parser config and the byte-exact
56-
producer. Public; see
57-
[Public Markdown Library](../markdown-library.md).
55+
parse/produce surface (CommonMark+PI);
56+
`pkg/markdown/flavor` adds extensions.
57+
Public; see [Public Markdown Library](../markdown-library.md).
5858
- `internal/mdtext` — walk an
5959
already-parsed AST (slugging, TOC,
6060
plain-text). `pkg/markdown` produces

docs/development/architecture/index.md

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -97,13 +97,35 @@ cmd/mdsmith internal/lsp
9797
surface; public,
9898
imports no
9999
internal/ package)
100+
└─> pkg/markdown/flavor
101+
(extended-syntax
102+
parsers, the
103+
Flavor / Feature
104+
support model,
105+
and Detect;
106+
public,
107+
imports no
108+
internal/ package)
109+
└─> pkg/markdown/flavor/ext
110+
(the five
111+
custom
112+
extensions)
100113
```
101114

102-
`pkg/markdown` sits at the bottom: the
115+
`pkg/markdown` sits at the bottom. It
116+
owns the canonical CommonMark + PI
103117
parse path. `internal/lint` and
104-
`internal/release` depend on it; it
105-
depends on nothing in the tree. It is
106-
also a public cross-system surface — see
118+
`internal/release` depend on it. It
119+
depends on nothing in the tree.
120+
121+
The `flavor` sub-package adds the
122+
extended-syntax parsers. It owns every
123+
custom goldmark parser in the tree.
124+
The MDS034 rule and the schema engine
125+
both build on it.
126+
127+
Both packages are public surfaces.
128+
For details see
107129
[cross-system contracts](cross-system.md)
108130
and [Public Markdown Library](../markdown-library.md).
109131

docs/development/markdown-library.md

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -75,10 +75,17 @@ that matter:
7575
extension. The closing `---` is matched only
7676
at the start of a line. So a `---` row inside
7777
a YAML block scalar does not end it early.
78-
- **CommonMark only.** The canonical parser
79-
enables no GFM extensions. Tables,
80-
strikethrough, and the rest are composed
81-
separately by the MDS034 flavor detector.
78+
- **CommonMark by default, extensions in
79+
`pkg/markdown/flavor`.** The canonical parser
80+
at `markdown.NewParser` enables no GFM
81+
extensions. The `pkg/markdown/flavor`
82+
sub-package adds GFM tables, strikethrough,
83+
task lists, footnotes, definition lists, the
84+
heading-ID attribute parser, and five
85+
custom extensions (superscript, subscript,
86+
math block, math inline, abbreviations). Use
87+
`flavor.NewParser` (or `flavor.NewParserWith`
88+
for a subset) to opt in.
8289
- **Byte-stability contract.** `Splice` output
8390
is pinned byte-for-byte (see the policy
8491
below). goldmark gives no such cross-version
@@ -213,12 +220,28 @@ the changelog.
213220
The stable surface:
214221

215222
- `Parse` and `Document` (its fields).
216-
- `ParseContext`, `NewParser`.
223+
- `ParseContext`, `NewParser`, `NewPooledParser`.
217224
- `StripFrontMatter`, `CountLines`.
218225
- `Splice` and `Edit` (its fields).
219226
- `ProcessingInstruction` (its exported fields
220227
and methods), `KindProcessingInstruction`,
221228
`NewPIBlockParser`, `PIBlockParserPrioritized`.
229+
- Sub-package `pkg/markdown/flavor`: the
230+
`Flavor` type and constants, the `Feature`
231+
type and constants, `AllFeatures`, `Supports`,
232+
`ParseFlavor`, the `Finding` and
233+
`HeadingIDExtra` shapes, `Detect`, the
234+
`NewPooledParser` / `NewPooledParserWith`
235+
constructors, the `WithSharedParser` callback
236+
for borrowing from the package-shared pool,
237+
and the small rewriter helpers
238+
(`FindHeadingID`, `IsGitHubAlert`, `LineCol`,
239+
`NearestBlockAncestor`).
240+
- Sub-package `pkg/markdown/flavor/ext`: the
241+
five custom extension Extender singletons
242+
(`Superscript`, `Subscript`, `MathBlock`,
243+
`MathInline`, `Abbreviation`) and their node
244+
kinds.
222245

223246
Policy:
224247

internal/convention/flavor.go

Lines changed: 31 additions & 91 deletions
Original file line numberDiff line numberDiff line change
@@ -1,101 +1,41 @@
1-
// Package convention owns the convention and flavor data shapes
2-
// independent of any rule. A convention pairs a Markdown flavor with
3-
// a table of rule presets; the config loader consults this package
4-
// at load time so a top-level `convention:` selection becomes a base
5-
// layer beneath the user's own rule config. Rule packages (notably
1+
// Package convention owns the convention data shape (a Markdown
2+
// flavor paired with a table of rule presets) independent of any
3+
// rule. A convention pairs a Markdown flavor with a table of rule
4+
// presets; the config loader consults this package at load time so a
5+
// top-level `convention:` selection becomes a base layer beneath the
6+
// user's own rule config. Rule packages (notably
67
// internal/rules/markdownflavor) consume these data shapes — they do
78
// not own them — which keeps internal/config from importing a rule.
9+
//
10+
// The Flavor identity itself lives in pkg/markdown/flavor; this
11+
// package re-exports it via type and constant aliases so existing
12+
// callers under internal/ keep importing convention.Flavor.
813
package convention
914

10-
// Flavor identifies a target Markdown flavor.
11-
type Flavor int
15+
import "github.com/jeduden/mdsmith/pkg/markdown/flavor"
1216

13-
// Flavor constants. The zero value is intentionally invalid so that
14-
// unparsed settings are caught.
17+
// Flavor is an alias for pkg/markdown/flavor.Flavor; convention.Flavor
18+
// and flavor.Flavor name the same underlying type.
19+
type Flavor = flavor.Flavor
20+
21+
// Flavor constants are re-exported from pkg/markdown/flavor so that
22+
// existing callers (internal/config, the markdown-flavor rule, the
23+
// convention table below) keep working with the convention.Flavor*
24+
// names. Add a new flavor by extending the canonical list in
25+
// pkg/markdown/flavor and adding one alias here.
1526
const (
16-
flavorInvalid Flavor = iota
17-
FlavorCommonMark
18-
FlavorGFM
19-
FlavorGoldmark
20-
// FlavorAny accepts every tracked feature. Useful when the
21-
// document is destined for an unknown or permissive renderer and
22-
// the user wants to disable flavor reporting without disabling
23-
// the rule.
24-
FlavorAny
25-
// FlavorPandoc is Pandoc's default markdown dialect. Accepts
26-
// GFM's four features plus footnotes, definition lists, heading
27-
// IDs, superscript, subscript, math block, and inline math;
28-
// rejects abbreviations (a non-default Pandoc extension).
29-
FlavorPandoc
30-
// FlavorPHPExtra is PHP Markdown Extra. Accepts tables,
31-
// footnotes, definition lists, heading IDs, and abbreviations;
32-
// rejects GFM's task lists, strikethrough, bare-URL autolinks,
33-
// and every math / sub/superscript feature.
34-
FlavorPHPExtra
35-
// FlavorMultiMarkdown extends PHP Markdown Extra with math
36-
// block and inline math. Like PHP Extra, rejects GFM task lists,
37-
// strikethrough, bare-URL autolinks, and sub/superscript.
38-
FlavorMultiMarkdown
39-
// FlavorMyST is the MyST flavor used by the Sphinx documentation
40-
// toolchain. Accepts tables, strikethrough, footnotes,
41-
// definition lists, heading IDs, math block, and inline math;
42-
// rejects GFM task lists, bare-URL autolinks, sub/superscript,
43-
// and abbreviations.
44-
FlavorMyST
27+
FlavorCommonMark = flavor.FlavorCommonMark
28+
FlavorGFM = flavor.FlavorGFM
29+
FlavorGoldmark = flavor.FlavorGoldmark
30+
FlavorAny = flavor.FlavorAny
31+
FlavorPandoc = flavor.FlavorPandoc
32+
FlavorPHPExtra = flavor.FlavorPHPExtra
33+
FlavorMultiMarkdown = flavor.FlavorMultiMarkdown
34+
FlavorMyST = flavor.FlavorMyST
4535
)
4636

47-
// IsValid reports whether f names a recognised flavor. The zero
48-
// value (reserved for "unparsed/unset") and any out-of-range integer
49-
// cast to Flavor both return false. Implemented in terms of String
50-
// so the two stay in lock-step: every recognised flavor has a name,
51-
// and adding a new constant only requires updating the switch in
52-
// String.
53-
func (f Flavor) IsValid() bool { return f.String() != "" }
54-
55-
// String returns the canonical lowercase name of the flavor.
56-
func (f Flavor) String() string {
57-
switch f {
58-
case FlavorCommonMark:
59-
return "commonmark"
60-
case FlavorGFM:
61-
return "gfm"
62-
case FlavorGoldmark:
63-
return "goldmark"
64-
case FlavorAny:
65-
return "any"
66-
case FlavorPandoc:
67-
return "pandoc"
68-
case FlavorPHPExtra:
69-
return "phpextra"
70-
case FlavorMultiMarkdown:
71-
return "multimarkdown"
72-
case FlavorMyST:
73-
return "myst"
74-
}
75-
return ""
76-
}
77-
78-
// ParseFlavor converts a config string into a Flavor. The match is
79-
// case-sensitive to reject typos like "GFM" that would otherwise
80-
// silently validate against the wrong flavor.
37+
// ParseFlavor delegates to pkg/markdown/flavor.ParseFlavor so callers
38+
// that already import internal/convention do not need a second import.
8139
func ParseFlavor(s string) (Flavor, bool) {
82-
switch s {
83-
case "commonmark":
84-
return FlavorCommonMark, true
85-
case "gfm":
86-
return FlavorGFM, true
87-
case "goldmark":
88-
return FlavorGoldmark, true
89-
case "any":
90-
return FlavorAny, true
91-
case "pandoc":
92-
return FlavorPandoc, true
93-
case "phpextra":
94-
return FlavorPHPExtra, true
95-
case "multimarkdown":
96-
return FlavorMultiMarkdown, true
97-
case "myst":
98-
return FlavorMyST, true
99-
}
100-
return 0, false
40+
return flavor.ParseFlavor(s)
10141
}

0 commit comments

Comments
 (0)