Skip to content

Commit 23be2a0

Browse files
committed
schema: flag in-stream extras + clarify proto.md migration state
Three review findings: 1. validate.go::handleNonMatch — when a doc heading doesn't match the current scope, `findOutOfOrderIdx` only looks at unclaimed later scopes. A schema [A, B] with doc [B, B, A] used to silently consume the second B (B already claimed out-of-order; A's matchScope walked past it). Consult `claimedScopeMatches` after the out-of-order check fails so the duplicate surfaces as "exceeds allowed occurrences". Test: TestPlan156_FlagsExtrasInIterationStream. 2. docs/reference/section-schema.md — the frontmatter summary and the proto.md section both implied `fmvar(...)` was a current proto.md feature. MDS020's file-schema check is still on the legacy parser, so `{field}` matches a wildcard run rather than resolving the document's frontmatter. Add a "Migration status" callout and reword the summary to scope the matcher promise to inline schemas. 3. docs/guides/schemas.md — the same caveat in the user- facing guide. Replace the "tokens desugar to the same matcher shape" wording with an explicit note that proto.md `{field}` is still a wildcard placeholder until the follow-up cutover lands. https://claude.ai/code/session_012GGH62fZUzLuzP8T4ocGkJ
1 parent 22df596 commit 23be2a0

7 files changed

Lines changed: 82 additions & 14 deletions

File tree

.github/copilot-instructions.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ row: "- [{summary}](../{filename})"
7878
- [Built-in Markdown conventions, the rule presets each one applies, and how user config layers on top via deep-merge.](../docs/reference/conventions.md)
7979
- [Glob pattern syntax across mdsmith config, directives, and CLI argument expansion, with the supported exclusion semantics for each surface.](../docs/reference/globs.md)
8080
- [Named field-type shortcuts for inline schema frontmatter values — the registered names, the canonical CUE each one resolves to, and example usage.](../docs/reference/schema-types.md)
81-
- [Section-schema reference: the entry-shape vocabulary used in inline `kinds.<name>.schema:` blocks and `proto.md` files. Covers the `heading:` discriminator, the `regex:` matcher (a Go RE2 body with `\#(digits)` and `\#(fmvar(...))` helpers), the `repeat: {min, max}` cardinality field, and the matching algorithm.](../docs/reference/section-schema.md)
81+
- [Section-schema reference for inline `kinds.<name>.schema:` blocks. Covers the `heading:` discriminator, the `regex:` matcher (a Go RE2 body with `\#(digits)` and `\#(fmvar(...))` helpers), the `repeat: {min, max}` cardinality field, and the matching algorithm. `proto.md` files are parsed into the same shape by the schema package, but MDS020's file-schema check still uses its legacy parser; see the proto.md section below for what is and is not migrated.](../docs/reference/section-schema.md)
8282
<?/catalog?>
8383

8484
### Development Workflow

AGENTS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ row: "- [{summary}]({filename})"
8484
- [Built-in Markdown conventions, the rule presets each one applies, and how user config layers on top via deep-merge.](docs/reference/conventions.md)
8585
- [Glob pattern syntax across mdsmith config, directives, and CLI argument expansion, with the supported exclusion semantics for each surface.](docs/reference/globs.md)
8686
- [Named field-type shortcuts for inline schema frontmatter values — the registered names, the canonical CUE each one resolves to, and example usage.](docs/reference/schema-types.md)
87-
- [Section-schema reference: the entry-shape vocabulary used in inline `kinds.<name>.schema:` blocks and `proto.md` files. Covers the `heading:` discriminator, the `regex:` matcher (a Go RE2 body with `\#(digits)` and `\#(fmvar(...))` helpers), the `repeat: {min, max}` cardinality field, and the matching algorithm.](docs/reference/section-schema.md)
87+
- [Section-schema reference for inline `kinds.<name>.schema:` blocks. Covers the `heading:` discriminator, the `regex:` matcher (a Go RE2 body with `\#(digits)` and `\#(fmvar(...))` helpers), the `repeat: {min, max}` cardinality field, and the matching algorithm. `proto.md` files are parsed into the same shape by the schema package, but MDS020's file-schema check still uses its legacy parser; see the proto.md section below for what is and is not migrated.](docs/reference/section-schema.md)
8888
<?/catalog?>
8989

9090
## Development Workflow

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ row: "- [{summary}]({filename})"
7070
- [Built-in Markdown conventions, the rule presets each one applies, and how user config layers on top via deep-merge.](docs/reference/conventions.md)
7171
- [Glob pattern syntax across mdsmith config, directives, and CLI argument expansion, with the supported exclusion semantics for each surface.](docs/reference/globs.md)
7272
- [Named field-type shortcuts for inline schema frontmatter values — the registered names, the canonical CUE each one resolves to, and example usage.](docs/reference/schema-types.md)
73-
- [Section-schema reference: the entry-shape vocabulary used in inline `kinds.<name>.schema:` blocks and `proto.md` files. Covers the `heading:` discriminator, the `regex:` matcher (a Go RE2 body with `\#(digits)` and `\#(fmvar(...))` helpers), the `repeat: {min, max}` cardinality field, and the matching algorithm.](docs/reference/section-schema.md)
73+
- [Section-schema reference for inline `kinds.<name>.schema:` blocks. Covers the `heading:` discriminator, the `regex:` matcher (a Go RE2 body with `\#(digits)` and `\#(fmvar(...))` helpers), the `repeat: {min, max}` cardinality field, and the matching algorithm. `proto.md` files are parsed into the same shape by the schema package, but MDS020's file-schema check still uses its legacy parser; see the proto.md section below for what is and is not migrated.](docs/reference/section-schema.md)
7474
<?/catalog?>
7575

7676
## Development Workflow

docs/guides/schemas.md

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -401,11 +401,20 @@ filename: "MDS*-*.md"
401401
The `# ?` (or `# {field}: {field}` form) acts as the
402402
title placeholder. `## ...` rows mark wildcard slots.
403403
Front-matter keys map directly to CUE expressions.
404-
`<?require?>` declares the filename pattern. The
405-
heading-row tokens desugar to the same matcher
406-
shape the inline form uses; see the
404+
`<?require?>` declares the filename pattern.
405+
406+
MDS020's file-schema check still routes through
407+
its legacy parser today: `{field}` in a proto.md
408+
heading row matches a non-empty run rather than
409+
resolving the document's frontmatter value via
410+
`fmvar(...)`. The schema package parses proto.md
411+
into the new Matcher shape (used by tests), and a
412+
follow-up plan will wire MDS020 through that
413+
parser. Until then, treat `{field}` in proto.md
414+
heading rows as a wildcard placeholder, not as a
415+
frontmatter substitution. The
407416
[section-schema reference](../reference/section-schema.md#protomd-file-syntax)
408-
for the row-by-row mapping.
417+
records the eventual mapping.
409418

410419
## Choosing a source
411420

docs/reference/section-schema.md

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,16 @@
11
---
22
summary: >-
3-
Section-schema reference: the entry-shape
4-
vocabulary used in inline `kinds.<name>.schema:`
5-
blocks and `proto.md` files. Covers the
3+
Section-schema reference for inline
4+
`kinds.<name>.schema:` blocks. Covers the
65
`heading:` discriminator, the `regex:` matcher
76
(a Go RE2 body with `\#(digits)` and
87
`\#(fmvar(...))` helpers), the
98
`repeat: {min, max}` cardinality field, and the
10-
matching algorithm.
9+
matching algorithm. `proto.md` files are
10+
parsed into the same shape by the schema
11+
package, but MDS020's file-schema check still
12+
uses its legacy parser; see the proto.md
13+
section below for what is and is not migrated.
1114
---
1215
# Section schema
1316

@@ -252,13 +255,24 @@ schema:
252255

253256
## `proto.md` file syntax
254257

258+
> **Migration status.** MDS020's file-schema
259+
> check still uses the legacy `parseSchema`
260+
> pipeline today: `## ?` and `## ...` already
261+
> behave as wildcards, and `{field}` in a
262+
> heading row matches a non-empty run rather
263+
> than resolving the frontmatter value via
264+
> `fmvar(...)`. The mapping below is what the
265+
> schema package parses. Tests exercise it so a
266+
> follow-up cutover plan can wire MDS020
267+
> through without docs churn. Until then,
268+
> proto.md authors should treat `{field}` as a
269+
> wildcard, not a frontmatter substitution.
270+
255271
Proto.md files use a literal-template surface
256272
distinct from the inline `regex:` form. Heading
257273
rows in the body act as the schema's
258274
`sections:` list. `{n}` and `{field}` survive
259-
here as template placeholders; they desugar to
260-
the same matcher the inline form produces with
261-
`digits` and `fmvar`.
275+
here as template placeholders.
262276

263277
| Row syntax | Equivalent inline entry |
264278
|-------------------|------------------------------------------------|

internal/schema/plan156_acceptance_test.go

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -459,6 +459,35 @@ func TestPlan156_OutOfOrderSequentialDiagFires(t *testing.T) {
459459
"the unenforced-sequential diagnostic")
460460
}
461461

462+
// TestPlan156_FlagsExtrasInIterationStream regresses a Copilot
463+
// finding: when a doc has [B, B, A] against schema [A, B], the
464+
// second B used to be silently consumed by handleNonMatch
465+
// (findOutOfOrderIdx skips claimed scopes, open schema → no
466+
// diag). The handler now consults claimedScopeMatches and
467+
// emits "exceeds allowed occurrences" for the duplicate.
468+
func TestPlan156_FlagsExtrasInIterationStream(t *testing.T) {
469+
raw := map[string]any{
470+
"sections": []any{
471+
map[string]any{"heading": "A"},
472+
map[string]any{"heading": "B"},
473+
},
474+
}
475+
sch, err := ParseInline(raw, "kind x")
476+
require.NoError(t, err)
477+
doc := newDocFile(t, "doc.md",
478+
"# T\n\n## B\n\nx\n\n## B\n\ny\n\n## A\n\nz\n")
479+
diags := Validate(doc, sch, nil, false, makeDiagForTest)
480+
var exceeded bool
481+
for _, d := range diags {
482+
if strings.Contains(d.Message, "exceeds scope") {
483+
exceeded = true
484+
}
485+
}
486+
assert.True(t, exceeded,
487+
"the second B must be flagged as exceeding scope B's "+
488+
"allowed occurrences, not silently consumed")
489+
}
490+
462491
// TestPlan156_UnboundedRepeatDoesNotFlagAsExceeded regresses a
463492
// Copilot finding: claimedScopeMatches used to fire for any
464493
// claimed scope, including unbounded matchers (`max == 0`). A

internal/schema/validate.go

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -552,6 +552,22 @@ func (s *matchRun) handleNonMatch(docHeads []DocHeading, docIdx int) (bool, int,
552552
s.diags = append(s.diags, ooDiags...)
553553
return false, newIdx, false
554554
}
555+
// If dh matches a scope already claimed (in-order or
556+
// out-of-order), it's an extra occurrence — surface it as
557+
// "exceeds allowed occurrences" so the user sees which scope
558+
// was over-filled. Without this, a sequence like [A, B] with
559+
// doc [B, B, A] silently consumes the second B because
560+
// findOutOfOrderIdx ignores claimed scopes.
561+
if idx := claimedScopeMatches(s.scopes, dh, s.claimed, s.docFM); idx >= 0 {
562+
sc := s.scopes[idx]
563+
s.diags = append(s.diags, s.mkDiag(s.f.Path, dh.Line,
564+
fmt.Sprintf(
565+
"section %q exceeds scope %q's allowed occurrences "+
566+
"(out-of-order recovery only counts one match)",
567+
formatHeading(dh.Level, dh.Text),
568+
formatHeading(s.expectedLevel, displayHeading(sc)))))
569+
return false, docIdx + 1, false
570+
}
555571
if !s.allowExtra && s.closed {
556572
sc := s.scopes[s.idx]
557573
s.diags = append(s.diags, s.mkDiag(s.f.Path, dh.Line,

0 commit comments

Comments
 (0)