Commit 79b8ad8
Add comprehensive mdsmith vs mdbase research documentation (#240)
* docs: add mdbase comparison to markdown-linters background doc
mdbase (mdbase.dev) is a spec for typed, queryable Markdown
collections. It overlaps with mdsmith's file-kinds and
front-matter schema (MDS020) but targets the data layer rather
than linting. Document the overlap, the differences (rename
refactor, SQLite cache, Obsidian Bases query syntax), and the
fact that the two tools coexist on disk but cannot share schemas.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): deep-dive comparison of mdsmith and mdbase
Replace the embedded mdbase section in markdown-linters.md with
a brief pointer, and add a four-document deep-dive at
docs/research/mdbase-vs-mdsmith/.
- README.md: TL;DR, when-to-use matrix, status snapshot,
the same-files-different-layer model, navigation
- features.md: 22 sections across distribution, configuration,
type/kind systems, file matching, field types, validation,
front matter, queries, links, rename refactoring, CRUD,
generated content, prose rules, conventions, cache, CLI,
LSP, output, conformance, security, determinism
- workflows.md: bootstrap, authoring, daily editor session,
schema evolution, file rename, query, CI, pre-commit,
Obsidian-vault, agent workflows, contributor onboarding
- interop.md: coexistence on disk, dual-schema problem,
four combination strategies (mdbase-as-truth,
mdsmith-as-truth, both-by-hand, generate-bridge),
recommended folder layout, sample CI, sketch of a
schema bridge, when to skip one tool
Sources cited from the mdbase 0.2.1 spec sections 0-15 and
appendices A-D plus the mdsmith codebase as of this branch.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): expand LSP comparison and add concrete example
Address review on PR #240:
- Expand features.md §17 (LSP) to cover both shipped mdsmith
LSP (diagnostics + code actions) and planned plans 122
(hover) and 131 (PR #238: documentSymbol, definition,
implementation, references, workspace/symbol, call
hierarchy). Add a "LSP for AI agents" subsection
contrasting the typed-vault view (mdbase-lsp) with the
document-graph view (mdsmith plan 131).
- Update workflows.md §3 (Daily editor session) with a
three-column "today vs planned vs mdbase-lsp" table.
- Update workflows.md §10 (LLM/agent workflows) with an
"Agent navigates over LSP" subsection that maps the
nine LSP methods Claude Code's LSP tool exposes onto
the planned mdsmith surface.
- Add a concrete worked example to the mdbase entry in
markdown-linters.md showing the same .md file and what
each tool does with the bytes (the shared front-matter
layer vs the split body / refactor surfaces).
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): correct kinds vs concepts; address Copilot feedback
- Replace "CUE schema files" framing with the actual mechanism:
mdsmith schemas are Markdown `proto.md` files (CUE in front
matter, heading template in body) referenced from kinds via
`required-structure.schema:`. internal/concepts/ is internal
implementation architecture documentation, not user-facing.
- Update §5 (field type system) and §3 (kind definitions) in
features.md to reflect the proto.md format.
- Update the dual-schema table, Strategy D bridge sketch, and
recommended folder layout in interop.md to point at proto.md
instead of fictional `internal/concepts/*.cue`.
- Fix glued bold lead "**Observation.**mdbase" → add the
missing space.
- Correct the workflows.md mental-model table: file creation
is humans or `mdbase create`, not "mdsmith + mdbase create".
- Reference plan 122 by file path and plan 131 explicitly via
PR #238 with a note that the plan file is on that branch
and not yet on main.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): add use-cases, learn-from-mdbase; address Copilot
New docs in docs/research/mdbase-vs-mdsmith/:
- use-cases.md — seven concrete worked scenarios
(open-source repo, Obsidian vault, RFC tracker,
task tracker, agent-maintained runbooks, knowledge
graph, mixed wiki+plan tracker) showing where each
tool fits and the three signals that predict the
fit (prose load, typed-record load, rename load)
- learn-from-mdbase.md — systematic enumeration of
every mdbase capability mdsmith doesn't have.
Triaged into in-scope / out-of-scope / in-flight,
with twelve mini-plans (S-1..S-6, M-1..M-3, C-4,
C-6, L-1, L-3, L-4, Q-1..Q-4, Q-7) covering goal,
sketch, surface, effort, depends-on, open
questions. Includes an extended P-1 (SQLite cache)
section that walks staleness checks, what queries
run on the index, and six alternative
implementations ranked by complexity, with a
recommendation that mdsmith land an in-memory link
graph in the LSP server before considering any
persistent cache.
README.md updated to point at both new docs.
Address Copilot review:
- Bump Go version reference 1.24 → 1.25 to match
current go.mod
- Replace branch-name source pin with a date-based
pointer + git-history reference
- Fix workflows.md onboarding install command:
recommend `go install ./cmd/mdsmith` from a clone
or `@vX.Y.Z` for a release pin
- Update interop.md CI sample to match Go 1.25 and
pin mdsmith to a version rather than `@latest`
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): fix orphan + list marker in use-cases.md
The blank line between "mdbase rename" and "+ mdbase watch"
made the + render as a list item. Replace with prose:
"mdbase rename plus mdbase watch round out the workflow."
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): drop scope assertions; add aggregation-use-cases doc
Reframe learn-from-mdbase.md to stop pretending mdsmith has a
closed feature set:
- Drop the "in-scope / out-of-scope" triage. Replace with
per-gap **Trigger** field naming the concrete condition
under which shipping the gap is worth the cost (real user
demand, profiling result, dependency from another feature,
coherence pull from a subsystem).
- Capability tables now carry Effort + Trigger / status,
not Status + Priority.
- Promote previously-out-of-scope items (S-5, C-1..C-3, C-5,
L-2, L-5, P-1, P-2, V-1, X-1, Q-5, Q-6) to mini-plans-lite
with the same Goal/Trigger/Sketch/Effort/Depends-on shape.
- Soften the SQLite section: remove "mdsmith does not plan
to ship those" assertion. Keep the option ranking, add three
alternative directions (stateless-fast, content-addressed,
tiered daemon), and frame the choice as measurable per
trigger rather than predetermined.
- Reframe closing observations as descriptive, not prescriptive.
New doc docs/research/mdbase-vs-mdsmith/aggregation-use-cases.md
addresses the "more use cases for aggregations + fzf/ripgrep
exploration" ask:
- 5 aggregation workload shapes (count, sum/avg, top-N, join,
time bucket) with cost drivers
- 6 worked use cases (sprint dashboard, reviewer load report,
knowledge-graph backlinks, time-bucketed velocity, cross-type
join, real-time editor decoration) — each with the query,
cost driver, mdbase approach, mdsmith stateless approach,
and verdict
- Cross-product table identifying the three signals that
predict whether an index pays: interactive latency,
repeated queries, multi-hop joins
- Deep dive on the fzf / ripgrep stateless-fast model:
what mdsmith could ripgrep-class (FM filter, body FTS,
light aggregations) and what it can't (backlinks at scale,
multi-hop joins, editor decoration)
- Pragmatic path: stateless by default, in-memory in the LSP,
persistent only when the workload proves it. Plus open
questions a real benchmark plan would need to answer.
Audit features.md for scope-asserting "out of scope" cells in
the LSP table — reframe completion / rename rows as "not yet
planned" / "candidate (L-3)".
README updated with pointers to both new perspectives.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): split conflated hover row in workflows.md table
Copilot caught that the "Type-name hover" row in the §3
editor-session table conflated two different hover capabilities.
Split into:
- "Rule / directive hover" — mdsmith planned via plan 122,
n/a for mdbase-lsp (no rule/directive concepts).
- "Type / kind hover" — mdsmith no, mdbase-lsp yes.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): incorporate cost-of-cache analysis; folded summaries
Two changes:
1. Substantive: aggregation-use-cases.md gets a new section
"What an index actually costs" addressing three confounders
to the simple "cold-start dominates" trigger:
- Sync-check overhead. A persistent cache must validate
against the filesystem before any query. mtime sweeps
run ~10us per stat (~500ms at 50k files); content-hash
fallback adds ~30us per file. Validation alone can
approach the cost of a fresh parse.
- OS file cache. Linux page cache, macOS UBC, Windows
system cache all keep recent files in RAM. mdsmith's
"stateless re-read" is in practice a re-read from RAM
for any corpus that fits in memory and any session
that runs more than once. The implication: an
application cache competes against work the OS
already does; it pays only on parsing cost, not IO.
- Background indexing with priority queries. Pattern
from IDEs (IntelliJ, TS server): start the index in
the background, queries that need not-yet-indexed
data jump the queue, the index iteratively completes.
Works only for long-lived processes (LSP, watch
daemon) but covers most apparent-latency wins
without persistent state.
Adds a benchmark plan sketch with five configurations
(stateless cold/warm, in-memory lazy, in-memory with
priority, persistent mtime/hash sync) on synthetic
1k/10k/50k corpora, plus a refined three-filter
trigger for P-1: OS-cache, sync-check, in-memory-with-
priority must all be insufficient before reaching for
a persistent on-disk store.
Updates the P-1 trigger in learn-from-mdbase.md to
point at this analysis.
2. Stylistic: convert all seven research docs' front-matter
summary fields to YAML folded block scalars (summary: >-)
to match the convention used elsewhere in the repo
(e.g. docs/guides/file-kinds.md). Copilot was right
about the convention even though MDS001 doesn't fire.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): mdbase index validation cost; query layer arch / security
Address four follow-up questions about the mdbase architecture
that affect both the cost analysis and the security picture.
1. Validation cost applies to mdbase too. Updated the
"Sync-check overhead" subsection: a one-shot
`mdbase query` invocation without watch mode running
has the same validation cost as a hypothetical
mdsmith-with-cache run. The SQLite cache wins
decisively only in two regimes: long-lived sessions
with watch mode (where validation amortizes), or
repeated queries within one process. The dominant
winning case is always-on watch-mode deployment,
which fits a smaller share of real workflows than
the spec-first framing implies.
2. Body queries beyond links. Per spec §10, mdbase can
match against `file.body` as a string (substring,
regex). It cannot do structural body queries
(heading count, first-paragraph patterns, code-block
language tags) because the body is a blob, not an
AST. mdsmith's lint engine has full AST access
internally but does not expose it to query;
surfacing it would be a Q-3 follow-on.
3. mdbase queries are NOT SQL. The user-facing
language is the Bases-compatible expression DSL
from spec §11. Implementations may compile to SQL
internally for SQLite-backed storage, but the
compilation is implementation-private; users have
no SQL drop-through. Storage is abstracted across
impls (SQLite, in-memory, Bolt-style KV).
4. Security posture. The spec mandates path sandboxing
(§8) and bounds traversal depth and expression
nesting (§11), but is silent on YAML attacks,
query injection during DSL→SQL compilation, cache
integrity, and untrusted type definitions. mdsmith's
10-finding adversarial review and YAML
anchor/alias rejection are not mandated by the spec;
each mdbase impl decides. Document this with a
side-by-side table.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): drop split framing; explicit operational modes
Two related corrections to the framing throughout the doc set:
1. Drop the "owns / split / overlap" framing.
Every gap and every workflow is a candidate for either
tool; the underlying substrate (FM, body, link graph)
supports all use cases. What differs is which surfaces
each tool currently exposes, and at what cost.
- README TL;DR rewritten as "Two tools, one substrate,
different surfaces today" — no more "complementary,
not competitive" or "owns prose / owns data" claims.
- "Same problem, different layer" diagram retitled to
"Same substrate, different surfaces" and redrawn
to show shared substrate at top, current surfaces
below, and a "candidate evolutions either way" row
explicitly named.
- "When to use which" table gains a "Candidate for the
other side" column referencing mini-plan IDs from
learn-from-mdbase.md, so the framing is "best fit
today, with a known evolutionary path" rather than
"use X for this".
- Drop status: 🔳 from all 7 research docs; existing
research notes in this repo (conciseness, corpus)
don't carry one and the field implied a completion
gate that doesn't apply to research.
2. Make operational-mode assumptions explicit in
aggregation-use-cases.md.
Previous numbers compared mdbase warm-cache against
mdsmith cold-stateless — implicitly granting mdbase a
running server. New "Operational mode matters more than
tool choice" section names two modes (cold-start /
long-lived-session) and notes that comparing fairly
means evaluating both tools under the same mode.
Each of A-1..A-6 now starts with a Mode assumption
and (for long-lived cases) explicit pre-conditions:
- A-1 sprint dashboard: cold-start one-shot CI;
mdbase ≈ 600ms cold (cache rebuild), mdsmith ≈ 600ms
stateless. Roughly equivalent.
- A-2 reviewer load: cold-start weekly cron; both
~300-400ms; mdbase wins only when CI preserves cache.
- A-3 backlinks panel: long-lived editor with three
pre-conditions (cold session, first query, repeat).
mdbase watch + mdsmith planned in-memory both serve
sub-ms after cold build. mdbase wins on cross-session
warmth.
- A-4 velocity: cold-start quarterly; both ~50ms; the
small corpus makes cache validation overhead and
fresh parse converge.
- A-5 cross-type join: either mode; cold-vs-cold
comparable (500-800ms); long-lived comparable.
- A-6 editor decoration: long-lived only with four
pre-conditions; stateless not applicable for either
tool. mdbase wins on cross-session warmth.
Updated cross-product table titled "When operational
mode + persistence pays" with explicit Mode column;
pattern observation distinguishes cold-shot vs
long-lived use cases. Cross-session warmth is named
as the actual SQLite payoff.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): tighten Strategy A; clarify ignore vs overrides; soften 'typed'
Three real Copilot catches:
1. interop.md Strategy A previously said "skip MDS020
entirely" with `required-structure: false`. That
disables both FM validation AND heading-template
enforcement. Split into A.1 (full disable) and A.2
(keep MDS020 with a permissive FM schema, so mdsmith
still owns body structure while mdbase owns FM).
2. interop.md folder-layout example previously used
`ignore: [_types/**]`, which skips ALL mdsmith rules
including prose / line-length / table-format. Show
both options: full ignore (intentional end-to-end
mdbase ownership) vs. overrides with required-structure
off (lint type files for prose / whitespace, just skip
schema conflicts).
3. markdown-linters.md mdbase pointer previously claimed
"both tools treat status, priority, due as typed
data". mdsmith treats FM as typed only when a CUE
schema is wired up via MDS020; without one it's just
YAML values. Rephrased as "structured fields" and
added the conditional.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): cache stores body? + correct A.2; fix orphan +
Address two Copilot catches plus a follow-up question about
whether mdbase stores body content in the SQLite cache.
1. Body in SQLite. Spec §13 says the cache tracks an FTS
index and link graph; it does not mandate storing the
full body verbatim. An SQLite-backed impl can use
content-in-FTS (body inline in FTS5 virtual table) or
external-content FTS (index only, body on disk). Both
are conformant. Crucially, A-3 backlinks and A-6
decoration do not need body content in the cache at
all — the link graph and FM are derived structures
sufficient for those workloads. Only Q-3 full-text
search needs body access (or an FTS index over it).
Building the cache reads every body once regardless
of storage mode; only steady-state size and re-read
patterns differ. Added a new subsection
"Does the cache store the full body?" to
aggregation-use-cases.md.
2. Broken A.2 example in interop.md. Copilot pointed
out that mdsmith schemas treat `[string]` as a
literal field-name key, not CUE's open-field syntax.
The "permissive FM schema" example as written would
require a literal field named `[string]` in the
document FM, not match-any. Reworked A.2 into three
honest partial routes (empty FM block; mirror mdbase
types in CUE; or fall back to A.1) with caveats on
each. Note that a clean "validate body, skip FM"
mdsmith config option would close the gap and is a
candidate for a future plan.
3. Orphan `+` list marker in use-cases.md U-7
("via a `plan` kind\n\n+ proto.md schema"). Same
bug pattern as the earlier use-cases.md fix:
collapsed into prose ("via a `plan` kind plus a
`proto.md` schema").
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): cover both mdbase impls; fix layer-order; YAML split
Two substantive additions, three Copilot fixes.
Substantive:
1. Add the actual implementation analysis in
aggregation-use-cases.md, covering both reference
impls. mdbase has TWO impls — TS (`mdbase`) used by
mdbase-cli, and Rust (`mdbase-rs`) used by mdbase-lsp
— and they make materially different choices:
- TS uses sql.js (WebAssembly), in-memory + flush-on-
close, populates only the `files` table; auxiliary
tables (field_values, links, tags) are scaffolded
but unused; backlinks queries rebuild a link index
in memory per query.
- Rust uses native rusqlite with bundled SQLite,
incremental disk writes, populates files + links +
file_types + unique_values; backlinks served via
B-tree probe on links(target_path); also caches
`effective_json` (FM with defaults applied) so
reads skip recomputation.
Both store body inline; neither uses FTS5; staleness
is mtime-based with no content-hash fallback in
either. Discusses what this means for each of the
A-3 backlinks / A-6 decoration / Q-3 body-FTS use
cases — the spec-permissive "indexed edge lookup,
sub-millisecond" claim fits the Rust impl but the
TS impl rebuilds per query.
Adds a side-by-side table of the two impls' storage
choices and a closing note on what mdsmith's
hypothetical cache would inherit from each shape.
Copilot fixes:
2. features.md config-layer diagram had `kinds` after
`overrides`; the actual code in
internal/config/merge.go applies kinds before
overrides (overrides win). Swapped and added a
citation comment.
3. interop.md folder-layout YAML had two `ignore:` keys
in one fenced block (invalid if copy-pasted whole).
Split into two separate code blocks, one per option.
4. workflows.md "spec doc on github" → "spec doc on
GitHub".
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): drop fictional <?required-structure?> directive
mdsmith does not have a `<?required-structure?>` PI. Heading-
template enforcement is the MDS020 rule (`required-structure`)
applied to a kind whose `required-structure.schema:` points at
a `proto.md`. The schema body carries the heading skeleton;
the rule checks real files against it. The schema-only PI is
`<?require?>` (filename and other constraints inside a
schema), not `<?required-structure?>`.
Two fixes:
- features.md directives table: removed the fictional row.
Reduced count from "five active directives" to four
(catalog, include, toc, build). Added a paragraph
pointing at MDS020 + proto.md schemas + the `<?require?>`
PI as the actual mechanism.
- workflows.md authoring section: reworded "no template
mechanism beyond `<?required-structure?>`" to point at
MDS020 schema files instead.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): add concrete query examples to features.md §8
The query language section described complex queries
abstractly (date arithmetic, cross-file traversal,
aggregation) but didn't show what they actually look like.
Added eight worked examples (Q-A through Q-H) that
exercise the same task corpus through both syntaxes:
- Q-A simple equality filter
- Q-B compound boolean
- Q-C date range — "due in next 7 days"
- Q-D list contains
- Q-E cross-file traversal via asFile()
- Q-F body content match
- Q-G sort and limit
- Q-H aggregation (group + count)
For each, the mdbase form sits next to the equivalent
mdsmith form (CUE struct literal where it works,
shell pipeline where it doesn't, or "not expressible
in query today, see L-5/Q-X" with a pointer to the
candidate plan in learn-from-mdbase.md).
The closing reading shows where the language differences
actually bite: mdsmith's query covers the common filter
case ergonomically and composes with shell tools for the
rest; mdbase covers more inside the DSL at the cost of a
richer language to learn.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): add query-and-types deep dive; drop pipe-to-X
New doc docs/research/mdbase-vs-mdsmith/query-and-types.md
covers:
- What each tool's type system actually types. Direct
answer to "is mdbase's type system limited to front
matter?": almost — with two named exceptions
(path_pattern filename templates, and the Rust impl's
unique_values cross-file table). Beyond those, every
typed assertion in mdbase reduces to a property of FM
or a derivation of it. mdsmith goes one layer further
by typing the body's heading structure too (schema
body carries the heading template; MDS020 enforces).
Neither tool types body content as a structured
value.
- A 12-row "what gets typed" matrix covering FM types,
field constraints, filename patterns, computed fields,
cross-field constraints, body structure, body content,
and cross-file value coherence.
- Side-by-side type-definition examples for the same
task type in both syntaxes.
- A "what each can express" matrix on twelve constraint
categories (regex, range, enum, optional, default,
generated, unique, cross-field, computed, filename,
body template, schema composition).
- When types fire (write-time vs lint-time), with
implications for agent loops.
- The query languages compared as languages (lexical,
semantic, composition, error handling, type-awareness),
not just feature lists.
- Migration table between Bases queries and CUE queries.
- Three capabilities missing in both tools (structural
body typing, schema-aware query validation,
cross-implementation schema portability).
Plus reframing in features.md §8 worked-query examples:
The Q-F (body content), Q-G (sort/limit), and Q-H
(aggregation) examples previously showed mdsmith
"workarounds" via shell pipelines to ripgrep, jq,
sort/head. That undercuts mdsmith's single-binary
identity and suggests external dependencies. Replaced
with prose noting that the right answer for mdsmith
is a native implementation of each capability (the
Q-1, Q-2, Q-3, Q-5 plans in learn-from-mdbase.md),
not a shell pipeline. Same fix in query-and-types.md
migration table.
mdsmith ships standalone; the candidate plans
describe native implementations. Closing this doc
section with that principle stated explicitly.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): fix CUE struct syntax in query-and-types
Copilot caught a real syntax error: I wrote
`mdsmith query 'status: "open" & priority: int & >=3'`
in two places, but mdsmith wraps the argument in
`{...}` per internal/query/query.go. Inside a CUE
struct literal, `&` is not a field separator;
fields are separated by commas (or newlines). The
written form would parse as `status` having value
`"open" & priority: int & >=3`, which is invalid.
Both occurrences fixed to use the comma form:
`status: "open", priority: int & >=3`. The
prose explanation already used the correct form
in the surrounding text.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): full-enforcement matrix; correct CUE syntax claims
Two corrections:
1. Broaden the "what gets typed" analysis to cover ALL
mdsmith mechanisms, not just MDS020. The first version
underrated mdsmith by treating only schema-driven
constraints; in reality mdsmith's broader rule set
(MDS001..MDS054) enforces many structural document
properties: heading rules, paragraph structure, tables,
code blocks, link integrity, file/section caps, prose
readability, etc. Replaced the single matrix with a
set of category-grouped tables (FM/type-assignment,
filename/headings, sections/prose, tables/code/lists,
links, style/formatting/generated) showing three
columns per row: mdbase type, mdsmith MDS020 schema,
mdsmith broader rule set. Closing observation: mdsmith
is broad-and-shallow (54 rules across many properties),
mdbase is narrow-and-deep (one type system, deep on FM
constraints + cross-file uniqueness + link graph).
2. Fix three Copilot-caught factual errors about CUE in
mdsmith:
- mdsmith query does not accept CUE imports. The CLI
argument is wrapped in {…} per internal/query/query.go
and compiled as an expression body; `import "..."`
and `package` declarations are not parsed at this
surface. Removed the "composable via CUE imports"
bullet and the "CUE imports" entry in the
side-by-side language table; replaced with "shell-
level reuse only".
- mdsmith schemas don't compose via CUE imports either.
Schema FM is parsed as YAML and converted into a
closed CUE struct (deriveFrontMatterCUE); per-field
CUE expressions are encoded as YAML scalar values.
Removed the bogus `import "github.com/jeduden/mdsmith/concepts"`
example. Replaced with the actual mechanism: schema
body include via `<?include?>` plus per-field CUE
expressions inline. S-3 in learn-from-mdbase.md is
where CUE-imports-in-schemas would be a candidate.
- features.md Q-D used `tags: ["urgent", ...]` to mean
"list contains urgent". CUE list pattern semantics
make that "starts with urgent". Fixed to use
`tags: [..., "urgent", ...]` (leading + trailing
ellipsis) for true contains, with prose explaining
the difference.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(background): correct MDS028 attribution in mdbase example table
MDS028 is the token-budget rule, not a conciseness or
readability rule. The row "Body conciseness, readability"
that cited (MDS023, MDS024, MDS028) conflated readability
with token budget. Renamed the row to "Body readability,
structure, token budget" and labelled each rule by what
it actually does:
- MDS023 ARI (paragraph readability)
- MDS024 sentences (paragraph structure)
- MDS028 token budget
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): cleaner editorial — drop tautology, fix lean phrasing
Three small Copilot fixes:
- workflows.md: drop redundant "mdsmith side:" /
"mdbase side:" labels after the bold section
leads. Replaced with content-bearing intros that
describe what's in the following code block
(e.g. "Init writes a default config; the first
lint runs straight away.").
- use-cases.md: standardize on "(lean)" everywhere.
The U-3 verdict said "mdsmith leans" while the
index used "mdsmith (lean)"; aligned to "(lean)"
for index/verdict consistency.
- README.md: collapse the redundant lead-in
"Both projects in detail share these
properties. Together, these properties hold for
both:" to a single plain "Shared by both projects:"
line before the bullet list.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): FM↔body sync mechanisms; widen schema framing
Address two related asks: schema analysis must check ALL
mdsmith rules (and mdbase mechanisms), and FM is structurally
disjoint from the body so what does each tool offer to keep
them in sync?
Front matter and body parse to different trees; nothing in
standard Markdown forces them to co-vary. A file can have
`title: Migration Plan` in FM and `# Outline` in the body and
still be syntactically valid. The honest comparison needs to
name what each tool ships to enforce coherence.
New section "Front matter and body — keeping them in sync"
in query-and-types.md, with five mdsmith mechanisms and one
mdbase mechanism walked side by side:
mdsmith:
- MDS020 heading template with FM placeholders
(`# {title}` → body H1 must equal FM title)
- MDS019 catalog directive (FM fields → body table cells,
drift-detected on lint)
- MDS021 include directive (FM fields → body include
variables)
- MDS039 build directive (FM-derived params)
- MDS038 toc directive (body headings → body TOC, not
strictly FM↔body but body-internal sync)
mdbase:
- `path_pattern` (FM fields → filename) and that's it.
Body is treated as opaque prose; no body templates,
no body-from-FM generation, no FM↔body validation.
Spec doesn't define mechanisms for this.
Plus a side-by-side table covering seven sync directions
(FM→filename, FM→body H1, FM→body table cells, FM→body
include vars, FM→body build artifact, body→body TOC,
FM→query-only check). mdsmith covers all directions
within the body; mdbase covers only the filename
direction.
Added a corresponding row to the front-matter section
of the enforcement matrix: "FM ↔ body sync (e.g. title
↔ H1)" — mdbase: no; mdsmith MDS020: yes (placeholders);
mdsmith broader rules: yes (catalog/include/build
directives drift-detect).
The closing observation: an mdbase team using FM as the
typed source of truth has no built-in way to keep body
in sync (they need conventions or external tools); an
mdsmith team gets it from the directive surface.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): fix S-2 import path inconsistency and count mismatch
Two real Copilot catches in S-2 (named field-type taxonomy):
1. The sketch said the library lives at
`internal/cue/types.cue` but the schema example imports
`github.com/jeduden/mdsmith/types`. Go's `internal/`
paths are not importable as public modules, so the two
are inconsistent. Reworded to describe the source as
living in a publicly importable module path with the
source-tree location as a sketch (e.g. `cue/types/types.cue`
or via an mdsmith CUE overlay), and flagged that the
exact module path / overlay shape is open for the real
plan to settle.
2. The open-question note said "Start with the seven mdbase
covers" but listed nine items. mdbase has 9 scalar field
types in spec §7 (string, int, number, bool, date,
datetime, time, enum, link), so the count was wrong —
updated to "nine".
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): add S-7 rich body schema candidate
Add a new S-7 mini-plan to learn-from-mdbase.md: rich
body schema for structural document types (presentation
decks, runbooks, training material). The user surfaced
this as a "needs work" sketch; this commit treats it as
a real candidate plan with goal / trigger / sketch /
surface / effort / depends / open questions.
The illustrative schema covers five capabilities current
MDS020 does not address:
- Hierarchical body structure (nested sections with
required/optional/repeatable; child patterns like
`### Slide {n}` with sequential or non-sequential
numbering)
- Per-field content rules (word/char counts, forbidden
starts/contains, required patterns, scoped skip rules
like "cover slide is exempt from action-title length cap")
- Cross-reference validation ("Slide 5" must resolve to
an actual heading; skip-rules for version-history
blockquotes)
- Acronym tracking with known-safe allowlist
- Index generation as a side-output of the parse pass
This is the largest schema-side candidate (effort L) and
the most ambitious. It would close the structural-body-
typing gap noted in query-and-types.md "What's missing
in both" — a gap mdbase cannot close without going
outside its data-layer scope.
Updates:
- learn-from-mdbase.md: new S-7 mini-plan after S-6
- learn-from-mdbase.md: schema-language capability table
gains an S-7 row
- query-and-types.md: "What's missing in both" §1
(structural body typing) now cross-links to S-7
Open questions called out explicitly: schema language
choice (YAML / CUE / new DSL); composition with MDS020
(extension or sister rule); index output location/
format; cross-reference scope and skip-rules; relation
to mdbase types (orthogonal but a project running both
wants one definition); performance at large file size.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
* docs(research): rework S-7 example to be less verbatim
The original S-7 sketch was a near-direct copy of a
specific presentation-deck schema (Action Title,
On-slide content, Speaker notes, TRANSITION marker,
leave-behind PDFs, etc.). It read more as the source
proposal than as a generic illustration.
Reworked to use a runbook example instead. Runbooks
are common across teams, generic enough to not feel
domain-specific, and exercise every capability the
deck schema demonstrated:
- Hierarchical: ## Diagnosis → ### Step {n} → fields
- Required/optional/aliases at section level
- Per-field word counts, forbidden starts/contains
- Required patterns ("see Step N") with skip_indices
for the last-step exemption
- Cross-reference validation with skip_lines_matching
for blockquotes
- Acronym tracking with infrastructure-term allowlist
- Index generation as JSON side-output
The schema is also explicitly labelled "illustrative —
not a final syntax" so readers don't read it as a
finished design. Trigger reworded to focus on the
signal — the constraint vocabulary repeating across
two-or-more projects — rather than enumerating
specific document types.
The five capabilities, surface, effort, depends-on,
and open questions stay the same. Only the example
and surrounding prose are reworked for less verbatim
feel.
https://claude.ai/code/session_01MaUmRwLMk5UDoxe5twpqAV
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 219766f commit 79b8ad8
9 files changed
Lines changed: 7079 additions & 0 deletions
File tree
- docs
- background
- research/mdbase-vs-mdsmith
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
169 | 169 | | |
170 | 170 | | |
171 | 171 | | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
172 | 232 | | |
173 | 233 | | |
174 | 234 | | |
| |||
739 | 799 | | |
740 | 800 | | |
741 | 801 | | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
742 | 805 | | |
743 | 806 | | |
744 | 807 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
0 commit comments