Skip to content

Commit 38692c5

Browse files
baonguyenNavabaonguyenNava
committed
Add initial documents from spike
1 parent dd3b7ab commit 38692c5

175 files changed

Lines changed: 14451 additions & 1 deletion

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude-plugin/marketplace.json‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"name": "strata-documentation-engine",
3+
"owner": {
4+
"name": "NavaPBC",
5+
"url": "https://github.com/navapbc"
6+
},
7+
"metadata": {
8+
"description": "Local marketplace for the Strata Documentation Engine plugin."
9+
},
10+
"plugins": [
11+
{
12+
"name": "strata-documentation-engine",
13+
"source": "./",
14+
"description": "Generates, links, and self-verifies documentation for the Strata project family.",
15+
"category": "documentation",
16+
"keywords": ["documentation", "strata", "automation"]
17+
}
18+
]
19+
}

‎.claude-plugin/plugin.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"name": "strata-documentation-engine",
3+
"version": "0.0.2",
4+
"description": "Generates, links, and self-verifies documentation for the Strata project family: documents each source in sources.md, builds a frontmatter-derived index and graph, then runs an adjudicated verify-fix loop.",
5+
"author": {
6+
"name": "NavaPBC",
7+
"url": "https://github.com/navapbc"
8+
},
9+
"repository": "https://github.com/navapbc/strata-documentation-engine",
10+
"keywords": ["documentation", "strata", "automation", "skill"],
11+
"skills": "./skills/"
12+
}

‎.github/workflows/generate-docs.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: Generate Strata Docs
2+
# Full regeneration is a deliberate, maintainer-run action — manual dispatch only.
3+
# Routine drift refresh + new-source pickup lives in update-docs.yml (cron + manual).
4+
on:
5+
workflow_dispatch:
6+
7+
permissions:
8+
contents: write
9+
pull-requests: write
10+
11+
jobs:
12+
generate:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v7
16+
- uses: actions/setup-python@v6
17+
with:
18+
python-version: "3.13"
19+
- run: pip install -r scripts/requirements.txt
20+
- name: Configure git
21+
run: |
22+
git config user.name "strata-docs-bot"
23+
git config user.email "strata-docs-bot@users.noreply.github.com"
24+
- name: Generate docs with Claude
25+
uses: anthropics/claude-code-action@v1
26+
env:
27+
GH_TOKEN: ${{ secrets.SOURCES_READ_TOKEN }} # read access to the source repos
28+
with:
29+
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
30+
claude_args: "--allowedTools 'Skill Workflow Agent Task Read Write Edit Bash' --max-turns 120"
31+
prompt: |
32+
Invoke the generate-strata-docs skill in full mode (the default) to regenerate and
33+
verify documentation for every source in sources.md. Follow the skill exactly.
34+
Do not commit or push.
35+
- name: Open or update PR
36+
uses: peter-evans/create-pull-request@v7
37+
with:
38+
branch: docs/full-regen
39+
title: "docs: full regeneration of Strata documentation"
40+
commit-message: "docs: full regeneration of Strata documentation"
41+
body: |
42+
Full regeneration by the Strata Documentation Engine.
43+
Review skipped sources, any docs marked `verified: needs-review`
44+
(see `docs/.verification/`), feature-key and platform-component gaps, and the curator's
45+
process notes (`docs/.curation/improvements.md`).
46+
add-paths: |
47+
docs/**

‎.github/workflows/lint.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Lint & Test
2+
on:
3+
push:
4+
paths: ["scripts/**", "tests/**", "skills/**", "sources.md", "docs/**", ".github/workflows/lint.yml"]
5+
pull_request:
6+
paths: ["scripts/**", "tests/**", "skills/**", "sources.md", "docs/**", ".github/workflows/lint.yml"]
7+
8+
permissions:
9+
contents: read
10+
11+
jobs:
12+
lint:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v7
16+
- uses: actions/setup-python@v6
17+
with:
18+
python-version: "3.13"
19+
- run: pip install -r scripts/requirements.txt pytest
20+
- name: Unit tests
21+
run: python -m pytest -v
22+
- name: Lint manifest
23+
run: python -m scripts.lint_manifest
24+
- name: Lint docs
25+
run: python -m scripts.lint_docs
26+
- name: Graph freshness (committed graph matches frontmatter)
27+
run: |
28+
python -m scripts.build_graph
29+
git diff --exit-code -- docs/INDEX.md docs/graph.json

‎.github/workflows/update-docs.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: Update Strata Docs
2+
# Manually-triggered drift refresh: re-document only sources whose upstream changed since they were
3+
# last documented, and full-generate any source newly added to sources.md. Full regeneration of
4+
# everything is a separate, manual-only workflow (generate-docs.yml).
5+
on:
6+
workflow_dispatch:
7+
8+
permissions:
9+
contents: write
10+
pull-requests: write
11+
12+
jobs:
13+
update:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/checkout@v7
17+
- uses: actions/setup-python@v6
18+
with:
19+
python-version: "3.13"
20+
- run: pip install -r scripts/requirements.txt
21+
- name: Configure git
22+
run: |
23+
git config user.name "strata-docs-bot"
24+
git config user.email "strata-docs-bot@users.noreply.github.com"
25+
- name: Update docs with Claude
26+
uses: anthropics/claude-code-action@v1
27+
env:
28+
GH_TOKEN: ${{ secrets.SOURCES_READ_TOKEN }} # read access to the source repos
29+
with:
30+
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
31+
claude_args: "--allowedTools 'Skill Workflow Agent Task Read Write Edit Bash' --max-turns 120"
32+
prompt: |
33+
Invoke the generate-strata-docs skill in update mode. It must re-document and
34+
re-verify only the sources whose upstream changed since they were last documented,
35+
full-generate any brand-new sources in sources.md, and skip unchanged sources.
36+
Follow the skill exactly. Do not commit or push.
37+
- name: Open or update PR
38+
uses: peter-evans/create-pull-request@v7
39+
with:
40+
branch: docs/auto-update
41+
title: "docs: update Strata documentation"
42+
commit-message: "docs: update Strata documentation"
43+
body: |
44+
Incremental update by the Strata Documentation Engine (changed + new sources only).
45+
Review skipped (unchanged) sources, any docs marked `verified: needs-review`
46+
(see `docs/.verification/`), feature-key and platform-component gaps, orphaned
47+
sources, and the curator's process notes (`docs/.curation/improvements.md`).
48+
add-paths: |
49+
docs/**

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.sources/
2+
.logs/
3+
__pycache__/
4+
*.pyc
5+
.pytest_cache/

‎CLAUDE.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## What this is
6+
7+
The Strata Documentation Engine generates and self-verifies documentation for the Strata project
8+
family (the SDK, Rails app template, infra template, `platform-cli`, and SDK-consuming apps like
9+
OSCER). It pulls source repos listed in `sources.md`, has Claude write per-source docs, then builds
10+
a frontmatter-derived index + graph and runs an adversarial verify→fix loop over each doc.
11+
12+
There are two distinct layers, and they are edited very differently:
13+
14+
1. **Generation layer** — non-deterministic, Claude-driven. The `generate-strata-docs` skill
15+
(`skills/generate-strata-docs/SKILL.md`) orchestrates two multi-agent `Workflow` runs. You
16+
*invoke* this; you rarely hand-edit its output.
17+
2. **Validation / graph layer** — deterministic Python in `scripts/`. This is the spine that
18+
enforces correctness and is what you'll most often edit and test. Output is reproducible from
19+
doc frontmatter alone.
20+
21+
## Commands
22+
23+
Requires Python 3.13.
24+
25+
```bash
26+
# Python setup
27+
pip install -r scripts/requirements.txt pytest
28+
29+
# Tests
30+
python -m pytest -v # all
31+
python -m pytest tests/test_lint_docs.py -v # one file
32+
python -m pytest tests/test_lint_docs.py::test_validate_doc_accepts_valid -v # one test
33+
python -m pytest -k frontmatter -v # by pattern
34+
35+
# Lint / build pipeline (run in this order; each prints a *_OK sentinel on success)
36+
python -m scripts.lint_manifest # validates sources.md -> "MANIFEST_OK"
37+
python -m scripts.lint_manifest --json # emits parsed sources (used by the skill at runtime)
38+
python -m scripts.lint_docs # validates doc frontmatter + registry usage -> "DOCS_OK"
39+
python -m scripts.build_graph # writes docs/INDEX.md + docs/graph.json -> "GRAPH_OK"
40+
python -m scripts.source_delta --json # update mode: classify sources new/changed/unchanged/throttled/orphaned (skill passes --shas-file; --now/--min-age-days tune the staleness throttle)
41+
```
42+
43+
Scripts are run as modules (`python -m scripts.x`), not as files — `pyproject.toml` sets
44+
`pythonpath = ["."]` so `scripts/` and `tests/` resolve.
45+
46+
## Architecture
47+
48+
### The manifest drives everything
49+
50+
`sources.md` is a markdown table (one row per source: `id`, `type`, `repo`, `ref`, `subpaths`,
51+
`notes`). A source's `type` selects a **profile** in
52+
`skills/generate-strata-docs/references/profiles/<type>.md` that tells the documenter agent how to
53+
treat that kind of repo. `lint_manifest` rejects a source whose `type` has no matching profile, so
54+
adding a new source type means adding both a manifest row and a profile file.
55+
56+
### The skill orchestration (SKILL.md)
57+
58+
```
59+
Setup (clone sources to .sources/) → Run 1 DOCUMENT → build graph
60+
→ Run 2 VERIFY→ADJUDICATE→FIX (per doc) → rebuild graph → CURATE → report
61+
```
62+
63+
- **Run 1** (`workflows/run-1-document.mjs`): one `general-purpose` agent per source, in parallel.
64+
Each reads `agents/source-doc.md` + its profile + the registries, then writes docs to
65+
`docs/sources/<id>/` and a distillation log to `.logs/<id>.distillation.md`.
66+
- **Run 2** (`workflows/run-2-verify-fix.mjs`): per doc, a bounded loop (`max_rounds`, default 2) of
67+
verify → adjudicate → fix using four agent roles (`agents/{verifier,adjudicator,fixer}.md`). A
68+
verifier finds claims unsupported by the source; an adjudicator confirms/rejects each by
69+
re-checking the source; a fixer edits only confirmed findings. Residual findings after the last
70+
round mark the doc `verified: needs-review`, with the audit trail in `docs/.verification/`.
71+
- The skill commits to the `Workflow` tool with **no fallback** — if `Workflow` is unavailable it
72+
stops and escalates. Subagents never talk to the user.
73+
- The skill runs in **full** mode (document every source) or **update** mode (only new + changed
74+
sources, computed by `scripts/source_delta.py`, with unchanged sources skipped); the invoking
75+
prompt picks the mode. See SKILL.md "Modes". Drift is detected by comparing each clone's resolved
76+
commit SHA against the `source_ref.ref` recorded in that source's existing docs. Update mode also
77+
**throttles**: a drifted source is re-documented only once its docs (`last_documented` date) are
78+
at least a week old, so a frequently-changing repo isn't re-documented every run.
79+
80+
### Frontmatter is the single source of truth for the graph
81+
82+
Every doc under `docs/sources/<id>/` starts with YAML frontmatter
83+
(contract: `references/doc-frontmatter-schema.md`). `build_graph.py` derives `docs/INDEX.md` and
84+
`docs/graph.json` *purely* from that frontmatter — never edit those two files by hand.
85+
86+
Two cross-link axes connect docs, both resolved through **registries** (fenced kebab-case key lists
87+
that the linter parses):
88+
89+
- **Feature axis** (`references/feature-keys.md`): an `sdk` doc *owns* a key via `feature_keys`; an
90+
`example` doc *uses* it via `demonstrates`. The graph builder resolves each `demonstrates` key to
91+
the owning SDK doc and emits an `example-of` edge.
92+
- **Platform axis** (`references/platform-components.md`): a doc *owns* a component id via
93+
`component_keys`; `platform-cli` docs declare `manages`, and app/infra docs declare
94+
`integrates_with`, both resolved to the owning doc (`manages` / `integrates-with` edges).
95+
96+
`lint_docs` **hard-fails** on any `feature_keys`/`demonstrates`/`component_keys`/`manages`/
97+
`integrates_with` value not present in its registry. So a doc may only reference a feature/component
98+
by a registry key — never by a not-yet-generated doc id. Adding a new feature/component means adding
99+
its key to the registry's fenced block first.
100+
101+
### "Never silently drop" invariant
102+
103+
The pipeline surfaces every gap instead of hiding it: a clone failure → the source is recorded
104+
**skipped** (not dropped); a registry-valid `demonstrates`/`manages` key that no doc owns →
105+
`build_graph` prints a `GAP:` line; unresolved verification findings → `verified: needs-review`;
106+
a `docs/sources/<id>/` whose source left `sources.md` → `source_delta` reports it **orphaned**
107+
(never auto-deleted); a drifted source documented less than a week ago → `source_delta` reports it
108+
**throttled** with a warning (skipped this run, not silently dropped). When editing the graph builder, linter, or delta classifier, preserve this —
109+
emit a visible record rather than discarding.
110+
111+
## CI (`.github/workflows/`)
112+
113+
- **lint.yml** (`scripts/`, `tests/`, `docs/`, `sources.md`, `skills/` changes): pytest →
114+
lint_manifest → lint_docs → **graph freshness** (`build_graph` then `git diff --exit-code` on
115+
`docs/INDEX.md` + `docs/graph.json`). The freshness check means **you must commit regenerated
116+
`INDEX.md`/`graph.json` whenever doc frontmatter changes**, or CI fails.
117+
- **generate-docs.yml**: **manual only** (`workflow_dispatch`). Runs the skill in **full** mode and
118+
opens a PR on `docs/full-regen`. For deliberate, maintainer-run full rebuilds — not a cron job,
119+
and not triggered by pushes/merges.
120+
- **update-docs.yml**: **manual only** (`workflow_dispatch`). Runs the skill in **update** mode
121+
(re-document only changed sources, full-generate brand-new ones, skip unchanged) and opens a PR
122+
on `docs/auto-update`. Both Claude workflows need `ANTHROPIC_API_KEY` and `SOURCES_READ_TOKEN`
123+
secrets.
124+
125+
## Conventions & gotchas
126+
127+
- `.sources/` (source checkouts) and `.logs/` (distillation logs) are runtime, gitignored
128+
directories — present during a skill run, absent in a clean tree.
129+
- `docs/.verification/` (per-doc findings) and `docs/.curation/improvements.md` (advisory process
130+
notes from the curator) are kept as audit trail under `docs/`.
131+
- `scripts/frontmatter.py` is the shared YAML-frontmatter parser used by both the linter and the
132+
graph builder — change it in one place.
133+
- The design spec and plan live in `docs/superpowers/{specs,plans}/`; section references like
134+
"§3.3" in the code and skill point back to that spec.

‎README.md‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,41 @@
1-
# strata-documentation-engine
1+
# Strata Documentation Engine
2+
3+
Generates and self-verifies documentation for the Strata project family from a list of
4+
sources, producing a linked, agent-queryable knowledge base.
5+
6+
## How it works
7+
8+
1. `sources.md` lists each source (`id`, `type`, `repo`, `ref`, optional `subpaths`) —
9+
the SDK, the Rails app template, SDK-consuming apps like OSCER, the `template-infra`
10+
infrastructure template, and the `platform-cli` (`nava-platform`) tool.
11+
2. The `generate-strata-docs` skill documents each source (one agent per source) using the
12+
type's profile in `skills/generate-strata-docs/references/profiles/`. The `sdk`,
13+
`infra-template`, and `platform-cli` documenters distill each source's own `docs/`
14+
(verifying against code); each doc is tagged with canonical `feature_keys`/`demonstrates`
15+
from `references/feature-keys.md` and `component_keys`/`manages`/`integrates_with` from
16+
`references/platform-components.md`.
17+
3. `scripts/build_graph.py` builds `docs/INDEX.md` and `docs/graph.json` from doc frontmatter,
18+
linking each example to the SDK feature it `demonstrates` (`example-of`), and the CLI/templates
19+
to the components they `manage` / `integrate with` (`manages` / `integrates-with`).
20+
4. An adjudicated verify→fix loop checks each doc against its source; unresolved docs are
21+
marked `verified: needs-review` with findings in `docs/.verification/`.
22+
5. A curator reviews the run's distillation logs into `docs/.curation/improvements.md`.
23+
24+
## Running it
25+
26+
- **Locally:** open this repo in Claude Code and invoke the `generate-strata-docs` skill
27+
(uses your own Claude auth; needs git access to the source repos and Python 3.13).
28+
- **In CI:** the `Generate Strata Docs` Action (manual, weekly, or on `sources.md` change)
29+
runs the skill and opens a PR.
30+
Requires the `ANTHROPIC_API_KEY` and `SOURCES_READ_TOKEN` secrets.
31+
32+
## Developing
33+
34+
```bash
35+
pip install -r scripts/requirements.txt pytest
36+
python -m pytest -v
37+
python -m scripts.lint_manifest
38+
python -m scripts.lint_docs
39+
```
40+
41+
See `docs/superpowers/specs/2026-06-18-strata-documentation-engine-design.md` for the design.

0 commit comments

Comments
 (0)