Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .agents/skills/assess-form-corpus/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
name: assess-form-corpus
description: Profile a batch of XDPs before anyone converts them - tier each form by size, binding, scripting and unsupported constructs, find the constructs and elements shared across the batch, and propose a conversion order and the rulebook or dictionary work that unlocks the most forms. Use when asked to "assess these forms", "size the batch", "which form next", or when a new set of XDPs arrives.
---

# Assess a form corpus

`python -m tools.corpus <dir>` gives the numbers. This skill is the reading of them: which forms share
a problem, which rule or dictionary element would clear the most findings, and in what order the per-form
sessions should run so later forms inherit earlier decisions.

## Procedure

1. Put the XDPs in one directory (outside Git unless cleared). Run
`python -m tools.corpus <dir> -o out/`. Exit non-zero means an XDP the extractor cannot parse; open an
issue with the filename and the traceback, do not skip the file silently.
2. Read `out/corpus.json`. For each form note the tier and the dominant finding. Then aggregate across
forms, by hand or with a short script you attach to the PR:
- unsupported construct codes (`UNS-*`, `LAY-*`) with the number of forms each appears in
- unbound-field captions and schema leaf names that recur (candidate canonical elements)
- script classifications that recur (candidate `VAL-*` translation rules)
- forms whose `revision_hint` is missing (G0 will not protect them)
3. Propose the batch plan:
- conversion order: `suggested_order` from the tool, adjusted so that a form introducing a shared
construct or element is converted before the forms that only reuse it; say why for each move
- rulebook work: one line per construct that appears in 2+ forms, with the rule family it belongs to and
the fixture form
- dictionary work: one line per recurring element with the forms and the evidence (caption, schema type)
- forms to hold: anything `complex` with `human_touch` above the batch median, until the shared rules exist
4. Write the plan as `docs/batches/<batch_id>.md` (form ids and counts only, no field values), open a PR,
link this session. Then launch `playbooks/convert-form.md` per form in the proposed order, one session
each, and reference the batch doc in each.

## Never

- Convert a form as part of the assessment.
- Reorder the batch to make the metrics look better; reorder to maximise reuse and say so.
- Treat tier as readiness. A `simple` form with a draft rule is not ready.
92 changes: 92 additions & 0 deletions .agents/skills/convert-xdp-form/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
name: convert-xdp-form
description: Convert one AEM Forms XDP end to end in a Devin session - run the deterministic pipeline, interpret every gate finding, write the form's decision record, propose dictionary rows and rules, prepare the Designer handoff, and open the PR. Use when asked to "convert form <id>", "run the pipeline on <form>", "process <form>", or to add a form to src/fixtures/pairs.yaml.
---

# Convert an XDP form

Every form goes through this procedure in its own Devin session. The Python tools are deterministic
and produce the facts; this skill is the judgment layer that turns each fact into a routed, reviewable
proposal. A form with an unrouted finding, a proposal without rationale, or a PR without a session link
has not been converted.

## Inputs (stop and ask if any is missing)

- the XDP file (`<digits>.xdp`; six digits are the form number, e.g. `181026.xdp` -> `18-1026`)
- if a Quadient reference exists: the exported XML and composed PDF, exact filenames
- the AEM-rendered PDF and one sanitized upstream payload, if available (they unlock G6 later)
- whether the artifacts may be committed (default: no, they stay under gitignored `src/fixtures/raw/`)
- who will import into Designer (name) and who reviews decisions (name)

## Procedure

### 1. Register the form (deterministic)

1. Copy the artifacts into `src/fixtures/raw/` or point `CCM_FIXTURE_ROOT` at them. Do not rename them.
2. Add the form to `src/fixtures/pairs.yaml` with exact filenames and `sha256sum` hashes;
`python -c "import fixtures; fixtures.load_pairs()"` must pass.
3. Run `python -m tools.pipeline -o out/`.
- Exit `2`: G1/G2 failed, the extractor lost data. Fix `src/converters/xdp_to_cim/extractor.py` with a test
in `tests/test_extractor.py` that fails before the fix. Nothing else until this is exit `0`.
- Exit `0`: open `out/readiness.json` for the form. Every finding below comes from there.

### 2. Read the form before routing anything (judgment)

Write five lines in your notes, you will need them for the PR and the decision rationales:
what the form is for (title, statics), how it is bound (bound %, data schema names), what the scripts do
(classes from `scripts`), what the target reference looks like if there is one (G4 sections, naming prefix),
and whether the revision hint matches the target (G0).

### 3. Route every blocker/major G3 finding into `src/decisions/<form_id>.yaml`

Create the file per `src/decisions/README.md` with `session` set to this session's URL. One record per
(node, finding), `status: proposed`, a one-sentence `rationale` that cites evidence (caption, schema type,
sibling field, reference-target object). Actions by finding:

| finding | usual action | also do |
|---|---|---|
| `G3-UNBOUND`, `G3-NONAME` | `bind_to_element` | add a `proposed` row in `src/dictionary/crosswalk.yaml` (and `elements.yaml` if the element is new) with `evidence`; if no evidence exists, `keep_unbound` and say why |
| `G3-UNRESOLVED-BIND` | `bind_to_element` or `add_rule` | if the binding syntax is one the normaliser (BND-01) misses, draft the rule with `fixtures: [<form_id>]` |
| `G3-VALIDATION`, `VAL-SCRIPT-MANUAL` | `translate_script` or `drop_script` | quote the script's intent (not its code) in `proposal`, name the Quadient construct it maps to; `drop_script` only for capture-channel behaviour that has no print equivalent |
| `UNS-*` unsupported construct | `add_rule` or `manual_design` | draft rule if it will recur; `manual_design` if it is a one-off for the designer |
| `G3-NOITEMS`, `G3-DUPNAME` | `manual_design` or `defer` | say which section disambiguates |
| `G3-DRAFT-RULE` | none (rule-level, not node-level) | list the rule ids in the PR; do not record decisions for them |

Re-run the pipeline. `findings routed` must equal `findings routable` for the form and `G3-STALE-DECISION`
must be absent. Do not set any decision to `accepted`.

### 4. Interpret G4 (reference pairs only)

For each unmatched field: if the target has the object under a name the matcher should have found, fix
`src/gates/matching.py` with a test. If the designer renamed it by convention (prefixes like `EFTAuth`), record the
convention as a `proposed` crosswalk row and note the pattern for OBJ-05/FLD-01 in the PR. If the target lacks
the field, write a `defer` decision against the G3 finding for that node if one exists, otherwise leave it in
the report. If static-text coverage is below GOV-01, stop routing names: the pair is a revision mismatch and
the PR must say so.

### 5. Prepare the Designer handoff

Follow `.agents/skills/prepare-designer-handoff/SKILL.md` for `out/<form_id>.plan.json`. It produces the
import package and the checklist the designer returns. Name the importer in the PR.

### 6. Pin and verify

Add the form's metrics to `tests/test_golden.py` `GOLDEN` from `out/readiness.json` (copy, do not round).
Run `ruff check . && pytest -q`.

### 7. Open the PR

Branch `devin/<timestamp>-convert-<form_id>`. Commit `pairs.yaml`, `src/decisions/<form_id>.yaml`, dictionary
and rulebook changes, golden metrics, tests. Attach or paste `out/readiness.md` and the handoff package; do
not commit `out/`. The PR body reports, from `readiness.json`: every gate verdict; static-text coverage,
field match and exact-name rate; findings routable/routed with the count per action; scripts by class;
draft rules depended on; the decision reviewer and the Designer importer by name; G5/G6 as `external`.
Link this session. Do not describe the form as converted or ready.

## Never

- Hand-write Quadient XML or edit `<form>.plan.json` by hand.
- Change `GOV-01` / G4 thresholds, exit codes, or `decisions` loader checks to make the run green.
- Mark any rule, dictionary row or decision `verified`/`accepted`.
- Write a decision without a rationale, or a rationale that restates the finding.
- Say the form "converted" or "is ready". Say what the gates said and what you proposed.
63 changes: 63 additions & 0 deletions .agents/skills/prepare-designer-handoff/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
name: prepare-designer-handoff
description: Turn a form's build plan into a Quadient Designer import package and return checklist for the person with Designer access, then record what comes back as the G5 import record. Use when asked to "hand off <form> to the designer", "prepare the import for <form>", or when a designer returns a Designer import result.
---

# Prepare the Designer handoff

Devin has no Designer licence. G5 is decided by a person importing an artifact into Designer and telling
us what happened. This skill makes that round trip unambiguous in both directions so nothing is lost
between the session and the designer's inbox.

## Outbound (after `convert-xdp-form` step 5)

1. Confirm `out/<form_id>.plan.json` exists and `out/readiness.json` shows G1 and G2 `pass`. If not, stop.
2. Build the package directory `handoff/<form_id>/` (gitignored, attach to the PR):
- `<form_id>.plan.json` and its `sha256`
- `CimBuildPlanImport.groovy` from `src/converters/cim_to_inspire/groovy/` with the migration-stack version
pinned in `knowledge/quadient-designer.md` (if the version is `unknown`, say so in the checklist)
- `src/decisions/<form_id>.yaml` filtered to `manual_design` and `defer` records: these are the things the
designer must do by hand, in order
- `CHECKLIST.md` (template below)
3. Read the plan once as the designer would: for every `decisions_required` entry, check a decision record
exists; for every `skipped` block, check the skip reason is a rule id. Anything else is a bug in
`build_plan.py`, fix it with a test before sending.
4. Name the importer in the PR and attach the package.

### CHECKLIST.md template

```
Form: <form_id> plan sha256: <hash> session: <devin session url>
Migration-stack version used: <x.y.z or unknown> Designer version: <from knowledge/quadient-designer.md>

1. Run the Groovy import against the plan. Paste the full console output below, unedited.
2. Export the resulting WFD as XML (Designer's generic XML export). Record its filename and sha256.
3. For each manual_design / defer item listed, say done / not done / not applicable.
4. Compose once with the fixed input data for this form (if you have it). Record the output PDF sha256.

Console output:
<paste>

Manual items:
<list>

Exported XML: <filename> <sha256> Composed PDF: <filename> <sha256>
```

## Inbound (when the checklist comes back)

1. If the console output has errors: invoke `.agents/skills/triage-designer-errors/SKILL.md`. Every error
class becomes an import-lint check, a rule and a fixture. Do not fix the form by hand and re-send.
2. Write `src/fixtures/import_logs/<xml_sha256>.json` per `src/fixtures/import_logs/README.md` with the importer's
name, date, Designer version, verdict and the raw console text. Re-run the pipeline; G5 must now read the
record instead of `external`.
3. If the designer changed anything by hand that is not a `manual_design` decision, add a `proposed`
decision for it with `rationale: "designer edit during import, undocumented"` so the gap is visible.
4. If a composed PDF and input data came back, invoke `.agents/skills/validate-pdf-output/SKILL.md`.
5. Update the PR: G5 verdict, new import-lint checks, new rules, and the count of undocumented edits.

## Never

- Send a package whose plan was edited by hand.
- Summarise the designer's console output; store it raw.
- Record an import log without the importer's name and Designer version.
54 changes: 54 additions & 0 deletions .agents/skills/review-rulebook-change/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
name: review-rulebook-change
description: Review a PR that adds or changes rules under src/rulebook/rules, the CIM schema, gates, or the dictionary. Use when asked to review a rulebook PR, to assess whether a rule can move from draft to verified, or to check a Devin-drafted rule before a Quadient designer signs it off.
---

# Review a rulebook / CIM / gate change

Two audiences use this: a Devin session doing a first-pass review (it may only *recommend*), and the named
Quadient/customer reviewer who is the only person allowed to set `status: verified` and `reviewed_by`.

## Checklist

For every rule touched (`src/rulebook/rules/*.yaml`):

1. **Origin is real.** `origin.type` is one of the lint-accepted types and `origin.refs` points at something a
reviewer can open: a form id in `src/fixtures/pairs.yaml`, an import-log hash, a document. "Devin inferred it"
is `analysis` and must say what was compared.
2. **The fixture exercises the rule.** Open the CIM (`out/<form>.cim.json`) for each id in `fixtures` and find
the construct the rule describes. If the rule fires on nothing, reject; if it fires on one form only, ask
whether it is a rule or a patch (`AGENTS.md` rule 8).
3. **Class matches implementation.** `AUTO`/`AUTO*` rules must list a resolvable `implemented_in` and that code
must be covered by a test in `tests/`. A rule described as deterministic but implemented nowhere is
`ASSIST` at best. `MANUAL` rules must say in `target` where the human decision lands (a G3 finding, a
dictionary row, a CIM `data_dictionary` entry) so it is persisted as data (GOV-05).
4. **Target claim is evidenced from the target.** The `target` and `evidence` fields must come from a Designer
export, official Quadient documentation, or the migration-stack source, not from the XDP. Two reference
exports are examples, not a grammar; say so in the review if the rule generalises beyond them.
5. **Unsupported is honest.** If the rule narrows what is converted, the excluded case must appear in
`src/rulebook/rules/unsupported.yaml` or as a G3 finding, not disappear.
6. **Thresholds unchanged.** Any edit to `GOV-01`, the G4 floor, `INTEGRITY_GATES`, or pipeline exit codes
needs a written rationale from the customer, not a green CI run.

For CIM schema changes (`src/cim/schema/cim.schema.json`): `CIM_VERSION` bumped, `tests/test_golden.py` counts
re-pinned from a fresh run, and every extractor field that feeds the changed node still has a test.

For gate changes (`src/gates/`): the failure path is tested (a fixture that *should* fail still fails), and the
gate still distinguishes `fail` / `external` / `skipped` as documented in `src/gates/common.py`.

For dictionary changes (`src/dictionary/*.yaml`): `verified` rows carry `reviewed_by`; `evidence` names the
source (Designer export variable, data master, Factory map); `rejected` rows stay in the file so the same
proposal is not re-made.

## Verdicts

- Devin: comment "recommend verify" / "keep draft: <reason>" / "reject: <reason>". Do not edit `status`.
- Human reviewer: to verify, set `status: verified`, `reviewed_by: <name>`, `reviewed_on: <YYYY-MM-DD>` in the
same PR. `pytest -q` enforces that verified rules carry a reviewer and draft rules do not.

## Run before commenting

```
ruff check . && pytest -q
python -m tools.pipeline -o out/ # confirm the golden metrics moved only where the PR says they should
```
44 changes: 44 additions & 0 deletions .agents/skills/triage-designer-errors/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
name: triage-designer-errors
description: Turn a Quadient Inspire Designer import error log into permanent assets - an import-lint check, a rulebook entry, a fixture, and a G5 import record - then open a PR. Use when someone pastes Designer import errors, asks to "triage import errors for <form>", or supplies an import log.
---

# Triage Designer import errors

Input: the raw error output from a Designer import of a generated artifact, plus who imported it, which
Designer version, and on what date. If the raw text is paraphrased ("it complained about tables"), ask
for the verbatim messages before proceeding; the catalogue stores messages, not summaries.

## Procedure

1. Record the import as evidence first. Compute `sha256sum <layout.xml>` for the artifact that was imported
and write `src/fixtures/import_logs/<sha256>.json` (format in `src/fixtures/import_logs/README.md`) with
`verdict: fail` and every message in `errors`. This is the only thing that makes G5 say anything other
than `external`.
2. Classify each message against `src/gates/import_lint/catalogue.yaml`:
- **Known code** (`IMP-00x`): set that entry's `status: observed`, paste the message into
`designer_message`, and extend `evidence`. If the offline check did *not* flag the artifact before
import, the check is wrong or too narrow: fix it in `src/gates/import_lint/__init__.py` with a test in
`tests/test_gates.py` that fails on the offending XML before the fix.
- **New class**: add a catalogue entry with the next code, `status: observed`, the verbatim
`designer_message`, and a `check` function you implement in `src/gates/import_lint/__init__.py`. The check
receives the target inventory (`gates.target_inventory.inventory`) and returns offending node ids. Add
it to `CHECKS`. Add a failing-then-passing test using a mutated copy of `tests/data/99-0001.xml`.
- **Not structural** (licensing, font not installed, version mismatch): do not add a check. Record it in
the PR and, if it recurs, in `knowledge/quadient-designer.md`.
3. For each new or changed check, find or add the rulebook rule that prevents the error at build-plan time:
`src/rulebook/rules/<family>.yaml`, `class` `AUTO` if `src/converters/cim_to_inspire/build_plan.py` can enforce
it (then implement and list it in `implemented_in`), otherwise `ASSIST`/`MANUAL`. `origin` is
`{type: designer_error, refs: ["<import_log sha256 prefix>"]}`. Status stays `draft`.
4. Re-run `python -m tools.pipeline -o out/`. The catalogue must now flag the imported artifact offline
(G5 `fail` with the new code) and must still be clean on the two reference exports (they were produced
by Designer, so any new finding against them is a false positive).
5. `ruff check . && pytest -q`. Open a PR titled `import-lint: <IMP-code> <short title>` listing: the error
text, the check, the rule, and what a reviewer must confirm in Designer to mark the rule `verified`.

## Never

- Fix the generated XML by hand to make the import pass.
- Mark a catalogue entry `observed` without the verbatim Designer message.
- Add a check that also fires on `17-0574.xml` or `18-1026_Quad_2.xml`.
- Set a rule to `verified`.
Loading
Loading