contract: align SCHEMA.md and the converter with the runner bundle manifests

[marwen-abid]

This PR is the bottom of the campaign-runner stack. It writes down the data contract between the campaign runner and the converter, and it makes the converter match. The runner writes four JSON manifests into a result bundle. SCHEMA.md now documents all four. This layer documents plan.json and leg.json before the code that writes them lands in parts 3 and 4. The contract comes first, the implementation goes above it.

What is in this PR

• SCHEMA.md documents the campaign bundle manifests: metadata.json (adds finished_at, campaign.resumed, status), the new plan.json and leg.json (runner-owned, additive, ignored by the converter today), and invocation.json with the camelCase keys that stellar-rpc#907 writes.
• converter/convert.py reads binary identity in both spellings. _normalize_binary maps commitHash and buildTimestamp onto the snake_case names, so pre-#907 bundles still resolve.
• load_invocations warns loudly when an invocation carries an error field. A failed run writes invocation.json too, and its CSVs are partial.
• .github/workflows/ingest.yml becomes a thin wrapper over scripts/ingest.sh. The workflow passes the bundle path and --dataset-kind. Run identity comes from the bundle's own metadata.json. The run arrives as a run/<run_id> PR, not a push to main.
• converter/tests/ covers both binary spellings and the failed-run warning. test_golden.py skips when the dataset directory is unreadable, not only when it is absent.

Review focus

• The invocation.json section of SCHEMA.md: it is a cross-repo contract with stellar-rpc.
• _normalize_binary and resolve_binary: the commit cross-check must still fire on a mismatch in either spelling.
• ingest.yml word-splits extra_args on purpose. The shellcheck disable=SC2086 marks the documented limit of that input.

Verification

python3 -m unittest discover converter/tests
shellcheck scripts/ingest.sh

Part 7 adds the shellcheck.yml CI gate that covers scripts/.

---

Stack: part 1 of 8 · base main · replaces a slice of #16

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-02 00:20 UTC

[Copilot]

Pull request overview

Defines and documents the campaign bundle manifest “contract” (especially invocation.json) and updates the converter + CI ingest path to match that contract and improve resilience/diagnostics when bundles contain failed or legacy-shaped invocations.

Changes:

• Expanded SCHEMA.md to document all campaign bundle manifests (including new plan.json/leg.json) and updated invocation.json to camelCase keys with legacy-compat notes.
• Updated converter/convert.py to warn on failed-run invocations and normalize binary identity keys across legacy/new spellings.
• Refactored the GitHub Actions ingest workflow to delegate end-to-end ingest/convert/PR creation to scripts/ingest.sh, and extended tests/fixtures to cover the schema changes.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
scripts/ingest.sh	Updates CI documentation and usage-header range to reflect the workflow delegating full ingest behavior to this script.
SCHEMA.md	Documents the full campaign bundle manifest set and aligns the written contract with the runner/converter expectations.
converter/tests/test_golden.py	Improves golden-test skipping when local datasets are missing or unreadable.
converter/tests/test_campaign.py	Adds tests for legacy snake_case binary keys and failed-run invocation warnings.
converter/tests/fixtures.py	Updates invocation fixtures to mirror the camelCase invocation.json schema.
converter/convert.py	Adds failed-run warnings for invocation.json and normalizes binary identity key spellings.
.github/workflows/ingest.yml	Converts the workflow into a thin wrapper around scripts/ingest.sh, including optional passthrough args.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[marwen-abid]

@claude[agent] review

[Claude]

@claude[agent] review

Reviewed the PR at aa24c83. No code changes needed — the five prior reviewer threads are all resolved with the author's rationale, and there are no open actionable items. Verified locally: python3 -m unittest discover converter/tests passes (71 tests, OK, 13 skipped) and shellcheck scripts/ingest.sh is clean. _normalize_binary/resolve_binary handle both key spellings and still fire the commit-mismatch warning; load_invocations warns on failed-run manifests.