Add kya governance skill and optional governance break-signal (#4)

Add kya governance skill and optional governance break-signal

Author: satra

README.md

@@ -13,6 +13,7 @@ The list below is kept in alphabetical order by skill name.
 | Skill | Description |
 | --- | --- |
 | [`duct`](./skills/duct/SKILL.md) | Wrap any command with [con/duct](https://github.com/con/duct) to capture wall-clock time, CPU, and memory usage as structured logs, so agents and reviewers can inspect what a run actually consumed. |
+| [`kya`](./skills/kya/SKILL.md) | Govern and review agents with [veldt-kya](https://github.com/veldtlabs/veldt-kya) (Know Your Agents) — risk-score, consensus-judge, and drift-check an agent, emit compliance evidence, and write a governance verdict that the `labnb` loop can break on. |
 | [`labnb`](./skills/labnb/SKILL.md) | Create and maintain a concurrency-safe global lab notebook outside project roots, with startup summaries of related prior work, first-class idea capture and promotion, isolated experiment workspaces, focused companion skills, and append-only indexing across projects, investigations, and tasks. |
 | [`labnb-idea`](./skills/labnb-idea/SKILL.md) | Record a promising but not-yet-implemented experiment idea in the shared lab notebook index. |
 | [`labnb-promote`](./skills/labnb-promote/SKILL.md) | Promote a lab notebook idea into a concrete experiment with explicit budgets, source links, and provenance. |

skills/kya/SKILL.md

@@ -0,0 +1,100 @@
+---
+name: kya
+description: Govern and review autonomous agents with veldt-kya (Know Your Agents) — risk-score an agent, run multi-judge consensus, detect configuration drift, and emit compliance evidence — then turn the result into a verdict that labnb's loop can act on. Use when a run needs a trust/authorization/policy check, not just a resource or metric check.
+---
+
+# Know Your Agents (KYA) Governance Review
+
+Use this skill to answer a different question from "is the run fast?" (see [`duct`](../duct/SKILL.md)) or "is the metric improving within budget?" (see [`labnb`](../labnb/SKILL.md)): **is the agent authorized, policy-compliant, and uncompromised?**
+
+It wraps [`veldt-kya`](https://github.com/veldtlabs/veldt-kya) ("Know Your Agents", Apache-2.0), a governance SDK that risk-scores agents, runs consensus judges over their behavior, detects drift from an approved definition, and produces regulator-grade compliance evidence.
+
+This is the third review axis in this repo: `duct` = resource accounting, `labnb` = budget / experimental-validity accounting, `kya` = trust / authorization / compliance accounting.
+
+## When To Use
+
+- Before granting an agent a privileged action (writing outside scope, executing SQL, calling external services), score or consensus-check it first.
+- When a long-running labnb experiment should stop if the agent drifts from its approved configuration or trips a policy judge — not only when the metric stalls.
+- When a run touches regulated or PII-bearing data and needs auditable authorization evidence (GDPR, HIPAA, EU AI Act, and similar).
+
+Skip it for ordinary, low-stakes local iteration; governance has real cost (see Guardrails).
+
+## Install
+
+```bash
+pip install veldt-kya   # Apache-2.0
+```
+
+`veldt-kya` is a heavier, pre-1.0 dependency (database, identity bindings, optional external judges). Keep it **optional** — it is never required by the `labnb` core. Install it only in environments that need governance.
+
+## Core Capabilities
+
+```python
+from kya import score_agent, normalize_agent_def
+from kya import canonical_hash, detect_drift
+from kya.scorer_orchestrator import check_consensus, register_available_adapters
+from kya import compliance_summary
+```
+
+- **Pre-deployment scoring** — `score_agent(...)` returns an object with `score`, a severity `bucket` (e.g. `"critical"`), and per-factor `factors` (each with `.name` / `.delta`).
+- **Runtime consensus** — `check_consensus(...)` runs multiple judges in parallel and returns `consensus` (`OK` / `SPLIT` / `UNCLEAR` / `BREACH`), `per_dimension` scores (input_safety, safety, faithfulness), and per-`judges` detail.
+- **Drift detection** — `canonical_hash(definition)` plus `detect_drift(...)` flag unauthorized configuration changes against an approved baseline.
+- **Compliance evidence** — `compliance_summary(...)` emits model cards / breach-notification artifacts across many regimes.
+
+## Producing A labnb Governance Verdict
+
+`labnb`'s `monitor_slice.py check` can break a slice on a **governance** signal, but it stays dependency-free: it reads a small JSON **verdict file** rather than importing KYA. This skill's job is to run a governance check and write that verdict.
+
+Canonical verdict schema (all keys optional):
+
+```json
+{
+  "decision": "allow | warn | block",
+  "trust_score": 0.0,
+  "drift": false,
+  "reasons": ["short human-readable reasons"]
+}
+```
+
+Map a KYA result onto it, then write it with the bundled helper:
+
+```python
+from kya.scorer_orchestrator import check_consensus
+verdict = check_consensus(...)              # consensus: OK | SPLIT | UNCLEAR | BREACH
+decision = "block" if verdict.consensus == "BREACH" else (
+    "warn" if verdict.consensus in ("SPLIT", "UNCLEAR") else "allow")
+```
+
+```bash
+python skills/kya/scripts/write_verdict.py \
+  --output "$EXPERIMENT_DIR/artifacts/governance.json" \
+  --decision "$decision" \
+  --trust-score 0.41 \
+  --reason "consensus=BREACH on faithfulness judge"
+```
+
+`write_verdict.py` is deterministic and tool-agnostic: it validates the fields and writes the canonical schema, so you can drive it from KYA, `detect_drift`, or any other governance source.
+
+## Feeding It Into The Loop
+
+Point `labnb`'s monitor at the verdict so a compromised or unauthorized slice breaks (with the highest break priority — governance outranks budget, engineering, and validity):
+
+```bash
+python skills/labnb/scripts/monitor_slice.py check \
+  --experiment-dir "$EXPERIMENT_DIR" \
+  --governance-file "$EXPERIMENT_DIR/artifacts/governance.json" \
+  --min-trust-score 0.6 --break-on-drift
+```
+
+A `block` decision, a `trust_score` below `--min-trust-score`, or `drift: true` (with `--break-on-drift`) breaks the slice and moves the entry to `blocked`; a `warn` decision is advisory. This also fits `labnb`'s Action Gate: run the check before any write-bearing or privileged step.
+
+The gate fails closed: once `--governance-file` is set, if the verdict file is missing, unreadable, or malformed the monitor treats it as a `block`. So always write a verdict before the check — including a conservative one (e.g. `decision: "warn"` or `"block"`) when KYA or its judges are unavailable — rather than leaving the file absent.
+
+## Guardrails
+
+1. Respect any parent constitution, project policy, or task-level write constraint already in scope.
+2. Keep `veldt-kya` optional. Never make it a hard dependency of the `labnb` core or of offline CI; the verdict-file seam exists precisely so labnb stays dependency-free.
+3. Governance has cost. Consensus judging may call LLMs or external services (e.g. Fiddler, Presidio); count that time and spend against the same labnb budget, per labnb's budget policy.
+4. Handle a degraded/offline path: if KYA or its judges are unavailable, emit a conservative verdict (for example `decision: "warn"`) rather than silently treating the run as authorized.
+5. Treat the project as pre-1.0 and verify its behavior before relying on it for real compliance decisions; do not present its output as legal/regulatory advice.
+6. Do not send secrets or unminimized PII to external judges; use KYA's PII scanning and scrub before sharing.

skills/kya/agents/openai.yaml

@@ -0,0 +1,7 @@
+interface:
+  display_name: "Know Your Agents (KYA)"
+  short_description: "Govern, trust-score, and drift-check an agent, and gate the loop on the verdict"
+  default_prompt: "Use $kya to run a governance check (score, consensus, or drift) on the agent, write a labnb governance verdict, and gate privileged actions or the experiment loop on the result."
+
+policy:
+  allow_implicit_invocation: false

skills/kya/scripts/write_verdict.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+"""Write a labnb-consumable governance verdict JSON.
+
+Deterministic and tool-agnostic: it validates the fields and writes the
+canonical schema that ``labnb``'s ``monitor_slice.py check --governance-file``
+reads. Drive it from a veldt-kya result (``score_agent`` / ``check_consensus`` /
+``detect_drift``) or any other governance source.
+
+Schema (all keys optional, but at least one of decision/trust_score/drift
+should be set to be useful):
+
+    {"decision": "allow|warn|block", "trust_score": 0.0, "drift": false,
+     "reasons": ["..."]}
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+DECISIONS = ("allow", "warn", "block")
+
+
+def build_verdict(args: argparse.Namespace) -> dict[str, object]:
+    verdict: dict[str, object] = {}
+    if args.decision is not None:
+        verdict["decision"] = args.decision
+    if args.trust_score is not None:
+        verdict["trust_score"] = args.trust_score
+    if args.drift is not None:
+        verdict["drift"] = args.drift
+    if args.reason:
+        verdict["reasons"] = list(args.reason)
+    return verdict
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--output",
+        required=True,
+        help="Path to write the verdict JSON (parent dirs are created).",
+    )
+    parser.add_argument("--decision", choices=DECISIONS, default=None)
+    parser.add_argument(
+        "--trust-score",
+        type=float,
+        default=None,
+        help="Trust score in the inclusive range 0.0–1.0.",
+    )
+    drift = parser.add_mutually_exclusive_group()
+    drift.add_argument("--drift", dest="drift", action="store_true", default=None)
+    drift.add_argument("--no-drift", dest="drift", action="store_false")
+    parser.add_argument(
+        "--reason",
+        action="append",
+        default=[],
+        help="Human-readable reason (repeatable).",
+    )
+    args = parser.parse_args(argv)
+
+    if args.trust_score is not None and not (0.0 <= args.trust_score <= 1.0):
+        print("error: --trust-score must be between 0.0 and 1.0", file=sys.stderr)
+        return 2
+    if not any(v is not None for v in (args.decision, args.trust_score, args.drift)) and not args.reason:
+        print("error: provide at least one of --decision/--trust-score/--drift/--reason", file=sys.stderr)
+        return 2
+
+    verdict = build_verdict(args)
+    output = Path(args.output).expanduser()
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(json.dumps(verdict, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+    print(json.dumps(verdict, sort_keys=True))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

skills/labnb-run/SKILL.md

@@ -66,7 +66,7 @@ python skills/labnb/scripts/monitor_slice.py start \
    - if comparing two alternatives by a metric, where one side may be one or more prior runs, design the smallest asynchronous evaluation path that can decide the comparison
    - if parallel follow-up experiments help answer the comparison, you may use subagents to run them as separate experiments, but count their resource usage against the same budget
 10. Unless absolutely necessary, do not run long-lived work blindly; add enough logging, checkpoints, and external observability to inspect progress and consumption while it runs.
-11. Before leaving any background command unattended, run `monitor_slice.py check` and decide whether a timer or watchdog should be started within the remaining budget. `check` can break the slice not only on budget but on engineering (pace too slow, stalled, runaway memory), correctness (repeated failures), or validity (no improvement, metric guardrail) signals; pass the relevant flags (`--reserve-seconds`, `--patience`, `--stall-seconds`, `--max-failures`, `--metric-guardrail`, `--usage-file`) and treat a non-zero exit as "stop, do not iterate again".
+11. Before leaving any background command unattended, run `monitor_slice.py check` and decide whether a timer or watchdog should be started within the remaining budget. `check` can break the slice not only on budget but on governance (unauthorized/drifted, via `--governance-file`), engineering (pace too slow, stalled, runaway memory), correctness (repeated failures), or validity (no improvement, metric guardrail) signals; pass the relevant flags (`--reserve-seconds`, `--patience`, `--stall-seconds`, `--max-failures`, `--metric-guardrail`, `--usage-file`, `--governance-file`/`--min-trust-score`/`--break-on-drift`) and treat a non-zero exit as "stop, do not iterate again".
 12. Before scheduling a new wait job, check whether this experiment already has a pending wait:
    - if the earlier wait still covers the needed follow-up, do not submit a duplicate; just wait on it
    - if the new wait supersedes the older one, cancel or replace the earlier wait first

skills/labnb/SKILL.md

@@ -95,7 +95,7 @@ Treat the budget as applying to the whole proposed path, including any parallel
 6. Track labnb-managed creates and updates with best-effort provenance using W3C PROV-O terms inside each entry directory.
 7. Require explicit user confirmation before labnb performs deletions.
 8. Treat provenance as the source of truth for monitored slice state; do not rely on a separate mutable loop-state file.
-9. Let `monitor_slice.py check` decide when to break a slice, and honor a break: it can stop on budget, engineering (pace, stall, resource), correctness (repeated failures), or validity (no improvement, guardrail) signals, and a non-zero exit means stop rather than start another iteration.
+9. Let `monitor_slice.py check` decide when to break a slice, and honor a break: it can stop on governance (unauthorized, policy-violating, or drifted), budget, engineering (pace, stall, resource), correctness (repeated failures), or validity (no improvement, guardrail) signals, and a non-zero exit means stop rather than start another iteration.
 
 ## Local Guardrails
 
@@ -324,14 +324,15 @@ while python skills/labnb/scripts/monitor_slice.py check \
 done   # the loop ends the moment check decides to break
 ```
 
-A check can break the slice for four classes of reason, not just elapsed time:
+A check can break the slice for five classes of reason, not just elapsed time:
 
-- **Budget** (`budget` category): the loop or overall budget is spent. Use `--reserve-seconds` to break while slack remains for verification, logging, and summary, and `--warn-fraction` (default `0.8`) to get an advisory `warn` before the cap.
+- **Governance** (`governance` category, highest priority): the agent is unauthorized, policy-violating, or drifted from its approved definition. Point `--governance-file` at a JSON verdict (`decision`/`trust_score`/`drift`) written by a governance tool such as the [`kya`](../kya/SKILL.md) skill; `--min-trust-score` and `--break-on-drift` turn it into breaks. labnb stays dependency-free and only consumes the verdict. This gate fails closed: once `--governance-file` is set, a missing, unreadable, or malformed verdict is treated as a `block` rather than silently skipped.
+- **Budget** (`budget`): the loop or overall budget is spent. Use `--reserve-seconds` to break while slack remains for verification, logging, and summary, and `--warn-fraction` (default `0.8`) to get an advisory `warn` before the cap.
 - **Engineering** (`engineering`): the slice is too slow or wasteful even if time remains. Pace projection breaks when another iteration at the recent cadence will not fit the remaining loop budget (disable with `--no-pace`); `--stall-seconds` breaks when no new `results.tsv` row has appeared for too long; `--usage-file` with `--max-rss-bytes` / `--max-pmem` breaks on runaway memory read from a con/duct log.
 - **Correctness** (`correctness`): `--max-failures` breaks after that many consecutive failed/crashed iterations.
 - **Validity** (`validity`): `--patience` breaks when the metric has not improved for that many logged iterations (a plateau or drift, using the recorded `direction`), and `--metric-guardrail` breaks when the latest metric crosses a hard bound in the wrong direction.
 
-When several conditions trip at once, correctness outranks budget, then engineering, then validity, and that primary reason chooses the terminal status (`crashed` for correctness, `budget_exhausted` for budget, `stopped` otherwise). Override the status with `--status-on-break`, keep the legacy always-exit-0 behavior with `--exit-zero`, and treat a `warn` decision as advisory (it does not change status and exits `0`). The full decision, signals, and diagnostics are written into the provenance state snapshot for later review.
+When several conditions trip at once, governance outranks correctness, then budget, then engineering, then validity, and that primary reason chooses the terminal status (`blocked` for governance, `crashed` for correctness, `budget_exhausted` for budget, `stopped` otherwise). Override the status with `--status-on-break`, keep the legacy always-exit-0 behavior with `--exit-zero`, and treat a `warn` decision as advisory (it does not change status and exits `0`). The full decision, signals, and diagnostics are written into the provenance state snapshot for later review.
 
 Notes on the break semantics:
 
@@ -404,7 +405,7 @@ Use the same tight loop pattern that powers autoresearch, but anchor it in the g
    - note what command to restart or re-check on resume
 16. Run `monitor_slice.py check` before continuing, and `monitor_slice.py finish` when the slice ends. Pass the break conditions that matter for this run (`--reserve-seconds`, `--patience`, `--stall-seconds`, `--max-failures`, `--metric-guardrail`, `--usage-file`); if `check` exits non-zero, stop the slice instead of starting another iteration. See "Breaking A Slice Early".
 17. Decide whether another iteration is justified, rather than expanding work to fill the remaining budget.
-18. Repeat until `check` breaks the slice, the stop condition is reached, the next checkpoint fails, or the user interrupts. A break can come from budget, engineering (too slow, stalled, or resource-hungry), correctness (repeated failures), or validity (no improvement or a guardrail breach), not only from elapsed time.
+18. Repeat until `check` breaks the slice, the stop condition is reached, the next checkpoint fails, or the user interrupts. A break can come from governance (unauthorized or drifted), budget, engineering (too slow, stalled, or resource-hungry), correctness (repeated failures), or validity (no improvement or a guardrail breach), not only from elapsed time.
 
 When the work diverges materially, register a child experiment instead of overloading the current one.