docs: add public benchmark documentation

Author: jroell

README.md

@@ -9,10 +9,11 @@ The Agentic Epistemology Framework (AEF) provides a principled approach to model
 ## Key Features
 
 - **Epistemic Modeling**: Explicit representation of beliefs, justifications, confidence levels, and frames.
-- **Frame-Based Reasoning**: Support for different cognitive perspectives that influence belief formation and update.
-- **Multi-Agent Interactions**: Mechanisms for conflict detection and resolution through justification exchange.
+- **Facet-Conditioned Auditability**: First-class support for auditing whether synthetic personas faithfully express selected facet values and composed frames.
+- **Facet Influence Ledger**: Records selected facets, prompt hashes, runtime weights, marker coverage, contradictions, and recommended debug actions.
+- **Counterfactual Diagnosis**: Supports facet removal/swap probes to estimate whether persona behavior changes in the expected direction.
 - **Observer Pattern**: Comprehensive tracing of epistemic events for transparency and debugging.
-- **Modular Architecture**: Flexible, extensible components that can be integrated into various agent architectures.
+- **Public Benchmarks**: Deterministic benchmark harnesses for root-cause diagnosis, facet-frame faithfulness, and the AEF audit service.
 
 ## Installation
 
@@ -27,6 +28,16 @@ npm run build
 
 ## Getting Started
 
+Run the public benchmarks:
+
+```bash
+npx ts-node --transpile-only examples/root-cause-benchmark.ts --seed=42 --cases=160
+npx ts-node --transpile-only examples/facet-frame-faithfulness-benchmark.ts --seed=42 --cases=200
+npx ts-node --transpile-only examples/facet-faithfulness-audit-experiment.ts
+```
+
+See `benchmarks/README.md` for result schemas and current reference results.
+
 Clone the repository and run the Debate Simulation using ts-node:
 
 ### macOS / Linux

benchmarks/README.md

@@ -0,0 +1,117 @@
+# AEF Public Benchmarks
+
+This directory documents the reproducible benchmark harnesses included with the Agentic Epistemology Framework.
+
+The benchmark code lives in `examples/` so it can run directly against the TypeScript source without a package publish step.
+
+## Benchmarks
+
+### 1. Generic root-cause benchmark
+
+Tests whether structured epistemic traces recover injected agent failure causes.
+
+```bash
+npx ts-node --transpile-only examples/root-cause-benchmark.ts --seed=42 --cases=160
+```
+
+Outputs:
+
+```text
+results/root-cause-benchmark/
+├── cases.json
+├── metrics.json
+├── metrics.csv
+├── metrics-table.tex
+├── confusion-full.json
+├── confusion-event-only.json
+├── qualitative-example.json
+└── accuracy-by-condition.svg
+```
+
+Primary metric: diagnosis accuracy by trace condition.
+
+### 2. Facet-frame faithfulness benchmark
+
+Tests whether facet-aware traces improve diagnosis of synthetic persona failures such as facet omission, facet misweighting, facet conflict, persona drift, and response-style violation.
+
+```bash
+npx ts-node --transpile-only examples/facet-frame-faithfulness-benchmark.ts --seed=42 --cases=200
+```
+
+Outputs:
+
+```text
+results/facet-frame-faithfulness/
+├── cases.json
+├── metrics.json
+├── metrics.csv
+├── metrics-table.tex
+├── qualitative-example.json
+└── accuracy-by-condition.svg
+```
+
+Primary metric: diagnosis accuracy and macro F1 by trace condition.
+
+### 3. Facet faithfulness audit service experiment
+
+Tests the implemented `FacetFaithfulnessAuditService` on Vurvey-style persona cases with prompt provenance, marker coverage, contradictions, counterfactual sensitivity, calibration, and signal-to-noise scoring.
+
+```bash
+npx ts-node --transpile-only examples/facet-faithfulness-audit-experiment.ts
+```
+
+Outputs:
+
+```text
+results/facet-faithfulness-audit-experiment/
+├── cases.json
+├── audits.json
+├── metrics.json
+├── metrics-table.tex
+└── diagnosis-accuracy.svg
+```
+
+Primary metric: diagnosis accuracy by evidence condition.
+
+## Reproducibility notes
+
+- All public benchmark inputs are synthetic fixtures.
+- The harnesses require no Vurvey credentials, no private database, and no model API calls.
+- The Vurvey real-generation counterfactual results included in the paper are reported only as sanitized aggregates under `results/vurvey-*`; raw persona text and raw database fixture rows are intentionally excluded.
+
+## Current reference results
+
+### Generic root-cause benchmark
+
+| Condition | Accuracy | Macro F1 | Coverage |
+|---|---:|---:|---:|
+| full | 1.000 | 1.000 | 1.000 |
+| no_frame | 0.750 | 0.750 | 0.750 |
+| no_justification | 0.500 | 0.500 | 0.500 |
+| event_only | 0.000 | 0.000 | 0.000 |
+
+### Facet-frame faithfulness benchmark
+
+| Condition | Accuracy | Macro F1 | Coverage |
+|---|---:|---:|---:|
+| full | 0.665 | 0.680 | 0.725 |
+| no_facet_weights | 0.465 | 0.466 | 0.595 |
+| no_facet_prompts | 0.200 | 0.200 | 0.200 |
+| event_only | 0.000 | 0.000 | 0.000 |
+
+### Facet faithfulness audit service experiment
+
+| Condition | Accuracy | Faithfulness | SNR |
+|---|---:|---:|---:|
+| prompt_only | 0.125 | 0.162 | 0.17 |
+| facet_list | 0.125 | 0.382 | 0.83 |
+| facet_weights | 0.750 | 0.526 | 2.20 |
+| full_aef | 1.000 | 0.536 | 4.11 |
+
+## How to add a new benchmark
+
+1. Add a new `examples/<benchmark-name>.ts` runner.
+2. Make it deterministic by accepting `--seed`.
+3. Write outputs under `results/<benchmark-name>/`.
+4. Include at least `cases.json`, `metrics.json`, and a qualitative example.
+5. Add a short section to this README with the command and expected outputs.