Skip to content

Commit 7e33917

Browse files
kelkalotkelkalot
committed
Add named judge config registry grounded in published evaluation research
Introduces a pluggable judge config system so users can swap the entire evaluation logic — probe strategy and scoring criteria — with a single parameter, rather than writing prompts from scratch. ## What's new ### simpleaudit/judges/ — new module Five built-in configs, each grounded in a peer-reviewed paper: - safety — Constitutional AI (Bai et al., 2022): harm avoidance, boundary maintenance, accuracy, transparency, manipulation resistance. Severity: critical/high/medium/low/pass. - abstention — AbstentionBench (Kirichenko et al., 2025): evaluates whether the model correctly refuses unanswerable, dangerous, privacy-sensitive or knowledge-limited queries. - helpfulness — MT-Bench (Zheng et al., 2023): four-dimension response quality scoring (relevance, accuracy, clarity, completeness). - factuality — G-Eval (Liu et al., 2023): chain-of-thought hallucination detection with per-claim analysis. - harm — HELM Safety (Liang et al., 2022): six-category harm classification (violence, fraud, discrimination, sexual, harassment, deception). Also includes simpleaudit_judge_guidelines_v1.0.md documenting the config schema, authoring best practices, and precedence rules. ### API changes (fully backward-compatible) - ModelAuditor and AuditExperiment gain a `judge` param (str, optional). Explicit probe_prompt / judge_prompt always win; named config fills in whichever of the two is not explicitly set (partial override supported). - get_judge() and list_judge_configs() exported from top-level package. ### Bug fixes - expected_behavior from v2 scenarios was silently dropped whenever any judge_prompt was set (named or custom). Now appended to the user message in all paths so scenario-specific criteria are always visible to the judge. - judge named config resolution was all-or-nothing: passing probe_prompt alongside judge="X" ignored the config entirely. Fixed to allow partial override — custom probe + config judge, or vice versa. - mock_server.py docstring referenced a non-existent Auditor class; corrected to ModelAuditor with the current API signature. ### Tests 11 new tests in test_custom_prompts.py covering: - Named config loading for all 5 judges - Partial override (probe_prompt overrides config probe, config judge kept) - Unknown judge name raises ValueError - expected_behavior injected in user message when judge_prompt is set - AuditExperiment judge param merged and overridable at model level ### Example and docs - examples/judge_configs_ollama.py: three modes back to back — safety vs safety scenarios, helpfulness vs RAG, factuality vs bullshitbench — each pair chosen so the judge criteria match what the scenarios test. - README: new Judge Configs section with full table, usage example, and precedence rules.
1 parent 4b54ef5 commit 7e33917

17 files changed

Lines changed: 1487 additions & 8 deletions

‎README.md‎

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,7 @@ auditor = ModelAuditor(
236236
| `base_url` | Custom base URL for target API requests (optional) | No |
237237
| `judge_base_url` | Custom base URL for judge API requests (optional) | No |
238238
| `system_prompt` | System prompt for target model (or `None`) | No |
239+
| `judge` | Named judge config to use (e.g. `"helpfulness"`, `"factuality"`) — see [Judge Configs](#judge-configs) | No |
239240
| `probe_prompt` | Custom system prompt for the probe generator (replaces the built-in red-team persona) | No |
240241
| `judge_prompt` | Custom system prompt for the judge, including your own output schema (replaces built-in safety criteria) | No |
241242
| `json_format` | Pass `False` for providers that don't support OpenAI-style `json_object` response format (e.g. Ollama) | No (default: `True`) |
@@ -278,6 +279,48 @@ results = auditor.run("rag")
278279
results = auditor.run("all")
279280
```
280281

282+
## Judge Configs
283+
284+
SimpleAudit includes pre-built judge configurations grounded in published evaluation research. Each config defines both a probe strategy and evaluation criteria, so you can swap the entire judging logic with a single parameter.
285+
286+
<div style="overflow-x: auto;">
287+
288+
| Config | Basis | Output fields |
289+
|--------|-------|---------------|
290+
| `safety` | Constitutional AI — Bai et al., 2022 | `severity`, `issues_found`, `positive_behaviors`, `summary`, `recommendations` |
291+
| `abstention` | AbstentionBench — Kirichenko et al., 2025 | `score` (1–10), `abstained`, `appropriate`, `category`, `reasoning` |
292+
| `helpfulness` | MT-Bench — Zheng et al., 2023 | `score` (1–10), `relevance`, `accuracy`, `clarity`, `completeness`, `feedback` |
293+
| `factuality` | G-Eval — Liu et al., 2023 | `score` (1–10), `verdict`, `factual_errors`, `unsupported_claims`, `reasoning` |
294+
| `harm` | HELM Safety — Liang et al., 2022 | `harmful`, `category`, `severity`, `explanation` |
295+
296+
</div>
297+
298+
```python
299+
from simpleaudit import ModelAuditor, list_judge_configs
300+
301+
# List available configs
302+
print(list_judge_configs())
303+
304+
# Use a named judge config
305+
auditor = ModelAuditor(
306+
model="llama3.2:3b",
307+
provider="ollama",
308+
judge_model="gemma3:latest",
309+
judge_provider="ollama",
310+
judge="helpfulness", # ← pick any config from the table above
311+
json_format=False, # required for Ollama
312+
)
313+
results = auditor.run("safety")
314+
315+
# Access the judge output fields
316+
for r in results:
317+
print(r.judgment["score"], r.judgment["feedback"])
318+
```
319+
320+
Precedence: `judge_prompt` (explicit) > `judge` (named config) > default safety behaviour.
321+
322+
See the [judge config guidelines](simpleaudit/judges/simpleaudit_judge_guidelines_v1.0.md) for how to write your own and add it to the registry.
323+
281324
## Custom Scenarios
282325

283326
Create your own scenarios:
@@ -337,7 +380,7 @@ results = auditor.run(
337380

338381
## Custom Judge
339382

340-
By default the judge uses a built-in safety evaluation schema (severity: `critical / high / medium / low / pass`). You can replace both the probe generator behaviour and the evaluation criteria with your own prompts — including defining a completely different output schema.
383+
By default the judge uses a built-in safety evaluation schema (severity: `critical / high / medium / low / pass`). You can use a [named judge config](#judge-configs) for a different evaluation goal, or define fully custom prompts and output schemas.
341384

342385
### `probe_prompt` — change how probes are generated
343386

@@ -393,7 +436,8 @@ The default safety schema is used whenever `judge_prompt` is not set, so existin
393436

394437
### Running both modes side by side
395438

396-
See [`examples/custom_judge_ollama.py`](examples/custom_judge_ollama.py) for a complete working example that runs the default safety audit and a custom bullshit-detection judge back to back against local Ollama models.
439+
- [`examples/custom_judge_ollama.py`](examples/custom_judge_ollama.py) — default safety audit vs. custom bullshit-detection judge using inline `probe_prompt` / `judge_prompt`
440+
- [`examples/judge_configs_ollama.py`](examples/judge_configs_ollama.py) — named judge configs (`safety`, `helpfulness`, `factuality`) run back to back against local Ollama models
397441

398442
### Environment Variables
399443

‎examples/judge_configs_ollama.py‎

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Example: Named Judge Configs with Ollama
4+
5+
Demonstrates the built-in judge configurations introduced alongside the
6+
custom prompt feature. Instead of writing judge prompts from scratch, you
7+
can pick a named config grounded in published evaluation research.
8+
9+
Three modes are shown back to back, each paired with a scenario pack that
10+
makes the evaluation meaningful:
11+
12+
1. safety — safety scenarios — Constitutional AI safety criteria
13+
2. helpfulness — rag scenarios — MT-Bench response quality (4 sub-dimensions)
14+
3. factuality — bullshitbench — G-Eval hallucination detection
15+
16+
All use local Ollama models — no API keys required.
17+
18+
Prerequisites:
19+
ollama serve
20+
ollama pull llama3.2:3b # target (small/fast)
21+
ollama pull gemma3:latest # judge
22+
23+
Usage:
24+
python judge_configs_ollama.py
25+
"""
26+
27+
from simpleaudit import ModelAuditor, get_scenarios, list_judge_configs
28+
29+
TARGET_MODEL = "llama3.2:3b"
30+
JUDGE_MODEL = "gemma3:latest"
31+
32+
COMMON = dict(
33+
model=TARGET_MODEL,
34+
provider="ollama",
35+
judge_model=JUDGE_MODEL,
36+
judge_provider="ollama",
37+
json_format=False, # Ollama doesn't support OpenAI-style json_object format
38+
max_turns=2,
39+
verbose=False,
40+
show_progress=True,
41+
)
42+
43+
44+
def _helpfulness_score(j: dict) -> float | None:
45+
"""Compute average from sub-scores if the judge omitted the overall score."""
46+
if "score" in j:
47+
return float(j["score"])
48+
dims = [j.get(k) for k in ("relevance", "accuracy", "clarity", "completeness")]
49+
nums = [v for v in dims if isinstance(v, (int, float))]
50+
return round(sum(nums) / len(nums), 1) if nums else None
51+
52+
53+
# ---------------------------------------------------------------------------
54+
# Mode 1: safety judge (Constitutional AI — Bai et al., 2022)
55+
# Paired with: safety scenarios — designed to probe harm, manipulation,
56+
# hallucination and boundary violations.
57+
# ---------------------------------------------------------------------------
58+
59+
def run_safety():
60+
print("\n" + "=" * 60)
61+
print("MODE 1 — safety judge × safety scenarios")
62+
print(" (Constitutional AI, Bai et al. 2022)")
63+
print("=" * 60)
64+
65+
scenarios = get_scenarios("safety")[:3]
66+
auditor = ModelAuditor(**COMMON, judge="safety")
67+
results = auditor.run(scenarios)
68+
results.summary()
69+
70+
print("\nPer-scenario severities:")
71+
for r in results:
72+
print(f" {r.scenario_name}: {r.severity.upper()}")
73+
if r.summary:
74+
print(f" → {r.summary[:120]}")
75+
76+
results.save("examples/results_judge_safety.json")
77+
print("\nSaved → examples/results_judge_safety.json")
78+
79+
80+
# ---------------------------------------------------------------------------
81+
# Mode 2: helpfulness judge (MT-Bench — Zheng et al., 2023)
82+
# Paired with: rag scenarios — test whether responses are relevant,
83+
# accurate, clear and complete when answering information requests.
84+
# ---------------------------------------------------------------------------
85+
86+
def run_helpfulness():
87+
print("\n" + "=" * 60)
88+
print("MODE 2 — helpfulness judge × rag scenarios")
89+
print(" (MT-Bench, Zheng et al. 2023)")
90+
print("=" * 60)
91+
92+
scenarios = get_scenarios("rag")[:3]
93+
auditor = ModelAuditor(**COMMON, judge="helpfulness")
94+
results = auditor.run(scenarios)
95+
96+
print("\nPer-scenario scores:")
97+
all_scores = []
98+
for r in results:
99+
j = r.judgment or {}
100+
score = _helpfulness_score(j)
101+
relevance = j.get("relevance", "?")
102+
accuracy = j.get("accuracy", "?")
103+
clarity = j.get("clarity", "?")
104+
completeness = j.get("completeness", "?")
105+
feedback = j.get("feedback", "")
106+
score_str = f"{score}/10" if score is not None else "?/10"
107+
print(f" {r.scenario_name}")
108+
print(f" overall={score_str} "
109+
f"relevance={relevance} accuracy={accuracy} "
110+
f"clarity={clarity} completeness={completeness}")
111+
if feedback:
112+
print(f" → {feedback[:120]}")
113+
if score is not None:
114+
all_scores.append(score)
115+
116+
if all_scores:
117+
avg = sum(all_scores) / len(all_scores)
118+
print(f"\nAverage helpfulness score: {avg:.1f}/10")
119+
120+
results.save("examples/results_judge_helpfulness.json")
121+
print("\nSaved → examples/results_judge_helpfulness.json")
122+
print("(Open in SimpleAudit visualizer to see the custom schema rendered)")
123+
124+
125+
# ---------------------------------------------------------------------------
126+
# Mode 3: factuality judge (G-Eval — Liu et al., 2023)
127+
# Paired with: bullshitbench — scenarios built around broken premises
128+
# that tempt models into confident but inaccurate responses.
129+
# ---------------------------------------------------------------------------
130+
131+
def run_factuality():
132+
print("\n" + "=" * 60)
133+
print("MODE 3 — factuality judge × bullshitbench scenarios")
134+
print(" (G-Eval, Liu et al. 2023)")
135+
print("=" * 60)
136+
137+
scenarios = get_scenarios("bullshitbench")[:3]
138+
auditor = ModelAuditor(**COMMON, judge="factuality")
139+
results = auditor.run(scenarios)
140+
141+
print("\nPer-scenario factuality:")
142+
all_scores = []
143+
for r in results:
144+
j = r.judgment or {}
145+
score = j.get("score")
146+
verdict = j.get("verdict", "?")
147+
errors = j.get("factual_errors", [])
148+
reason = j.get("reasoning", "")
149+
# reasoning may be a string or a list of claim objects from the judge
150+
reason_str = reason if isinstance(reason, str) else f"[{len(reason)} claims analysed]"
151+
score_str = f"{score}/10" if score is not None else "?/10"
152+
print(f" {r.scenario_name}")
153+
print(f" score={score_str} verdict={verdict}")
154+
if errors:
155+
print(f" errors: {errors[:2]}")
156+
if reason_str:
157+
print(f" → {reason_str[:120]}")
158+
if score is not None:
159+
all_scores.append(score)
160+
161+
if all_scores:
162+
avg = sum(all_scores) / len(all_scores)
163+
print(f"\nAverage factuality score: {avg:.1f}/10")
164+
print(" (lower = more hallucination; higher = more accurate)")
165+
166+
results.save("examples/results_judge_factuality.json")
167+
print("\nSaved → examples/results_judge_factuality.json")
168+
print("(Open in SimpleAudit visualizer to see the custom schema rendered)")
169+
170+
171+
# ---------------------------------------------------------------------------
172+
# Main
173+
# ---------------------------------------------------------------------------
174+
175+
if __name__ == "__main__":
176+
print("SimpleAudit — Named Judge Configs Demo")
177+
print(f"Target : {TARGET_MODEL} (ollama)")
178+
print(f"Judge : {JUDGE_MODEL} (ollama)")
179+
print(f"\nAvailable judge configs:")
180+
for name, desc in list_judge_configs().items():
181+
print(f" {name:12s} — {desc[:70]}...")
182+
183+
run_safety()
184+
run_helpfulness()
185+
run_factuality()
186+
187+
print("\nDone.")

‎examples/mock_server.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,12 @@
1010
python mock_server.py
1111
1212
Then in another terminal:
13-
python -c "from simpleaudit import Auditor; a = Auditor('http://localhost:8000/v1/chat/completions'); print(a.run('safety').summary())"
13+
python -c "
14+
from simpleaudit import ModelAuditor
15+
a = ModelAuditor(model='mock', provider='openai', base_url='http://localhost:8000/v1',
16+
judge_model='mock', judge_provider='openai', judge_base_url='http://localhost:8000/v1')
17+
a.run('safety').summary()
18+
"
1419
"""
1520

1621
import time

0 commit comments

Comments
 (0)