Commit 67cae86
authored
ops(e2e): enable IS-060 datamarking in shadow mode for nightly validation (#216)
Stage 1 of the IS-060 PR-2 validation chain. The datamarking transform
landed in PR #214 with `enabled: false` defaults — this PR turns it
ON in shadow mode for the nightly e2e harness (config-e2e-judge.yaml,
the canonical proxy config for both PR-gate and scheduled nightly).
Configuration applied:
- boundary_defense.enabled: true (parent — datamarking depends on zone wrapping)
- boundary_defense.randomize_nonce: true (defense in depth)
- boundary_defense.datamarking.enabled: true
- boundary_defense.datamarking.shadow_mode: true (compute+metrics, forward original)
- boundary_defense.datamarking.marker_strategy: randomized
Shadow mode means: the transform runs, emits the four
`llmtrace_spotlighting_*_total` counters, and emits the audit-trail
`spotlighting_applied` finding — but the bytes forwarded upstream are
unchanged. This isolates the "transform runs without breaking the
proxy / upstream API" signal from the "transform changes the model's
behaviour" signal.
Acceptance criteria for this stage (validated post-merge):
- Existing E2E PR Gate + E2E Tests stay green (no regression with
shadow mode active).
- The nightly's `Run full corpus` step completes without 4xx-rate
shift vs prior nightly (`e2e_2026-05-15.json` is the comparison
baseline).
- `llmtrace_spotlighting_zones_total{shadow=\"true\"}` > 0 in the run.
- `llmtrace_spotlighting_byte_delta_total{shadow=\"true\"}` > 0.
- `llmtrace_spotlighting_failures_total` == 0.
If all four hold after one nightly cycle, a follow-up ops PR flips
shadow_mode to false (active mode) and begins the 3-5 night ASR
delta collection per IS-060 PR-2's acceptance criteria.
Refs: #214 (PR-2 implementation), #213 (PR-3 corpus expansion),
docs/architecture/SPOTLIGHTING_INDIRECT_INJECTION.md §6.2 + §6.4
(evidence requirements + baseline).1 parent 0069991 commit 67cae86
1 file changed
Lines changed: 19 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
57 | 76 | | |
58 | 77 | | |
59 | 78 | | |
| |||
0 commit comments