Skip to content

feat: add anchor-drift-monitor skill - real-time intent drift detection for Oz runs#16

Open
HasRahm wants to merge 1 commit into
warpdotdev:mainfrom
HasRahm:main
Open

feat: add anchor-drift-monitor skill - real-time intent drift detection for Oz runs#16
HasRahm wants to merge 1 commit into
warpdotdev:mainfrom
HasRahm:main

Conversation

@HasRahm

@HasRahm HasRahm commented Apr 29, 2026

Copy link
Copy Markdown

What

Adds a new anchor-drift-monitor skill that detects and prevents cumulative intent drift during multi-step Oz agent runs.

Why

Oz agents can experience cumulative intent drift across multi-step tasks. An agent starts with a clear instruction - "refactor the auth module" - and by step 6 it is modifying unrelated files, changing environment variables, or introducing dependencies outside the original scope. This happens because multi-step context causes gradual faithfulness erosion.

How It Works

  1. At the start of a run, the skill creates an intent contract capturing the task + constraints
    1. After each step, it scores the action against the contract using HHEM-2.1-Open
    1. When faithfulness drops below threshold, it pauses the run and reports what drifted

Benchmark Results

Validated on CNN/DailyMail corpus using Gemma 4 31B + HHEM-2.1-Open:

Metric Baseline + Anchor Delta
High-fidelity grounding (HHEM >= 0.95) 47.1% 56.3% +9.2%
Mean HHEM faithfulness score 0.9257 0.9307 +0.005
Hallucination rate (<0.50) 0.8% 0.7% -0.1%

3:1 improvement ratio - interventions are grounded, not random.
0% regression on IFEval (instruction following) and Tau2 (task completion).

Setup

Requires ANCHOR_KEY environment variable. Free keys at anchor-app-one.vercel.app.

Links

Add documentation for the Anchor Drift Monitor skill, detailing its purpose, usage, prerequisites, workflow, and scoring guidelines.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant