Skip to content

feat: behavioral drift detection (agent-strace drift)#73

Merged
Siddhant-K-code merged 1 commit into
mainfrom
feat/drift-detection
May 17, 2026
Merged

feat: behavioral drift detection (agent-strace drift)#73
Siddhant-K-code merged 1 commit into
mainfrom
feat/drift-detection

Conversation

@Siddhant-K-code
Copy link
Copy Markdown
Owner

What

Adds agent-strace drift — a command that detects when agent behavior has shifted across sessions, without requiring an LLM.

Closes #70

How it works

Computes a behavioral fingerprint for a window of sessions: a compact JSON summary of six behavioral dimensions. Then measures how much the current window's fingerprint has diverged from a baseline using Jensen-Shannon divergence.

Six dimensions tracked:

Dimension What it measures
tool_mix Fraction of calls per tool type (Bash, Read, Write, etc.)
error_rate Errors per session (mean, p50, p95)
retry_rate Consecutive same-tool calls per session
blast_radius Distinct files written per session
session_duration_s Wall-clock time per session
decision_depth log_decision() events per session

Overall drift score: weighted average of per-dimension JSD, normalized to [0, 1]. tool_mix is weighted 2x (most sensitive signal).

Usage

# Detect drift over the last 30 days (splits window in half automatically)
agent-strace drift --since 30d

# Compare against a saved baseline
agent-strace drift --baseline .agent-traces/baselines/behavior-main.json

# Save current fingerprint as baseline
agent-strace drift --since 30d --save-baseline .agent-traces/baselines/behavior-main.json

# JSON output for CI
agent-strace drift --since 30d --format json

Exits non-zero when drift score exceeds --threshold (default: 0.20).

Files changed

  • src/agent_trace/drift.py — new module: BehavioralFingerprint, DriftReport, compute_fingerprint(), compute_drift(), print_report(), cmd_drift()
  • src/agent_trace/cli.pydrift subcommand wired in
  • tests/test_drift.py — 27 tests covering statistical helpers, fingerprint computation, drift scoring, serialization, and end-to-end store integration

Test results

Ran 593 tests in 3.4s — OK (27 new)

Adds agent-strace drift — detects when agent behavior shifts across
sessions using Jensen-Shannon divergence across six dimensions:
tool mix, error rate, retry pattern, blast radius, session duration,
and decision depth.

No LLM required. All analysis is structural. Behavioral fingerprints
are compact JSON files (<2KB) that can be committed as baselines.

- drift.py: BehavioralFingerprint, DriftReport, compute_fingerprint(),
  compute_drift(), print_report(), cmd_drift()
- cli.py: drift subcommand wired in with --since, --baseline,
  --current, --save-baseline, --threshold, --format flags
- 27 new tests covering statistical helpers, fingerprint computation,
  drift scoring, serialization, and end-to-end store integration

Closes #70

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code Siddhant-K-code merged commit b97d8c5 into main May 17, 2026
4 checks passed
@Siddhant-K-code Siddhant-K-code deleted the feat/drift-detection branch May 17, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: behavioral drift detection — alert when agent patterns shift across sessions (agent-strace drift)

1 participant