Skip to content

Avoid duplicate guardrail metric history - #3007

Open
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix/guardrail-tuner-metrics-history
Open

Avoid duplicate guardrail metric history#3007
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix/guardrail-tuner-metrics-history

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

  • Stop appending the current metric snapshot a second time inside ThresholdAdjuster.calculate_adjustment().
  • Keep threshold history, adjustment history, and explicit update_metrics() behavior unchanged.

Motivation

The feedback loop already records every evaluation result through ThresholdAdjuster.update_metrics(): once after the initial eval and once after each subsequent eval. calculate_adjustment() then appends that same GuardrailMetrics object again before changing the threshold.

As a result, metrics_history contains duplicate entries for every metric snapshot that triggers an adjustment. That makes the tuning history inaccurate for diagnostics or future consumers even though the current report's first/last metric lookup happens to mask the duplication.

There is already one explicit method responsible for recording metric observations, so adjustment calculation should not record the same observation again.

Validation

  • initial eval metric -> recorded once by update_metrics()
  • each re-eval metric -> recorded once by update_metrics()
  • threshold/adjustment history -> unchanged
  • convergence and threshold calculation -> unchanged

Self-review

  • One-line deletion in one guardrail-tuner helper.
  • No API, dependency, notebook, registry, or configuration changes.
  • Searched open PRs for an existing metric-history duplication fix and found none.

Maintainers may modify the branch if needed.

@sylvesterkaczmarek
sylvesterkaczmarek requested a review from a team as a code owner August 21, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant