Skip to content

fix(scorers): reject non-finite normalized scores #38

Description

@knisar

Problem

ScoreNormalizer.from_scale() currently allows non-finite values. In particular, NaN reaches the clamping logic and becomes 1.0, so malformed judge output can turn into a passing score.

Scope

Validate numeric inputs at the shared normalization boundary.

Acceptance criteria

  • Reject NaN, positive infinity, and negative infinity.
  • Reject a non-finite max_value.
  • Preserve the existing clamping and inversion behavior for finite values.
  • A mocked judge response of "NaN" cannot produce an assessed pass through the core pipeline.

Non-goals

Likely files

  • rai_toolkit/scorers/normalizer.py
  • tests/test_score_normalizer.py

Testing

Tests must be deterministic and make no API calls.

Starting work

Before starting, check this issue's Development section and the open pull requests for linked work. Follow CONTRIBUTING.md by opening a draft pull request with Closes #38, your proposed validation behavior, and a short test plan. Keep the change limited to normalization and its regression tests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is neededstatus: availableMaintainer-scoped and ready for contribution. Check for linked open pull requests before starting.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions