Skip to content

[NA] [SDK] test: add unit test coverage for IsJson metric - #7879

Open
hasnain1241 wants to merge 3 commits into
comet-ml:mainfrom
hasnain1241:hasnain1241/issue-7878-add-isjson-tests
Open

[NA] [SDK] test: add unit test coverage for IsJson metric#7879
hasnain1241 wants to merge 3 commits into
comet-ml:mainfrom
hasnain1241:hasnain1241/issue-7878-add-isjson-tests

Conversation

@hasnain1241

Copy link
Copy Markdown

Details

Adds unit test coverage for the IsJson heuristic metric (sdks/python/src/opik/evaluation/metrics/heuristics/is_json.py), which previously had none. Covers valid JSON (object, array, and primitive types like number/string/boolean/null), invalid JSON (plain text, empty string, malformed JSON), and non-string input (None, int), documenting the metric's current behavior on each. is_json.py itself was not modified — this PR is test-only.

Change checklist

  • User facing
  • Documentation update

Issues

AI-WATERMARK

AI-WATERMARK: yes

  • Tools: Claude Code
  • Model(s): Claude
  • Scope: Drafted the test cases and test file structure, following existing conventions in test_heuristics.py / test_sentiment.py.
  • Human verification: Reviewed the metric's source code and existing test patterns manually before requesting tests; ran the full test suite locally and verified all 11 tests pass before submission.

Testing

Ran the new test file locally in a virtual environment:
pytest sdks/python/tests/unit/evaluation/metrics/test_is_json.py -v

All 11 tests passed. Scenarios validated:

  • Valid JSON: object, array, number, string, boolean, null → all score 1.0
  • Invalid JSON: plain text, empty string, malformed JSON, None input, int input → all score 0.0

No regressions expected since is_json.py was not modified. Also ran make precommit (ruff, ruff-format, trailing whitespace, end-of-file checks) — all passed on the new file.

Documentation

N/A — this PR only adds test coverage, no user-facing behavior or docs changed.

@github-actions github-actions Bot added python Pull requests that update Python code tests Including test files, or tests related like configuration. Python SDK labels Aug 18, 2026
@@ -0,0 +1,41 @@
import pytest

from opik.evaluation.metrics.heuristics.is_json import IsJson

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test bypasses public IsJson API

IsJson is imported from the internal opik.evaluation.metrics.heuristics.is_json, so this test can pass while the documented opik.evaluation.metrics export is broken — should we import IsJson from opik.evaluation.metrics instead, as .agents/skills/python-sdk/testing.md requires?

Severity

Want Baz to fix this for you? Activate Fixer You can also update your AI coding guidelines based on this comment by apply pr to [branch name]

Other fix methods

Fix in Cursor

Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`sdks/python/tests/unit/evaluation/metrics/test_is_json.py` around line 3, update the
`IsJson` import used by these JSON metric tests to come from the public
`opik.evaluation.metrics` package rather than the internal `heuristics.is_json` module.
Keep the existing test cases unchanged so collection and execution verify that the
documented package-level export works correctly.

@hasnain1241

Copy link
Copy Markdown
Author

I've reviewed the CLA (CLA.md) and agree to its terms for this contribution.

@hasnain1241
hasnain1241 marked this pull request as ready for review August 18, 2026 05:28
@hasnain1241
hasnain1241 requested a review from a team as a code owner August 18, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

baz: pending Python SDK python Pull requests that update Python code 🟢 size/S tests Including test files, or tests related like configuration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FR]: [SDK] Add missing unit test coverage for IsJson metric

1 participant