[NA] [SDK] test: add unit test coverage for IsJson metric - #7879
Open
hasnain1241 wants to merge 3 commits into
Open
[NA] [SDK] test: add unit test coverage for IsJson metric#7879hasnain1241 wants to merge 3 commits into
hasnain1241 wants to merge 3 commits into
Conversation
| @@ -0,0 +1,41 @@ | |||
| import pytest | |||
|
|
|||
| from opik.evaluation.metrics.heuristics.is_json import IsJson | |||
Contributor
There was a problem hiding this comment.
Test bypasses public IsJson API
IsJson is imported from the internal opik.evaluation.metrics.heuristics.is_json, so this test can pass while the documented opik.evaluation.metrics export is broken — should we import IsJson from opik.evaluation.metrics instead, as .agents/skills/python-sdk/testing.md requires?
Want Baz to fix this for you? Activate Fixer You can also update your AI coding guidelines based on this comment by apply pr to [branch name]
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`sdks/python/tests/unit/evaluation/metrics/test_is_json.py` around line 3, update the
`IsJson` import used by these JSON metric tests to come from the public
`opik.evaluation.metrics` package rather than the internal `heuristics.is_json` module.
Keep the existing test cases unchanged so collection and execution verify that the
documented package-level export works correctly.
Author
|
I've reviewed the CLA (CLA.md) and agree to its terms for this contribution. |
hasnain1241
marked this pull request as ready for review
August 18, 2026 05:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
Adds unit test coverage for the
IsJsonheuristic metric (sdks/python/src/opik/evaluation/metrics/heuristics/is_json.py), which previously had none. Covers valid JSON (object, array, and primitive types like number/string/boolean/null), invalid JSON (plain text, empty string, malformed JSON), and non-string input (None,int), documenting the metric's current behavior on each.is_json.pyitself was not modified — this PR is test-only.Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
test_heuristics.py/test_sentiment.py.Testing
Ran the new test file locally in a virtual environment:
pytest sdks/python/tests/unit/evaluation/metrics/test_is_json.py -vAll 11 tests passed. Scenarios validated:
Noneinput,intinput → all score 0.0No regressions expected since
is_json.pywas not modified. Also ranmake precommit(ruff, ruff-format, trailing whitespace, end-of-file checks) — all passed on the new file.Documentation
N/A — this PR only adds test coverage, no user-facing behavior or docs changed.