Skip to content

Change "greater than" to "greater than or equal to" in results validation to handle problem subtests#59

Merged
malucius-rh merged 1 commit into
mainfrom
allow_zero_as_error
Jun 24, 2026
Merged

Change "greater than" to "greater than or equal to" in results validation to handle problem subtests#59
malucius-rh merged 1 commit into
mainfrom
allow_zero_as_error

Conversation

@malucius-rh

Copy link
Copy Markdown
Contributor

Description

This allows the validator to consider a zero result for a subtest as "valid but error", allowing the "good" results to clear validation and be saved off. One example is if a system doesn't support the CPU_avx512 test a zero is returned, which is a valid "something went wrong here but everyone else is okay" result.

Before/After Comparison

Before:
If one of the subtests returns a zero reflecting an error in one of the subtests, the validation rejects the entire run
Sample:
CPU_fma,144077.57151551355,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx,118863.51315704771,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx512,0,2026-06-08T15:37:26Z,2026-06-08T15:40:29ZCPU_fma,144077.57151551355,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx,118863.51315704771,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx512,0,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_SHA,219491544665.70303,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_AES,141409149289.58951,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_SHA,219491544665.70303,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_AES,141409149289.58951,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z

1 validation error for list[Passmark_Results]
13.Operations
Input should be greater than 0 [type=greater_than, input_value=0.0, input_type=float]

After:
After changing the greater_than to greater_equal the validation returns:
Results verified
updating: results_passmark_throughput-performance.tar (deflated 96%)

and the full results are supported.

Clerical Stuff

This closes #58
Relates to JIRA: RPOPC-1276

This allows the validator to consider a zero result for a subtest as "valid but error", allowing the "good" results to clear validation and be saved off.
@malucius-rh malucius-rh requested a review from dvalinrh June 10, 2026 12:26
@malucius-rh malucius-rh self-assigned this Jun 10, 2026
@malucius-rh malucius-rh added the bug Something isn't working label Jun 10, 2026
@github-actions

Copy link
Copy Markdown

This relates to RPOPC-1276

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: b791dd03-5ad5-4345-acd4-b98b084f9d08

📥 Commits

Reviewing files that changed from the base of the PR and between 374ccc3 and db6fea9.

📒 Files selected for processing (1)
  • passmark/results_schema.py

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Updated validation to accept zero values for Operations field, where previously only positive values were permitted.

Walkthrough

The Operations field validation in Passmark_Results is relaxed from requiring values strictly greater than 0 to accepting values greater than or equal to 0. This permits zero values to represent unsupported subtests while maintaining rejection of infinite and NaN values.

Changes

Operations Field Validation Update

Layer / File(s) Summary
Operations field constraint update
passmark/results_schema.py
Passmark_Results.Operations field constraint relaxed from gt=0 to ge=0, allowing zero values for unsupported subtests while still rejecting infinite/NaN values.

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: relaxing the Operations field validation from 'greater than 0' to 'greater than or equal to 0' to handle problem subtests.
Description check ✅ Passed The description covers all required template sections: summary of changes, before/after comparison with examples, and clerical information linking to issue #58 and JIRA ticket RPOPC-1276.
Linked Issues check ✅ Passed The code change directly addresses issue #58 by modifying the Operations field constraint from gt=0 to ge=0, enabling zero results to be treated as valid errors for unsupported subtests.
Out of Scope Changes check ✅ Passed The change is narrowly scoped to the specific requirement: only the Operations field constraint in Passmark_Results was modified; all other validation behavior remains unchanged.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@dvalinrh dvalinrh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dvalinrh dvalinrh added the group_review_lgtm Indicates approval after a group review meeting label Jun 23, 2026
@malucius-rh malucius-rh merged commit 8cf1ea7 into main Jun 24, 2026
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working group_review_lgtm Indicates approval after a group review meeting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validation doesn't handle "0 as 'valid but error'" in results

2 participants