Skip to content

test: add validator decision contract coverage - #927

Open
blackdragoon26 wants to merge 1 commit into
kubeedge:mainfrom
blackdragoon26:test/validator-contracts
Open

blackdragoon26 wants to merge 1 commit into
kubeedge:mainfrom
blackdragoon26:test/validator-contracts

Conversation

@blackdragoon26

Copy link
Copy Markdown

Hi maintainers,

This adds a focused, dependency-light contract suite for the example validator and its CI decision policy.

The change covers:

  • inventory filtering, target selection, empty selections, and POSIX/Windows path normalization;
  • PASS, FAIL, ERROR, WARNING, and SKIP aggregation, including executed-versus-skipped semantics;
  • base/head classification for new, pre-existing, fixed, warning-only, and skipped results;
  • malformed, missing, legacy, and incompatible result artifacts through a versioned result contract;
  • explicit mock-runtime labeling and base/head workflow artifact isolation; and
  • a normal pull-request workflow that runs without external APIs, model downloads, datasets, GPUs, or specialized hardware.

Verification:

python -m unittest discover -s .github/workflows/validator/tests -p 'test_*.py' -v
Ran 28 tests: OK

python -m compileall -q .github/workflows/validator

I also exercised static result generation, report aggregation, and identical base/head regression comparison end to end. The generated artifact used schema version 1, recorded execution, and remained non-blocking for an unchanged comparison.

Closes #838

@kubeedge-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: blackdragoon26
To complete the pull request process, please assign moorezheng after the PR has been reviewed.
You can assign the PR to them by writing /assign @moorezheng in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubeedge-bot kubeedge-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Aug 25, 2026
Signed-off-by: blackdragoon26 <sankalp.jha9643@gmail.com>
@blackdragoon26
blackdragoon26 force-pushed the test/validator-contracts branch from 96934bc to 6616863 Compare August 25, 2026 17:22
@blackdragoon26

blackdragoon26 commented Sep 16, 2026

Copy link
Copy Markdown
Author

Pinging for the review, thanks.
@hsj576 @MooreZheng

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Phase IV] Add contract tests for the example validator and CI decision policy

2 participants