Skip to content

Add Stateful API testing for Model Registry#132

Merged
dbasunag merged 15 commits intoopendatahub-io:mainfrom
lugi0:feature/stateful_api_testing
Apr 11, 2025
Merged

Add Stateful API testing for Model Registry#132
dbasunag merged 15 commits intoopendatahub-io:mainfrom
lugi0:feature/stateful_api_testing

Conversation

@lugi0
Copy link
Copy Markdown
Contributor

@lugi0 lugi0 commented Feb 7, 2025

Description

This adds a test for stateful API testing of Model Registry. It also bumps Schemathesis to V4, which is still an alpha release but appears to work fine for this workload.
I will keep this as a draft until the openAPI spec is updated in the Model Registry repo to support Stateful testing, and we can then decide if we are ok with an alpha release being pulled or if we want to wait / downgrade to V3 (requires some rework due to breaking changes between the two major versions).
I have also added a new mark type, "fuzzer", which I'd use to indicate tests which are bound to fail due to their random nature.

How Has This Been Tested?

Multiple runs made from local with an updated openAPI spec file which have already identified some failures to address in Model Registry.

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

Signed-off-by: lugi0 <lgiorgi@redhat.com>
@lugi0 lugi0 self-assigned this Feb 7, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 7, 2025

The following are automatically added/executed:

Available user actions:

  • To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
  • To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
  • To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
    lgtm label removed on each new commit push.
  • To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
    verified label removed on each new commit push.
Supported labels

{'/wip', '/lgtm', '/hold', '/verified'}

@lugi0
Copy link
Copy Markdown
Contributor Author

lugi0 commented Feb 7, 2025

On hold until kubeflow/model-registry#773 is merged

lugi0 added 3 commits April 8, 2025 17:40
Signed-off-by: lugi0 <lgiorgi@redhat.com>
Signed-off-by: lugi0 <lgiorgi@redhat.com>
@github-actions github-actions bot added size/l and removed size/xl labels Apr 8, 2025
@lugi0 lugi0 marked this pull request as ready for review April 8, 2025 15:44
@lugi0 lugi0 requested a review from a team as a code owner April 8, 2025 15:44
@lugi0
Copy link
Copy Markdown
Contributor Author

lugi0 commented Apr 8, 2025

/hold

@lugi0
Copy link
Copy Markdown
Contributor Author

lugi0 commented Apr 8, 2025

The test now works as we've updated the openAPI spec file in upstream, however I sometimes get a library error (hypothesis.errors.FlakyStrategyDefinition) I'm still trying to figure out. Opening the PR up for review but keeping it on hold until I get to the root case of the error

Signed-off-by: lugi0 <lgiorgi@redhat.com>
adolfo-ab
adolfo-ab previously approved these changes Apr 9, 2025
Signed-off-by: lugi0 <lgiorgi@redhat.com>
Signed-off-by: lugi0 <lgiorgi@redhat.com>
@github-actions github-actions bot added size/xxl and removed size/l labels Apr 11, 2025
Signed-off-by: lugi0 <lgiorgi@redhat.com>
@github-actions github-actions bot added size/l and removed size/xxl labels Apr 11, 2025
dbasunag
dbasunag previously approved these changes Apr 11, 2025
Signed-off-by: lugi0 <lgiorgi@redhat.com>
@lugi0
Copy link
Copy Markdown
Contributor Author

lugi0 commented Apr 11, 2025

/verified

@opendatahub-tests-bot opendatahub-tests-bot added the Verified Verified pr in Jenkins label Apr 11, 2025
Copy link
Copy Markdown
Contributor

@fege fege left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@dbasunag dbasunag merged commit 69a5123 into opendatahub-io:main Apr 11, 2025
7 checks passed
@github-actions
Copy link
Copy Markdown

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants