Add Testsuite for Triton KServe Models (Rest and gRPC) by rpancham · Pull Request #414 · opendatahub-io/opendatahub-tests

rpancham · 2025-07-09T08:12:55Z

Addresses: https://issues.redhat.com/browse/RHOAIENG-21956

…constants) and supporting utils

coderabbitai · 2025-07-09T08:13:04Z

📝 Walkthrough

Summary by CodeRabbit

New Features
- Added comprehensive test suite for Triton model serving, supporting ONNX model inference validation via both REST and gRPC protocols in raw and serverless deployment modes.
- Introduced utilities and fixtures for dynamic test environment setup, including S3 storage access, Kubernetes resource management, and protocol selection.
- Provided reference snapshot files for validating model inference outputs.
Chores
- Added new runtime image option for Triton in test configuration.
- Updated constants to support Triton REST and gRPC runtime templates and AMD GPU identifiers.
Refactor
- Centralized and streamlined S3 secret and protocol fixtures for model serving tests.

Summary by CodeRabbit

New Features
- Added comprehensive test support for Triton model serving with KServe, including fixtures for runtime setup, inference services, and S3 integration.
- Introduced ONNX model inference tests for both REST and gRPC protocols, covering raw and serverless deployment modes.
- Provided utility functions and constants for managing test configurations, S3 secrets, and inference validation.
- Added snapshot files to validate ONNX model inference outputs.
Chores
- Added new pytest command-line option for specifying the Triton runtime image.
- Updated shared constants to support Triton runtime and AMD GPU identifiers.
Refactor
- Centralized S3 secret and protocol fixtures for model serving tests, removing duplicates from MLServer-specific files.

Walkthrough

This update introduces comprehensive support for Triton model serving runtime tests, including new pytest fixtures, utility functions, and constants for configuring and validating inference via REST and gRPC protocols. It adds ONNX model inference tests with snapshot validation, centralizes S3 secret creation for test environments, and refines runtime image configuration options.

Changes

Files / Groups	Change Summary
tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py	Added pytest fixtures and context managers for Triton ServingRuntime and InferenceService setup.
tests/model_serving/model_runtime/triton/basic_model_deployment/test_onnx_model.py	Introduced ONNX model inference tests for Triton via REST and gRPC with snapshot validation.
tests/model_serving/model_runtime/triton/basic_model_deployment/utils.py	Added utility functions for S3 secret management, inference requests, response validation, and helpers.
tests/model_serving/model_runtime/triton/constant.py	Introduced constants for Triton test configuration, paths, ports, templates, and resource specs.
utilities/constants.py	Added Triton runtime template identifiers and AMD GPU label constants; refactored long string constants.
conftest.py	Added `--triton-runtime-image` pytest option for runtime image configuration.
tests/conftest.py	Added `triton_runtime_image` session-scoped fixture for runtime image retrieval.
tests/model_serving/conftest.py	Added fixtures and context manager for S3 secret creation; added `root_dir` and `protocol` fixtures.
tests/model_serving/model_runtime/mlserver/conftest.py tests/model_serving/model_runtime/mlserver/utils.py	Removed S3 secret fixtures/context manager and related imports; centralized in shared conftest.
tests/model_serving/model_runtime/triton/basic_model_deployment/snapshots/test_onnx_model/*	Added JSON snapshot files for ONNX model inference outputs (REST and gRPC, raw and serverless modes).

Suggested labels

size/l

Suggested reviewers

lugi0
mwaykole

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9e57005 and e725e5b.

📒 Files selected for processing (1)

tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: adolfo-ab
PR: opendatahub-io/opendatahub-tests#334
File: tests/model_explainability/trustyai_service/test_trustyai_service.py:52-65
Timestamp: 2025-06-05T10:05:17.642Z
Learning: For TrustyAI image validation tests: operator image tests require admin_client, related_images_refs, and trustyai_operator_configmap fixtures, while service image tests would require different fixtures like trustyai_service_with_pvc_storage, model_namespace, and current_client_token.

Learnt from: israel-hdez
PR: opendatahub-io/opendatahub-tests#346
File: tests/model_serving/model_server/inference_graph/conftest.py:85-92
Timestamp: 2025-06-11T16:40:11.593Z
Learning: The helper `create_isvc` (used in tests/model_serving utilities) already waits until the created InferenceService reports Condition READY=True before returning, so additional readiness waits in fixtures are unnecessary.

tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py (6)

Learnt from: adolfo-ab
PR: opendatahub-io/opendatahub-tests#334
File: tests/model_explainability/trustyai_service/test_trustyai_service.py:52-65
Timestamp: 2025-06-05T10:05:17.642Z
Learning: For TrustyAI image validation tests: operator image tests require admin_client, related_images_refs, and trustyai_operator_configmap fixtures, while service image tests would require different fixtures like trustyai_service_with_pvc_storage, model_namespace, and current_client_token.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#401
File: tests/model_registry/rest_api/mariadb/conftest.py:89-110
Timestamp: 2025-07-04T00:17:47.799Z
Learning: In tests/model_registry/rest_api/mariadb/conftest.py, the model_registry_with_mariadb fixture should always use OAUTH_PROXY_CONFIG_DICT for the oauth_proxy parameter regardless of the is_model_registry_oauth parameter value, based on expected product behavior for MariaDB-backed ModelRegistry instances.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#354
File: tests/model_registry/rbac/conftest.py:166-175
Timestamp: 2025-06-16T11:25:39.599Z
Learning: In tests/model_registry/rbac/conftest.py, predictable names are intentionally used for test resources (like RoleBindings and groups) instead of random names. This design choice prioritizes exposing cleanup failures from previous test runs through name collisions rather than masking such issues with random names. The philosophy is that test failures should be observable and informative to help debug underlying infrastructure or cleanup issues.

Learnt from: israel-hdez
PR: opendatahub-io/opendatahub-tests#346
File: tests/model_serving/model_server/inference_graph/conftest.py:85-92
Timestamp: 2025-06-11T16:40:11.593Z
Learning: The helper `create_isvc` (used in tests/model_serving utilities) already waits until the created InferenceService reports Condition READY=True before returning, so additional readiness waits in fixtures are unnecessary.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#338
File: tests/model_registry/rbac/test_mr_rbac.py:24-53
Timestamp: 2025-06-06T12:22:57.057Z
Learning: In the opendatahub-tests repository, prefer keeping test parameterization configurations inline rather than extracting them to separate variables/constants, as it makes triaging easier by avoiding the need to jump between different parts of the file to understand the test setup.

Learnt from: fege
PR: opendatahub-io/opendatahub-tests#320
File: tests/model_registry/rest_api/conftest.py:200-216
Timestamp: 2025-06-05T14:32:40.247Z
Learning: In the opendatahub-tests repository, the test fixtures should raise exceptions on cleanup failures rather than just logging warnings. The user fege prefers strict cleanup behavior where tests fail if cleanup doesn't work properly, rather than silently continuing.

🔇 Additional comments (9)

tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py (9)

1-38: Imports look comprehensive and well-organized.

The imports are properly structured and include all necessary dependencies for the Triton model serving test fixtures.

41-82: Template fixtures are well-designed with proper separation.

The protocol-specific template fixtures use a clean shared context manager pattern that ensures proper resource cleanup and maintainability.

84-151: ServingRuntime creation function is comprehensive and protocol-aware.

The function properly handles all protocol-specific configurations including volumes, ports, container arguments, and supported model formats. The conditional logic for gRPC vs REST is clean and maintainable.

154-170: ServingRuntime fixture is properly parameterized.

The fixture uses the ServingRuntimeFromTemplate utility correctly and supports flexible parameterization through request.param.

173-219: InferenceService fixture handles complex configurations correctly.

The fixture properly manages GPU resources using copy.deepcopy to avoid side effects, correctly configures multi-GPU volumes, and leverages the create_isvc utility which already handles readiness waiting.

222-232: ServiceAccount fixture is simple and correct.

The fixture properly creates a ServiceAccount linked to the S3 secret for model storage access.

235-237: Snapshot fixture is correctly configured.

The fixture properly configures the JSON snapshot extension for response validation.

240-248: Pod resource fixture has appropriate error handling.

The fixture correctly retrieves pods using the utility function and raises a descriptive ResourceNotFoundError when no pods are found.

1-248: Overall file structure follows pytest best practices.

The conftest.py file is well-organized with appropriate fixture scoping, proper resource management using context managers, and clear separation of concerns between different test setup components.

✨ Finishing Touches

📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

github-actions · 2025-07-09T08:13:18Z

The following are automatically added/executed:

PR size label.
Run pre-commit
Run tox
Add PR author as the PR assignee
Build image based on the PR

Available user actions:

To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
lgtm label removed on each new commit push.
To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
verified label removed on each new commit push.
To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.
To build and push image to quay, add /build-push-pr-image in a comment. This would create an image with tag
pr-<pr_number> to quay repository. This image tag, however would be deleted on PR merge or close action.

Supported labels

{'/lgtm', '/cherry-pick', '/wip', '/build-push-pr-image', '/verified', '/hold'}

coderabbitai

Actionable comments posted: 6

♻️ Duplicate comments (3)

tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-raw-grpc-deployment].json (2)

10-11: Same “shape as string” inconsistency as the other gRPC snapshot

See earlier comment – make the shape element consistently integer or string across snapshots.

14-16: Large base-64 payload – repository size concern

Same comment as for the serverless-grpc snapshot.

tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-serverless-rest-deployment].json (1)

6-1013: Same large-float-array concern as raw-REST snapshot

See previous comment – prefer toleranced checks or trimmed expectations to keep the repo slim and maintenance-friendly.

🧹 Nitpick comments (8)

tests/model_serving/model_runtime/triton/constant.py (2)

77-92: Type annotation is unreadable & wrong

PREDICT_RESOURCES is annotated as
dict[str, Union[list[dict[str, Union[str, dict[str, str]]]], dict[str, dict[str, str]]]],
which doesn’t match the actual structure (volumes, volume_mounts, resources).
Prefer a TypedDict or dataclass, or simply drop the overly complex Union[…] here.

55-58: Hard-coded path prefixes block reuse

MODEL_PATH_PREFIX_* are cluster-specific. Consider moving them to a fixture or env-config so the same test-suite works on non-GPU clusters without code edits.
tests/model_serving/model_runtime/triton/basic_model_deployment/test_onnx_model.py (1)
83-83: Fix class naming convention.

The class name should follow PascalCase convention for better readability.
-class TestdensenetonnxModel:
+class TestDensenetOnnxModel:
tests/model_serving/model_runtime/triton/basic_model_deployment/triton_rest_serving_template.yaml (1)
54-55: Consider enabling autoSelect for PyTorch model format.

For consistency with other model formats, consider enabling autoSelect for PyTorch.
-        - name: pytorch
+        - autoSelect: true
+          name: pytorch
+          priority: 1
           version: "1"
tests/model_serving/model_runtime/triton/basic_model_deployment/triton_grpc_serving_template.yaml (1)
58-59: Consider enabling autoSelect for PyTorch model format.

For consistency with other model formats and the REST template, consider enabling autoSelect for PyTorch.
-        - name: pytorch
+        - autoSelect: true
+          name: pytorch
+          priority: 1
           version: "1"
tests/model_serving/model_runtime/triton/basic_model_deployment/utils.py (1)
83-83: Extract magic number to a constant.

The threshold value for stdin should be defined as a constant for better maintainability.

Add at the top of the file after imports:
GRPC_INLINE_PAYLOAD_THRESHOLD = 8000  # bytes
Then update line 83:
-    use_stdin = len(input_str.encode("utf-8")) > 8000
+    use_stdin = len(input_str.encode("utf-8")) > GRPC_INLINE_PAYLOAD_THRESHOLD
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-serverless-grpc-deployment].json (1)
14-16: Enormous base-64 payload bloats the repository

rawOutputContents stores ~4 MB of base-64 text in every gRPC snapshot.
This balloons the Git history and slows cloning.

Typical approaches:
-  "rawOutputContents": [
-    "<4 MB base-64>"
-  ]
+  "rawOutputContents": "<sha256 digest of tensor>"   // compare against digest at runtime
or store the blob under tests/_artifacts/ and read it on demand.

Consider one of the above to keep the repo lean.
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-raw-rest-deployment].json (1)

6-1013: Snapshot dwarfs source – consider toleranced assertions instead

Storing 1 000 FP32 logits (~8 kB) is fine, but four nearly-identical snapshots multiply quickly.
You can instead:

Assert only the top-%K class indices / probabilities, or

Compare each output with pytest.approx(tol) instead of literal snapshot equality, generating inputs on-the-fly.

Benefits: smaller repo, no manual updates when framework changes minutely affect floats.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e36131c and a6ed983.

📒 Files selected for processing (10)

tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-raw-grpc-deployment].json (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-raw-rest-deployment].json (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-serverless-grpc-deployment].json (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-serverless-rest-deployment].json (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/test_onnx_model.py (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/triton_grpc_serving_template.yaml (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/triton_rest_serving_template.yaml (1 hunks)
tests/model_serving/model_runtime/triton/basic_model_deployment/utils.py (1 hunks)
tests/model_serving/model_runtime/triton/constant.py (1 hunks)

🧰 Additional context used

🧠 Learnings (9)

📓 Common learnings

Learnt from: adolfo-ab
PR: opendatahub-io/opendatahub-tests#334
File: tests/model_explainability/trustyai_service/test_trustyai_service.py:52-65
Timestamp: 2025-06-05T10:05:17.642Z
Learning: For TrustyAI image validation tests: operator image tests require admin_client, related_images_refs, and trustyai_operator_configmap fixtures, while service image tests would require different fixtures like trustyai_service_with_pvc_storage, model_namespace, and current_client_token.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch. This file should be excluded from Buf linting as it's a reference implementation.

tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-serverless-grpc-deployment].json (2)

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch. This file should be excluded from Buf linting as it's a reference implementation.

tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-raw-grpc-deployment].json (2)

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch. This file should be excluded from Buf linting as it's a reference implementation.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

tests/model_serving/model_runtime/triton/basic_model_deployment/test_onnx_model.py (1)

Learnt from: adolfo-ab
PR: opendatahub-io/opendatahub-tests#334
File: tests/model_explainability/trustyai_service/test_trustyai_service.py:52-65
Timestamp: 2025-06-05T10:05:17.642Z
Learning: For TrustyAI image validation tests: operator image tests require admin_client, related_images_refs, and trustyai_operator_configmap fixtures, while service image tests would require different fixtures like trustyai_service_with_pvc_storage, model_namespace, and current_client_token.

tests/model_serving/model_runtime/triton/basic_model_deployment/triton_rest_serving_template.yaml (3)

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch. This file should be excluded from Buf linting as it's a reference implementation.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:15-17
Timestamp: 2025-06-03T06:31:57.214Z
Learning: The file utilities/manifests/common/grpc_predict_v2.proto is from the KServe V2 inference protocol standard and should not be modified to comply with local directory structure conventions.

tests/model_serving/model_runtime/triton/basic_model_deployment/triton_grpc_serving_template.yaml (3)

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch. This file should be excluded from Buf linting as it's a reference implementation.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:15-17
Timestamp: 2025-06-03T06:31:57.214Z
Learning: The file utilities/manifests/common/grpc_predict_v2.proto is from the KServe V2 inference protocol standard and should not be modified to comply with local directory structure conventions.

tests/model_serving/model_runtime/triton/basic_model_deployment/utils.py (3)

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

Learnt from: israel-hdez
PR: opendatahub-io/opendatahub-tests#346
File: tests/model_serving/model_server/inference_graph/conftest.py:85-92
Timestamp: 2025-06-11T16:40:11.593Z
Learning: The helper `create_isvc` (used in tests/model_serving utilities) already waits until the created InferenceService reports Condition READY=True before returning, so additional readiness waits in fixtures are unnecessary.

Learnt from: adolfo-ab
PR: opendatahub-io/opendatahub-tests#334
File: tests/model_explainability/trustyai_service/test_trustyai_service.py:52-65
Timestamp: 2025-06-05T10:05:17.642Z
Learning: For TrustyAI image validation tests: operator image tests require admin_client, related_images_refs, and trustyai_operator_configmap fixtures, while service image tests would require different fixtures like trustyai_service_with_pvc_storage, model_namespace, and current_client_token.

tests/model_serving/model_runtime/triton/constant.py (5)

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#328
File: tests/model_serving/model_runtime/mlserver/constant.py:20-25
Timestamp: 2025-06-16T05:50:12.556Z
Learning: In the MLServer test suite, the PROTO_FILE_PATH constant is intentionally kept as a relative path string and is used together with the root_dir fixture to construct absolute paths at runtime when accessing the grpc_predict_v2.proto file.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#338
File: tests/model_registry/rbac/test_mr_rbac.py:24-53
Timestamp: 2025-06-06T12:22:57.057Z
Learning: In the opendatahub-tests repository, prefer keeping test parameterization configurations inline rather than extracting them to separate variables/constants, as it makes triaging easier by avoiding the need to jump between different parts of the file to understand the test setup.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:15-17
Timestamp: 2025-06-03T06:31:57.214Z
Learning: The file utilities/manifests/common/grpc_predict_v2.proto is from the KServe V2 inference protocol standard and should not be modified to comply with local directory structure conventions.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch.

Learnt from: Snomaan6846
PR: opendatahub-io/opendatahub-tests#325
File: utilities/manifests/common/grpc_predict_v2.proto:16-16
Timestamp: 2025-06-03T06:31:46.614Z
Learning: The grpc_predict_v2.proto file in utilities/manifests/common/ is from the official KServe V2 inference protocol specification and should not be modified. The package declaration "inference" is standardized and should remain unchanged even if it triggers Buf linting errors about package directory mismatch. This file should be excluded from Buf linting as it's a reference implementation.

tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py (6)

Learnt from: adolfo-ab
PR: opendatahub-io/opendatahub-tests#334
File: tests/model_explainability/trustyai_service/test_trustyai_service.py:52-65
Timestamp: 2025-06-05T10:05:17.642Z
Learning: For TrustyAI image validation tests: operator image tests require admin_client, related_images_refs, and trustyai_operator_configmap fixtures, while service image tests would require different fixtures like trustyai_service_with_pvc_storage, model_namespace, and current_client_token.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#354
File: tests/model_registry/rbac/conftest.py:166-175
Timestamp: 2025-06-16T11:25:39.599Z
Learning: In tests/model_registry/rbac/conftest.py, predictable names are intentionally used for test resources (like RoleBindings and groups) instead of random names. This design choice prioritizes exposing cleanup failures from previous test runs through name collisions rather than masking such issues with random names. The philosophy is that test failures should be observable and informative to help debug underlying infrastructure or cleanup issues.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#401
File: tests/model_registry/rest_api/mariadb/conftest.py:89-110
Timestamp: 2025-07-04T00:17:47.799Z
Learning: In tests/model_registry/rest_api/mariadb/conftest.py, the model_registry_with_mariadb fixture should always use OAUTH_PROXY_CONFIG_DICT for the oauth_proxy parameter regardless of the is_model_registry_oauth parameter value, based on expected product behavior for MariaDB-backed ModelRegistry instances.

Learnt from: israel-hdez
PR: opendatahub-io/opendatahub-tests#346
File: tests/model_serving/model_server/inference_graph/conftest.py:85-92
Timestamp: 2025-06-11T16:40:11.593Z
Learning: The helper `create_isvc` (used in tests/model_serving utilities) already waits until the created InferenceService reports Condition READY=True before returning, so additional readiness waits in fixtures are unnecessary.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#338
File: tests/model_registry/rbac/test_mr_rbac.py:24-53
Timestamp: 2025-06-06T12:22:57.057Z
Learning: In the opendatahub-tests repository, prefer keeping test parameterization configurations inline rather than extracting them to separate variables/constants, as it makes triaging easier by avoiding the need to jump between different parts of the file to understand the test setup.

Learnt from: dbasunag
PR: opendatahub-io/opendatahub-tests#354
File: tests/model_registry/rbac/test_mr_rbac.py:64-77
Timestamp: 2025-06-16T11:26:53.789Z
Learning: In Model Registry RBAC tests, client instantiation tests are designed to verify the ability to create and use the MR python client, with actual API functionality testing covered by separate existing tests.

🪛 GitHub Actions: Tox Tests

tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_onnx_model/TestdensenetonnxModel.test_densenetonnx_inference[densenetonnx-serverless-grpc-deployment].json