feat: add model car (OCI image) deployment tests#1162
Merged
Raghul-M merged 3 commits intoopendatahub-io:mainfrom Apr 10, 2026
Merged
feat: add model car (OCI image) deployment tests#1162Raghul-M merged 3 commits intoopendatahub-io:mainfrom
Raghul-M merged 3 commits intoopendatahub-io:mainfrom
Conversation
|
The following are automatically added/executed:
Available user actions:
Supported labels{'/lgtm', '/verified', '/cherry-pick', '/wip', '/hold', '/build-push-pr-image'} |
Contributor
Author
|
/hold wait for opendatahub-io/odh-model-controller#703 |
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds MLServer “model car” (OCI image) test support: a class-scoped pytest fixture to create OCI-backed InferenceService instances, a parameterized smoke test exercising sklearn/xgboost/lightgbm/onnx model-car deployments, utility updates to handle model-car storage and deployment types, new constants for KServe deployment and MLServer OCI images, README and snapshot fixtures. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Actionable issues (security-first, direct):
🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py (1)
119-125: Select a ready predictor pod instead of the first listed pod.Line 125 uses
pods[0], which is non-deterministic during restarts/rollouts and can produce flaky inference failures.Proposed fix
pods = get_pods_by_isvc_label( client=mlserver_model_car_inference_service.client, isvc=mlserver_model_car_inference_service, ) if not pods: raise RuntimeError(f"No pods found for InferenceService {mlserver_model_car_inference_service.name}") - pod = pods[0] + ready_pods = [ + p + for p in pods + if any( + cs.ready + for cs in (getattr(p.instance.status, "containerStatuses", None) or []) + ) + ] + if not ready_pods: + raise RuntimeError( + f"No ready pods found for InferenceService {mlserver_model_car_inference_service.name}" + ) + pod = ready_pods[0]As per coding guidelines, "REVIEW PRIORITIES: 3. Bug-prone patterns and error handling gaps".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py` around lines 119 - 125, The test currently picks pods[0] which is non-deterministic; change it to choose a ready predictor pod from the list returned by get_pods_by_isvc_label (mlserver_model_car_inference_service) by filtering pods for readiness (e.g., pod.status.phase == "Running" and pod.status.conditions contains condition type "Ready" == "True" or a container_status with ready==True) and/or label identifying the predictor container, then assign that ready pod to pod; if no ready predictor pod is found, raise a clear RuntimeError stating no ready predictor pod for the InferenceService instead of using pods[0].
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/model_serving/model_runtime/mlserver/conftest.py`:
- Around line 169-173: The test fixture currently allows storage_uri to be None
which delays the failure to create_isvc and obscures the real error; add an
explicit validation right after storage_uri = params.get("storage-uri") to raise
a ValueError (or similar) when storage_uri is missing/empty, mirroring the
existing model_format check so callers fail fast before invoking create_isvc.
- Around line 170-184: The code reads params["deployment-mode"] into
deployment_mode which misses callers using "deployment_type" and can silently
mis-handle Standard deployments; change the lookup to normalize both keys (e.g.,
check "deployment-mode" then "deployment_type" or map/alias them) before passing
deployment_mode into create_isvc, and change the wait_for_predictor_pods default
in the create_isvc call to a deterministic True (use
params.get("wait_for_predictor_pods", True)) so pod readiness is awaited; update
references to the variables deployment_mode and the create_isvc call sites
accordingly.
In `@tests/model_serving/model_runtime/mlserver/utils.py`:
- Around line 206-220: Validate model_format_name before using getattr: ensure
model_format_name is a non-empty string and that ModelCarImage has the dynamic
attribute f"MLSERVER_{model_format_name.upper()}" (use hasattr or catch
AttributeError) and if missing raise a clear ValueError mentioning the
unsupported model_format_name and the expected constant name; then proceed to
fetch the storage_uri via getattr(ModelCarImage, ...) and build the config as
before.
In `@utilities/constants.py`:
- Around line 300-304: Constants MLSERVER_SKLEARN, MLSERVER_XGBOOST,
MLSERVER_LIGHTGBM currently point to a personal quay.io namespace and
MLSERVER_ONNX is empty; replace these hardcoded URIs with
organization-owned/mirrored registry URIs (e.g., change quay.io/jooholee/... to
the org mirror like quay.io/opendatahub-io/...) or make them configurable via
environment variables and fall back to sensible defaults, and ensure
MLSERVER_ONNX is populated before use (or add a runtime guard in the code that
references MLSERVER_ONNX to raise a clear error if it is empty). Update the
constants MLSERVER_SKLEARN, MLSERVER_XGBOOST, MLSERVER_LIGHTGBM, and
MLSERVER_ONNX and/or add validation logic where these symbols are consumed to
prevent CI/production breakage if the values are missing or still pointing at
personal namespaces.
---
Nitpick comments:
In
`@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py`:
- Around line 119-125: The test currently picks pods[0] which is
non-deterministic; change it to choose a ready predictor pod from the list
returned by get_pods_by_isvc_label (mlserver_model_car_inference_service) by
filtering pods for readiness (e.g., pod.status.phase == "Running" and
pod.status.conditions contains condition type "Ready" == "True" or a
container_status with ready==True) and/or label identifying the predictor
container, then assign that ready pod to pod; if no ready predictor pod is
found, raise a clear RuntimeError stating no ready predictor pod for the
InferenceService instead of using pods[0].
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: cf351258-cc36-4c0f-9b97-1de88a52e681
📒 Files selected for processing (11)
tests/model_serving/model_runtime/mlserver/conftest.pytests/model_serving/model_runtime/mlserver/model_car/__init__.pytests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-raw-deployment-modelcar_text_type].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.pytests/model_serving/model_runtime/mlserver/utils.pyutilities/constants.pyutilities/general.pyutilities/inference_utils.py
Contributor
Author
Jooho
force-pushed
the
mlserver/modelcar_test
branch
from
2cc3167 to
56d9f09
Compare
Jooho
force-pushed
the
mlserver/modelcar_test
branch
2 times, most recently
from
b23d7ef to
a875c9c
Compare
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/model_serving/model_runtime/mlserver/utils.py (1)
272-275:⚠️ Potential issue | 🟠 MajorFail fast for unsupported deployment type instead of returning empty config.
Lines [272]-[275] return
{}for unknowndeployment_type, which defers errors and obscures root cause. RaiseValueErrorimmediately with supported options.Proposed fix
def get_deployment_config_dict( model_format_name: str, deployment_type: str = RAW_DEPLOYMENT_TYPE, ) -> dict[str, str]: @@ - deployment_config_dict = {} - - if deployment_type == RAW_DEPLOYMENT_TYPE: - deployment_config_dict = {"name": model_format_name, **BASE_RAW_DEPLOYMENT_CONFIG} - - return deployment_config_dict + if deployment_type == RAW_DEPLOYMENT_TYPE: + return {"name": model_format_name, **BASE_RAW_DEPLOYMENT_CONFIG} + + raise ValueError( + f"Unsupported deployment_type: {deployment_type}. " + f"Supported values: ({RAW_DEPLOYMENT_TYPE},)" + )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/utils.py` around lines 272 - 275, The code currently falls through and returns an empty deployment_config_dict when deployment_type is not recognized; change this to fail fast by raising a ValueError that names the unsupported deployment_type and lists the supported options (e.g., RAW_DEPLOYMENT_TYPE and any others expected) instead of returning {} — update the branch around RAW_DEPLOYMENT_TYPE/model_format_name/BASE_RAW_DEPLOYMENT_CONFIG in tests/model_serving/model_runtime/mlserver/utils.py to raise ValueError with a clear message when deployment_type is unknown.
♻️ Duplicate comments (1)
tests/model_serving/model_runtime/mlserver/utils.py (1)
206-211:⚠️ Potential issue | 🟡 MinorValidate
ModelCarImageconstant existence before dynamic lookup.Line [210] does an unchecked dynamic
getattr, so unsupportedmodel_format_namefails with an opaqueAttributeError. Validate and raise a clearValueErrorthat includes the expected constant name.Proposed fix
if modelcar: from utilities.constants import ModelCarImage - # Get OCI image URI from ModelCarImage constant - storage_uri = getattr(ModelCarImage, f"MLSERVER_{model_format_name.upper()}") + # Get OCI image URI from ModelCarImage constant + attr_name = f"MLSERVER_{model_format_name.upper()}" + if not model_format_name.strip(): + raise ValueError("model_format_name must be a non-empty string") + if not hasattr(ModelCarImage, attr_name): + raise ValueError( + f"Unsupported model format for modelcar: {model_format_name}. " + f"Expected ModelCarImage.{attr_name}" + ) + storage_uri = getattr(ModelCarImage, attr_name)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/utils.py` around lines 206 - 211, The dynamic lookup using getattr(ModelCarImage, f"MLSERVER_{model_format_name.upper()}") can raise an opaque AttributeError for unsupported formats; before calling getattr in the modelcar branch, compute const_name = f"MLSERVER_{model_format_name.upper()}" and check hasattr(ModelCarImage, const_name); if missing, raise a clear ValueError referencing model_format_name and the expected constant name (const_name), otherwise assign storage_uri = getattr(ModelCarImage, const_name).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@tests/model_serving/model_runtime/mlserver/utils.py`:
- Around line 272-275: The code currently falls through and returns an empty
deployment_config_dict when deployment_type is not recognized; change this to
fail fast by raising a ValueError that names the unsupported deployment_type and
lists the supported options (e.g., RAW_DEPLOYMENT_TYPE and any others expected)
instead of returning {} — update the branch around
RAW_DEPLOYMENT_TYPE/model_format_name/BASE_RAW_DEPLOYMENT_CONFIG in
tests/model_serving/model_runtime/mlserver/utils.py to raise ValueError with a
clear message when deployment_type is unknown.
---
Duplicate comments:
In `@tests/model_serving/model_runtime/mlserver/utils.py`:
- Around line 206-211: The dynamic lookup using getattr(ModelCarImage,
f"MLSERVER_{model_format_name.upper()}") can raise an opaque AttributeError for
unsupported formats; before calling getattr in the modelcar branch, compute
const_name = f"MLSERVER_{model_format_name.upper()}" and check
hasattr(ModelCarImage, const_name); if missing, raise a clear ValueError
referencing model_format_name and the expected constant name (const_name),
otherwise assign storage_uri = getattr(ModelCarImage, const_name).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 38c6c746-6970-4469-8180-07916196ebad
📒 Files selected for processing (13)
tests/model_serving/model_runtime/mlserver/conftest.pytests/model_serving/model_runtime/mlserver/model_car/README.mdtests/model_serving/model_runtime/mlserver/model_car/__init__.pytests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-raw-deployment-modelcar_text_type].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[onnx-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-raw-deployment-modelcar].jsontests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.pytests/model_serving/model_runtime/mlserver/utils.pyutilities/constants.pyutilities/general.pyutilities/inference_utils.py
✅ Files skipped from review due to trivial changes (8)
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-raw-deployment-modelcar].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-raw-deployment-modelcar].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-raw-deployment-modelcar_text_type].json
- tests/model_serving/model_runtime/mlserver/model_car/README.md
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[onnx-raw-deployment-modelcar].json
- tests/model_serving/model_runtime/mlserver/conftest.py
- tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py
- utilities/constants.py
🚧 Files skipped from review as they are similar to previous changes (2)
- utilities/general.py
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-raw-deployment-modelcar].json
Contributor
|
This PR depends on opendatahub-io/MLServer#117 |
Snomaan6846
reviewed
Apr 1, 2026
Snomaan6846
reviewed
Apr 1, 2026
Snomaan6846
reviewed
Apr 1, 2026
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
utilities/constants.py (1)
303-309:⚠️ Potential issue | 🟠 MajorReplace personal OCI registries for MLServer model-car images (CWE-829).
Lines 304-309 pin critical test images to a personal namespace (
quay.io/jooholee/...). That is a supply-chain and availability risk for CI/disconnected mirroring paths. Use org-owned mirrored images or env-configured URIs with a fail-fast guard.Suggested hardening diff
+import os + class ModelCarImage: @@ - MLSERVER_SKLEARN: str = "oci://quay.io/jooholee/mlserver-sklearn@sha256:ec9bc6b520909c52bd1d4accc2b2d28adb04981bd4c3ce94f17f23dd573e1f55" - MLSERVER_XGBOOST: str = "oci://quay.io/jooholee/mlserver-xgboost@sha256:5b6982bdc939b53a7a1210f56aa52bf7de0f0cbc693668db3fd1f496571bff29" - MLSERVER_LIGHTGBM: str = "oci://quay.io/jooholee/mlserver-lightgbm@sha256:77eb15a2eccefa3756faaf2ee4bc1e63990b746427d323957c461f33a4f1a6a3" - MLSERVER_ONNX: str = ( - "oci://quay.io/jooholee/mlserver-onnx@sha256:d0ad00fb6f2caa8f02a0250fc44a576771d0846b2ac8d164ec203b10ec5d604b" - ) + MLSERVER_SKLEARN: str = os.getenv("MLSERVER_SKLEARN_IMAGE", "") + MLSERVER_XGBOOST: str = os.getenv("MLSERVER_XGBOOST_IMAGE", "") + MLSERVER_LIGHTGBM: str = os.getenv("MLSERVER_LIGHTGBM_IMAGE", "") + MLSERVER_ONNX: str = os.getenv("MLSERVER_ONNX_IMAGE", "")#!/bin/bash # Verify MLServer model-car constants are not sourcing from personal registries. rg -n '^\s*MLSERVER_(SKLEARN|XGBOOST|LIGHTGBM|ONNX)\s*:.*quay\.io/jooholee/' utilities/constants.pyAs per coding guidelines, "REVIEW PRIORITIES: 1. Security vulnerabilities (provide severity, exploit scenario, and remediation code)".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@utilities/constants.py` around lines 303 - 309, The four hardcoded constants MLSERVER_SKLEARN, MLSERVER_XGBOOST, MLSERVER_LIGHTGBM, and MLSERVER_ONNX currently point at a personal quay.io namespace; replace them so CI and production don't depend on a personal registry by reading their URIs from environment variables (e.g. MLSERVER_SKLEARN_URI, etc.) with a sane org-owned default, and add a fail-fast validation that rejects or logs and aborts if the resolved URI matches the disallowed pattern (quay.io/jooholee) so builds cannot silently use personal images. Ensure the change is made in utilities/constants.py updating the named symbols and include a clear error message when validation fails.tests/model_serving/model_runtime/mlserver/conftest.py (1)
173-187:⚠️ Potential issue | 🟠 MajorNormalize deployment key handling and restore deterministic pod readiness waits.
Line 173 only reads
deployment-mode, while related fixtures usedeployment_type; this can silently drop the caller’s intended mode. Line 187 defaultswait_for_predictor_podstoFalse, which reintroduces race-prone test behavior.Suggested fix
- deployment_mode = params.get("deployment-mode", KServeDeploymentType.RAW_DEPLOYMENT) + deployment_mode = ( + params.get("deployment_type") + or params.get("deployment-mode") + or KServeDeploymentType.RAW_DEPLOYMENT + ) @@ - external_route=params.get("enable_external_route"), - wait_for_predictor_pods=params.get("wait_for_predictor_pods", False), + external_route=params.get("enable_external_route", False), + wait_for_predictor_pods=params.get("wait_for_predictor_pods", True),As per coding guidelines, "REVIEW PRIORITIES: 2. Architectural issues and anti-patterns" and "3. Bug-prone patterns and error handling gaps".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/conftest.py` around lines 173 - 187, The code uses params.get("deployment-mode") (assigned to deployment_mode) but other fixtures expect "deployment_type", and it sets wait_for_predictor_pods default to False which reintroduces race conditions; update the params handling before the create_isvc call to normalize the key (e.g., check params for "deployment_type" first then fallback to "deployment-mode" and set deployment_mode accordingly) and change the wait_for_predictor_pods default to True when calling create_isvc so the test waits deterministically for predictor pods; adjust references to deployment_mode, params, and the create_isvc call as needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/model_serving/model_runtime/mlserver/conftest.py`:
- Around line 169-171: The fixture currently only checks presence of storage_uri
(variable storage_uri) but accepts any scheme; update the validation in the
storage_uri handling block to explicitly enforce that storage_uri starts with
the "oci://" scheme and raise a ValueError with a clear message if it does not
(e.g., "storage-uri must use oci:// scheme"); modify the branch where
storage_uri is fetched (the if not storage_uri check) to first validate
storage_uri.startswith("oci://") and raise when false so non-OCI inputs are
rejected immediately.
---
Duplicate comments:
In `@tests/model_serving/model_runtime/mlserver/conftest.py`:
- Around line 173-187: The code uses params.get("deployment-mode") (assigned to
deployment_mode) but other fixtures expect "deployment_type", and it sets
wait_for_predictor_pods default to False which reintroduces race conditions;
update the params handling before the create_isvc call to normalize the key
(e.g., check params for "deployment_type" first then fallback to
"deployment-mode" and set deployment_mode accordingly) and change the
wait_for_predictor_pods default to True when calling create_isvc so the test
waits deterministically for predictor pods; adjust references to
deployment_mode, params, and the create_isvc call as needed.
In `@utilities/constants.py`:
- Around line 303-309: The four hardcoded constants MLSERVER_SKLEARN,
MLSERVER_XGBOOST, MLSERVER_LIGHTGBM, and MLSERVER_ONNX currently point at a
personal quay.io namespace; replace them so CI and production don't depend on a
personal registry by reading their URIs from environment variables (e.g.
MLSERVER_SKLEARN_URI, etc.) with a sane org-owned default, and add a fail-fast
validation that rejects or logs and aborts if the resolved URI matches the
disallowed pattern (quay.io/jooholee) so builds cannot silently use personal
images. Ensure the change is made in utilities/constants.py updating the named
symbols and include a clear error message when validation fails.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 5175c9a3-1ad5-4684-a9f0-1228bfb7e448
📒 Files selected for processing (8)
tests/model_serving/model_runtime/mlserver/conftest.pytests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment_text_type].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[onnx-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/utils.pyutilities/constants.py
✅ Files skipped from review due to trivial changes (5)
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[onnx-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment_text_type].json
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/model_serving/model_runtime/mlserver/utils.py
Contributor
|
@Jooho, I think we can squash all the commits into one for this PR. |
Jooho
force-pushed
the
mlserver/modelcar_test
branch
from
272beba to
2e82e48
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py (4)
89-95: Class docstring also omits ONNX.Same issue as module docstring - update to include all tested formats.
- xgboost, and lightgbm model formats. + xgboost, lightgbm, and onnx model formats.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py` around lines 89 - 95, The class docstring on TestMLServerModelCar omits the ONNX format; update the docstring inside the TestMLServerModelCar class to list all tested OCI image model formats (sklearn, xgboost, lightgbm, and ONNX) so it matches the module-level docstring and accurately documents the test coverage.
60-72: Inconsistent parametrization pattern for text-type case.Lines 60-72 use an inline dict
{"name": f"{ModelFormat.LIGHTGBM}-model-car-text-type"}while other cases useget_model_namespace_dict(). Consider adding a parameter to the helper or documenting why this case requires manual construction.Not blocking, but reduces maintainability.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py` around lines 60 - 72, The test case uses an inline dict for the pytest.param name ({"name": f"{ModelFormat.LIGHTGBM}-model-car-text-type"}) while other cases use get_model_namespace_dict(), which is inconsistent; update the test to either (a) extend get_model_namespace_dict() to accept a flag/param for the text-type naming and use that helper here, or (b) replace the inline dict with a call to get_model_namespace_dict(model_format_name=ModelFormat.LIGHTGBM, suffix="model-car-text-type") so the test uses get_model_namespace_dict consistently alongside get_model_storage_uri_dict and get_deployment_config_dict and the id generation via get_test_case_id.
125-127: Pod selection assumes single pod.
pods[0]silently picks the first pod if multiple exist (e.g., under scaling). Acceptable for smoke tests, but consider adding a comment or logging which pod is selected for debugging multi-replica scenarios.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py` around lines 125 - 127, The test currently picks the first pod blindly using pods[0] which can hide which replica was used; update the code around the pods/pod selection to either (a) add a short explanatory comment that selection of pods[0] is intentional for smoke tests, or (b) emit a debug/log message showing which pod is being selected (e.g., log pod.metadata.name or relevant identifier) before assigning pod = pods[0], referencing the variables pods, pod and mlserver_model_car_inference_service so future debugging of multi-replica runs can identify the chosen pod.
1-6: Docstring incomplete: ONNX format missing.Module docstring lists sklearn, xgboost, and lightgbm but the test also covers ONNX (line 73-81).
-for sklearn, xgboost, and lightgbm formats. +for sklearn, xgboost, lightgbm, and onnx formats.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py` around lines 1 - 6, The module docstring currently lists sklearn, xgboost, and lightgbm but omits ONNX even though the tests exercise ONNX; update the top-of-file docstring in test_mlserver_model_car.py to include ONNX (and ensure it accurately reflects all formats tested) so the description matches the test coverage.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py`:
- Around line 129-137: The test is passing an empty model_version to
validate_inference_request which bypasses versioned endpoint checks; update the
call in test_mlserver_model_car.py (the validate_inference_request invocation)
to pass model_format_config["model_version"] instead of "" so it uses the
correct version from MODEL_CONFIGS and matches the basic_model_deployment
behavior and the versioning logic in utils.py:73.
---
Nitpick comments:
In
`@tests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.py`:
- Around line 89-95: The class docstring on TestMLServerModelCar omits the ONNX
format; update the docstring inside the TestMLServerModelCar class to list all
tested OCI image model formats (sklearn, xgboost, lightgbm, and ONNX) so it
matches the module-level docstring and accurately documents the test coverage.
- Around line 60-72: The test case uses an inline dict for the pytest.param name
({"name": f"{ModelFormat.LIGHTGBM}-model-car-text-type"}) while other cases use
get_model_namespace_dict(), which is inconsistent; update the test to either (a)
extend get_model_namespace_dict() to accept a flag/param for the text-type
naming and use that helper here, or (b) replace the inline dict with a call to
get_model_namespace_dict(model_format_name=ModelFormat.LIGHTGBM,
suffix="model-car-text-type") so the test uses get_model_namespace_dict
consistently alongside get_model_storage_uri_dict and get_deployment_config_dict
and the id generation via get_test_case_id.
- Around line 125-127: The test currently picks the first pod blindly using
pods[0] which can hide which replica was used; update the code around the
pods/pod selection to either (a) add a short explanatory comment that selection
of pods[0] is intentional for smoke tests, or (b) emit a debug/log message
showing which pod is being selected (e.g., log pod.metadata.name or relevant
identifier) before assigning pod = pods[0], referencing the variables pods, pod
and mlserver_model_car_inference_service so future debugging of multi-replica
runs can identify the chosen pod.
- Around line 1-6: The module docstring currently lists sklearn, xgboost, and
lightgbm but omits ONNX even though the tests exercise ONNX; update the
top-of-file docstring in test_mlserver_model_car.py to include ONNX (and ensure
it accurately reflects all formats tested) so the description matches the test
coverage.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: f27e9e13-4803-4a0f-998f-e177fd432d64
📒 Files selected for processing (11)
tests/model_serving/model_runtime/mlserver/conftest.pytests/model_serving/model_runtime/mlserver/model_car/README.mdtests/model_serving/model_runtime/mlserver/model_car/__init__.pytests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment_text_type].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[onnx-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/__snapshots__/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-modelcar-raw-deployment].jsontests/model_serving/model_runtime/mlserver/model_car/test_mlserver_model_car.pytests/model_serving/model_runtime/mlserver/utils.pyutilities/constants.py
✅ Files skipped from review due to trivial changes (7)
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[xgboost-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[sklearn-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[lightgbm-modelcar-raw-deployment_text_type].json
- tests/model_serving/model_runtime/mlserver/model_car/snapshots/test_mlserver_model_car/TestMLServerModelCar.test_mlserver_model_car_inference[onnx-modelcar-raw-deployment].json
- tests/model_serving/model_runtime/mlserver/model_car/README.md
- tests/model_serving/model_runtime/mlserver/utils.py
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/model_serving/model_runtime/mlserver/conftest.py
Jooho
force-pushed
the
mlserver/modelcar_test
branch
4 times, most recently
from
db60ed9 to
6cf3fe3
Compare
Snomaan6846
force-pushed
the
mlserver/modelcar_test
branch
3 times, most recently
from
8d7e914 to
4c89d0b
Compare
mwaykole
approved these changes
Apr 10, 2026
- Add MLServer model car tests for sklearn, xgboost, and lightgbm using OCI images. - Add mlserver_model_car_inference_service fixture with env variable support. - Add Standard deployment mode support in constants and utils Signed-off-by: Jooho Lee <jlee@redhat.com>
Signed-off-by: Snomaan6846 <syedali@redhat.com> rh-pre-commit.version: 2.3.2 rh-pre-commit.check-secrets: ENABLED
Snomaan6846
force-pushed
the
mlserver/modelcar_test
branch
from
4c89d0b to
06d1464
Compare
Raghul-M
approved these changes
Apr 10, 2026
|
Status of building tag latest: success. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
Related Issues
How it has been tested
Additional Requirements
Summary by CodeRabbit
Tests
Documentation
Chores