Skip to content

Commit 0d9e2c7

Browse files
authored
Model Validation automation - v2 (#464)
* feat(model-validation-v2): add gpu count for larger models * feat(model-validation): serving arguments per model * feat(model-validation-v2): resolve pr comments * feat(model-validation-v2): remove pytest txt file * feat(model-validation-v2): pr comments pass gpu count directly * feat(model-validation-v2): remove snapshots folder
1 parent 50e3681 commit 0d9e2c7

15 files changed

Lines changed: 109 additions & 463 deletions

File tree

tests/conftest.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -201,10 +201,13 @@ def ci_s3_bucket_endpoint(pytestconfig: pytest.Config) -> str:
201201

202202

203203
@pytest.fixture(scope="session")
204-
def serving_argument(pytestconfig: pytest.Config, modelcar_yaml_config: dict[str, Any] | None) -> list[str]:
204+
def serving_argument(pytestconfig: pytest.Config, modelcar_yaml_config: dict[str, Any] | None) -> tuple[list[str], int]:
205205
if modelcar_yaml_config:
206-
arg = modelcar_yaml_config.get("serving_argument", [])
207-
return arg if isinstance(arg, list) else [arg]
206+
val = modelcar_yaml_config.get("serving_arguments", {})
207+
if isinstance(val, dict):
208+
args = val.get("args", [])
209+
gpu_count = val.get("gpu_count", 1)
210+
return args, gpu_count
208211

209212
raw_arg = pytestconfig.option.serving_argument
210213
try:

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarRaw.test_oci_modelcar_raw_openai_inference[modelcar-granite-3-1-8b-base-quantized-w4a161.5-raw].1.json

Lines changed: 0 additions & 50 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarRaw.test_oci_modelcar_raw_openai_inference[modelcar-granite-3-1-8b-base-quantized-w4a161.5-raw].json

Lines changed: 0 additions & 24 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarRaw.test_oci_modelcar_raw_openai_inference[modelcar-llama-3-1-8b-instruct1.5-raw].1.json

Lines changed: 0 additions & 50 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarRaw.test_oci_modelcar_raw_openai_inference[modelcar-llama-3-1-8b-instruct1.5-raw].json

Lines changed: 0 additions & 24 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarRaw.test_oci_modelcar_raw_openai_inference[modelcar-mistral-7b-instruct-v0-3-quantized-w4a161.5-raw].1.json

Lines changed: 0 additions & 50 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarRaw.test_oci_modelcar_raw_openai_inference[modelcar-mistral-7b-instruct-v0-3-quantized-w4a161.5-raw].json

Lines changed: 0 additions & 24 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarServerless.test_oci_modelcar_serverless_openai_inference[modelcar-granite-3-1-8b-base-quantized-w4a161.5-serverless].1.json

Lines changed: 0 additions & 50 deletions
This file was deleted.

tests/model_serving/model_runtime/model_validation/__snapshots__/test_modelvalidation/TestVLLMModelcarServerless.test_oci_modelcar_serverless_openai_inference[modelcar-granite-3-1-8b-base-quantized-w4a161.5-serverless].json

Lines changed: 0 additions & 24 deletions
This file was deleted.

0 commit comments

Comments
 (0)