Add Test Suite for FIL Model#463
Conversation
|
Caution Review failedThe pull request is closed. 📝 WalkthroughSummary by CodeRabbit
WalkthroughThis change introduces new tests and supporting files for validating Forest Inference Library (FIL) model inference served by Triton via KServe. It adds parameterized test cases covering REST and gRPC protocols in both raw and serverless deployment modes, corresponding input payloads, expected output snapshots, and constants for input file paths. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes
Suggested labels
Suggested reviewers
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. 📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (8)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
The following are automatically added/executed:
Available user actions:
Supported labels{'/lgtm', '/hold', '/verified', '/cherry-pick', '/wip', '/build-push-pr-image'} |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
tests/model_serving/model_runtime/triton/constant.py (1)
25-26: Consider grouping & derive constants to curb ever-growing duplicationThe new FIL constants follow the existing naming/layout pattern, so functionally they’re fine.
However, this file is turning into a long, near-duplicate list for every (model × protocol) combination. Generating these paths programmatically (e.g.f"{TRITON_INPUT_BASE_PATH}/kserve-triton-{model}-{proto}-input.json") or grouping per‐model dicts would cut copy/paste churn and lower the risk of missing a future addition.No action required for this PR, just food for thought before the list becomes unmanageable.
tests/model_serving/model_runtime/triton/basic_model_deployment/kserve-triton-fil-gRPC-input.json (1)
1-1: Consider formatting the JSON for better readability.While the JSON structure is correct for gRPC inference requests, consider formatting it with proper indentation to improve maintainability and readability, especially given the long base64-encoded input data.
Example formatting:
-{"id":"test1","model_name":"fil","model_version":"1","inputs":[{"name":"input__0","datatype":"FP32","shape":[1,393]}],"outputs":[{"name":"output__0"}],"raw_input_contents":["lzwpP+wwvj6VwRs/iuB8PyCNRT8L4tU+MAQgPwDZSz+xf80+3dYxP/dbTz/iI2M/ZFd7P2J2gz3C8Gk/p8B2P9ZWgz2HsxM+kWdkP8S98D7U/Ok+KgGvPs4gLjzWn98+z6OcPsOh4jy9Ftc+hFQaPjs6ND+TLA4/lURDP5quPT+foAA/o0dPP5vFEz/BREs/mX5dP8S6KD9yjyI/lmFxPi7EbT25z3I/gcIYP9bLCD9fzek+oz7mPtmfvz7IINo+j+xkP+AvTD/aSDs/XEQnPwdDED/PiDs/YaEtP2QW0j5S+n0/koJGPhwy5z0xo8w9LmxMP/EK1j1cOiI/VS2LPedKij7UpSE/77TxPpdAXzmGEno/8Ft9P7vVAD8rGgM/9oMQPpvZED+rodk++3G3Pn3OYT9bFaE+2SrGPX7C/T5SVSY/7rC1Pbhq0j5Xqg0/RM0EP/CPCT+qyAc/DcvLPnZtlT6kiTo+WE1/P2WFRD/rHLo+wIGqPm7aPz/wrpM+q6H3PjV6TT9zIhQ/ADM4OxaXGD8gUi4+AvqxPjS7YT9pJPk+VV06P3qwcj+GzFw/zWr0Ps1cfD8cFSU+uKpzPoUk2T5EEIM99IwmP0acyT4mclI/tSmCPvt8Qz9C7UA/G4YIP1L5Oz984Xo+ye/5Ppu+5jzuQUc/xv41PmammT13Wew+W9sHPyKriT0dvyc+nBKiOx0MbD4Cj4Q8zvfaPLaGPT5I4GY+2s8SPwZd6D60t28/NYFHP6hTQj58wTU/z11IP79oVj/sYxY/Y5JkOx+oLz7Xa4k9qLksP9hDmz6I0HM/hcEVP9B6nz6FGFg/sQ81PzaJuD7BBUo/oIa6PqH1Sj+erKg9eij0PWBDwj5kcFM/KmpfP87/1D77f6I+zqoMPz9AHD545EI/gzF5PzqOij5bqNM+H9yfPspKOD9OqPk+YPSePrvDnD7mCEE+UTGyPHJuAz7k4Gc/WAC8PlzhdD3HJYM+KNIdP/bIfT+rH6M9WXW8PgfxEz/9eQk+BY66PjfBJz8enoA+UPBkPtfBXD9U7kQ8Fx1LPuXDIj/mDMw+OQlHPyU7NT9COJ0+WWMePvYCFD08ZOI+V50bPwDgKD9BGcA+DgP7Pjjkpz7wMEI/qbTRPjejJT5OoiA+zxJxP+VVyz60ph4/o2kcPiF3xD6b1iA/oouJO8IWID/18rQ+qoAYPy3qGj2YiVg/InkTP3hjXD/MdJs+37YKPswoHD/+tvE8h1ipPrLGFz+ghzs/ME05P/Jw7j6HlvU8pwo6P8RVLD4imqo+HetxPyQLqz38wwI/NlHMPgvWHz8K7NE+cSZxP3N0cD8TLr0+ZgPvPoDnCj+Ug4g+oeVgPzqICD/EHXQ/7/RfP1yPhz2no2I/cYJpP64QlD1P0Ac+37ijPGCePz6WgzA/25mWPvAAjz4IMrc+YPUaPVzaDz/jnF8+2/WTPeZvZD9IimA/13LOPtg6Fz9xiDI/wzNNP0cntz4552A/zEM/PrawHz9KEng/PLkCP8VKXj9diWM/SCVzPguHDz+GE24+4uR7P4XGmj6Rdmk/3HBbPt7TOT/DKMo+VumVPuxjQz8NasY+HBFMP40gsz46q0M/hGsyPcUGfz+zPws/UK9ZP/yNyz398GE/AMwGP5dZZz/pzRE/YiVSP2bUUD8aUQw/QwRNP1fVdD63tWk/8+xiP1UJZD9D+Ns+DG9gPnrvQT9LtVE/13xcPsuAXD40fpM+w5TOPXyfkj5Zx0Y/zAixPiBnRT/eawU/QpPXPhbvJj9xVKc+BXZ9PmhxxT5PFg0/zIZtPyjnHD9aEBo/4hQUPnjjCj7fgDU9ZLxDPeBoRj9rRBE/buY5P16fCj8V3SE/FMYmPxzUeT/+LQ49oH72Pq32XT/ayQk/b/rLPYNvwz4ZAXI/gp0hP51DuD3s6Rc/uMQ5PNI2+j12XG0/IcNCPQtj1z4Qrx4/RxV2P89fYT8O4Us/i8VOP56lWD+/itw+LH8hP7JmQT7bCR88uihkPh9lLD9Zd0Y/v004Pz/6PD34dFE/fG1KPxRoij673kk+mPjNPszE4j5WU+M+nShBPrRunTylmds8"]} +{ + "id": "test1", + "model_name": "fil", + "model_version": "1", + "inputs": [ + { + "name": "input__0", + "datatype": "FP32", + "shape": [1, 393] + } + ], + "outputs": [ + { + "name": "output__0" + } + ], + "raw_input_contents": [ + "lzwpP+wwvj6VwRs/iuB8PyCNRT8L4tU+MAQgPwDZSz+xf80+3dYxP/dbTz/iI2M/ZFd7P2J2gz3C8Gk/p8B2P9ZWgz2HsxM+kWdkP8S98D7U/Ok+KgGvPs4gLjzWn98+z6OcPsOh4jy9Ftc+hFQaPjs6ND+TLA4/lURDP5quPT+foAA/o0dPP5vFEz/BREs/mX5dP8S6KD9yjyI/lmFxPi7EbT25z3I/gcIYP9bLCD9fzek+oz7mPtmfvz7IINo+j+xkP+AvTD/aSDs/XEQnPwdDED/PiDs/YaEtP2QW0j5S+n0/koJGPhwy5z0xo8w9LmxMP/EK1j1cOiI/VS2LPedKij7UpSE/77TxPpdAXzmGEno/8Ft9P7vVAD8rGgM/9oMQPpvZED+rodk++3G3Pn3OYT9bFaE+2SrGPX7C/T5SVSY/7rC1Pbhq0j5Xqg0/RM0EP/CPCT+qyAc/DcvLPnZtlT6kiTo+WE1/P2WFRD/rHLo+wIGqPm7aPz/wrpM+q6H3PjV6TT9zIhQ/ADM4OxaXGD8gUi4+AvqxPjS7YT9pJPk+VV06P3qwcj+GzFw/zWr0Ps1cfD8cFSU+uKpzPoUk2T5EEIM99IwmP0acyT4mclI/tSmCPvt8Qz9C7UA/G4YIP1L5Oz984Xo+ye/5Ppu+5jzuQUc/xv41PmammT13Wew+W9sHPyKriT0dvyc+nBKiOx0MbD4Cj4Q8zvfaPLaGPT5I4GY+2s8SPwZd6D60t28/NYFHP6hTQj58wTU/z11IP79oVj/sYxY/Y5JkOx+oLz7Xa4k9qLksP9hDmz6I0HM/hcEVP9B6nz6FGFg/sQ81PzaJuD7BBUo/oIa6PqH1Sj+erKg9eij0PWBDwj5kcFM/KmpfP87/1D77f6I+zqoMPz9AHD545EI/gzF5PzqOij5bqNM+H9yfPspKOD9OqPk+YPSePrvDnD7mCEE+UTGyPHJuAz7k4Gc/WAC8PlzhdD3HJYM+KNIdP/bIfT+rH6M9WXW8PgfxEz/9eQk+BY66PjfBJz8enoA+UPBkPtfBXD9U7kQ8Fx1LPuXDIj/mDMw+OQlHPyU7NT9COJ0+WWMePvYCFD08ZOI+V50bPwDgKD9BGcA+DgP7Pjjkpz7wMEI/qbTRPjejJT5OoiA+zxJxP+VVyz60ph4/o2kcPiF3xD6b1iA/oouJO8IWID/18rQ+qoAYPy3qGj2YiVg/InkTP3hjXD/MdJs+37YKPswoHD/+tvE8h1ipPrLGFz+ghzs/ME05P/Jw7j6HlvU8pwo6P8RVLD4imqo+HetxPyQLqz38wwI/NlHMPgvWHz8K7NE+cSZxP3N0cD8TLr0+ZgPvPoDnCj+Ug4g+oeVgPzqICD/EHXQ/7/RfP1yPhz2no2I/cYJpP64QlD1P0Ac+37ijPGCePz6WgzA/25mWPvAAjz4IMrc+YPUaPVzaDz/jnF8+2/WTPeZvZD9IimA/13LOPtg6Fz9xiDI/wzNNP0cntz4552A/zEM/PrawHz9KEng/PLkCP8VKXj9diWM/SCVzPguHDz+GE24+4uR7P4XGmj6Rdmk/3HBbPt7TOT/DKMo+VumVPuxjQz8NasY+HBFMP40gsz46q0M/hGsyPcUGfz+zPws/UK9ZP/yNyz398GE/AMwGP5dZZz/pzRE/YiVSP2bUUD8aUQw/QwRNP1fVdD63tWk/8+xiP1UJZD9D+Ns+DG9gPnrvQT9LtVE/13xcPsuAXD40fpM+w5TOPXyfkj5Zx0Y/zAixPiBnRT/eawU/QpPXPhbvJj9xVKc+BXZ9PmhxxT5PFg0/zIZtPyjnHD9aEBo/4hQUPnjjCj7fgDU9ZLxDPeBoRj9rRBE/buY5P16fCj8V3SE/FMYmPxzUeT/+LQ49oH72Pq32XT/ayQk/b/rLPYNvwz4ZAXI/gp0hP51DuD3s6Rc/uMQ5PNI2+j12XG0/IcNCPQtj1z4Qrx4/RxV2P89fYT8O4Us/i8VOP56lWD+/itw+LH8hP7JmQT7bCR88uihkPh9lLD9Zd0Y/v004Pz/6PD34dFE/fG1KPxRoij673kk+mPjNPszE4j5WU+M+nShBPrRunTylmds8" + ] +}tests/model_serving/model_runtime/triton/basic_model_deployment/test_fil_model.py (2)
7-23: Consider organizing imports by category.The imports are functional but could be better organized for improved readability.
Consider grouping imports by category:
""" Test module for FIL model served by Triton via KServe. Validates inference using REST and gRPC protocols with both raw and serverless deployment modes. """ from typing import Any import pytest from ocp_resources.inference_service import InferenceService from ocp_resources.pod import Pod from simple_logger.logger import get_logger from utilities.constants import Protocols -from tests.model_serving.model_runtime.triton.basic_model_deployment.utils import validate_inference_request, load_json +from tests.model_serving.model_runtime.triton.basic_model_deployment.utils import ( + load_json, + validate_inference_request, +) from tests.model_serving.model_runtime.triton.constant import ( BASE_RAW_DEPLOYMENT_CONFIG, BASE_SERVERLESS_DEPLOYMENT_CONFIG, MODEL_PATH_PREFIX, TRITON_GRPC_FIL_INPUT_PATH, TRITON_REST_FIL_INPUT_PATH, )
24-24: Consider removing unused logger.The logger is initialized but not used in the test method. Either use it for logging test execution or remove it to avoid dead code.
If logging is not needed, remove:
-LOGGER = get_logger(name=__name__)Or if you want to add logging, consider adding it to the test method:
def test_fil_inference(self, ...): LOGGER.info(f"Running FIL inference test with protocol: {protocol}") # ... rest of the method
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-raw-grpc-deployment].json(1 hunks)tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-raw-rest-deployment].json(1 hunks)tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-serverless-grpc-deployment].json(1 hunks)tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-serverless-rest-deployment].json(1 hunks)tests/model_serving/model_runtime/triton/basic_model_deployment/kserve-triton-fil-gRPC-input.json(1 hunks)tests/model_serving/model_runtime/triton/basic_model_deployment/kserve-triton-fil-rest-input.json(1 hunks)tests/model_serving/model_runtime/triton/basic_model_deployment/test_fil_model.py(1 hunks)tests/model_serving/model_runtime/triton/constant.py(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
tests/model_serving/model_runtime/triton/basic_model_deployment/test_fil_model.py (4)
utilities/constants.py (1)
Protocols(91-98)tests/model_serving/model_runtime/triton/basic_model_deployment/utils.py (1)
load_json(171-173)tests/model_serving/model_runtime/triton/basic_model_deployment/conftest.py (3)
triton_inference_service(174-219)triton_pod_resource(241-248)triton_response_snapshot(236-237)tests/model_serving/conftest.py (2)
protocol(26-36)root_dir(12-22)
🔇 Additional comments (11)
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-raw-grpc-deployment].json (1)
1-18: Snapshot acceptedNothing to flag in the generated snapshot; structure matches gRPC payload expectations.
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-serverless-rest-deployment].json (1)
1-18: Snapshot acceptedContents and precision look consistent with the other REST snapshots.
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-raw-rest-deployment].json (1)
1-18: Snapshot acceptedNo issues observed; matches the expected REST schema.
tests/model_serving/model_runtime/triton/basic_model_deployment/kserve-triton-fil-rest-input.json (1)
1-1: Large test fixture – notedHuge numeric array is expected for the FIL feature vector; no review comments needed.
tests/model_serving/model_runtime/triton/basic_model_deployment/__snapshots__/test_fil_model/TestFILModel.test_fil_inference[fil-serverless-grpc-deployment].json (1)
1-18: LGTM: Well-structured snapshot for Triton inference response.The JSON snapshot correctly captures the expected response structure for FIL model inference via gRPC serverless deployment, including proper field names, data types, and base64-encoded output content.
tests/model_serving/model_runtime/triton/basic_model_deployment/test_fil_model.py (6)
1-6: LGTM: Clear module documentation.The module docstring provides a concise and accurate description of the test purpose and scope.
26-28: LGTM: Well-defined constants.The constants are appropriately defined and follow good naming conventions.
29-32: LGTM: Proper fixture setup.The pytest fixtures are correctly specified for the test requirements.
34-83: LGTM: Comprehensive parameterization covers all test scenarios.The parameterization effectively covers all four combinations of protocol (REST/gRPC) and deployment mode (raw/serverless) with clear, descriptive test IDs.
84-93: LGTM: Well-documented test class.The class docstring clearly describes the test scope and coverage.
94-123: LGTM: Well-implemented test method with proper protocol handling.The test method demonstrates good practices:
- Clear parameter typing and documentation
- Protocol-specific input file selection
- Proper use of utility functions for validation
- Clean separation of concerns
The logic correctly selects the appropriate input file based on the protocol and delegates the actual validation to the utility function.
for more information, see https://pre-commit.ci
|
Status of building tag latest: success. |
6 participants
Adresses:
FIL REST: RHOAIENG-23398
FIL GRPC: RHOAIENG-23399