Skip to content

Model Serving Inference Endpoint #224

@adnan-drina

Description

@adnan-drina

Describe the bug
When following the “Serve the model / call the model server” step, the workshop suggests using the internal Model Mesh service URL. In environments where the model server is deployed in a different Kubernetes namespace than the attendee’s workbench (for example, model in ic-shared-img-det, workbench in user1), the hostname constructed in the notebook is wrong (e.g., http://modelmesh-serving.user1:8008/...).
This causes a DNS resolution failure in the notebook and makes it appear that Model Mesh is down. In addition, the example uses a placeholder model name that doesn’t match the actual deployed model, and the health endpoints shown in some docs return 404 even when the model is healthy, which adds to the confusion.

To Reproduce
Complete 4.5 Model serving as per workshop (multi-model serving with Model Mesh + OpenVINO).
In our lab, the model server and InferenceService were created in ic-shared-img-det (shared namespace).

Open notebook 04-04-accident-recognition.ipynb and set the inference URL based on the instructions (or copy the “Internal Service” base URL).
Example (what we used, following the placeholder names):
infer_url = "http://modelmesh-serving.user1:8008/v2/models/my-first-model/infer"

Run:
img = process_image(image_path, infer_url)

Error:
ConnectionError: HTTPConnectionPool(host='modelmesh-serving.user1', port=8008):
Max retries exceeded ... Failed to establish a new connection: [Errno -2] Name or service not known

From a terminal, verify that there is no modelmesh-serving Service in user1:
oc get svc -n user1 | grep modelmesh-serving

But there is one in ic-shared-img-det:
oc get svc -n ic-shared-img-det modelmesh-serving

Expected behavior
The workshop instructions and notebook should guide attendees to:

  • Use the correct namespace in the hostname (fully qualified service name), even when the model server is deployed in a shared namespace different from their workbench.
  • Use the actual deployed model name in the path (/v2/models//infer), not a placeholder.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions