Problem
In statusPredictor, when an InferenceService becomes Ready but Status.URL is nil (which is the common case in RawDeployment mode, where KServe only creates an in-cluster ClusterIP Service), the provider returns controller.Ready() with no connection details at all. This means:
- The user sees "Ready" but has no idea how to reach the model.
- The OpenEverest UI/CLI cannot show an endpoint for the predictor.
- External tools or other OpenEverest resources that depend on connection details (e.g., API Gateway routing, downstream consumers) get an empty host/port.
The LLM topology solves this elegantly by creating a provider-owned external Service (LoadBalancer or NodePort) when the user selects a non-ClusterIP type, and populating connection details from that. The predictor topology does no such thing.
Suggested approach
Align predictor with LLM:
- Read
comp.Service.ServiceType for predictor topology (ClusterIP, LoadBalancer, NodePort).
- When LoadBalancer or NodePort is selected, create a provider-owned external
Service fronting the predictor pods (similar to externalServiceSuffix logic in llminferenceservice.go).
- Populate connection details from the external Service's endpoint (or from
Status.URL if KServe eventually populates it).
- When ClusterIP is selected, continue to return
Ready() with no connection details (document this as "use port-forward or enable external exposure").
KServe predictor pods use the label selector kserve.io/inferenceservice: <name> — use that for the external Service's selector.
Acceptance criteria
Problem
In
statusPredictor, when anInferenceServicebecomesReadybutStatus.URLis nil (which is the common case in RawDeployment mode, where KServe only creates an in-cluster ClusterIP Service), the provider returnscontroller.Ready()with no connection details at all. This means:The LLM topology solves this elegantly by creating a provider-owned external
Service(LoadBalancer or NodePort) when the user selects a non-ClusterIP type, and populating connection details from that. The predictor topology does no such thing.Suggested approach
Align predictor with LLM:
comp.Service.ServiceTypefor predictor topology (ClusterIP, LoadBalancer, NodePort).Servicefronting the predictor pods (similar toexternalServiceSuffixlogic inllminferenceservice.go).Status.URLif KServe eventually populates it).Ready()with no connection details (document this as "use port-forward or enable external exposure").KServe predictor pods use the label selector
kserve.io/inferenceservice: <name>— use that for the external Service's selector.Acceptance criteria
ClusterIP,LoadBalancer, andNodePortviaservice.serviceType.<node-ip>:<nodePort>in connection details.serviceTypematches the LLM allowed set (no unsupported types).