Problem
The provider emits a PodMonitor for LLM topology instances (when ENABLE_POD_MONITOR=true), but the predictor topology does not get any Prometheus Operator scraping configuration. This means:
- KServe predictor pods running Triton, TorchServe, SKLearn, XGBoost, etc. are not scraped by Prometheus unless users manually create ServiceMonitors or PodMonitors.
- There's an observability gap between LLM and predictor workloads.
- The provider already has the logic (
podmonitor.go); it's just not wired up for predictor topology.
KServe serving runtimes (including Triton, TorchServe, sklearnserver) export Prometheus metrics on standard ports. The PodMonitor should work similarly to the LLM case.
Suggested approach
- Reuse the existing
PodMonitor logic in internal/provider/podmonitor.go.
- Add predictor topology support in the provider's
Sync/Cleanup methods.
- Ensure the
PodMonitor targets the correct label selector for predictor pods (which may differ from LLM pods).
- Gate it on the same
ENABLE_POD_MONITOR env var.
- Optionally allow per-topology
enableMetrics flag in the topology parameters.
Acceptance criteria
Problem
The provider emits a
PodMonitorfor LLM topology instances (whenENABLE_POD_MONITOR=true), but the predictor topology does not get any Prometheus Operator scraping configuration. This means:podmonitor.go); it's just not wired up for predictor topology.KServe serving runtimes (including Triton, TorchServe, sklearnserver) export Prometheus metrics on standard ports. The
PodMonitorshould work similarly to the LLM case.Suggested approach
PodMonitorlogic ininternal/provider/podmonitor.go.Sync/Cleanupmethods.PodMonitortargets the correct label selector for predictor pods (which may differ from LLM pods).ENABLE_POD_MONITORenv var.enableMetricsflag in the topology parameters.Acceptance criteria
PodMonitorin the Instance's namespace.PodMonitorEnabled()returns the right result for both topologies.