Skip to content

Emit PodMonitor for predictor topology #20

Description

@tartkri-bot

Problem

The provider emits a PodMonitor for LLM topology instances (when ENABLE_POD_MONITOR=true), but the predictor topology does not get any Prometheus Operator scraping configuration. This means:

  • KServe predictor pods running Triton, TorchServe, SKLearn, XGBoost, etc. are not scraped by Prometheus unless users manually create ServiceMonitors or PodMonitors.
  • There's an observability gap between LLM and predictor workloads.
  • The provider already has the logic (podmonitor.go); it's just not wired up for predictor topology.

KServe serving runtimes (including Triton, TorchServe, sklearnserver) export Prometheus metrics on standard ports. The PodMonitor should work similarly to the LLM case.

Suggested approach

  1. Reuse the existing PodMonitor logic in internal/provider/podmonitor.go.
  2. Add predictor topology support in the provider's Sync/Cleanup methods.
  3. Ensure the PodMonitor targets the correct label selector for predictor pods (which may differ from LLM pods).
  4. Gate it on the same ENABLE_POD_MONITOR env var.
  5. Optionally allow per-topology enableMetrics flag in the topology parameters.

Acceptance criteria

  • Creating a predictor Instance also creates a PodMonitor in the Instance's namespace.
  • The PodMonitor's label selector matches KServe predictor pods.
  • Deleting the predictor Instance garbage-collects the PodMonitor.
  • PodMonitorEnabled() returns the right result for both topologies.
  • Chainsaw tests verify PodMonitor existence for predictor topology.
  • Existing LLM PodMonitor behavior is unchanged.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions