RHOAI Metrics Dashboard for Single Serving Models

Enable RHOAI User Workload Metrics for Single Serving Models

Prerequisites

• OpenShift 4.10 or later
• OpenShift AI 2.10+ installed
• OpenShift AI KServe installed and configured
• NVIDIA GPU Operator installed and configured

Installation

• Configure Monitoring for the Single Model Serving Platform
• Configure GPU Monitoring Dashboard
• Install the RHOAI Metrics Grafana and Dashboards for Single Serving Models

Usage

Grab the Grafana route and open it in a browser:

NS="user-grafana"
GRAFANA_URL=$(oc get route -n $NS grafana-route -o jsonpath='{.spec.host}')
echo $GRAFANA_URL

• vLLM Model Metrics Dashboard: Provides Model metrics for vLLM Single Serving Models dashboard.

• vLLM Service Performance Dashboard: Provides Service Performance metrics for vLLM Single Serving Models dashboard.

• OpenVino Service Model Metrics Dashboard: Provides metrics for OpenVino Single Serving Models

• OpenVino Model Metrics Dashboard: Provides Service Performance metrics for OpenVino Single Serving Models.

• GPU / vLLM Dashboard: Provides GPU and vLLM performance metrics for Single Serving Models.