Add DCGM data source and extractor for GPU utilization#1895
Conversation
github-actions
Bot
added
size/XL
c45c304 to
81e187d
Compare
| func NewHTTPDataSource[T any](scheme, path string, skipCertVerification bool, | ||
| // portOverride, when non-zero, makes the source scrape podIP:portOverride | ||
| // instead of the endpoint's MetricsHost. | ||
| func NewHTTPDataSource[T any](scheme, path string, portOverride int, skipCertVerification bool, |
There was a problem hiding this comment.
Can we create an Option struct for optional args such as the port override, so NewHTTPDataSource(existingargs, opts... Option)
81e187d to
0c4d7cf
Compare
|
@noalimoy PR has been review and needs rebase/conflict resolution to progress. |
There was a problem hiding this comment.
Clean, well-tested addition - the extractor logic (missing metric, empty samples, multi-GPU max aggregation, normalization) is covered thoroughly, and the port-override mechanism on HTTPDataSource is backward-compatible for existing callers (variadic option, no signature break for source/metrics or source/models).
|
Thanks @elevran for the review! |
0c4d7cf to
0efefb0
Compare
elevran
left a comment
elevran
left a comment
There was a problem hiding this comment.
PR is well scoped and follows repo's conventions.
A few points below worth a look before merge; none are hard blockers.
The EPP has no visibility into GPU hardware pressure. This adds the data-layer plugins that poll DCGM Exporter and store per-endpoint GPU utilization, along with an HTTPDataSource portOverride to target sidecar containers on a different port. Part of llm-d#1706 Signed-off-by: noalimoy <nlimoy@redhat.com>
Add WithUseNodeAddress option to HTTPDataSource, scraping NodeAddress:port instead of PodIP:port for node-level exporters (DCGM DaemonSet). The extractor filters samples by pod label when present. Add source/dcgm factory tests covering defaults, custom params, and invalid-scheme validation. Signed-off-by: noalimoy <nlimoy@redhat.com>
- Reject WithUseNodeAddress without a non-zero WithPortOverride at construction time. - gaugeValue returns an error for non-gauge metrics instead of silent 0. - Add direct tests for NewHTTPDCGMDataSource. Signed-off-by: noalimoy <nlimoy@redhat.com>
0efefb0 to
f95e9de
Compare
fix lint issue Signed-off-by: Noa Limoy <84776878+noalimoy@users.noreply.github.com>
Head branch was pushed to by a user without write access
3 participants
What type of PR is this?
/kind feature
What this PR does / why we need it:
The EPP routes requests using application-level metrics (KV cache, queue
depth) but has no GPU hardware utilization signal. This adds the
data-layer infrastructure to collect GPU metrics from a DCGM Exporter endpoints:
HTTPDataSourceport override to scrape a different port on the samepod IP (backward-compatible, default 0 preserves existing behavior)
dcgm-data-sourceplugin that polls and parses Prometheus metricsdcgm-extractorplugin that readsDCGM_FI_DEV_GPU_UTIL, aggregatesacross GPUs (max), and stores normalized utilization in the endpoint's
AttributeMapGPUUtilizationattribute type withProduces/Consumesdata keyA follow-up PR adds a filter and scorer that consume this attribute.
Which issue(s) this PR fixes:
Part of #1706
Release note (write
NONEif no user-facing change):