Skip to content

通过独占GPU的方式调度,31992监控指标中无pod显存使用指标 #931

Open
@Wangyxiang

Description

@Wangyxiang

使用的hami版本:2.5.0
yaml文件部分内容:
resources: limits: cpu: "96" memory: 1Ti nvidia.com/gpu: "8" requests: cpu: "96" memory: 1Ti nvidia.com/gpu: "8" nodeName: node14-h20
node14对应的31992指标页面似乎并没有
Device_memory_desc_of_container和Device_utilization_desc_of_container指标
`
curl node14-h20:31992/metrics

HELP HostCoreUtilization GPU core utilization

TYPE HostCoreUtilization gauge

HostCoreUtilization{deviceidx="0",deviceuuid="GPU-b230f4ad-7dd4-c343-7dcc-fc2217f7ed7f",zone="vGPU"} 13
HostCoreUtilization{deviceidx="1",deviceuuid="GPU-6ff6801c-e914-5780-5576-a38fc99433e5",zone="vGPU"} 72
HostCoreUtilization{deviceidx="2",deviceuuid="GPU-bade86be-5a20-ff41-4ced-e078acb5abfa",zone="vGPU"} 70
HostCoreUtilization{deviceidx="3",deviceuuid="GPU-61dc4819-fa25-0e13-ab49-1baebdcfce54",zone="vGPU"} 69
HostCoreUtilization{deviceidx="4",deviceuuid="GPU-4436e555-2dab-98f7-e090-94f1f8fb7dcf",zone="vGPU"} 72
HostCoreUtilization{deviceidx="5",deviceuuid="GPU-6dc15c0c-b6d7-3dc7-aee4-555816d3b4ce",zone="vGPU"} 72
HostCoreUtilization{deviceidx="6",deviceuuid="GPU-0d12c911-d9aa-5765-6faa-c08aec4c66a2",zone="vGPU"} 72
HostCoreUtilization{deviceidx="7",deviceuuid="GPU-4b97f297-7f8b-81fb-3a21-7d832004eec8",zone="vGPU"} 72

HELP HostGPUMemoryUsage GPU device memory usage

TYPE HostGPUMemoryUsage gauge

HostGPUMemoryUsage{deviceidx="0",deviceuuid="GPU-b230f4ad-7dd4-c343-7dcc-fc2217f7ed7f",zone="vGPU"} 7.9258648576e+10
HostGPUMemoryUsage{deviceidx="1",deviceuuid="GPU-6ff6801c-e914-5780-5576-a38fc99433e5",zone="vGPU"} 7.9380283392e+10
HostGPUMemoryUsage{deviceidx="2",deviceuuid="GPU-bade86be-5a20-ff41-4ced-e078acb5abfa",zone="vGPU"} 7.9380283392e+10
HostGPUMemoryUsage{deviceidx="3",deviceuuid="GPU-61dc4819-fa25-0e13-ab49-1baebdcfce54",zone="vGPU"} 7.9380283392e+10
HostGPUMemoryUsage{deviceidx="4",deviceuuid="GPU-4436e555-2dab-98f7-e090-94f1f8fb7dcf",zone="vGPU"} 7.9380283392e+10
HostGPUMemoryUsage{deviceidx="5",deviceuuid="GPU-6dc15c0c-b6d7-3dc7-aee4-555816d3b4ce",zone="vGPU"} 7.9380283392e+10
HostGPUMemoryUsage{deviceidx="6",deviceuuid="GPU-0d12c911-d9aa-5765-6faa-c08aec4c66a2",zone="vGPU"} 7.9380283392e+10
HostGPUMemoryUsage{deviceidx="7",deviceuuid="GPU-4b97f297-7f8b-81fb-3a21-7d832004eec8",zone="vGPU"} 7.8876966912e+10
`

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions