Open
Description
Hello, i am a new user, i really liked the project.
I am having some problems with the cluster wide metrics that are show on the dashboard:
I am using kube-prometheus-stack to handle prometheus and grafana and i am using prometheus-adapter for the metrics API.
To get the headlamp to even show anything i had to add a few settings to the chart's values:
kube-prometheus-stack
kubelet:
serviceMonitor:
metricRelabelings:
- action: replace
sourceLabels:
- node
targetLabel: instance
prometheus-node-exporter:
prometheus:
monitor:
attachMetadata:
node: true
relabelings:
- sourceLabels:
- __meta_kubernetes_endpoint_node_name
targetLabel: node
action: replace
regex: (.+)
replacement: ${1}
metricRelabelings:
- action: replace
regex: (.*)
replacement: $1
sourceLabels:
- __meta_kubernetes_pod_node_name
targetLabel: kubernetes_nodeprometheus-adapter (which is normal to get the metrics apis)
resource:
cpu:
containerQuery: |
sum by (<<.GroupBy>>) (
rate(container_cpu_usage_seconds_total{container!="",<<.LabelMatchers>>}[3m])
)
nodeQuery: |
sum by (<<.GroupBy>>) (
rate(node_cpu_seconds_total{mode!="idle",mode!="iowait",mode!="steal",<<.LabelMatchers>>}[3m])
)
resources:
overrides:
node:
resource: node
namespace:
resource: namespace
pod:
resource: pod
containerLabel: container
memory:
containerQuery: |
sum by (<<.GroupBy>>) (
avg_over_time(container_memory_working_set_bytes{container!="",<<.LabelMatchers>>}[3m])
)
nodeQuery: |
sum by (<<.GroupBy>>) (
avg_over_time(node_memory_MemTotal_bytes{<<.LabelMatchers>>}[3m])
-
avg_over_time(node_memory_MemAvailable_bytes{<<.LabelMatchers>>}[3m])
)
resources:
overrides:
node:
resource: node
namespace:
resource: namespace
pod:
resource: pod
containerLabel: container
window: 3mIndividual node's CPU values are correct, the memory value is correct as well but the unit is different:
Is this a headlamp problem or this a prometheus(me) problem?
Thanks for the help and the project have a nice day.