Skip to content

Commit fc7acd7

Browse files
committed
Fina lslides for 20231025
1 parent 26b2915 commit fc7acd7

8 files changed

+46
-30
lines changed

100_monitoring/prometheus/alerting.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
## Alerting
2+
3+
XXX Prometheus recording rules [](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/#recording-rules)

100_monitoring/prometheus/cluster_scraping.drawio.svg

+3-3
Loading

100_monitoring/prometheus/grafana.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Supports many datasources including Prometheus
66

77
Supports datasource-specific query languages
88

9-
Prometheus community offers pre-created dashcoards [](https://github.com/kubernetes-monitoring/kubernetes-mixin)
9+
Prometheus community offers pre-created dashboards [](https://github.com/kubernetes-monitoring/kubernetes-mixin)
1010

1111
### Demo
1212

100_monitoring/prometheus/prometheus.drawio.svg

+3-3
Loading

100_monitoring/prometheus/promql.md

+26-4
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,6 @@ sum by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m]))
7676

7777
## PromQL: Fun with kube-state-metrics
7878

79-
kube-state-metrics creates additional metrics from cluster state information
80-
81-
Queries:
82-
8379
```plaintext
8480
# Error reasons for pods
8581
count by (reason) (kube_pod_status_reason)
@@ -112,10 +108,36 @@ Filter out sleeping POD container:
112108
container_memory_usage_bytes{namespace="ingress-nginx",container!="",container!="POD"}
113109
```
114110

111+
This also removes the kumulative pod metrics
112+
115113
### `kube_pod_labels` is empty
116114

117115
`kube-state-metrics` does not aggregate labels anymore [#1501](https://github.com/kubernetes/kube-state-metrics/issues/1501)
118116

119117
Set `--metric-labels-allowlist=pods=[*]` in arguments
120118

121119
Or `metricLabelsAllowlist` in Helm chart [](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-state-metrics/values.yaml#L184)
120+
121+
---
122+
123+
## PromQL: Joining metrics
124+
125+
Metrics can be joined...
126+
127+
...to add labels from another metric
128+
129+
Joined metrics must have at least one common label
130+
131+
### Example
132+
133+
Node metrics only reference the instance and do not contain the nodename
134+
135+
`node_uname_info` helps to get the nodename
136+
137+
```plaintext
138+
node_memory_Active_bytes * on(instance) group_left(nodename) node_uname_info
139+
```
140+
141+
Use label `instance` to join metrics
142+
143+
Use all labels from left metrics (`group_left`) and add label `nodename`

100_monitoring/prometheus/values-grafana.yaml

+9-9
Original file line numberDiff line numberDiff line change
@@ -563,15 +563,15 @@ plugins: []
563563
## Configure grafana datasources
564564
## ref: http://docs.grafana.org/administration/provisioning/#datasources
565565
##
566-
datasources: {}
567-
# datasources.yaml:
568-
# apiVersion: 1
569-
# datasources:
570-
# - name: Prometheus
571-
# type: prometheus
572-
# url: http://prometheus-prometheus-server
573-
# access: proxy
574-
# isDefault: true
566+
datasources:
567+
datasources.yaml:
568+
apiVersion: 1
569+
datasources:
570+
- name: Prometheus
571+
type: prometheus
572+
url: http://prometheus-operated.kube-system:9090
573+
access: proxy
574+
isDefault: true
575575
# - name: CloudWatch
576576
# type: cloudwatch
577577
# access: proxy

2023-10-25_heise-Kubernetes-ueberwachen.html

+1
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@
106106
2023-11-23 / 2023-11-30 / 2023-11-07: Workshops [GitLab Ops](https://heise-academy.de/schulungen/einfuehrung-in-gitlab) (1 day) and [GitLab CI](https://heise-academy.de/schulungen/cicd-gitlab) (2 days)
107107
</span>
108108
</textarea></section>
109+
<section data-markdown="100_monitoring/prometheus/promql.md" data-separator="^---$" data-separator-vertical="^--$"></section>
109110

110111
</div>
111112
</div>

2023-10-25_heise-Kubernetes-ueberwachen.md

-10
This file was deleted.

0 commit comments

Comments
 (0)