You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> Metrics emitted by the `Microsoft.Extensions.Diagnostics.ResourceMonitoring` meter are in experimental stage. This means that there could be breaking changes to them.
81
+
> Some metrics emitted by the `Microsoft.Extensions.Diagnostics.ResourceMonitoring` meter (such as disk I/O) are in experimental stage and not documented here.
81
82
82
83
##### Metric: `container.cpu.limit.utilization`
83
84
84
85
The instrument is only available on a system running on containers both on Windows and Linux.
85
86
86
87
| Name | Instrument Type | Unit (UCUM) | Description |
|`container.cpu.limit.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The CPU consumption of the running containerized application relative to resource limit in range `[0, 1]`. |
89
+
|`container.cpu.limit.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The CPU consumption of the running containerized application relative to resource limit. Linux: range `[0, 1]` by default. Windows: range `[0, 100]` by default. See `UseZeroToOneRangeForMetrics` and `UseZeroToOneRangeForLinuxMetrics` options to change the default behavior. |
89
90
90
91
Available starting in `Microsoft.Extensions.Diagnostics.ResourceMonitoring` 8.8.0.
91
92
92
93
##### Metric: `container.cpu.request.utilization`
93
94
94
-
The instrument is only available on a system running on containers on Linux.
95
+
The instrument is only available on a system running on containers both on Windows and Linux.
95
96
96
97
| Name | Instrument Type | Unit (UCUM) | Description |
|`container.cpu.request.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The CPU consumption of the running containerized application relative to resource request in range `[0, 1]`. |
99
+
|`container.cpu.request.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The CPU consumption of the running containerized application relative to resource request. Linux: range `[0, 1]` by default. Windows: range `[0, 100]` by default. See `UseZeroToOneRangeForMetrics` and `UseZeroToOneRangeForLinuxMetrics` options to change the default behavior. |
99
100
100
101
Available starting in `Microsoft.Extensions.Diagnostics.ResourceMonitoring` 8.8.0.
101
102
@@ -115,10 +116,20 @@ The instrument is only available on a system running on containers both on Windo
115
116
116
117
| Name | Instrument Type | Unit (UCUM) | Description |
|`container.memory.limit.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The memory consumption of the running containerized application relative to resource limit in range `[0, 1]`. |
119
+
|`container.memory.limit.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The memory consumption of the running containerized application relative to resource limit. Linux: range `[0, 1]` by default. Windows: range `[0, 100]` by default. See `UseZeroToOneRangeForMetrics` and `UseZeroToOneRangeForLinuxMetrics` options to change the default behavior. |
119
120
120
121
Available starting in `Microsoft.Extensions.Diagnostics.ResourceMonitoring` 8.8.0.
|`container.memory.request.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The memory consumption of the running containerized application relative to resource request. Linux: range `[0, 1]` by default. Windows: range `[0, 100]` by default. See `UseZeroToOneRangeForMetrics` and `UseZeroToOneRangeForLinuxMetrics` options to change the default behavior. |
130
+
131
+
Available starting in `Microsoft.Extensions.Diagnostics.ResourceMonitoring` 9.8.0.
132
+
122
133
##### Metric: `container.memory.usage`
123
134
124
135
The instrument is only available on a system running on containers either on Windows or Linux.
@@ -133,15 +144,15 @@ Available starting in `Microsoft.Extensions.Diagnostics.ResourceMonitoring` 9.8.
133
144
134
145
| Name | Instrument Type | Unit (UCUM) | Description |
|`process.cpu.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The CPU consumption of the running application in range `[0, 1]`. |
147
+
|`process.cpu.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The CPU consumption of the running application. Linux: range `[0, 1]` by default. Windows: range `[0, 100]` by default. See `UseZeroToOneRangeForMetrics` and `UseZeroToOneRangeForLinuxMetrics` options to change the default behavior. |
|`dotnet.process.memory.virtual.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The memory consumption of the running application in range `[0, 1]`. |
155
+
|`dotnet.process.memory.virtual.utilization`|<xref:System.Diagnostics.Metrics.ObservableGauge`1>|`1`| The memory consumption of the running application. Linux: range `[0, 1]` by default. Windows: range `[0, 100]` by default. See `UseZeroToOneRangeForMetrics` and `UseZeroToOneRangeForLinuxMetrics` options to change the default behavior. |
Copy file name to clipboardExpand all lines: docs/core/diagnostics/diagnostic-resource-monitoring.md
+172-1Lines changed: 172 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -98,8 +98,9 @@ For the source code of this example, see the [Resource monitoring sample](https:
98
98
Since the <xref:Microsoft.Extensions.Diagnostics.ResourceMonitoring.IResourceMonitor> interface is deprecated, migrate to the metrics-based approach. The `Microsoft.Extensions.Diagnostics.ResourceMonitoring` package provides several metrics that you can use instead, for instance:
99
99
100
100
-`container.cpu.limit.utilization`: The CPU consumption share of the running containerized application relative to resource limit in range `[0, 1]`. Available for containerized apps on Linux and Windows.
101
-
-`container.cpu.request.utilization`: The CPU consumption share of the running containerized application relative to resource request in range `[0, 1]`. Available for containerized apps on Linux.
101
+
-`container.cpu.request.utilization`: The CPU consumption share of the running containerized application relative to resource request in range `[0, 1]`. Available for containerized apps on Linux and Windows.
102
102
-`container.memory.limit.utilization`: The memory consumption share of the running containerized application relative to resource limit in range `[0, 1]`. Available for containerized apps on Linux and Windows.
103
+
-`container.memory.request.utilization`: The memory consumption share of the running containerized application relative to resource request in range `[0, 1]`. Available for containerized apps on Linux and Windows.
103
104
104
105
For more information about the available metrics, see the [Built-in metrics: Microsoft.Extensions.Diagnostics.ResourceMonitoring](built-in-metrics-diagnostics.md#microsoftextensionsdiagnosticsresourcemonitoring) section.
105
106
@@ -124,6 +125,176 @@ The following is an example of the output from the preceding code:
124
125
125
126
For the complete source code of this example, see the [Resource monitoring with manual metrics sample](https://github.com/dotnet/docs/tree/main/docs/core/diagnostics/snippets/resource-monitoring-with-manual-metrics).
126
127
128
+
## Kubernetes resource monitoring
129
+
130
+
When your application runs inside a Kubernetes cluster, you typically configure resource limits and requests in your pod specification. The [Microsoft.Extensions.Diagnostics.ResourceMonitoring.Kubernetes](https://www.nuget.org/packages/Microsoft.Extensions.Diagnostics.ResourceMonitoring.Kubernetes) NuGet package extends the base resource monitoring library to read these values from environment variables exposed by the [Kubernetes Downward API](https://kubernetes.io/docs/concepts/workloads/pods/downward-api/).
131
+
132
+
This package automatically detects your container's CPU and memory boundaries and emits accurate utilization metrics relative to those values.
133
+
134
+
> [!TIP]
135
+
> If your cluster runs Kubernetes v1.32 or later with cgroup v2, always prefer `AddKubernetesResourceMonitoring()` over `AddResourceMonitoring()` for accurate request-based utilization metrics. The Kubernetes package reads CPU and memory requests directly from environment variables, bypassing the cgroup weight inversion that can produce inaccurate values. See [Known limitations](#known-limitations-on-cgroup-v2) for details.
136
+
137
+
### How it works
138
+
139
+
The Kubernetes Downward API can expose pod resource limits and requests as environment variables. The `Microsoft.Extensions.Diagnostics.ResourceMonitoring.Kubernetes` package reads these environment variables at startup and uses them to calculate utilization metrics.
140
+
141
+
You choose a prefix for your environment variables (for example, `MY_APP_`). The library then looks for the following variables:
142
+
143
+
| Environment variable | Description |
144
+
| --- | --- |
145
+
|`<PREFIX>LIMITS_CPU`| CPU limit in millicores (for example, `2000` for 2 cores) |
146
+
|`<PREFIX>LIMITS_MEMORY`| Memory limit in bytes |
147
+
|`<PREFIX>REQUESTS_CPU`| CPU request in millicores (optional, defaults to limit value) |
148
+
|`<PREFIX>REQUESTS_MEMORY`| Memory request in bytes (optional, defaults to limit value) |
149
+
150
+
At minimum, you must set `<PREFIX>LIMITS_CPU` and `<PREFIX>LIMITS_MEMORY` to non-zero values. If you omit the request variables or set them to zero, the library defaults them to the corresponding limit values.
151
+
152
+
### Configure the Downward API
153
+
154
+
To expose resource limits as environment variables, add `resourceFieldRef` entries to your Kubernetes deployment manifest:
155
+
156
+
```yaml
157
+
apiVersion: apps/v1
158
+
kind: Deployment
159
+
metadata:
160
+
name: my-app
161
+
spec:
162
+
template:
163
+
spec:
164
+
containers:
165
+
- name: my-app
166
+
resources:
167
+
requests:
168
+
cpu: "500m"
169
+
memory: "256Mi"
170
+
limits:
171
+
cpu: "1000m"
172
+
memory: "512Mi"
173
+
env:
174
+
- name: MY_APP_LIMITS_CPU
175
+
valueFrom:
176
+
resourceFieldRef:
177
+
resource: limits.cpu
178
+
divisor: "1m"
179
+
- name: MY_APP_LIMITS_MEMORY
180
+
valueFrom:
181
+
resourceFieldRef:
182
+
resource: limits.memory
183
+
divisor: "1"
184
+
- name: MY_APP_REQUESTS_CPU
185
+
valueFrom:
186
+
resourceFieldRef:
187
+
resource: requests.cpu
188
+
divisor: "1m"
189
+
- name: MY_APP_REQUESTS_MEMORY
190
+
valueFrom:
191
+
resourceFieldRef:
192
+
resource: requests.memory
193
+
divisor: "1"
194
+
```
195
+
196
+
The `divisor: "1m"` for CPU fields ensures Kubernetes expresses the value in millicores (for example, a `500m` limit becomes `500`). The `divisor: "1"` for memory fields returns the value in bytes.
197
+
198
+
### Register Kubernetes resource monitoring
199
+
200
+
In your application code, call <xref:Microsoft.Extensions.DependencyInjection.KubernetesResourceQuotaServiceCollectionExtensions.AddKubernetesResourceMonitoring*> with the environment variable prefix that matches your Kubernetes manifest:
The `"MY_APP_"` prefix tells the library to look for environment variables named `MY_APP_LIMITS_CPU`, `MY_APP_LIMITS_MEMORY`, `MY_APP_REQUESTS_CPU`, and `MY_APP_REQUESTS_MEMORY`.
205
+
206
+
> [!IMPORTANT]
207
+
> Don't call `AddResourceMonitoring()` in addition to `AddKubernetesResourceMonitoring()`. The Kubernetes method already registers all necessary base resource monitoring components. Calling both methods can result in conflicting service registrations.
208
+
209
+
### Emitted metrics
210
+
211
+
Once registered, the library emits the following metrics under the `Microsoft.Extensions.Diagnostics.ResourceMonitoring` meter. It uses the Kubernetes resource boundaries you configured for its calculations:
212
+
213
+
| Metric name | Type | Description |
214
+
| --- | --- | --- |
215
+
| `container.cpu.limit.utilization` | ObservableGauge | CPU usage relative to the configured limit, in the range [0, 1] |
216
+
| `container.cpu.request.utilization` | ObservableGauge | CPU usage relative to the configured request, in the range [0, 1] |
217
+
| `container.cpu.time` | ObservableCounter | Total CPU time consumed in seconds, with a `cpu.mode` dimension (user/system) |
218
+
| `container.memory.limit.utilization` | ObservableGauge | Memory usage relative to the configured limit, in the range [0, 1] |
219
+
| `container.memory.request.utilization` | ObservableGauge | Memory usage relative to the configured request, in the range [0, 1] |
220
+
| `container.memory.usage` | ObservableUpDownCounter | Memory usage in bytes |
221
+
| `process.cpu.utilization` | ObservableGauge | Process CPU usage relative to the CPU limit, in the range [0, 1] |
222
+
| `dotnet.process.memory.virtual.utilization` | ObservableGauge | Process memory usage relative to the memory limit, in the range [0, 1] |
223
+
224
+
For the full list of metrics emitted by the base resource monitoring library, see [.NET extensions metrics: Microsoft.Extensions.Diagnostics.ResourceMonitoring](built-in-metrics-diagnostics.md#microsoftextensionsdiagnosticsresourcemonitoring).
225
+
226
+
### Example output
227
+
228
+
When you run your application inside a Kubernetes pod with the environment variables configured, the metrics produce values such as:
229
+
230
+
```
231
+
Instrument: container.cpu.limit.utilization
232
+
Value: 0.23
233
+
234
+
Instrument: container.cpu.request.utilization
235
+
Value: 0.46
236
+
237
+
Instrument: container.memory.limit.utilization
238
+
Value: 0.61
239
+
240
+
Instrument: container.memory.request.utilization
241
+
Value: 0.78
242
+
243
+
Instrument: container.memory.usage
244
+
Value: 312475648 (By)
245
+
246
+
Instrument: container.cpu.time
247
+
Value: 142.35 (s), cpu.mode=user
248
+
Value: 28.91 (s), cpu.mode=system
249
+
```
250
+
251
+
In this example, the pod uses 23% of its CPU limit and 61% of its memory limit. Because the CPU request is lower than the CPU limit, `container.cpu.request.utilization` shows a higher value (46%) for the same absolute CPU usage.
252
+
253
+
### Collect metrics with OpenTelemetry
254
+
255
+
To export these metrics to your observability backend, register the meter with OpenTelemetry:
builder.AddOtlpExporter(); // Or any other metrics exporter
263
+
});
264
+
```
265
+
266
+
### Known limitations on cgroup v2
267
+
268
+
Starting with Kubernetes v1.32 (using runc 1.3.2+ or crun 1.23+), the OCI runtime uses a new quadratic formula to convert cgroup v1 CPU shares to cgroup v2 CPU weight. This change affects the accuracy of the base `AddResourceMonitoring()` method when it attempts to derive CPU request values from cgroup parameters on Linux.
269
+
270
+
The core issue is that the base resource monitoring library reads `cpu.weight` from the cgroup v2 filesystem and reverse-converts it to estimate the original CPU request in millicores. However, the new conversion formula is **many-to-one**: multiple milliCPU values map to the same `cpu.weight`. For example, milliCPU values from 90 through 109 all produce `cpu.weight = 17`. Reversing this mapping cannot recover the exact original value, which means `container.cpu.request.utilization` may report inaccurate values.
271
+
272
+
The following metrics are affected:
273
+
274
+
| Metric | Affected? | Reason |
275
+
| --- | --- | --- |
276
+
| `container.cpu.request.utilization` | Yes | Relies on inferred CPU request from `cpu.weight` |
277
+
| `container.memory.request.utilization` | Yes | Relies on inferred memory request from cgroup parameters |
278
+
| `container.cpu.limit.utilization` | No | Uses `cpu.max`, not `cpu.weight` |
279
+
| `container.memory.limit.utilization` | No | Uses memory limit directly from cgroup |
280
+
281
+
**How the Kubernetes package solves this:** `AddKubernetesResourceMonitoring()` reads the actual CPU and memory request values directly from environment variables you configure through the Downward API. It never reverse-converts cgroup parameters, so utilization metrics are always accurate regardless of which OCI runtime conversion formula your cluster uses.
282
+
283
+
For more information, see:
284
+
285
+
- [New cgroup v1 to v2 CPU conversion formula (Kubernetes blog)](https://kubernetes.io/blog/2026/01/30/new-cgroup-v1-to-v2-cpu-conversion-formula/)
When deploying .NET applications to Kubernetes, consider the following recommendations:
291
+
292
+
- **Use the Kubernetes package in Kubernetes environments.** Always prefer `AddKubernetesResourceMonitoring()` over `AddResourceMonitoring()` when running in a Kubernetes cluster. It provides accurate resource utilization metrics by reading CPU and memory request values directly from environment variables rather than inferring them from cgroup parameters.
293
+
294
+
- **Expose resource metadata through the Downward API.** Configure your deployment manifests to expose CPU and memory limits and requests as environment variables. This ensures the library has access to the exact values you specified in your pod spec.
295
+
296
+
- **Test metric accuracy after cluster upgrades.** When upgrading your Kubernetes cluster or OCI runtime (runc, crun), verify that your resource utilization metrics still report expected values, especially if you use the base `AddResourceMonitoring()` method.
297
+
127
298
## Kubernetes probes
128
299
129
300
In addition to resource monitoring, apps that exist within a Kubernetes cluster report their health through diagnostic probes. The [Microsoft.Extensions.Diagnostics.Probes](https://www.nuget.org/packages/Microsoft.Extensions.Diagnostics.Probes) NuGet package provides support for Kubernetes probes. It externalizes various [health checks](diagnostic-health-checks.md) that align with various Kubernetes probes, for example:
0 commit comments