You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add persist_resource_version documentation to k8sobjects receiver README
- Added persist_resource_version to example configurations
- Documented persist_resource_version configuration property
- Added dedicated "ResourceVersion Persistence" section with:
- Key features and benefits
- Complete configuration example with storage extension
- Storage key format explanation
- Deployment considerations with PVC example
- Updated ConfigMap and Deployment examples to include persistence setup
Co-Authored-By: Dhruv Shah <dhruv.shah@sumologic.com>
Copy file name to clipboardExpand all lines: receiver/k8sobjectsreceiver/README.md
+118-3Lines changed: 118 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,6 +26,7 @@ The following is example configuration
26
26
```yaml
27
27
k8sobjects:
28
28
auth_type: serviceAccount
29
+
storage: file_storage
29
30
k8s_leader_elector: k8s_leader_elector
30
31
objects:
31
32
- name: pods
@@ -39,6 +40,7 @@ The following is example configuration
39
40
mode: watch
40
41
group: events.k8s.io
41
42
namespaces: [default]
43
+
persist_resource_version: true
42
44
```
43
45
44
46
Brief description of configuration properties:
@@ -67,8 +69,8 @@ use this config to specify the group to select. By default, it will select the f
67
69
For example, `events` resource is available in both `v1` and `events.k8s.io/v1` APIGroup. In
68
70
this case, it will select `v1` by default.
69
71
- `k8s_leader_elector` (default: none): if specified, will enable Leader Election by using `k8sleaderelector` extension
70
-
- `storage` (default: none): if specified, will enable the receiver to store the last resource version seen in a file. The resource version will be
71
-
persisted when the watch mode is configured for the given object.
72
+
- `storage` (default: none): specifies the storage extension to use for persisting resourceVersions. Required when `persist_resource_version` is enabled.
73
+
- `persist_resource_version` (default: `false`): When set to `true` (watch-mode only), the receiver persists the resourceVersion after processing each event. On restart, the receiver resumes watching from the persisted resourceVersion, preventing duplicate events. Requires a `storage` extension to be configured.
72
74
73
75
74
76
The full list of settings exposed for this receiver are documented in [config.go](./config.go)
@@ -97,6 +99,90 @@ The `k8sobjectsreceiver` supports collecting a wide range of standard Kubernetes
97
99
98
100
This receiver supports both `pull` and `watch` modes, allowing for flexible and real-time monitoring of these objects. Please note that custom resources are supported only if their CRDs are available in the cluster.
99
101
102
+
### ResourceVersion Persistence
103
+
104
+
When using `watch` mode, the receiver can persist the resourceVersion after processing each event. This ensures that on collector restart, the receiver resumes from where it left off, preventing duplicate events from being collected.
105
+
106
+
#### Key Features
107
+
- **Prevents Duplicate Events**: On restart, watches resume from the last persisted resourceVersion
108
+
- **Per-Namespace Tracking**: Separate resourceVersions are maintained for each namespace
109
+
- **410 Gone Error Recovery**: Automatically handles expired resourceVersions by fetching a fresh one
110
+
- **Opt-in**: Disabled by default; enable via `persist_resource_version: true`
111
+
112
+
#### Configuration Example
113
+
114
+
To enable resourceVersion persistence, you need:
115
+
1. A storage extension (e.g., `file_storage`)
116
+
2. The `storage` field pointing to the extension
117
+
3. `persist_resource_version: true` on objects you want to persist
118
+
119
+
```yaml
120
+
extensions:
121
+
file_storage:
122
+
directory: /var/lib/otelcol/storage
123
+
timeout: 1s
124
+
125
+
receivers:
126
+
k8sobjects:
127
+
auth_type: serviceAccount
128
+
storage: file_storage # Reference to storage extension
0 commit comments