Skip to content

Commit 7451d93

Browse files
committed
fix(dogstatsd): change stream handler task name to avoid unbounded cardinality (#1899)
As stated in the PR title. For DSD stream handlers tasks, we were naming them like so: `dogstatsd-stream-handler-{}-{}`, where the first portion was the listener type (`udp`, `unix`, etc) and the second was a "stream index.". For UDP and UDS datagram, this basically meant a single value: zero. For UDS streams, however, this value would grow over time as new connections were established. On systems with a high volume of connect churn, this can lead to a number of orphaned metrics where the task name is used as a tag, thus creating a unique metric... which in turn becomes a memory leak, and CPU hog as more and more metrics have to be processed, copied during map resizes, and so on. While we'd like to solve the problem of orphaned metrics not being automatically cleaned up, we're making a tactical fix here since the unbounded cardinality being removed will solve our immediate problem of said memory leakage. - [x] Bug fix - [ ] New feature - [ ] Non-functional (chore, refactoring, docs) - [ ] Performance - Manual local testing. DADP-2 Co-authored-by: toby.lawrence <toby.lawrence@datadoghq.com>
1 parent c2ff310 commit 7451d93

1 file changed

Lines changed: 1 addition & 4 deletions

File tree

  • lib/saluki-components/src/sources/dogstatsd

lib/saluki-components/src/sources/dogstatsd/mod.rs

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1078,7 +1078,6 @@ async fn process_listener(
10781078
}
10791079

10801080
let mut stream_shutdown_coordinator = DynamicShutdownCoordinator::default();
1081-
let mut stream_idx: u32 = 0;
10821081

10831082
info!(%listen_addr, "DogStatsD listener started.");
10841083

@@ -1108,11 +1107,9 @@ async fn process_listener(
11081107
};
11091108

11101109
let task_name = format!(
1111-
"dogstatsd-stream-handler-{}-{}",
1110+
"dogstatsd-stream-handler-{}",
11121111
listen_addr.listener_type(),
1113-
stream_idx,
11141112
);
1115-
stream_idx = stream_idx.wrapping_add(1);
11161113
spawn_traced_named(task_name, process_stream(stream, source_context.clone(), handler_context, stream_shutdown_coordinator.register(), enabled_filter));
11171114
}
11181115
Err(e) => {

0 commit comments

Comments
 (0)