Proposal: discovery.snmp for prometheus.exporter.snmp target discovery

[Mesverrum]

Component(s)

prometheus.exporter.snmp, discovery.file, discovery.http, discovery.relabel

Background

When you compare Alloy + prometheus.exporter.snmp to competitive solutions that monitor network hardware, a major user expectation is some mechanism for device discovery.

In the field, very few network teams have what they consider a reliable external source of truth for inventory, and even fewer of those systems emit a Prometheus HTTP SD endpoint. That leaves operators to hand-maintain target lists or DIY a sidecar (scripts, CMDB exports, a second collector).

prometheus.exporter.snmp already polls well. The missing piece is finding devices and mapping each one to a named auth and a module= list. SNMP credentials and MIB modules are not Kubernetes labels; they have to come from a Discoverer.

This should stay Prometheus-shaped. Named auth= / module= on targets, secrets only in snmp.yml auths:. Not a side-channel device file, and not community strings on labels.

Related prior art: SuperQ’s fingerprint-based module matching in snmp_exporter#1468 (still open). Until that lands in the exporter, the same matching can run at SD time so stock prometheus.exporter.snmp stays unchanged.

Related Alloy requests, out of scope here (separate proposals):

• SNMP trap listener — #440
• OTel netflow contrib receiver — #6304

Those benefit from a shared target catalog later. This issue is only “find the estate and emit scrape targets.”

Proposal

Add an experimental discovery.snmp component: a Prometheus Discoverer that probes CIDRs (and optionally crawls LLDP/CDP neighbors), fingerprints sysObjectID, and exports targets for prometheus.exporter.snmp.

Why a component, not a sidecar

Operators can already approximate this with a CLI that writes file_sd / HTTP SD and discovery.file / discovery.http. That works for a lab. It does not work well for Fleet / remotecfg: Fleet can only push config for components in the running binary. Discovery groups (CIDRs + auth names) are operator config and belong in River.

A small CLI that shares the same library is still useful for vanilla Prometheus. That can be a follow-up; the Alloy-native form is the component.

Shape

discovery.snmp "fabric" {
  refresh_interval = "15m"

  group {
    name  = "hq"
    cidrs = ["172.20.20.0/24"]
    auths = ["public_v2"]   // names from snmp.yml auths: — never the secret
  }
}

prometheus.exporter.snmp "fabric" {
  targets = discovery.snmp.fabric.targets
}

Operator input is groups: CIDRs and/or crawl seeds, plus named auths scoped to that group. Do not dump every community/user onto every subnet.

Discovery loop

1. Sweep — expand group CIDRs (default cap ~1024 hosts: IPv4 /22, IPv6 /118; wider only with allow_large). Optional ICMP pre-filter (ping, default true; needs CAP_NET_RAW — document that, and that a missing ping is not false).
2. Identity probe — SNMP GET of sysObjectID / sysName / sysDescr using the group’s named auths in order (v1/v2c and v3 USM as already modeled in snmp_exporter auths:).
3. Fingerprint — SuperQ-style matchers (#1468): regex on probe labels → module= list. Unknown sysObjectID falls back to a default (e.g. if_mib). Intersect emitted module names with the live snmp.yml modules: keys so unknown names never reach the exporter (empty walk / panic).
4. Crawl (optional) — from seeds + catalog, walk LLDP/CDP, probe neighbors that fall inside the group CIDRs. One hop per interval.
5. Publish — export discovery.Exports targets. Failed scans keep the last catalog. Overlapping ticks skip. Drop after N silent cycles (misses, persistable via state_path).

Exported labels (Discoverer contract)

Label	Meaning
address	Canonical SNMP IP
auth	Auth name from snmp.yml
module	Comma-separated snmp_exporter modules
name / device_name	From sysName (or the address)
sysObjectID	When the probe returned one
snmp_group	Discovery group name

Never export community strings or v3 secrets.

Optional: collapse duplicate sysNames to one scrape identity (lowest IP wins). Opt out with allow_duplicate_sysname for cloned IoT hostnames.

Optional: a snmp_tier label (hot / cold / topology) so operators can discovery.relabel and scrape octets/oper on 60s vs names/errors/LLDP on minutes–hours. A single-list fingerprinter still works (tier = "hot" or one scrape).

Safety / ops

• Secrets stay in snmp.yml auths:. River only names them.
• Default CIDR cap; allow_large is explicit.
• ICMP is a filter, not a requirement — ping = false for unprivileged containers.
• Health: Unknown until first scan; Healthy with discovered N devices, M targets; Unhealthy on config/scan failure, last targets retained.
• Debug metrics for scan duration, skips, probe success/error reasons, catalog size (same pattern as other discovery.* components).
• Live Debugging: publish the current target list on successful scans.
• Start experimental. Integration tests with a fake SNMP agent before GA.

Non-goals (this issue)

• Changing prometheus.exporter.snmp (no CIDR walker inside the exporter).
• Shipping a curated vendor MIB library (separate discussion; discovery works with stock if_mib + any snmp.yml).
• Trap receiver, syslog wiring, or NetFlow (#440, #6304).
• ARP / OSPF / BGP as neighbor sources.
• Acting as a CMDB / NetBox replacement.

Alternatives considered

Approach	Why not as the product
Hand-written prometheus.exporter.snmp targets / discovery.file	What people do today; does not scale; new gear is a ticket.
HTTP SD from NetBox / CMDB	Fine when it exists; most network teams do not have a reliable one.
Wait for snmp_exporter #1468	Solves module selection given a target, not “find the estate.” Complementary — we would drop in-process fingerprinting if/when the exporter does it.
Sidecar CLI only	Portable to Prometheus, not Fleet-manageable, extra process/image story.

Open questions for the team

1. In-tree vs community component — this is core network collection next to prometheus.exporter.snmp. I’d rather it be first-party experimental than a community module, but that is your call.
2. ICMP default — default ping = true matches field practice (don’t SNMP-timeout a /24 of dark IPs) but surprises unprivileged containers. Prefer default true + loud docs, or default false?
3. Fingerprinter YAML in-tree — ship a small default matcher file, or require the operator to provide one?
4. CLI / HTTP SD in the same PR, or component-only first?

I have a working prototype (library + discovery.snmp + docs/metrics/health) used in a lab against SR Linux. Happy to open a draft PR against this issue or trim it to whatever slice you want to review first.

AI disclosure

• This issue was substantially generated with AI assistance.

Tip

_{React with 👍 if this issue is important to you.}

[Mesverrum]

Sibling proposals (discovery-only here):

• Traps as logs: #6943
• NetFlow wrap (comment on existing request): Feature Request: Add OTel Netflow Receiver #6304 (comment)