Sanitize index-derived label values to valid UTF-8#1646
Merged
Conversation
Invalid UTF-8 bytes in index values previously caused NewConstMetric to fail, dropping the metric and emitting a scrape error. Replace invalid sequences instead, matching how pduValueAsString handles metric values. Signed-off-by: lukeod <l.odonnell11@gmail.com>
SuperQ
approved these changes
Jul 2, 2026
Member
There was a problem hiding this comment.
Good find, thanks!
I think the limitation of duplicates is fine. For the most part we recommend users keep the numeric indexes (i.e ifIndex) to avoid string collisions already (i.e. ifDescr).
If the collision is from the new enum mapping, it's at least easier to fix in the MIB. For #1610, it could be more difficult to avoid collisions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#968 replaced invalid UTF-8 in metric values with U+FFFD, but only on the
pduValueAsStringpath. Labels built from table indexes go throughindexesToLabelsdirectly, so they never get sanitized. When a device returns non-UTF-8 bytes in a string-typed index (e.g. aDisplayStringindex containing Latin-1 or raw binary),NewConstMetricrejects the label value, the metric is replaced with an invalid metric, and the gather fails with ansnmp_error, so the whole scrape errors out.This applies the same
strings.ToValidUTF8(..., "�")treatment to index label values, so those metrics survive with a replacement character instead of erroring out. Valid non-ASCII UTF-8 in index values passes through unchanged; only invalid byte sequences are replaced.One known limitation:
ToValidUTF8replaces each run of invalid bytes with a single replacement character, so two rows whose indexes differ only within an invalid run (say one-byteDisplayStringindexes 0xFE and 0xFF) end up with identical label sets, and the gatherer rejects the duplicate series. That still fails the scrape, but it requires colliding garbage rather than any garbage, and it's the same trade-off #968 already accepted for values. Lookups are unaffected either way, since they key off the raw index OIDs rather than the sanitized string.This does change behavior asserted by two existing tests, which expected the invalid-UTF-8 error. Those assertions date back to #303, which was about turning a panic into a graceful error rather than about erroring being the desired end state. #968 later settled on sanitizing for the value path, and this brings the index path in line with that. I've updated the two test cases accordingly.
The practical motivation is to ensure safety for #1610, which adds
display_hintformatting for index labels - thea/thint formats write device bytes into labels and would hit that error path.