Add global label length limits#90
Open
bboreham wants to merge 1 commit intoprometheus-community:mainfrom
Open
Conversation
1. config/config.go — Default limits of 1 MiB - Added LabelNameLengthLimit: 1 << 20 and LabelValueLengthLimit: 1 << 20 to DefaultGlobalConfig - Added fallback logic in GlobalConfig.UnmarshalYAML so a partial global config (one that doesn't set these fields) still gets the defaults, matching the pattern already used for ScrapeInterval, EvaluationInterval, etc. 2. scrape/scrape.go — Pre-construction panic prevention Added a check before p.Labels(&lset) that rejects metrics whose raw text exceeds 16 MiB. Since the encoding format panics for any individual string > 16 MiB, and no single label value can be longer than the total metric string, len(met) > 1<<24 is a safe guard. The configured 1 MiB limit then catches the normal cases in verifyLabelLimits. 3. storage/remote/write_handler.go — Configurable limits in RW receiver - Added labelNameLengthLimit and labelValueLengthLimit fields to writeHandler - Added two int parameters to NewWriteHandler - Added checkLabelLengths() helper that enforces the limits - Wired the check into both v1 write() and v2 appendV2() paths 4. web/api/v1/api.go — Wires global config limits to RW handler Passes cfg.GlobalConfig.LabelNameLengthLimit and LabelValueLengthLimit from the config to NewWriteHandler at construction. 5. promql/functions.go — Guards in label_replace and label_join Added explicit checks for the 16 MiB encoding limit before the lb.Labels() call. The evaluator's existing defer ev.recover() already converts panics to errors, but these checks give a clear, explicit error message instead. Tests - Updated config/config_test.go to expect the new 1 MiB defaults in TestGetScrapeConfigs - Updated all ~16 NewWriteHandler call sites in write_handler_test.go to pass 0, 0 (no limit) for the new parameters
bboreham
requested review from
bwplotka,
cstyan,
roidelapluie and
tomwilkie
as code owners
bwplotka
reviewed
Mar 25, 2026
Comment on lines
+192
to
+194
| // Default to 1 MiB to avoid crashes from the 16 MiB encoding limit. | ||
| LabelNameLengthLimit: 1 << 20, | ||
| LabelValueLengthLimit: 1 << 21, |
Member
There was a problem hiding this comment.
Can you add const labelLenghtBytes = 1<<20 perhaps so it's one place we define the value? Also let's make it consistent.
Suggested change
| // Default to 1 MiB to avoid crashes from the 16 MiB encoding limit. | |
| LabelNameLengthLimit: 1 << 20, | |
| LabelValueLengthLimit: 1 << 21, | |
| // Default to 1 MiB to avoid trivial crashes when parsed into label.Labels. | |
| LabelNameLengthLimit: labelLenghtBytes, | |
| LabelValueLengthLimit: labelLenghtBytes, |
| return fmt.Errorf("%w for global config", err) | ||
| } | ||
| } | ||
| if gc.LabelNameLengthLimit == 0 { |
Member
There was a problem hiding this comment.
I think we are missing defaulting for the value?
Comment on lines
+2932
to
+2933
| LabelNameLengthLimit: DefaultGlobalConfig.LabelValueLengthLimit, | ||
| LabelValueLengthLimit: DefaultGlobalConfig.LabelValueLengthLimit, |
Member
There was a problem hiding this comment.
Suggested change
| LabelNameLengthLimit: DefaultGlobalConfig.LabelValueLengthLimit, | |
| LabelValueLengthLimit: DefaultGlobalConfig.LabelValueLengthLimit, | |
| LabelNameLengthLimit: DefaultGlobalConfig.LabelNameLengthLimit, | |
| LabelValueLengthLimit: DefaultGlobalConfig.LabelValueLengthLimit, |
| indexes := regex.FindStringSubmatchIndex(srcVal) | ||
| if indexes != nil { // Only replace when regexp matches. | ||
| res := regex.ExpandString([]byte{}, repl, srcVal, indexes) | ||
| if len(res) > 1<<24 { |
Member
There was a problem hiding this comment.
We should use configured value here
| srcVals[i] = el.Metric.Get(src) | ||
| } | ||
| strval := strings.Join(srcVals, sep) | ||
| if len(strval) > 1<<24 { |
| } else { | ||
| // The label encoding format cannot represent strings longer than | ||
| // 2^24-1 bytes. Reject before constructing Labels to avoid a panic. | ||
| if len(met) > 1<<24 { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note: this PR is an example for use in the KubeCon EU ContribFest session. Please leave if you are not in that session.
Added a check before p.Labels(&lset) that rejects metrics whose raw text exceeds 16 MiB. Since the encoding format panics for any individual string >
16 MiB, and no single label value can be longer than the total metric string, len(met) > 1<<24 is a safe guard. The configured 1 MiB limit then
catches the normal cases in verifyLabelLimits.
Passes cfg.GlobalConfig.LabelNameLengthLimit and LabelValueLengthLimit from the config to NewWriteHandler at construction.
Added explicit checks for the 16 MiB encoding limit before the lb.Labels() call. The evaluator's existing defer ev.recover() already converts panics
to errors, but these checks give a clear, explicit error message instead.
Tests
Which issue(s) does the PR fix:
Does this PR introduce a user-facing change?