lookupplan: add SampleValues to Statistics interface #13783
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Copy the Statistics interface from index.Statistics into the lookupplan package to decouple query planning from the Prometheus TSDB index package.
Add SampleValues method to the Statistics interface and implement sample collection in BlockStatistics. This enables selectivity estimation of regex matchers by running them against a representative sample of actual label values. Sample collection uses: - 1% sampling probability - Max 1024 samples per label - Max 64KB total bytes per label - Deterministic seeding for reproducibility
Tests verify: - Sample values are collected with ~1% probability - Sampling is deterministic (same input -> same output) - Non-existent labels return nil - Small datasets behave correctly
Add configurable parameters for sample values collection: - SampleValuesProbability (default 1%) - SampleValuesMaxCount (default 1024) - SampleValuesMaxBytes (default 64KB) Also refactor Stats() to take CostConfig instead of separate threshold parameters.
Document that sampled values are held in memory and that setting any of the sample values parameters to 0 disables sampling (falling back to 10% selectivity estimation for regexes).
Move string sampling logic from Stats() into a dedicated StringsSampler struct with MaybeSample() and Sampled() methods. This allows for more focused unit testing of the sampling behavior.
dimitarvdimitrov
added
the
changelog-not-needed
Contributor
|
💻 Deploy preview available (lookupplan: add SampleValues to Statistics interface): |
Base automatically changed from
dimitar/lookupplan/define-local-statistics-interface
to
main
December 8, 2025 21:15
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Collects a sample of label values during statistics generation to enable better regex selectivity estimation.
This PR:
SampleValuesmethod toStatisticsRelated to #13782