feat(tag_cardinality_limit): add per-tag cache_size_per_key override in probabilistic mode#25650
feat(tag_cardinality_limit): add per-tag cache_size_per_key override in probabilistic mode#25650ArunPiduguDD wants to merge 1 commit into
Conversation
3bc46fc to
b845b1c
Compare
b845b1c to
9850e50
Compare
…in probabilistic mode
9850e50 to
ce0e240
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ce0e240502
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| type: { | ||
| required: true | ||
| type: string: enum: { | ||
| file: "Exposes data from a static file as an enrichment table." | ||
| memory: """ | ||
| Exposes data from a memory cache as an enrichment table. The cache can be written to using | ||
| a sink. | ||
| """ | ||
| geoip: """ | ||
| Exposes data from a [MaxMind][maxmind] [GeoIP2][geoip2] database as an enrichment table. | ||
|
|
||
| [maxmind]: https://www.maxmind.com/ | ||
| [geoip2]: https://www.maxmind.com/en/geoip2-databases | ||
| """ | ||
| mmdb: """ | ||
| Exposes data from a [MaxMind][maxmind] database as an enrichment table. | ||
|
|
||
| [maxmind]: https://www.maxmind.com/ | ||
| """ | ||
| } | ||
| type: string: enum: file: "Exposes data from a static file as an enrichment table." | ||
| description: "enrichment table type" |
There was a problem hiding this comment.
Restore generated metadata for supported config types
This regenerated reference metadata narrows enrichment_tables.*.type to only file, even though the code still defines memory, geoip, and mmdb enrichment table variants in src/enrichment_tables/mod.rs; the same hunk also drops unrelated API and aws_secrets_manager metadata. As committed, the generated configuration reference will stop documenting valid production configuration options, so this file should be regenerated with the full feature set or reverted except for the tag-cardinality changes.
Useful? React with 👍 / 👎.
Summary
In probabilistic mode, the bloom filter cache size (
cache_size_per_key) was a single global/per-metric setting inherited by every tag. However, when specifying a per-tag override, it doesn't really make sense to inherit this value, especially in cases where the per-tag override specifies a much higher limit than the global/per-metric limit.This adds an optional
cache_size_per_keyfield to per-taglimit_overrideentries. When set, it overrides the bloom filter size for that tag only. When omitted, it inherits from the enclosing per-metric or global config as before. Setting it inexactmode is silently ignored.Example:
Vector configuration
See example above. Works in both top-level
per_tag_limitsand insideper_metric_limits.How did you test this PR?
apply_cache_size_overridecovering probabilistic+Some, probabilistic+None, and exact+Some (no-op).per_metric_limitsand globalper_tag_limitsscopes.Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.