fix(config): Apply common object storage config to the ruler storage#22366
fix(config): Apply common object storage config to the ruler storage#22366sakiphan wants to merge 1 commit into
Conversation
|
The problem affects more components than just the ruler. See #20741 |
Thanks for the pointer. I went through the call sites on main to double check this. The compactor (incl. the delete request store), index gateway and bloom components all build their clients from The ruler was the odd one out, since it reads from the separate So I couldn't find a code path on current main where the compactor would ignore the TLS settings if #20741 still reproduces on a recent version it's probably something else, happy to dig into that separately. |
What this PR does / why we need it:
When
use_thanos_objstoreis enabled, the ruler reads its bucket configuration from the separateruler_storagesection. Unlike every legacy storage type (s3, gcs, azure, swift, ...), thecommon.storage.object_storeconfig was never propagated there, so settings such as TLS CA, credentials and endpoints configured in the common section were silently ignored by the ruler. In a simple scalable deployment this surfaces astls: failed to verify certificate: x509: certificate signed by unknown authorityerrors on the backend target (which runs the ruler), while the read and write components work fine.This PR applies the common object storage config to
ruler_storageas well, mirroring how the legacy storage types have always been applied to the ruler:use_thanos_objstoreis disabled, so legacy setups are unaffected.ruler_storagesection still takes precedence, since the config file is re-applied on top of the dynamic config.This also resolves two long-standing
TODO: common config should be set on ruler bucket configcomments in the config wrapper tests.Which issue(s) this PR fixes:
Fixes #22319
Special notes for your reviewer:
The new test "when common object_store config is provided, the http config such as TLS settings should be applied to the ruler storage" reproduces the exact scenario from the issue (custom CA via
s3.http.tls_ca_path). Since the ruler may now start in setups wherecommon.storage.object_storeis set butruler_storagewas intentionally left unconfigured, this behavior change is documented in the upgrade guide.Checklist
CONTRIBUTING.mdguide (required)docs/sources/setup/upgrade/_index.mddeprecated-config.yamlanddeleted-config.yamlfiles respectively in thetools/deprecated-config-checkerdirectory. Example PR