Skip to content

feat(db): clickhouse failover support#1230

Merged
invis-bitfly merged 2 commits intostagingfrom
BEDS-1097/add-failover-support-for-clickhouse
Jan 8, 2025
Merged

feat(db): clickhouse failover support#1230
invis-bitfly merged 2 commits intostagingfrom
BEDS-1097/add-failover-support-for-clickhouse

Conversation

@invis-bitfly
Copy link
Contributor

@invis-bitfly invis-bitfly commented Jan 2, 2025

also includes a switch to a more sane env lib so i dont have to define everything 45 times
first commit might not build, need to remove all the now useless casting everywhere

@invis-bitfly
Copy link
Contributor Author

seems to work with some basic testing:

  • invalid host as primary, valid host as fallback: connection succeeds after a small delay (~8s)
  • valid host as primary, invalid host as fallback: connection succeeds instantly, no attempt made to invalid host

should note that the logic only gets applied when a new connection gets opened. this means that a faulty host that accepts/holds connections but fails to execute any queries will continue to cause issues until the faulty host gets stopped/fixed.
a potential fix is setting the max connection lifetime to something low (5m?) and adding a custom dial function that does a sanity check each time to confirm that the host is safe to use. sounds annoying tho as you would only have the raw dial to do that.
at some point it makes more sense to just write our own wrapper tho which would allow for much more flexibility.

also this pr is breaking as it makes the ssl flag configurable instead of always forcing ssl with clickhouse.

@invis-bitfly invis-bitfly marked this pull request as ready for review January 7, 2025 12:06
@invis-bitfly invis-bitfly merged commit 04b67b9 into staging Jan 8, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant