Skip to content

Conversation

@jubrad
Copy link
Contributor

@jubrad jubrad commented Dec 15, 2025

Currently a large number of connections per second can overwhelm environmentd impacting work for existing connections. In order to isolate the impact of connection floods while still maintaining a reasonably high max_connections I propose that we add connection rate limiting. This will send a too many connections error to the client which can then retry with backoffs spreading connection requests a bit. This should reduce impact somewhat, however, large CPS floods will still be making TLS connections which may still lead to environmentd health impairment.

Adds connection rate limiting through governor handled by burstable quotas added to the pgwire server config. Limits are checked in handle_connection, and controlled by the following system params.

  • PGWIRE_CONNECTION_RATE_LIMIT
  • PGWIRE_CONNECTION_RATE_LIMIT_BURST
  • PGWIRE_CONNECTION_RATE_LIMIT_PER_IP
  • PGWIRE_CONNECTION_RATE_LIMIT_BURST_PER_IP

Environmentd must be restarted to update params.

Motivation

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

@jubrad jubrad force-pushed the feature/connection-rate-limits branch from 2486be0 to bd763e7 Compare December 15, 2025 21:01
Adds connection rate limiting through governor handled by
burstable quotas added to the pgwire server config. Limits are
checked in handle_connection, and controlled by the following
system params.
- PGWIRE_CONNECTION_RATE_LIMIT
- PGWIRE_CONNECTION_RATE_LIMIT_BURST
- PGWIRE_CONNECTION_RATE_LIMIT_PER_IP
- PGWIRE_CONNECTION_RATE_LIMIT_BURST_PER_IP

Environmentd must be restarted to update params.
@jubrad jubrad force-pushed the feature/connection-rate-limits branch from bd763e7 to 4480c39 Compare December 15, 2025 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant