SMG provides several mechanisms to ensure high availability and stability.
- Circuit Breakers: Automatically detect failing workers and temporarily stop sending traffic to them until they recover.
- Retries & Backoff: Configurable retry logic with exponential backoff to handle transient network issues or worker busy states.
- Rate Limiting: Protect your workers from being overwhelmed by controlling the concurrency and request rate.
- Priority Scheduling: Admit higher-priority traffic first with reserved slots, per-class queues, and TTFT-aware preemption.
- Health Checks: Active and passive monitoring of worker health to remove unhealthy nodes from the rotation.
- Graceful Shutdown: Ensure in-flight requests complete before the server stops.