Skip to content

Commit befc2fa

Browse files
author
Nathan Flurry
committed
docs(self-host): document autoscaling for every platform
1 parent e5bff0d commit befc2fa

20 files changed

Lines changed: 89 additions & 15 deletions

src/content/self-host/control-plane/aws-ecs.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,8 @@ Use `normal` for long-lived workers. See <SelfHostLink to="workers">Runner confi
145145

146146
EFS-backed file system storage is single-node: RocksDB cannot be shared between tasks. Before raising the desired count above one, move to RDS PostgreSQL and add NATS for pub/sub. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
147147

148+
Once on RDS, ECS Service Auto Scaling can drive the desired count. Use target tracking on average CPU at 60%, matching the <SelfHostLink to="control-plane/kubernetes">Kubernetes</SelfHostLink> manifests, and keep a minimum of two tasks so the service survives losing one. Leave the desired count at one for as long as the task is on EFS.
149+
148150
## Next steps
149151

150152
- <SelfHostLink to="control-plane/production-checklist">Production checklist</SelfHostLink>

src/content/self-host/control-plane/custom.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ docker run -d \
4848
rivetdev/engine:2.3.3
4949
```
5050

51-
For multi-node, drop the file system variable and set `RIVET__POSTGRES__URL` instead. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
51+
For multi-node, drop the file system variable, set `RIVET__POSTGRES__URL` instead, and add NATS for pub/sub. Do not replicate or autoscale a node that is still on the file system backend: RocksDB is single-node. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
5252

5353
</Step>
5454

src/content/self-host/control-plane/docker-compose.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ volumes:
147147
postgres-data:
148148
```
149149
150-
Multi-node deployments also need NATS. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
150+
This file still runs a single engine container, so it is not yet a multi-node deployment. It is the prerequisite for one: the file system backend is single-node RocksDB and cannot be shared, while PostgreSQL can. Multi-node deployments also need NATS. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
151151
152152
## Using a config file
153153

src/content/self-host/control-plane/kubernetes.mdx

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,18 @@ To use Amazon RDS, Cloud SQL, or Azure Database instead of the bundled Postgres:
114114

115115
Use the direct connection string, not a pooled one. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
116116

117+
## Autoscaling
118+
119+
`05-rivet-engine-hpa.yaml` is a HorizontalPodAutoscaler on the engine Deployment. It scales between **2 and 10 replicas**, targeting **60% average CPU** and **80% average memory**.
120+
121+
```bash
122+
kubectl -n rivet-engine get hpa rivet-engine
123+
```
124+
125+
It needs the metrics server from the requirements above. Without it the targets read `<unknown>` and the HPA never scales.
126+
127+
Autoscaling is only safe here because this bundle runs on PostgreSQL and NATS. The file system (RocksDB) backend is single-node, so an engine using it must stay at one replica. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
128+
117129
## Applying configuration changes
118130

119131
The ConfigMap is read at startup, so a change needs a restart:

src/content/self-host/control-plane/production-checklist.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ We recommend passing this page to your coding agent to verify your configuration
2323

2424
- **Run two or more nodes.** A single control plane node is a single point of failure. Deploy at least two behind a load balancer.
2525
- **Do not run multiple file system nodes.** RocksDB is single-node. Multi-node deployments need PostgreSQL or FoundationDB. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
26-
- **Configure autoscaling.** Target 70% CPU and 80% memory to leave headroom for traffic spikes. In Kubernetes this is a Horizontal Pod Autoscaler.
26+
- **Configure autoscaling, on a shared backend only.** Target 60% CPU and 80% memory to leave headroom for traffic spikes. In Kubernetes this is a Horizontal Pod Autoscaler, and the shipped manifests include one at 2 to 10 replicas. Never autoscale a node on the file system backend: adding replicas to a single-node RocksDB deployment corrupts it.
2727
- **Check the rate limit on your serverless worker platform.** Actor start requests all originate from your control plane nodes, so they come from a small set of IPs. Per-IP rate limits throttle the control plane long before they would throttle end-user traffic. Size the limit to your peak actor create and wake rate.
2828

2929
## Storage

src/content/self-host/control-plane/railway.mdx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,10 @@ If your start command is a wrapper (`npm start`, `yarn start`, a shell script),
8484

8585
Railway's HTTP proxy supports WebSockets. If you add your own reverse proxy inside the service, raise its idle and read timeouts to `3600`. See <SelfHostLink to="control-plane/ports">Ports</SelfHostLink>.
8686

87+
## Scaling past one instance
88+
89+
Both templates provision a single control plane service on PostgreSQL, which is a fine starting point. Before raising the replica count, add NATS for pub/sub so replicas coordinate. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
90+
8791
## Next steps
8892

8993
- <SelfHostLink to="control-plane/production-checklist">Production checklist</SelfHostLink>

src/content/self-host/control-plane/render.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,8 @@ Render's proxy supports WebSockets but closes connections that sit idle past its
137137

138138
The blueprint above is a single instance on managed PostgreSQL, which is a fine starting point. Before scaling out, add NATS for pub/sub so instances coordinate. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>.
139139

140+
With NATS in place, instance count can be raised or handed to Render's autoscaling on a CPU target. Target 60% CPU, matching the <SelfHostLink to="control-plane/kubernetes">Kubernetes</SelfHostLink> manifests, and keep a minimum of two instances. Autoscaling a control plane that is still on the file system backend corrupts it, but that does not apply here: this blueprint is on PostgreSQL from the start.
141+
140142
## Next steps
141143

142144
- <SelfHostLink to="control-plane/production-checklist">Production checklist</SelfHostLink>

src/content/self-host/control-plane/vm.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,8 @@ From a container on the same host, use `host.docker.internal` instead of the hos
9898

9999
## Using PostgreSQL
100100

101+
The file system backend above is RocksDB on local disk: single-node, and impossible to share between hosts. Scaling past one control plane node means moving to PostgreSQL first, then adding NATS for pub/sub. See <SelfHostLink to="control-plane/storage">Storage</SelfHostLink>. The commands below are still one engine container on one host; PostgreSQL is what makes adding a second host possible later.
102+
101103
```bash
102104
docker network create rivet-net
103105

src/content/self-host/workers/aws-ecs.mdx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,12 @@ Once tasks are running they appear under **Runners** in the dashboard. Nothing n
4242

4343
<ProductSlot name="verify" />
4444

45+
## Autoscaling
46+
47+
Runner-mode tasks are interchangeable, so ECS Service Auto Scaling can drive the desired count. Use target tracking on `ECSServiceAverageCPUUtilization` at 60%, with a minimum of two tasks.
48+
49+
Scale-in is the caveat. It is bound by the same 120 second `stopTimeout` cap, so actors on a removed task are cut off rather than drained. Set a long scale-in cooldown, or leave the desired count fixed and change capacity at deploy time instead.
50+
4551
## Next steps
4652

4753
- <SelfHostLink to="workers/production-checklist">Production checklist</SelfHostLink>

src/content/self-host/workers/cloudflare.mdx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,10 @@ Set the Worker URL with the `/api/rivet` path as the serverless runner URL in th
6060

6161
<ProductSlot name="verify" />
6262

63+
## Scaling
64+
65+
Cloudflare autoscales Workers for you: there are no instances or replicas to configure. The one scaling bound you own is `maxConcurrentActors` on the serverless runner config, which caps concurrent actors across the whole pool and defaults to `1000`. See <SelfHostLink to="workers/production-checklist">the production checklist</SelfHostLink>.
66+
6367
## Next steps
6468

6569
- <SelfHostLink to="workers/production-checklist">Production checklist</SelfHostLink>

0 commit comments

Comments
 (0)