Skip to content
Open
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
cf0eae7
feat(server): enable downscaler for docker swarm watcher
SCBionicle Jul 6, 2026
d00f6d1
refactor(swarm): introduce routableSwarmService struct
SCBionicle Jul 6, 2026
2913289
feat(swarm): parse autoscale and MOTD labels for services
SCBionicle Jul 6, 2026
8551a0a
feat(swarm): implement service scale-down sleeper function
SCBionicle Jul 6, 2026
72929ac
feat(swarm): implement service scale-up waker function with task IP r…
SCBionicle Jul 6, 2026
859a768
feat(swarm): map wakers, sleepers, and custom MOTDs to route table
SCBionicle Jul 6, 2026
cf90b66
fix(swarm): resolve correct task IP by matching container network ID
SCBionicle Jul 7, 2026
8bd5d5c
fix(swarm): register empty backend endpoint when replicas are 0 to al…
SCBionicle Jul 7, 2026
cc2d803
feat(swarm): parse timeout and failed MOTD labels
SCBionicle Jul 7, 2026
6f3ab89
feat(swarm): detect restart delays and crash loops during service par…
SCBionicle Jul 7, 2026
db8cfcd
feat(swarm): extend waker timeout on restart delay and fail fast on c…
SCBionicle Jul 7, 2026
00d6855
docs(readme): document Swarm wait-timeout and failed-motd labels
SCBionicle Jul 7, 2026
3eb7e38
fix(swarm): resolve loading MOTD bypassing and stuck restart delay co…
SCBionicle Jul 7, 2026
3821c56
refactor(swarm): remove unused maxAttempts variable from waker and pa…
SCBionicle Jul 7, 2026
c184593
feat(swarm): resolve dynamic MOTD countdowns on-the-fly and listen to…
SCBionicle Jul 7, 2026
7534567
fix(swarm): remove event action filters and preserve failed MOTD temp…
SCBionicle Jul 7, 2026
a64b933
feat(swarm): separate restart delay motd from permanent failed motd
SCBionicle Jul 7, 2026
9a8a1e9
fix(routes): enforce case insensitivity in all mapping lookups and de…
SCBionicle Jul 8, 2026
1f29679
fix(swarm): simplify state machine using undocumented task StateReady
SCBionicle Jul 8, 2026
7607543
docs: document Docker Swarm task states mapping logic for autoscaling
SCBionicle Jul 8, 2026
ac8671a
fix(swarm): simplify state machine using actual and desired task states
SCBionicle Jul 8, 2026
1f57e33
fix(swarm): resolve scale-up race conditions and VIP routing propagat…
SCBionicle Jul 9, 2026
c631f9d
fix(swarm): document reconciliation block and add settling delay to t…
SCBionicle Jul 9, 2026
d805558
fix(swarm): replace task events with 5-second background polling ticker
SCBionicle Jul 9, 2026
690c462
docs: update README to clarify support for native Swarm scaling and V…
SCBionicle Jul 9, 2026
38e5d68
fix(swarm): remove fragile timestamp filters to make waker and watche…
SCBionicle Jul 11, 2026
c9dc320
fix(swarm): add state transition logging to reconcileServices
SCBionicle Jul 11, 2026
7a2f9ec
docs(swarm): document wait timeout client limit and api polling details
SCBionicle Jul 12, 2026
b0ef162
refractor: convert to switch statement for label parsing
SCBionicle Jul 18, 2026
2e8dd41
fix: update service update to only scale up when replicas is set to 0.
SCBionicle Jul 19, 2026
f7f0a04
refractor: update parseServiceData symbol name to accurately represen…
SCBionicle Jul 19, 2026
b07ccc6
chore: clean whitespace
SCBionicle Jul 19, 2026
89e327a
refactor: extract resolveTargetNetwork helper from evaluateSwarmService
SCBionicle Jul 19, 2026
1d3a37e
fix: build networkAliases once via getNetworkAliases helper
SCBionicle Jul 19, 2026
1e37154
refactor: extract queryServiceTasks helper from evaluateSwarmService
SCBionicle Jul 19, 2026
496b60e
refactor: extract classifyServiceState helper from evaluateSwarmService
SCBionicle Jul 19, 2026
ba992c4
fix: restart delay and permafailture motd not showing
SCBionicle Jul 20, 2026
3d9e107
fix: changed statement that might have prevented the scaledown timer …
SCBionicle Jul 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,11 +194,13 @@ These are the labels scanned:
- `mc-router.port`: This value must be set to the port the Minecraft server is listening on. The default value is 25565.
- `mc-router.default`: Set this to a truthy value to make this server the default backend. Please note that `mc-router.host` is still required to be set.
- `mc-router.network`: Specify the network you are using for the router if multiple are present in the container/service. You can either use the network ID, it's full name or an alias.
- `mc-router.auto-scale-up`: Per-container override to enable/disable auto scale up for Docker. When true (or left unspecified and the global `-auto-scale-up` flag is enabled), mc-router will start or unpause this container when a client connects to the declared hostname(s).
- `mc-router.auto-scale-down`: Per-container override to enable/disable auto scale down for Docker. When true (or left unspecified and the global `-auto-scale-down` flag is enabled), mc-router will stop this container after it has been idle for the configured `-auto-scale-down-after` duration.
- `mc-router.auto-scale-asleep-motd`: Per-container override for MOTD to show when container is scaled to zero. If empty or not set the host will
appear unresponsive.
- `mc-router.auto-scale-loading-motd`: Per-container override for MOTD to show while the container is waking and not yet reachable. If empty or not set, the global `-auto-scale-loading-motd` value is used.
- `mc-router.auto-scale-up`: Per-container/service override to enable/disable auto scale up for Docker/Swarm. When true (or left unspecified and the global `-auto-scale-up` flag is enabled), mc-router will start the container or scale up the Swarm service when a client connects.
- `mc-router.auto-scale-down`: Per-container/service override to enable/disable auto scale down for Docker/Swarm. When true (or left unspecified and the global `-auto-scale-down` flag is enabled), mc-router will stop the container or scale down the Swarm service to 0 after it has been idle.
- `mc-router.auto-scale-asleep-motd`: Per-container/service override for MOTD to show when scaled to zero. If empty or not set the host will appear unresponsive.
- `mc-router.auto-scale-loading-motd`: Per-container/service override for MOTD to show while waking up. Supports replacing the `{duration}` token with the remaining Swarm restart delay if the task is waiting to retry. If empty or not set, the global `-auto-scale-loading-motd` value is used.
- `mc-router.auto-scale-wait-timeout`: Configure the maximum duration the router waits for the container or Swarm task to become reachable after scaling up (e.g. `"5m"` or `"300s"`). Defaults to 60s. Note: Since the Minecraft Java client has a strict connection timeout of 30 seconds, configuring this value above 30s is not recommended for player join connections.
- `mc-router.auto-scale-restart-delay-motd`: MOTD to show while the service is in a temporary restart delay (e.g. `"Server failed to start. Retrying in {duration}."`). Supports the `{duration}` countdown token, which dynamically updates.
- `mc-router.auto-scale-failed-motd`: MOTD to show if the container/service fails to start permanently or Swarm exhausts its restart policy (e.g. `"Server crashed and stopped retrying."`). Does not support the countdown token.

#### Docker Auto Scale Up/Down

Expand All @@ -224,7 +226,8 @@ Behavior:
- While that wake-up is in progress and status pings are received, mc-router can return a loading MOTD (per-container override or `-auto-scale-loading-motd`).
- When no clients remain connected and the idle timer elapses (`-auto-scale-down-after`), mc-router gracefully stops the container.

Note: Docker Swarm deployments can use auto scaling via the [Webhook Auto Scale](#webhook-auto-scale) integration. Native Swarm service scaling via `-auto-scale-up`/`-auto-scale-down` is not supported.
> [!NOTE]
> Native Swarm service scaling via `-auto-scale-up`/`-auto-scale-down` is supported in both VIP (Virtual IP) and DNSRR (DNS Round-Robin) modes. Bypassing the VIP (by routing directly to the task container IP) is automatically enabled when `replicas == 1` to prevent VIP routing delay. Note that DNSRR mode has not been actively tested.

#### Example Docker deployment

Expand Down Expand Up @@ -539,7 +542,7 @@ To override the MOTD shown when the server is scaled down or scaling up, you can
- `mc-router.itzg.me/autoScaleLoadingMOTD`

You can also customize how long the router will wait for a scaling backend to become reachable (default: 60s):
- `mc-router.itzg.me/autoScaleWaitTimeout` (e.g. `2m`, `30s`)
- `mc-router.itzg.me/autoScaleWaitTimeout` (e.g. `2m`, `30s`). Note: Since the Minecraft Java client has a strict connection timeout of 30 seconds, configuring this value above 30s is not recommended for player join connections.

Example server with custom MOTD and timeout:
```yaml
Expand Down
55 changes: 55 additions & 0 deletions docs/docker-swarm-states.md
Comment thread
SCBionicle marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Docker Swarm Task States for Autoscaling

When building plugins or routers (like `mc-router`) that autoscale and monitor Docker Swarm services, interpreting task scheduling states is historically challenging due to sparse Docker documentation.

This document defines how to map Swarm's **Actual Task Status State** (`task.Status.State`) and **Desired Task State** (`task.DesiredState`) to logical route status classifications.

---

## Task State Matrix

| State | Status State (`task.Status.State`) | Desired State (`task.DesiredState`) | Description |
| :--- | :--- | :--- | :--- |
| **Running** | `running` | `running` | The container is alive, healthy, and fully running. Connections can be routed directly. |
| **Restart Delay** | `ready` | `ready` | The service task crashed and Swarm is holding the next task retry until the restart delay timer expires. |
| **Starting / Waking** | `new`, `pending`, `assigned`, `preparing`, `ready`, `starting` | `running` | Swarm has scaled the replicas to `> 0` and is actively preparing or starting the container. |
| **Stopping** | `running` | `shutdown` or `remove` | Swarm has commanded the task to shut down or be removed, but the container is still cleaning up. |
| **Permanently Failed** | `failed`, `shutdown`, `rejected` | `shutdown` | Swarm has commanded a shutdown on the last failed container and is no longer attempting to schedule a new task. |

---

## State Classifications & Logic

### 1. Sleeping State
* **Condition**: `replicas == 0`
* **Interpretation**: The service is explicitly scaled down to zero replicas.
* **Router Action**: Keep the backend route empty (`""`) and display the configured asleep MOTD to prompt a wake-up on join.

### 2. Running (Healthy) State
* **Condition**: At least one task has `Status.State == running` and `DesiredState == running`.
* **Interpretation**: The server is fully online and ready for traffic.
* **Router Action**: Resolve the route to the service's Virtual IP (VIP) or DNS Round Robin (DNSRR) name.

### 3. Restart Delay State
* **Condition**: `replicas > 0` AND at least one task has `Status.State == ready` and `DesiredState == ready`.
* **Interpretation**: Swarm has scheduled a retry but is waiting for the configured restart policy delay to expire before spawning the container.
* **Router Action**: Keep the backend route empty (`""`) to route connection attempts to the waker, and display the restart delay countdown MOTD.

### 4. Permanently Failed State
* **Condition**: `replicas > 0` AND the most recently updated task in history has `DesiredState == shutdown` AND no active tasks exist.
* **Interpretation**: Swarm has exhausted its restart attempts and stopped trying.
* **Router Action**: Keep the backend route empty (`""`) and display the custom permanently failed MOTD.

### 5. Starting / Waking State
* **Condition**: `replicas > 0` AND the desired state of the task is `running` but the actual status is not yet `running` (and not delayed or failed).
* **Interpretation**: The container is actively being assigned, prepared, or started by Swarm.
* **Router Action**: Keep the backend route empty (`""`) to preserve the waker, and display the loading/waking MOTD.

---

## Polling Implementation Details

> [!NOTE]
> **Why Polling is Used Instead of API Events:**
> Although `mc-router` subscribes to the Docker Event Stream, the Docker Engine API does **not** broadcast task-level scheduling transitions (such as `task.Status.State` moving from `preparing` to `ready` or `running`) over the event system.
> To reliably capture these low-level task state changes and resolve the direct task IP without transient routing failures, `mc-router` utilizes a background polling ticker that queries the Swarm Service and Task API every 5 seconds.
4 changes: 4 additions & 0 deletions server/docker.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ const (
DockerRouterLabelAutoScaleDown = "mc-router.auto-scale-down"
DockerRouterLabelAutoScaleAsleepMOTD = "mc-router.auto-scale-asleep-motd"
DockerRouterLabelAutoScaleLoadingMOTD = "mc-router.auto-scale-loading-motd"
DockerRouterLabelAutoScaleWaitTimeout = "mc-router.auto-scale-wait-timeout"
DockerRouterLabelAutoScaleFailedMOTD = "mc-router.auto-scale-failed-motd"
DockerRouterLabelAutoScaleRestartDelayMOTD = "mc-router.auto-scale-restart-delay-motd"
DockerRouterEventTypeTask = "task"
)

type dockerWatcherConfig struct {
Expand Down
Loading
Loading