24 orphaned deployments still serve /oauth/* on public hostnames — shipping the OAuth fixes retires none of them

[tonychang04]

Every deployment ever made for the insforge-mcp server is still answering HTTP on a public hostname. 25 machines are live right now, all under the mcp-bf35bbc3-* app prefix; only one is the production target behind mcp.insforge.dev. The other 24 are orphaned predecessors.

The hostname inventory is deliberately not in this issue. This repo is public and #106 carries a complete unauthenticated reproduction; a list of live pre-fix hosts next to it is a recipe. Full inventory, per-machine versions and exposure status are in Notion: insforge-mcp orphaned deployments — live inventory and exposure (2026-08-03).

The exposure is live, not theoretical

Shipping the fixes for #102, #106 and #110 retires nothing — the vulnerable copies keep answering on their own hostnames.

Running #106's actual payload (/oauth/callback?error=x&error_description=a&error_description=b with Accept: text/html) against all 25 machines:

• 13 of the 25 return the stack trace — TypeError: text.replace is not a function with absolute /app/dist/... paths, to an unauthenticated request, right now.
• mcp.insforge.dev and the three newest machines are clean. Production is genuinely fixed.

So the fix shipped and the vulnerable surface did not shrink. That is the whole problem in one sentence.

Two control-plane facts that break the obvious triage

1. Deployment status does not mean liveness. GET /api/v1/deployments?serverId=… reports 5 running, 43 stopped, 33 failed. 20 of the machines the API calls stopped answer HTTP right now. stopped means "we stopped tracking it". Anything that infers live surface from deployment status — including our own tooling — gets a wrong answer.

2. The reported version is not a security-posture signal. Two machines report 1.2.12 and leak; three report 1.2.12 and are clean. The package version was bumped independently of the fix landing, so there is no safe subset selectable by version, and no version-ordered reap. All 24 get retired together. (Mechanism: the escapeHtml2 frame lands on a different dist line on nearly every machine — the bundle genuinely differs build to build while the version string stays put.)

Corollary for anything we build on top of this: a direct probe is the only true statement about a machine. Neither the control plane nor the container's own version field can be trusted for it.

They are not idle

Runtime logs for a 15-minute window show four orphans in a permanent Redis reconnect loop, ~20 error lines/minute each, indefinitely. The live production machine logs zero — that contrast is the fastest way to tell an orphan from the real one in a runtime log.

Retire path

The platform API exposes one, and documents it as automatic:

POST   /api/v1/deployments/{id}/stop      "Stop a running deployment"
DELETE /api/v1/deployments/{id}           "Delete a deployment and tear down its infrastructure"
POST   /api/v1/deployments/{id}/activate  "…and tears down the previous production deployment"

activate is supposed to reap its predecessor. It demonstrably has not — 24 predecessors survive. That is a platform bug and it is the root cause of this issue existing.

Proposed

1. Reap the 24. stop first rather than DELETE — it ends the exposure without tearing down infrastructure, so it is the cheaper move to get wrong. Destructive and prod-adjacent either way: needs a human go-ahead. Oldest first; the production machine never.
2. Report upstream to the vendor (private channel — their SECURITY.md asks for exactly that, not a public issue): activate does not tear down its predecessor, and status: stopped is returned for machines still serving traffic.
3. Assert the live-machine count in verify-deploy.mjs, so a deploy that fails to retire its predecessor is caught at deploy time rather than a month later. This is the only item here that prevents recurrence.

Related: #102, #106, #110 — all of which the orphans keep serving unfixed.

[tonychang04]

One correction to the table, because it affects which machines you reap and in what order: the version column cannot be used as the triage key.

I probed the reported build behaviour across all 25 live machines. The reported version in /health does not correspond to which OAuth fixes a machine carries — machines reporting the same version string differ in behaviour, and some reporting 1.2.12 behave like pre-fix builds. The package version was bumped independently of the fixes landing, so 1.2.10 vs 1.2.12 tells you nothing about whether a given machine has the #106 / #110 remediation.

Practical consequence: there is no safe subset. Anything that reasons "reap the 1.2.10 ones first, the 1.2.12 ones are current" reaches the wrong answer. All 24 non-production machines should be treated as carrying the unfixed OAuth surface and reaped together, not triaged by version.

This compounds the stopped-means-untracked point already in this issue: neither the control plane's status nor the container's own reported version is a reliable statement about what a machine is actually serving. The only reliable signal is a direct probe of the machine.

Per-machine detail is in the Slack thread rather than here — see the note below on this issue's visibility.

Visibility: this repository is public, and so is #106, which carries a complete unauthenticated reproduction. This issue lists 25 live, reachable hostnames running builds that predate that fix. The two together are more actionable to an outside reader than either alone. Suggest trimming the hostname list here to a count plus the app prefix, keeping the inventory in Notion, and prioritising the reap over the write-up — the exposure ends when the machines do.