Skip to content

Latest commit

 

History

History
146 lines (136 loc) · 99.9 KB

threat-model.md

File metadata and controls

146 lines (136 loc) · 99.9 KB

Armada Threat Model (v0.1)

Status: draft, 31 Oct 2026. Updated whenever PKI, secrets, certificates, the proxy renderer, or API authentication change (see the security rules in CLAUDE.md).

Method: STRIDE per trust boundary.

1. System being modeled

                Internet
                   │  80/443
        ┌──────────▼──────────┐        private network (VPC)
        │  HAProxy (per node) │◄──────────────────────────────┐
        └──────────┬──────────┘                               │
                   │ private IP : host port                   │
        ┌──────────▼──────────┐   gRPC mTLS   ┌───────────────┴──┐
        │ app container       │               │  armadad server  │◄── armada CLI (mTLS)
        └──────────▲──────────┘               │  (state, CA,     │
                   │ containerd (root)        │   certmagic)     │
        ┌──────────┴──────────┐   gRPC mTLS   └──────────────────┘
        │   armadad agent     ├──────────────────────►
        └─────────────────────┘   (agent initiates the connection)

2. Assets

Asset Location Impact if leaked
Cluster CA key server data directory; backup archives; the control plane backup bucket (sealed, ADR 0051) Attacker can issue node/admin certificates → full control of the cluster
Secret master key server data directory (master.key, 0600); backup archives; the control plane backup bucket (sealed) All app secrets exposed (together with the state)
Decrypted secrets server memory; agent memory; node tmpfs (secret files); the runtime's container config on node disk (env secrets) That app's credentials exposed
Admin certificate /etc/armada/admin.config on each server (root, readable by the armada group, which init and server join add the user who ran them through sudo to); ~/.armada/config of each person given one Full control of the cluster: every member of the armada group on that server holds it. The server renews the one in /etc/armada/admin.config only while the cluster accepts that user, so deleting or replacing the user still ends it
DNS provider token In the state, sealed with the master key (ADR 0031); clusters from before keep a 0600 file on the server until its leader moves it over Domain hijacking
Registry credentials store (encrypted, as an app secret) Theft of private images
Domain TLS private keys certmagic storage, PEM on every node Eavesdropping on/impersonation of user sites
Container runtime socket (embedded containerd at /run/armada/containerd/containerd.sock) every node Root on the node
armada-shim sockets (/run/armada/shim/*.sock, 0600 in a 0700 directory; supervisor runtime, ADR 0046) supervisor nodes Create, start, signal, and delete that app's containers
State (embedded etcd, etcd/ in the data directory) server; backup archives and the control plane backup bucket (as a snapshot, sealed in the bucket) Cluster topology, metadata; secret ciphertext; writing to it controls what every node runs
etcd member key server data directory (etcd/member.key, 0600) Read and write access to the state, from wherever the member's address is reachable
Server join token shown once by armada server join-token; stored only as a hash While valid (single use, at most 1 h): redeemable for the CA key and the master key
Control plane backup bucket and key (ADR 0051) the bucket (snapshots sealed with the key); URL and key sealed with the master key in the state, and wherever the operator keeps them Both together: the CA key, the master key, and the state, so full control of the cluster and every secret. The bucket alone: sizes and times, and deleting or withholding backups

3. Threat actors

  1. Internet attacker: can only reach HAProxy on 80/443.
  2. Other VPC tenant: another machine on the same private network (owned by the same user but compromised, or a provider misconfiguration).
  3. Compromised app: malicious code inside one of the containers Armada runs.
  4. Compromised node: an attacker gains root on one agent machine.
  5. Supply chain: tampered Go dependencies, HAProxy image, bundled runtime binaries (containerd, runc, CNI plugins, tini), or release artifacts.

4. Threats and mitigations

# Boundary STRIDE Threat Mitigation Status
T1 Internet → HAProxy Tampering / EoP HAProxy config injection via user-supplied domain/path/app name, or via addresses reported by a node Whitelist validation in spec, revalidated in proxy.Config.Validate (agent) before rendering; user input is used only as literal strings (-m str/-m beg, prefix stripping by segment count, no regex); IP addresses must be canonical and carry no IPv6 zone (a zone may hold any bytes, including newlines); the renderer refuses to emit a configuration if any interpolated value is not a single token (no whitespace or control characters); golden tests, FuzzRender fuzzing, haproxy -c before reload and in CI Mitigated (phase 2, hardened in phase 7)
T2 Internet → HAProxy Spoofing Custom domain hijacking (adding someone else's domain) Verification of a TXT record _armada-verify.<domain> containing a random 128-bit per-domain token before routing and certificate requests; two apps cannot claim the same host+path. Verification is bound to the space that proved it, not the space's name: the record goes when the last app naming the domain is deleted or when its space is deleted, so a space created again under the name (by any space-creator) gets a new token Mitigated (phase 2)
T3 Internet → HAProxy DoS Exhausting ACME rate limits with arbitrary domains Certificates are requested only for verified domains and <app>.<root>; certmagic's built-in backoff Mitigated (phase 2)
T4 Internet → internal services Info disclosure gRPC/stats/DNS/container ports exposed publicly The API, join, ACME, metrics and DNS listeners and published container ports bind only to private addresses (0.0.0.0/::, public, and zoned addresses refused); auto-detection skips container and VM bridges; armadad doctor reports cluster ports listening on wildcard or public addresses, the host firewall state, and private-IP problems; internal ingress and DNS bind to the bridge gateway, not the private IP (ADR 0007). On providers with 1:1 NAT, anything on the private IP can still be reachable from the internet unless the provider firewall blocks it (see limitations) Partially mitigated (phase 1, 6, 7)
T5 VPC → server Spoofing Rogue node joins the cluster Single-use join tokens (256-bit secret, 1 h default, at most 24 h) stored as SHA-256 hashes and compared in constant time; redeemed in the same transaction that registers the node; separate join listener that serves only Join, is rate limited and size capped; node names and private IPs validated (canonical, private, not loopback, no IPv6 zone), duplicates refused only after the token checks out so unauthenticated callers learn nothing about existing nodes; the CA chooses the certificate identity, not the CSR Mitigated (phase 5, hardened in phase 7)
T6 VPC → agent Spoofing (MITM) Rogue server during join The join command carries the CA hash; armadad join learns the CA from the handshake, checks it against the pin and verifies the server certificate with it before sending anything (no InsecureSkipVerify); the token is read from a prompt, stdin, or a file, never from argv Mitigated (phase 5)
T7 VPC Info disclosure Eavesdropping on server ↔ agent traffic Mandatory mTLS 1.3 with RequireAndVerifyClientCert on the main API, enforced at construction; no plaintext fallback or insecure flag Mitigated (phase 1)
T8 Compromised node → server EoP Agent requests another node's assignments/secrets, or a certificate for another identity Authorization based on certificate identity; the server only sends data for that node; renewal always issues a certificate for the caller, whatever the CSR says Mitigated (phase 5)
T9 Compromised node Repudiation / persistence Stolen node certificate keeps being used 30-day node certificates renewed automatically; serial allowlist checked on every RPC; a renewal revokes every other certificate of the node, so a stolen certificate dies at the next legitimate renewal; renewals are rate limited per node; armada node delete deletes the serials and closes the agent's stream; streams end when their certificate expires; certificates not issued by the server are refused. Nodes created before the allowlist existed accept any CA-signed certificate for their identity until their first renewal Mitigated (phase 5, hardened in phase 7)
T10 CLI → server Spoofing Unauthorized admin API access API binds privately by default; mTLS with a user or service certificate; 0600 permission on the config. Each person has a user certificate under their own name (1 year, ADR 0028, ADR 0033); armada user delete refuses every certificate of a name at once, armada user credentials refuses the certificates a user was given before, including ones the old key renewed up to the second the replacement was recorded (it is bound by serial), and every API call and metrics scrape checks this. armada user renew signs a new one only for a caller holding a valid user certificate, under the caller's own name, keeps its roles, is audited, is rate limited per user, and is checked and recorded in one transaction with any replacement, so a stolen key cannot renew past one; it gives a stolen certificate nothing it lacked. Renewing an expired one needs the CA key on a server (armadad user renew, root), which records the name first. A user certificate the cluster has no record of is refused unless it was issued before schema 14 (meta/user-cutoff), so a restore from an earlier backup refuses users created after it instead of making them cluster-admins; armadad restore says that users and services are as of the backup, and warns for backups older than schema 14 Mitigated (phase 1, ADR 0028, ADR 0033)
T11 Store Info disclosure Secrets readable from the DB, logs, CLI output, or the audit log AES-256-GCM envelope encryption (random data key per version, wrapped by the master key); values are write-only (no RPC returns them); secrets.Value redacts itself in fmt, slog, and JSON; the audit log records only name and version; tests plant values and grep etcd's files (WAL, snapshots, database), the server log, CLI output, and container labels Mitigated (phase 4)
T12 Container → host EoP Container escape to the host Always no-new-privileges, never privileged, host network, or a container runtime socket; memory limit (default 256Mi) and PID limit (1024); NET_RAW, MKNOD and AUDIT_WRITE dropped. The other default capabilities remain, containers run as the image's user (often root) and the root filesystem is writable Partially mitigated (phase 1)
T13 Container → container, node, platform Info disclosure / EoP A compromised app reaches another app's containers or internal service, another space's apps, the nodes, the control plane, the private network, or the cloud metadata service With the cluster network (ADR 0025, ADR 0026): nftables tables on every node (ip and bridge families, rebuilt atomically) let traffic from one container address straight to another pass only between replicas of the same app, on one node and across nodes, and drop it towards unknown addresses; containers of every space but default (and of default once armada space update default --network-isolation is on) are kept from the private, shared, loopback, and link-local ranges, every node's private address, and every port of their own node except the DNS server on the bridge gateway and the proxy's ports 80 and 443; containers have IPv4 only and the bridge has IPv6 turned off (embedded.DisableIPv6, doctor's isolation check), so no IPv6 link-local address on it leads to the node's listeners; a container address the node does not know yet counts as isolated (fail closed); the agent's DNS server answers internal names only for the asking container's own space, and nothing for unknown addresses; HAProxy's internal frontend looks the caller's source address up in a map the agent keeps current through the Runtime API (validated again by the proxy driver: IPv4 in the node's subnet, valid app IDs, bounded size) and routes an internal name only for callers of the app's space, and with ingress.allow only for the listed apps of that space, so a forged Host header reaches nothing and unknown callers (including the node itself) get a 404. Addresses come from agents for their own replicas only, must lie in the node's own subnet (T21), and are validated again by every agent; an address two apps claim reaches nothing. Containers cannot forge their source address: they run without NET_ADMIN and NET_RAW. Established connections pass first, so when a container gives its address back (removal, a finished job attempt, a removed network sandbox) the node deletes the kernel connection tracking entries naming that address right after the container networking plugins release it (forgetConnections; a failure is logged): a peer that keeps a flow to the old owner alive cannot hand it to the next container given the address, of whatever space, and its next packet is checked again as new. Not covered: clusters without the cluster network are not filtered (their published ports bypass ingress.allow). Containers of every space share the node's kernel, so a kernel bug reachable from a container crosses this boundary: Armada does not contain code from people the operator does not trust, which needs a cluster of its own or dedicated nodes Partially mitigated (ADR 0026)
T14 Supply chain Tampering Tampered binary/installer/image Only repository admins may push v* tags (tag ruleset) and the release workflow refuses a tag whose commit is not on main; it then (GoReleaser) runs the CI gates, then publishes sha256 checksums, an SBOM per archive, a keyless cosign signature bundle of the checksum file, and GitHub build provenance; install.sh refuses archives whose checksum does not match and verifies the signature against the release workflow identity for that tag when cosign is installed (ARMADA_REQUIRE_SIGNATURE=1 makes it mandatory), and armadad upgrade does the same checks before it replaces anything (HTTPS only, redirects included, or a file:// mirror; only armadad, armada, and deploy/systemd/armada-*.service taken from the archive; no downgrades; ADR 0030); piped into sh, the installer runs nothing until the whole script has arrived (its body is a function called on the last line); CI actions pinned by commit SHA; HAProxy image pinned by digest; govulncheck, gosec and Dependabot Mitigated (phase 7; first signed release pending)
T15 Server Repudiation Untraceable changes Audit log with identity from the client certificate; automatic changes (rollout completed/failed) recorded with actor system Partial (deploy, scale, root domain, rollback, rollout, secret set/delete, join token, node join/cordon/label/restrict/remove)
T16 Server → agent Info disclosure App TLS private keys leak during distribution or on node disk Sent only over mTLS, in the assignment or, for current agents, fetched by reference with GetCertificates, which only node certificates may call and which answers only names the cluster serves (each checked against the SHA-256 the assignment names); written 0600 in a 0700 directory; never logged (the agent does not print ProxyConfig). Every node, including a newly joined one, receives the private key of every public domain, because any node can receive public traffic Mitigated (phase 2)
T17 VPC → server Spoofing / DoS ACME HTTP-01 responder abused Binds only to a private IP (port 0/0.0.0.0 rejected); answers only active challenge tokens, 404 otherwise Mitigated (phase 2)
T18 Host Info disclosure DNS provider token leaks (shell history, DB, logs) Read from a 0600 file (--dns-token-file), not stored in the DB, never shown in errors Mitigated (phase 2)
T19 Node EoP HAProxy Runtime API (admin level) used by another process Socket in a root-owned 0700 directory, mode 600; no TCP listener Mitigated (phase 2)
T20 Server → agent SSRF / Tampering A compromised server (or malicious spec) uses agent probes to send requests to other addresses on the node's network The agent only probes the container's own IP (bridge IP, or the app's published port); probe paths are revalidated on the agent with the same whitelist as the spec; the probe client does not follow redirects and ignores HTTP_PROXY; at most 4 KiB of the response body is read Mitigated (phase 3)
T21 Agent → server Tampering / DoS A compromised agent reports fake "ready" replicas to pull traffic to other containers, blocks rollouts, or breaks the proxy configuration of the whole cluster Reports only apply to replicas belonging to that node; addresses must be exactly <node private IP>:<port>, or with the cluster network (ADR 0025) an address in that node's own subnet other than its network, gateway, and broadcast addresses, in canonical form with a non-zero port, and a reported container address (which every node's firewall keys on, T13) likewise only in that node's subnet; ready is accepted only in state running; a server whose address still fails validation is left out of its backend instead of failing the whole configuration; HAProxy still runs its own health checks. Stale addresses: a node hands container addresses out again (after a reboot from the start of its range, since the allocation lives on tmpfs), so an address the server still holds for a replica whose container is gone could reach another app's container, of another space, with the first app's requests. The agent reports a replica whose container it recreates (after a crash, a failed liveness probe, a spec change, or a machine restart) as starting with no address before it starts any container, which usually takes the address out of the proxies first; nothing waits for every proxy to apply that, so a start quicker than the server's next pass can leave the address in a proxy for a moment. The server never puts an address or container IP that replicas of two apps claim into either backend (an AddressConflict event names the apps; it ends when the app is deleted), as the firewall leaves it out (T13), and the firewall and the internal frontend's space check still apply to whoever is reached. Removed addresses also lose their connection tracking entries (T13). A compromised agent can still serve any content from its own node (the "agent = root" limitation) Partially mitigated (phase 3, hardened in phase 7)
T22 CLI → server Tampering Rollback to another app's revision, or a rollback that grabs a domain already used by another app The revision must belong to the same app; rollback goes through the same host+path conflict check as deploy and re-registers domains with TXT verification Mitigated (phase 3)
T23 Store Tampering Someone with write access to the DB swaps ciphertexts between apps, secrets, or versions so one app receives another's secret Secret (its ID, an app's for the app's own, ADR 0052), key, and version are bound as AES-GCM additional data on both the data key and the value; a swapped or modified ciphertext fails authentication and the node's assignment is withheld Mitigated (phase 4)
T24 Host Info disclosure / DoS Master key lost, replaced, or world-readable Created 0600 with O_EXCL (never overwritten), loaded only if not group/other accessible; its ID is recorded in the DB and the server refuses to start with a missing or different key; init tells the operator to back it up (armadad backup includes it) Mitigated (phase 4)
T25 Admin machine Info disclosure Secret values end up in shell history or terminal scrollback secret set takes no value argument; reads from a hidden prompt, stdin, or --from-file Mitigated (phase 4)
T26 Node Info disclosure Secret files reach the node's disk or other local users Written only to a directory verified to be tmpfs/ramfs (otherwise secret files are disabled), root-owned 0700, removed when the replica goes; bind-mounted read-only; the runtime only mounts regular files at clean absolute paths (runtime.CheckFileMount), so they can never mount a runtime socket or a directory Mitigated (phase 4)
T27 Node Info disclosure Secrets leak through container metadata: the spec hash label, or environment variables in the runtime's on-disk container metadata (containerd's) The spec hash covers only name@vN references, never values. Env secrets remain visible to root on the node and are stored by containerd on disk; documented, with secretFiles as the alternative Partially mitigated (phase 4)
T28 Compromised node → server Info disclosure An agent obtains secrets of apps it does not run The server decrypts and sends secrets only inside the assignment for the node that runs the replica (the leader keeps the versions it opened in memory, never on disk, until no replica uses them); the admin secret RPCs require an admin certificate. Node capacity is self-reported: it is bounded and, beyond deciding whether a replica's CPU and memory reservation fits, only breaks ties in scheduling (spread and replica count come first), but a compromised node can still be chosen for new replicas and then receives their secrets. Labels that steer placement and placement.spreadBy are set by admins only; armada.os/armada.arch are agent-reported, so an app spread by them lets a node that misreports its architecture form a value of its own and take that value's share (image platforms rule out architectures the image lacks once known; spread sensitive apps by admin labels) Partially mitigated (phase 4, 7)
T29 VPC → server DoS / brute force Guessing join tokens or flooding the join listener 256-bit secrets; token buckets per source address (burst 5, then one attempt per 6 s) and overall (burst 30, one per second); at most 4 concurrent streams per connection and a 10 s handshake timeout; at most 256 connections in total and 16 per source address, refused as they are accepted, before TLS, and closed after 15 s idle or 2 minutes, so held connections cannot use up the server process's descriptors (the token buckets only see calls); the main API listener drops a peer that has not finished its TLS handshake within 10 s; failed attempts are logged with the token ID and source address, never the secret Mitigated (phase 5, hardened in phase 7)
T30 Node Info disclosure / availability Renewed key written insecurely, or a crash mid-renewal leaves the node without a usable certificate The new key and certificate are checked against the CA and the node identity before touching disk, written 0600 as .new files, then renamed into place; on start the agent completes or discards a half-finished renewal Mitigated (phase 5)
T31 Internet / VPC → internal apps Info disclosure / spoofing Internal apps reached from outside the cluster, for example through 1:1 NAT or a provider load balancer, or a public name hijacked onto an internal app The internal frontend binds only to the container bridge gateway, which is not reachable through NAT or load balancers; internal rules exist only in that frontend and public rules only in the public ones; the .internal zone is reserved in the spec and revalidated on the agent (<app>.internal only); internal apps get no public name, domain, or certificate. A VPC neighbour that routes the bridge subnet via the node could still reach it unless the provider's anti-spoofing or the host firewall stops it Mitigated (phase 6)
T32 Container / VPC → node DNS DoS / spoofing The node resolver abused as an open resolver or reflector, or poisoned Listens only on the bridge gateway; answers only the bridge subnet and stays silent for everyone else; with the cluster network a source in the node's subnet is accepted only from the bridge (or the node itself over loopback; ARMADA-NET-PRE drops it from any other interface, T57), so a machine on the private network cannot forge container sources to fill the forwarding slots or reach the internal names of every space (nodes without the cluster network have no such rule and no isolation between spaces, T13); answers .internal authoritatively only for apps the local HAProxy routes; forwards other names to the host's configured resolvers with a 2 s timeout, at most 128 at a time and 16 per client address, the client's share checked first so a client past it takes no node slot (SERVFAIL beyond either, .internal answers unaffected), and at most 16 open TCP connections per client address, so one container cannot take the forwarding capacity other spaces' containers on the node rely on; queries are not cached Mitigated (phase 6, hardened in phase 7)
T33 Host / backup storage Info disclosure A backup archive leaks: it holds the CA key, the master key, and the encrypted secrets together armadad backup writes 0600 and never overwrites; -o - refuses a terminal; the command, its help, and the operations guide tell operators to encrypt the archive (e.g. piping into age) before it leaves the host. Armada does not encrypt archives itself Partially mitigated (phase 7)
T34 Backup storage → server Tampering / EoP A modified or malicious archive is restored: path traversal, links, oversized files, swapped keys, or a forged CA Restore extracts into a fresh temporary directory and accepts only regular files with known names (no directories, no links), with per-file, total, and entry-count limits; every file must match the manifest's SHA-256 and nothing unlisted is accepted; the etcd snapshot must pass its own SHA-256 trailer, restore into a scratch member, and hold state with a schema this binary knows; the master key must match the ID recorded in the state; the CA key must match the CA certificate and the server certificate must chain to it; only then is the directory renamed into place, and never over a non-empty data directory. Checksums detect corruption, not a forger who rewrites the manifest: an archive is trusted as much as wherever it was stored Mitigated (phase 7)
T35 Admin → server DoS / tampering An older binary started against state written by a newer one misreads or corrupts it The store refuses a keyspace whose schema is newer than the binary knows; operators back up before upgrading and restore that backup to go back Mitigated (phase 7)
T36 Network / compromised node → server DoS Replicas are rescheduled away from healthy nodes (a partition of the control plane, or an attacker cutting its links), or a node flapping on purpose moves replicas around A node counts as lost only after --node-lost-timeout (default 5 min, at least 10 s); if more nodes are lost than ready, nothing is rescheduled; lost replicas are retired only after their replacements serve, so capacity never drops because of rescheduling; returning nodes only remove their stale containers, nothing moves back Mitigated (phase 7)
T37 Host / VPC → etcd Info disclosure / tampering / EoP Another process, another VPC host, or a node or admin certificate reads or rewrites the cluster state in etcd directly, bypassing the API's authorization, validation, and audit log etcd client and peer listeners require TLS 1.3 client certificates from the cluster CA and accept only the etcd member identity (AllowedCNs = etcd:member), which is issued only into control plane data directories, so node, admin, and server certificates are refused; the Armada API refuses the member identity in turn; members listen only on the server's private IP (public and wildcard addresses are refused), and peers must connect from an address named in their certificate (the member certificate names the machine's private addresses; the client SAN check stays on); the HTTP gateway and the separate metrics listener are off, and the health and metrics endpoints on the client port need the same client certificate; the member's directory is 0700 and its key 0600; secret values are stored only encrypted. Root on the control plane host can still read the member key, as it can read the CA key Mitigated (phase 7)
T38 Admin machine / network → server Info disclosure / EoP A server join token is stolen and redeemed to obtain the CA key and the master key, or a node token is used to join as a server Server tokens are a separate kind (node and server tokens are refused for each other's RPC), single use, 15 minutes by default and at most one hour, stored only as a SHA-256 hash and compared in constant time, read from a prompt, stdin, or a file, never from the command line; the join listener is rate limited and private; the joining machine pins the CA hash, so keys are only sent over a channel to the real cluster; the response's key fields are marked debug_redact and never logged; the join is audited (server.join). A stolen valid token still yields the cluster's keys: treat it like the CA key Partially mitigated (phase 7)
T39 Partition / paused process → server Tampering Two control plane servers act at once (split brain): an old leader that lost its lease keeps writing, rescheduling, or issuing certificates Leadership is an etcd election with a 10 s lease; every write of a leader's term is fenced by a compare on the election key's creation revision, so after the term ends writes fail with ErrNotLeader and change nothing; etcd itself needs a quorum to commit; a new term starts from fresh in-memory state; certmagic locks are lease-backed etcd mutexes Mitigated (phase 7)
T40 VPC → etcd peers Spoofing / DoS A machine on the private network joins the etcd cluster or disrupts it Peer traffic is TLS 1.3 with the member identity from the cluster CA; members are added only by the leader as learners after a server token is redeemed; a learner does not count for quorum until the leader promotes it; ports 2379/2380 are documented as open only between servers Mitigated (phase 7)
T41 Removed server Info disclosure / EoP A removed or decommissioned control plane server keeps its copies of the CA key and master key armada server delete removes its etcd member and record, so its member certificate no longer reaches the cluster; the operations guide says to wipe its data directory. CA and master key rotation are not supported in v0.1: a removed server that was compromised means rebuilding the cluster Accepted
T42 Supply chain → node Tampering Tampered containerd, runc, CNI plugin, tini, armada-log, or armada-shim binaries run as root on every node armada-log and armada-shim are built from logger/ and shim/ in the release workflow with the Rust toolchain pinned in their rust-toolchain.toml and the standard library only (a test refuses any crate in either Cargo.lock), and mkbundle refuses one built for the wrong architecture. Each downloaded artifact is pinned by version and SHA-256 in internal/runtime/embedded/lock.go; make runtime-bundle downloads only over HTTPS and refuses any checksum mismatch; the maintainer verifies upstream signatures (runc, tini) when bumping a version; the pack is embedded in the signed release binary, extracted into a root-owned 0700 directory, and every file is verified against the pack manifest on every start (names are checked, so a pack cannot write outside its directory). Advisories for the bundled components are checked at release time, since govulncheck does not see them Mitigated (phase 7)
T43 Container / host → embedded runtime EoP / tampering Another process or a container reaches the embedded containerd, or a container uses it to escape Armada runs its own containerd instance: socket in a root-owned 0700 directory, no TCP, CRI, debug, or metrics listener; app containers get Armada's defaults (capabilities minus NET_RAW/MKNOD/AUDIT_WRITE, no-new-privileges, default seccomp profile, AppArmor where enabled, cgroup v2 memory/CPU/PID limits, own network and cgroup namespaces) and never the runtime socket; published ports are DNAT rules bound to the private IP only (runtime.CheckContainerSpec); container IDs that end up in paths (logs, per-container files) are validated. On SELinux hosts every container (HAProxy included) runs as container_t with its own MCS categories, its rootfs mounted with the matching context and its bind-mounted files relabelled, so an escaped process cannot read host files or another container's secrets; hosts whose policy lacks the container types run containers unconfined by SELinux and armadad doctor warns Mitigated (phase 7)
T44 Compromised node / admin → scheduler Spoofing / info disclosure A node attracts replicas (and their secrets) meant for other machines by claiming labels, or labels are injected into status output and logs Only admins set labels, with armada node label or inside a join token they create (ADR 0014); a node never sends its own labels. The agent reports only its OS, architecture, and container runtime (armada.os, armada.arch, armada.runtime); the server stores them only if they are valid label values and the armada. prefix is reserved, so admins and tokens cannot set or remove those keys. armada.site comes only from the site an admin gives a node (a join token's --site or armada node site, audited, ADR 0050), never from the agent. Keys and values are whitelisted (lowercase letters, digits, ., -, _, / in keys only, 63 characters) on the CLI and again on the server; at most 32 labels per node; label changes are audited. A restricted node runs only apps whose placement names one of its admin labels, which keeps sensitive apps (and their secrets) to chosen machines and other apps off them. A compromised node can still claim another architecture or runtime and receive apps that require it. Image platforms, which restrict placement once known, also come from agents: they are accepted only for a replica the reporting node runs, only as a list of at most 32 os/arch entries whose parts are valid label values, and only the first valid list per revision is kept; the control plane does not contact registries to check them. A compromised node can report a wrong list first and keep an app off nodes that could run it A node's first reported image digest pins a revision's image for every node only when the spec names a tag, and only for the same repository; a spec that pins a digest keeps it, and a different digest a node reports is ignored. Partially mitigated (stage 1)
T47 Admin / node → app container EoP / info disclosure armada app exec (ADR 0015) is used to run commands with more privileges than the container has, in another node's or an infrastructure container, by a node that hijacks another node's session, or leaks secret values through the audit log, logs, or errors Only admin certificates may call AdminService.Exec, and only node certificates AgentService.ExecSession. The server sends each request, with a random ID, only to the node running the replica and accepts its session stream once, from that node only; the agent runs a command only for a replica assigned to it with a running container. The command runs with the container's own process settings (user, capabilities, no-new-privileges, seccomp, SELinux label, cgroup, namespaces); the request has no option to change them, and the conformance suite checks the exec process keeps them. The embedded runtime refuses system containers. Commands are checked on the CLI, server, agent, and runtime (at most 64 arguments of 4096 bytes, no NUL bytes); messages carry at most 32 KiB; an agent runs at most 8 sessions and the server ends a session after 30 minutes without traffic. The audit log and logs record the admin, app, replica, command name, and terminal flag, never the arguments. An admin can still read anything the app can, including its secrets: exec adds no authority beyond what an admin certificate already has (it controls images and secrets) Mitigated (stage 2)
T48 Node → control plane Tampering / DoS A node forges resource usage (ADR 0016) for replicas on other nodes to mislead operators (and later autoscaling), or floods the leader with usage reports Only node certificates may send on the Connect stream, and a usage report is stored under the sending node's identity; a replica's usage counts only from the node it is assigned to, checked when the sample is read. Values are bounded to plausible ranges, a report with more than 1024 replicas is dropped, and the leader keeps only the latest sample per node, in memory, for 35 seconds and until the node disconnects, so reports cost no store writes. A node can still misreport its own replicas' usage, as it can their state Mitigated (stage 2)
T49 Private network / stolen credential → control plane Info disclosure / EoP The metrics endpoint (ADR 0017) exposes app, replica, node, and revision names to anyone on the private network, or a metrics certificate is used for more than scraping The endpoint is off by default, binds only to a private address, and only the leader serves it; it requires TLS 1.3 and a client certificate from the cluster CA, and serves only metrics: identities and principals with cluster-admin (node and server certificates get 403). A metrics: certificate is refused by every API RPC and by etcd (AllowedCNs). armada metrics credentials generates the key on the admin's machine and sends only a CSR; the CA sets the identity and lifetime and the issue is audited. Label values are store-validated names, replica IP addresses, and the image each running revision's spec names (validated at deploy), never domains, environment values, agent messages, or secrets. Metrics certificates last a year; their serials are recorded and only recorded ones are accepted, so armada metrics revoke NAME refuses every certificate of a name at once (certificates issued before schema 13 stay accepted until their name is revoked or reissued); their holder learns names, images, addresses, and resource numbers, but registry credentials stay in secrets. Each node's HAProxy statistics, which name every space's apps too, listen only on the node's loopback address (127.0.0.1:8404), so neither the private network nor a container reaches them Mitigated (stage 2)
T50 Node / control plane load → admins Tampering / DoS Event history (ADR 0018): a node forges event text to mislead an operator or inject terminal sequences, or a crash loop or flapping node floods etcd with events Events are written only by the leader, from transitions it already validates (replica reports from the replica's own node); the agent protocol is unchanged. Messages are cut to 512 bytes with control characters replaced on the leader and again in the CLI, and carry no environment values, secrets, or exec arguments. The same holds for error text: the leader cleans what an agent answers to a volume operation (archives, restores, snapshots, where an app chooses the file names, which agents also quote) before it becomes an RPC error or a volume's last backup error, and the CLI replaces control characters in every RPC error message (keeping only the line breaks of the server's own lists). Repeats within 10 minutes update one stored event; lasting conditions are recorded once; events expire through an etcd lease after 24 hours and at most 2000 are kept. Writes are fenced on the leader's term and never block reconciles; followers that fall behind are disconnected. Only admin certificates read events Mitigated (stage 2)
T51 Admin / compromised node → cluster DoS / tampering Jobs (ADR 0019): a schedule that fires every minute with a long timeout piles up runs and fills nodes; an old agent restarts a job's container forever; a node reports a forged exit code; a job runs twice A run starts one container with the same security defaults, secrets handling, and capacity reservations as an app replica (no overcommit); concurrency: forbid (default) skips fires and refuses manual runs while a run is active, and replace cancels the active one, so at most one run per job is active; timeouts are 1s to 24h; at most 50 finished runs and only the latest attempt's container are kept, the containers for 24 hours. The leader alone schedules, and records each fire in the same transaction that creates its run, so a fire runs once across leader changes. Runs go only to nodes whose agent announces jobs. Exit codes are accepted only for the node's own replicas (as all reports) and only the first is kept; a node can fail or pass its own runs, which it could already do by running something else. Runs are at least once and documented as such. Only admin certificates run, cancel, or delete jobs; manual runs, cancels, and deletes are audited Mitigated (stage 3)
T52 Compromised node → cluster DoS / tampering Autoscaling (ADR 0020): a node reports false proxy load or usage to scale apps up until nodes fill, or down until they are overloaded; flapping churns replicas Request counts and samples are bounded (at most 1024 replicas per report, 1,000,000 requests per replica per node) and kept in the leader's memory only. CPU and memory count only from the replica's own node; request counts come from every node's proxy, so one node can raise any app's count, but only up to its scale.max, which the admin sets, and at most doubling the ready replicas per 15-second step. Scaling down needs every recommendation of the last 5 minutes to be lower, so a node cannot shrink an app another node's proxy sees busy; a new leader waits a full window. The count is written conditionally (unchanged count and revision, within bounds), audited as armada:autoscaler, and recorded as events. Replicas reserve capacity without overcommit as before; no new outbound connection is made (queue rules are rejected) Mitigated (stage 3)
T53 Client / container / compromised node → node Spoofing / DoS Scale to zero (ADR 0021): a client forges the wake header to reach another app; something on the network or the bridge reaches the wake listener; floods of requests to sleeping apps exhaust the agent; a node wakes apps or keeps them awake The wake listener is a Unix socket (mode 0600) in HAProxy's root-only (0700) run directory, so nothing on the network, on the bridge, or in an app container can connect; no port is opened. HAProxy sets X-Armada-Wake itself in the wk_<app> backend, overwriting any client value, and the agent accepts only valid app names. At most 1000 requests wait per node (100 per app, so one app cannot starve the others), each for at most 60 seconds, then get 503; the request body is read only when forwarded to a replica from the agent's own applied proxy configuration. Wakes from agents are rate limited per node (burst 20, then one a second), ignored for apps that do not scale to zero, and only raise an app from 0 to 1 with a conditional write; the app scales to zero again after idleAfter. Keeping an app awake by reporting requests is bounded like false load (T52) Mitigated (stage 3)
T54 Admin / spec author → proxy, DNS Tampering Traffic splitting (ADR 0022): revision labels become host names, backend names, and DNS names; a label collides with another app's name or host; weights or many targets bloat the configuration or starve replicas Labels pass the same whitelist as app names (lowercase letters, digits, single hyphens) in internal/spec, and the host <label>--<app> is checked again by proxy validation (spec.ValidHostName) and the DNS server; they are never turned into regular expressions. App names cannot contain --, so a label's host never equals an app's, and labels serve only under the root domain and .internal, never on custom domains. Traffic names only revisions of the same app that exist (checked in the store transaction), at most 10 targets, weights 0–100 adding up to 100; server weights are bounded to 1–256 in proxy validation; every revision runs at least one replica within the usual reservation and placement rules. armada app traffic is admin-only and audited Mitigated (stage 3)
T55 Spec author / compromised node / backup archive → node, data Tampering / information disclosure / DoS Persistent volumes (ADR 0023): a spec mounts a host directory or escapes the volume directory through its names; a container gains setuid programs or devices through its volume; another container reads it; a restore archive writes outside the volume or through links; two replicas write one volume; a node or admin reads another app's data through backups; a volume fills the disk A spec names only a volume and a container path: the host directory is always <data dir>/volumes/<app>/<name>, both names pass the app-name whitelist on the server and again in the agent, parents are root-owned 0700, and an existing link in place of a volume is refused; container paths follow the secret file rules (no /, /proc, /sys, /dev, runtime files) and are checked again in runtime.CheckContainerSpec. The agent mounts its volume directory nosuid,nodev, so every bind from it keeps both flags the embedded runtime also sets them on its mount, the conformance suite checks them, and SELinux relabels the directory to the container's MCS label on every start. Backups (archives, snapshots, and size measurements) walk the volume through a handle on its directory (os.Root), so a link the running app swaps in mid-backup cannot make the root agent read anything outside the volume; every file and directory is opened without blocking (O_NONBLOCK) and read only when the opened descriptor is the same entry, of the same type, as the lookup before it, so a pipe swapped in for a file cannot hold the agent's volume operations (2 per node) for every app: the backup fails with an error naming the entry, quoted, as the app chooses names; the leader replaces control characters in the agent's error before relaying it, and the CLI in every RPC error (T50). Restores unpack into a new directory with O_NOFOLLOW, accept only directories, regular files, and links with relative names that stay inside it, and replace the volume only on success; setuid and setgid bits are dropped. An app with volumes runs exactly one replica, a new one starts only after the old one is gone, the store binds volumes to a node in the replica's own transaction and refuses other nodes, and restore --node is refused while the old node is connected. Backups, restores, and removals are admin-only, audited, routed through the leader's mTLS streams (the agent opens no port), accepted only from the node a request was sent to, and at most 2 run per node; archives are as sensitive as the app's data and documented so. A restore (archive or snapshot, T72) refuses any entry that would leave less than 5% of the volumes' file system (at least 1 GiB) or 5% of its inodes free, so a small compressed archive cannot fill the node's disk for other apps. Disk use is measured and exported, not limited Mitigated (stage 4)
T56 VPC → replicas, traffic between nodes Info disclosure / tampering / spoofing Requests from HAProxy to replicas on other nodes, and their health checks, cross the private network as plain HTTP. Every port a replica publishes on its node's private IP (CNI portmap, 20000–32767) is reachable by any machine on the private network, not only by the cluster's HAProxy. Large transfers can stall when the private network's MTU is smaller than the container bridge's (1500) and path MTU discovery fails. Ports bind only to the private IP, never a public or wildcard address (T4); a replica's reported address must be on its own node (T21). With the cluster network (ADR 0025): replicas of the embedded runtime publish nothing and are reached at their own address through the tunnel; the node drops new connections into the container bridge that do not come through the tunnel (except ports published before the node joined it); the MTU follows the uplink; and the wireguard mode encrypts traffic between nodes (T57). Replicas on clusters created before the cluster network still publish ports. Mitigated (cluster network) / Accepted (published ports)
T57 VPC / compromised node or control plane → nodes, containers Spoofing / tampering / info disclosure The cluster network (ADR 0025). In the plain mode VXLAN between nodes is not encrypted or authenticated: a machine on the private network reads traffic between containers of different nodes, or forges VXLAN packets from a peer's address to inject traffic. In the wireguard mode a node announces another node's key to take its traffic, or a stolen network.key lets a machine pose as the node. In both, a forged peer list (a compromised leader or a bug) routes a public range or another network into the tunnel, or points a node's subnet at an attacker. The tunnel's port is accepted only on the private IP and only from the current peers' private IPs (spoofed source addresses on the private network remain possible in plain); containers cannot send to a peer's tunnel port (dropped in the raw table before the container networking plugins' accept rules), so a container cannot use its node's masqueraded address to inject frames with a forged inner source past the isolation of spaces, and what leaves the tunnel with a source outside the cluster's range is dropped. A packet with a source in the node's own container subnet is dropped in the same raw chain unless it arrives on the container bridge (or from the node itself over loopback), so neither the tunnel nor a machine on the private network that routes the subnet to the node can pose as one of its containers. VXLAN forwarding entries are static, with no learning or multicast. Packets from the tunnel are forwarded only to the container bridge. The agent validates the peer list before touching the host: valid names, private addresses outside the cluster CIDR, /24 subnets inside a private cluster CIDR, well-formed keys, no duplicates, at most 4096 peers, and the node itself at its own address. wireguard authenticates and encrypts every packet and allows each peer only its own subnet. The server takes a key only from the node's own certificate-bound stream, only well-formed, and refuses one another node holds. network.key is 0600, checked when read, never logged (it redacts itself), never sent, and not in backups; armada node delete drops a compromised node from every peer list, and armada node rotate-key (cluster-admin, audited) replaces a key that may have leaked, the new public half recorded only from the node's own stream and refused when another node holds it. Accepted (plain) / Mitigated (wireguard)
T58 User or service without cluster-admin / stolen certificate → cluster EoP / info disclosure / tampering A user or service with space roles (ADR 0033), or whoever steals its certificate, reaches apps, secrets, logs, or events of spaces it holds no role in or of the default space, does more than its role allows (a viewer deploys, a deployer execs), takes over nodes, tokens, servers, or backups, gives itself roles, routes another space's domain, or places apps on restricted nodes kept for others Every AdminService method has one entry in a table (methodNeeds): the space level it needs (viewer, deployer, admin), space-creator, or the caller's own certificate; any other method, including ones added later, is for cluster-admin only, and the agent API, the metrics endpoint (without cluster-admin), and etcd refuse principals. Roles are read from the store on every call, and every 5 seconds while a stream stays open (streamGuard: events and logs follows, exec sessions, volume transfers end with the call's error once the certificate is refused or expires, or no role reaches the method or the app the stream serves; a logs follow also checks before adding a replica); each handler checks the app or space it touches against the method's level, with the same answer whether or not the target exists, and listings hold only what the caller may see and no nodes; tests call every method outside the table, every allowed method with apps of other spaces, and each role with the next level's methods. deployer and admin cannot be given in the default space. Only cluster-admin manages users, services, and assignments; nobody deletes themselves or drops their own cluster-admin. Service certificates are signed from a CSR made by the tool (the CA decides the identity), last 90 days, count only while their serial is recorded, and are all revoked when the service is deleted; renewal revokes the others. A deleted space takes every role in it back, and a space created again under its name (by anyone) shows nobody but cluster-admins the events or log history its apps left from before it was created. A domain serves one space. placement.require is limited to the principal's allowed labels, also for stored revisions a principal makes serve again (a rollback, or each revision traffic splitting adds). Only cluster-admin changes a space's network isolation. Containers: T13 Mitigated (API); containers: T13
T59 Space / space-creator → cluster capacity DoS One space, or a user or service that creates spaces, reserves the cluster's free CPU and memory or piles up volumes, domains, or secrets, so other spaces cannot deploy or scale; one app's traffic takes a shared proxy's capacity or a node's uplink; one container fills a node's disk with logs Replicas reserve CPU and memory with no overcommit, so what runs keeps its share and what does not fit waits; scale.max bounds an app; container output is rotated (3 files of 10 MB) and limited to 1 MiB per second per container (4 MiB burst; the excess is dropped and counted in one log line). cluster-admin sees what each space uses (armada app top --spaces). Spaces have no limits of their own: capacity, the proxies, and uplinks are shared first come, first served, so operators who need more use separate clusters or restricted nodes Accepted
T60 Compromised node → operators Tampering / info disclosure Usage per space (ADR 0026): a node inflates or hides a space's request and egress counters to mislead whoever reads them, or a user or service without cluster-admin reads the usage of spaces it holds no role in Each report adds at most 2^30 requests and 1 TiB per replica to a saturating counter, kept in the leader's memory only. Reports count only for replicas the reconciler knows, and egress only from the replica's own node. Requests come from every node's proxy, so one compromised node can inflate any space's request count, and a node can misreport its own replicas' egress: the counters are operational figures, not proof of use. Top lists only the spaces the caller may manage, tested with an automation identity; metric labels are validated space names Mitigated
T61 Internet / app → node DoS Many short connections, from clients or from an app, fill the kernel's connection tracking table on a node, so the kernel drops new connections for every app there, and requests wait a second or more The agent raises the limit to 262144 (one entry per 4 KiB of memory on smaller machines, so a full table takes about 6% of it) and never lowers it. armadad doctor warns at 80% and the metrics endpoint shows the table per node, with values from nodes bounded. A sustained flood can still fill the table; there is no per-source or per-space limit, so this is accepted and left to the operator's limits in front of the node
T62 Internet → node DoS Clients hold connections open: silent WebSockets take proxy connections from other apps, and with frequent reloads each old proxy process stays in memory for what it still holds An app must accept the upgrade first. A silent tunnel closes after 1 hour (timeout tunnel), a reload's old process stops at most 1 hour later (hard-stop-after), and a request still gets 10 seconds to arrive and 60 seconds of silence. The node's proxy holds at most 20000 connections, so a client that keeps enough WebSockets open can still fill it until they time out.
T63 Internet → apps Spoofing A client sends X-Forwarded-For or CF-Connecting-IP with another address, to get around an app's rate limit or forge its audit trail Public ingress removes client address headers from every request unless the connection comes from a prefix in --trusted-proxies (admin only, audited, parsed on the server and again before rendering; a prefix matching every address is refused). Apps get the client in X-Real-IP, which the public frontend removes from every request (a trusted proxy's included) and sets itself to the address the client address rules judge (ADR 0035), so an app need not parse X-Forwarded-For, where only the last entries are proven. A client that reaches a node directly can only be stopped by limiting ports 80 and 443 in a firewall (operations guide).
T64 Network → operators Tampering / info disclosure The look for a newer release (ADR 0030) leaks that a machine or person runs Armada, or a forged answer pushes someone to install something else Only armada version, armadad doctor, armadad upgrade, and the client at most once a day on a terminal ask GitHub's API over HTTPS, sending nothing but the request; ARMADA_NO_UPDATE_CHECK=1 and CI turn it off; answers are bounded, and only a tag shaped like a release is believed. The answer only names a version: installing it still goes through the checksum and signature checks of T14. Servers and agents never look Mitigated
T65 User or service → apps Info disclosure / tampering The command line (ADR 0032): secret values typed as flags end up in shell history; an exported spec leaks secret values; two people changing one app at once silently undo each other's change; a saved space points commands at the wrong space --secret-env NAME=SECRET names a secret, never takes its value, and secret values are still read only from stdin or a file; --env is for plain values, which a spec holds in the clear anyway. GetAppSpec (app show -o yaml) returns the spec as written, with secret references but no values, and only for apps the caller may see; values envFrom added are left out. app deploy with flags sends the revision it started from, and the server refuses the deploy in the same transaction when another change landed in between; create refuses a name that exists and update one that does not. The space armada space use saves goes into the client config, written 0600 like the rest of it, and is checked against the spaces the caller sees; --space overrides it. Errors about a malformed --env never quote the value Mitigated
T66 App → autoscaler / node EoP / DoS / info disclosure A queue rule (ADR 0034) makes Armada run a command the spec names in a replica every 15 seconds, or connect to a Redis or RabbitMQ URL from the app's secret; a compromised or buggy app prints or reports a huge length to take capacity from other apps, sends output that breaks the agent or the leader, uses the root agent's network position to reach what the app cannot (other spaces, the platform, the provider's metadata service), makes the agent read files on the node through TLS options in the URL, or gets the URL's password into logs Redis and RabbitMQ are dialled from the replica's network namespace (runtime.ContainerDialer: the socket is created on a thread switched into it; only IP addresses are dialled there and names resolve through the container's nameserver from inside), so a rule reaches exactly what its app can, and isolated spaces stay isolated; TLS uses the node's roots with a config Armada sets, so certificate paths in the URL are never read; Redis replies are read one bounded line at a time and only LLEN's integer counts; errors name only the kind of failure, never the URL, password, or broker text; the URL is an app secret, pinned per revision, decrypted only into the node's assignment, and kept out of the spec hash and logs. The command runs only inside the app's own container, like a command probe: its own process settings, user, limits, and network (ADR 0015), never on the host, and bounded like exec commands (runtime.CheckExecCommand). It has 10 seconds and 64 bytes of output, and only a whole number between 0 and 2^40 counts. The 10 seconds hold even when the command leaves a process behind that keeps its output open: after the command exits the runtime copies output until none has moved for 2 seconds (output still on its way is never cut, and a reading cut by the deadline counts as timed out), and once the time is up it ends the command (and, on the supervisor runtime, the runc exec copying its output) with a bounded wait, so one app's rule cannot stop the readings of other apps; the agent also reads at most 4 rules at once, so a slow one does not delay the rest. Command probes are bounded the same way. Its arguments and output never reach logs or events. At most 32 commands run per agent pass. The leader bounds reported lengths, counts a reading only from the node the replica is assigned to, and an app can raise only its own count, never past scale.max, which capacity checks as for any replica (T28). Armada never connects to the queue or holds its credentials. Schedule rules use only the spec and the leader's clock Mitigated
T67 Internet → apps Spoofing / EoP A client outside an app's ingress.ipAllow (or inside its ipDeny) reaches it anyway: by writing an allowed address in X-Forwarded-For, Forwarded, X-Real-IP, or a CDN header; by prepending one before a trusted proxy's entry; by going through a trusted proxy that names no client; by another of the app's names, a revision label, the wake path, or a replica's port; or because a rule change never reaches a node, or a crafted range injects HAProxy configuration Only X-Forwarded-For counts, and only from --trusted-proxies: every client address header is removed first from other connections, whose own address is the client; from a trusted proxy the last entry (the one it appended) counts, stepping over trusted proxies for at most four hops, and a missing or invalid entry, or a chain of only trusted proxies, leaves the client unknown, which both modes refuse (fail closed). The check is the first rule of each of the app's backends (labels and wake included), so every name and path leading to the app passes it, and an app scaled to zero is not woken. Replicas publish no ports on the cluster network. Rules are part of the proxy's structural signature, so a change reloads. Ranges are parsed on the server (no zones, no IPv4-mapped IPv6, no whole-family prefixes, at most 64), sent as canonical masked prefixes, checked again by proxy.Config.Validate and the single-token check before rendering, and fuzzed (FuzzRenderIPRules). The e2e scenario ip-rules checks spoofed headers, prepended entries, hop stepping, and fail closed against a real HAProxy. Residual: a trusted proxy that lets its own users choose the appended address passes them (ADR 0035) Mitigated (ADR 0035)
T68 Web page → app Info disclosure / tampering A CORS policy lets a page on any origin read an app's answers with the user's cookies (* with credentials, or reflecting every Origin), or a crafted origin, method, or header name injects HAProxy configuration Origins must be exact http(s)://host[:port] (no paths, wildcards, or spaces), * only alone and never with allowCredentials; Access-Control-Allow-Origin is the exact listed origin, never an unlisted one reflected; methods come from a fixed list and header names are letters, digits, and hyphens; every value is checked again by proxy.Config.Validate and the single-token check before rendering; every response gets Vary: Origin. CORS does not keep other clients out (documented; ADR 0037) Mitigated (ADR 0037)
T69 Spec author / compromised node → node, other replicas EoP / info disclosure / DoS Init containers and sidecars (ADR 0038): an extra container is used to get more than the app's container has (privileges, host paths, another replica's files or network); a sidecar image is swapped under a mutable tag; a secret in a container's env leaks through the spec hash or the agent's logs; a stray container keeps running in a replica's namespace after it is gone; many sidecars take a node's capacity Every container gets the app container's security defaults from the same code path (appOpts: dropped capabilities, no-new-privileges, seccomp, AppArmor, its own cgroup with memory and PID limits, no host network, no runtime socket); the spec has no field for host paths, privileges, or ports, and runtime.CheckContainerSpec refuses any file or volume of a container that is not one of its replica's own mounts. A container joins only its own replica's network namespace and shares that replica's SELinux categories (and its volumes and secret files, which carry them); other replicas keep categories of their own, and the firewall treats the replica's address as one app. Container env takes secrets only by secretRef to keys of secrets of the app's space (T94), decrypted only into the assignment; the spec hash blanks their values like the app's env, and nothing logs a ContainerSpec. Images are validated like the app's and pulled with the app's registry credentials only for the same registry; they are not pinned to a digest, so the spec should pin them (documented). The runtime lists containers with their replica and removes one whose replica's container is gone; Remove takes them all before the network namespace. Each counts toward the replica's reservation (at most 4 init containers and 4 sidecars, each with a memory limit), so placement sees them Mitigated (ADR 0038); residual: images that are not pinned can differ between replicas, and a sidecar sees the app's localhost services by design
T70 App → apps / internet → apps / app → node EoP / info disclosure / DoS Internal TCP ingress (ADR 0039): an app of another space, or one allow leaves out, connects to a space's database through the node's proxy; the TCP listener is reachable from the internet or the private network; the port opened on the gateway for isolated containers reaches a host service listening on every address instead of the proxy; an app claims a port Armada uses, or one another program on the node holds, so the proxy fails to reload and every app on that node stops changing; a crafted port, backend, or caller injects HAProxy configuration protocol: tcp is valid only with type: internal, and TCP listeners bind only to the bridge gateway (frontend tcp_<port>, tested for every bind line); the caller's space and app come from the node's clients map by source address, the backend is picked by that space (one app per space and port, refused in the deploy transaction and again by proxy.Config.Validate), and an unknown caller, the node itself, another space, or an app outside allow is rejected at tcp-request connection before any data is read; TCP needs the cluster network (deploys are refused without it). Armada's own ports are refused in the spec, by Validate, and by the agent's network config parser. The agent leaves out ports another program listens on (every address, or the gateway while the proxy does not hold it), so a reload never fails on them. The firewall opens TCP ports on the gateway to isolated containers only for ports of TCP apps sent by the leader and re-checked, and only towards a socket bound to a specific address (socket wildcard 0), never a wildcard listener. Ports, backends, and callers pass the single-token check before rendering, and are fuzzed (FuzzRenderTCP). The e2e scenario tcp checks allowed, refused, other-space, and outside connections Mitigated (ADR 0039)
T71 Job → node / other spaces EoP / info disclosure / DoS A queue trigger (ADR 0040) makes a node dial the job's broker without a container: the dial reaches what the job could not (the node's own network, other spaces, the platform, the provider's metadata service); a crafted job ID names a path on the node; a forged report gives another job's sandbox address, or a huge length, or many entries; the URL's password reaches logs or events; stale sandboxes keep addresses after the job or node changes The agent dials only from a network sandbox: a namespace pinned under the runtime's run directory and wired by the same network plugins as a container, with an address of the node's subnet that the agent adds to its own containers as the job's ID before dialling, and that the leader adds to every node's firewall endpoints only from the node it picked, checked against that node's subnet (validReplicaIP) with clashing addresses left out. Isolation therefore applies to the sandbox as to the job's containers, which the e2e scenario queue-jobs checks for an isolated space. Sandboxes exist only on nodes of a cluster network with the embedded runtime; deploying a trigger without a cluster network is refused, and the agent never falls back to the host's network. Sandbox IDs are checked as app IDs (runtime.CheckSandboxID: lowercase letters, digits, dots, and hyphens, no ..) on the agent and in the driver. Only IP addresses over TCP or UDP are dialled from the namespace's thread, which is never unlocked while in it. The leader bounds reports (256 entries, valid IDs, lengths up to 2^40, ages up to an hour) and counts a reading only from the node it picked; a job can raise only its own runs, never past parallelism (at most 50), each placed under capacity checks (T28). One space cannot take queue reading from another: the leader places at most 64 sandboxes on a node, what an agent keeps, one space at a time (the first job of every space, then the second), and records QueueUnread for jobs that fit nowhere; an agent given more keeps them in the same order and reports the rest, which the leader records as QueueUnread; readings run at most 8 at once and 2 per space, without waiting for a whole round, so a broker that never answers holds one slot. URLs and errors are handled as in T66: the URL is a pinned job secret, decrypted only into that node's assignment, outside any hash, never logged; log lines and events name only the kind of failure and the length. Sandboxes not in the assignment are removed after the first assignment (never before), and armadad runtime reset removes the rest Mitigated (ADR 0040)
T72 Bucket holder / compromised app / spec author → backups, node Info disclosure / tampering / DoS Volume backups (ADR 0041): whoever reads the bucket reads an app's data; a changed or swapped object is restored into a volume; a crafted snapshot writes outside the volume through .. or links; the bucket URL's credentials or the key reach logs, events, or the app; an http:// URL sends them in clear over the internet; a backup that never finishes, or a full bucket, deletes the backups that exist; a crafted URL makes the node call something else Everything in the bucket but the repository's ID and key check is sealed with AES-256-GCM under a key derived (HKDF-SHA-256) from the app's key secret, with the object's name as additional data; chunk names are an HMAC of the content under another derived key, and chunk boundaries come from a keyed gear table, so the bucket shows only counts and sizes. Restoring opens every object and checks each chunk's plaintext against its name, so damaged, swapped, or moved objects fail the restore, which writes into a new directory and swaps it in only when everything was written; entries are relative, clean paths, written without following links and without setuid or setgid bits, as for volume archives (T55). The URL and key are pinned secrets of the app, decrypted only into the request to the volume's node (while it is not connected, a listing goes only to a connected node that is ready and allowed by the placement of the revision serving and the one the app is heading to, so never to a restricted node or one placement.require leaves out (T28, T44), and is refused when there is none; restore --node names a node that passes the same check), never in a spec hash, the app's environment (envFrom leaves them out), logs, events, or errors (ParseTarget and bucket errors never repeat the URL); http:// is accepted only for private IP addresses; loopback, link-local, unspecified, and multicast addresses are refused in the URL and when a name resolves to one (checked on the address dialled, and the bucket is always dialled directly, never through a proxy from the agent's environment); and because the node reaches the bucket from its own address, outside the space's isolation (T13), only cluster-admins may give an app a bucket on a private IP address (deploys, rollbacks, and URL secret changes by others may only keep the addresses of the revision the app is heading to). A name that resolves to a private address is still reached, over TLS only. Old backups and chunks are deleted only after a new backup was written, chunks younger than an hour never, and never on a partial read of the kept backups. The bucket is reached from the node's own address (documented): the access key should be limited to the bucket, and versioning or object lock keeps backups from someone holding it Mitigated (ADR 0041); residual: whoever may read the app's secrets may read and delete its backups
T73 Network partition / control plane outage → volume data Tampering / DoS Failover (ADR 0041) restores a volume elsewhere while the old node, only cut off from the control plane, still runs the app and takes writes from clients that reach it directly, so two copies diverge; a node that restarts while cut off starts the app again before anyone tells it otherwise; the old node returns and writes again Replicas of apps that fail over carry a fence time of half the node-lost timeout; the agent stops them once it has had no control plane session for that long, measured on its own clock from the end of its last session (or its start), also before its first assignment (by a container label), and keeps them stopped until it is back in touch. gRPC keepalives (30 s + 10 s) end a dead session well within that time. The leader restores elsewhere only after the full node-lost timeout and only with more nodes ready than lost, and binds the volumes only if the old node is still not connected; only local volumes are restored and bound (an app's NFS volumes stay on their server), and an unexpected error in a failover fails it with a VolumeFailoverFailed event instead of ending the server, so a spec cannot crash the leader (and each leader after it) by losing a node; a returning node no longer has the replica assigned and removes it, leaving its copy of the volume unused. Residual: writes since the last backup are lost, and such apps stop while the control plane is unreachable (documented) Mitigated (ADR 0041)
T74 Spec author → storage outside the cluster, node Info disclosure / tampering / EoP NFS volumes (ADR 0042): a deployer points an app at another team's export, or at a host outside the private network (the provider's metadata service, a public server); a crafted server or path injects mount options; an export holds setuid programs or device files; the mount hangs the agent; mounts pile up on nodes Only cluster-admins add an export to an app or make one writable (checkNFSExports); deploys and rollbacks that keep the exports of the revision the app is heading to are anyone's who may deploy it, and a rollback, and every revision armada app traffic or a spec's revisions.traffic adds to an app's traffic, re-runs this check and the node-label check on the stored spec, so a removed export or label cannot come back through an older revision (revisions already serving may be reweighted). server must be an IP address in a private range, path an absolute clean path of a fixed character set, checked in the spec and again by the agent before it calls the kernel; options are fixed by Armada (nosuid, nodev, hard, TCP), the source string is the only input. Mounts are bound into containers nosuid,nodev (and ro with readOnly) with the containers' usual defaults. A mount that does not finish within two minutes fails the start without holding the agent; exports no replica names are unmounted lazily. The server's exports and firewall still decide which nodes may mount what Mitigated (ADR 0042); residual: exports are reached by node address only (AUTH_SYS)
T75 Local user / control plane → client Spoofing / info disclosure The CLI learns control plane addresses from two new places: the armada-servers header of every answer, and /etc/armada/servers on Armada machines. A forged list sends the CLI, and its certificate, to another machine; a user who can write the file sends other users' commands elsewhere; the file leaks the cluster's addresses. armadad server join signs a user certificate for whoever runs it The CLI still verifies every server against its config's CA with the armada-server name, so a listed address that is not a cluster server gets no command and learns only that a client called; only private addresses are taken (CheckServers), from the header only after the call succeeded against a verified server, at most once per run. The file is read only when root owns it and nobody else may write it, at most 64 KiB, and only adds addresses after the config's own; it holds addresses, no credentials. The join runs as root with the CA key the cluster has just handed over, so the certificate it signs gives no more than that key already does; it names the sudo user (or --user-name), whose rights the cluster records decide, and existing configs are kept Mitigated
T76 Node disk → log history Info disclosure / DoS Container output kept after the container is gone (ADR 0043) holds sensitive data longer, or fills the node's disk History lives in the runtime's log directory (root, 0700, files 0600) like live logs, and leaves the node only through AdminService.Logs to a caller who may view the app's space; the agent answers searches only from its leader session. The rate limit per container still applies, and every node prunes history to its retention (7 days by default, 1 hour to 90 days) and size (10% of the disk, at most 5 GiB, by default) every minute, oldest first; searches read at most 1 GiB per node for 30 seconds, at most 2 run at once on a node and 4 on the leader (2 per caller), the leader keeps at most 32 MiB of a node's answer and only entries of the queried app's replicas inside the window that were placed on the answering node (a record of each replica's node, kept for the retention), so a compromised node cannot add entries for replicas it never ran or push genuine ones out, and --grep is plain text, never a pattern Mitigated (ADR 0043)
T77 Node → log store outside the cluster Info disclosure / spoofing Log export (ADR 0044) sends every app's output, which can hold secrets an app prints, to an endpoint outside the cluster; a plaintext URL, a redirect, or a leaked token exposes it, and a wrong endpoint receives it Only cluster-admins set the endpoint (audited, token never in the audit log); https:// required except to private IP addresses; no credentials or query in the URL, so it can be shown; token read from a file or stdin, sealed with the master key, opened only into assignments over mTLS, never logged (debug_redact, errors replace it); redirects not followed; error replies cut to one cleaned line; system containers never sent; a new destination never receives output written before it was set
T78 Control plane → chat or mail services Info disclosure / spoofing / DoS Alerts (ADR 0045) send event text about apps and nodes to third parties; a leaked webhook URL, bot token, or SMTP password lets others post as the cluster; a plaintext URL or SMTP session, or a redirect, exposes them; a flood of events floods a channel Only cluster-admins manage channels (audited without secrets); secrets read from a file or stdin, sealed with the master key, opened only on the leader, never logged or returned (debug_redact, errors replace them); https:// or private IPs only, no credentials in URLs, redirects not followed; SMTP TLS on 465, STARTTLS elsewhere, plaintext only to private IPs, passwords only over TLS; email headers only from checked addresses and a control-free subject; chat text sent as plain text (Slack escapes &, <, >; Discord escapes Markdown with mentions disabled; Telegram has no parse mode), so text from a registry, a command, or a file name is never a mention or a link whose label hides where it goes; notices hold only what events hold (no secrets or env values); per-channel queue with places and turns per source (each space, and the cluster), 20 per minute per source, so one space cannot crowd out another's notices or a lost node's; repeats within 10 minutes not sent
T79 App → other apps Spoofing A workload connects to its node's public frontend and claims an allowed client address in X-Forwarded-For, because a cluster-admin trusted a range (such as 10.0.0.0/8) that holds the cluster's container addresses The leader takes the container ranges (the cluster CIDR, or the default bridge subnet 10.121.0.0/24 without a cluster network) out of the trusted prefixes before they reach any node (proxy.ExcludePrefixes, Server.proxyTrusted), and SetClusterConfig refuses a prefix that holds only container addresses or that would split into more than 64 prefixes. Residual: a cluster without a cluster network whose agents use another --container-subnet must leave it out of the list itself (operations guide) Mitigated (ADR 0035)
T80 Internet → apps Spoofing A client sends X-Forwarded-Host (or -Port, -Prefix, -Server) with its own domain, and an app that builds links from it (password reset emails, redirects) sends users to the attacker The public frontends route on the request's Host (HTTPS on 443, and plain HTTP on 80 only when the cluster obtains no certificates), so they remove these headers from every request, a trusted proxy's included (proxies in front commonly pass a client's value on unchanged), and set X-Forwarded-Proto (https, or http on port 80) themselves. The internal frontend leaves them alone. Residual: an app that trusts Host from callers in its own space, or a client-chosen Forwarded: host= passed by a trusted proxy Mitigated
T81 Space viewer → node agents, leader DoS A caller who may view one space's logs opens many live log streams (armada app logs without --since), each holding a goroutine, a log file, and up to its tail in the root agent of every node that runs the app, nodes shared with other spaces The leader serves at most 128 live Logs calls at once and 8 per caller (ResourceExhausted beyond), each node streams at most 32 at once and answers more with a busy notice, and a tail keeps at most 4 MiB in memory however many entries are asked (at most 10,000) Mitigated
T82 Spec author → other spaces' images Info disclosure An app names another app's private image by digest and runs it on a node where that image already is, without registry credentials, either under the image's own name or under a name on a registry its author runs that answers only for the digest (content on a node is shared by digest across every name) The embedded runtime records on each image which credentials fetched it (a SHA-256 of the registry, user, and password, never the password, or anyone when the registry needed none); an image on the node is used without the registry only for the same credentials or when anyone may read it, else the registry is asked whether the app's credentials may read that digest, and the start fails when it refuses. Images pulled before this record existed are checked once. A pull under any name uses a blob already on the node (index, manifest, config, or layer) only when an image pulled with the same credentials, or with none, holds it; any other blob, including one another pull fetches at the same time, is fetched again from that pull's registry and checked against its digest (proveContent records what the pull's own fetcher downloads), so the registry must serve the content itself. Unpacked layers are shared by chain ID, computed from the config's diff IDs alone, and are reused without reading the layer blob; so every layer blob is fetched, and an image is kept only once each blob is shown to decompress to its diff ID (imagecheck: the unpacker's containerd.io/uncompressed label or reading it once), else it is removed; a config naming another image's diff IDs gets nothing Mitigated
T83 Spec author → node's network SSRF An app names an image on a registry its author runs, which answers with redirects (or a token service address) pointing the agent, which pulls as root from the machine's network, at the node's own services, the private network, cloud metadata, or other spaces' containers, and reads the target's status line back in the replica's message Every registry request, its redirects, and the token service's are HTTPS only (a redirect to plain HTTP is refused before it is sent, and the target needs a certificate the machine trusts for the name it is reached by), at most 10 redirects, and no connection to link-local (metadata), unspecified, or multicast addresses, checked when dialling (newResolver, guardRegistryClient). A machine whose environment names an HTTPS proxy reaches registries tunnelled through it; which addresses that proxy connects to is up to its operator Mitigated
T84 App → its own and other apps' containers (supervisor runtime) DoS An app keeps its node's shim busy (many short-lived processes, slow requests), so a request to the shim times out and the agent takes the shim for dead and kills every container it supervises, or makes the shim's table of exit codes grow without limit; or a corrupt runc pid file makes the shim signal process 0 or 1 The agent abandons a shim's containers only when nothing listens on its socket or its systemd unit is no longer active (Shims.gone); a timeout or a closed connection is returned as an error and asked again on the next pass, and starting and abandoning a group's shim are serialized per group. The shim keeps at most 4096 unclaimed exit codes (containers' own pinned), refuses a pid ≤ 1 from runc's pid file, and deletes a container whose pid file it cannot read. Its unit has the limits of armada-runtime.service (LimitNPROC, LimitCORE, LimitNOFILE, TasksMax unlimited; containers have their own pid and memory limits) Mitigated
T85 Network → apps without certificates Info disclosure / tampering With --tls=false port 80 serves apps over plain HTTP, so anyone on the path reads and changes requests, including cookies and credentials Only cluster-admins turn certificates off (audited); meant for private networks and labs, and documented as unencrypted; client address rules, trusted proxies, and X-Real-IP apply on port 80 as on 443; with certificates on, port 80 only redirects (and answers ACME challenges). Residual: an operator who turns certificates off on a machine reachable from the internet exposes its apps' traffic unless ingress.ipAllow narrows who connects Accepted (operator's choice)
T86 Network → apps Spoofing A client sends a PROXY protocol header of its own to name another address, passing ipAllow or forging X-Real-IP; or a container, inside a broad listed range, does Only connections from the prefixes of cluster update --proxy-protocol may start with the header (expect-proxy only if { src ... }); from anywhere else it is not HTTP and is refused; prefixes are canonical, none matches every address, set only by cluster-admins (audited), and the leader takes the container ranges out before they reach the proxies (a prefix of only container addresses is refused); values are re-validated and checked as plain tokens before rendering. Residual: whoever controls a listed address names any client, so only load balancers the operator runs belong in the list Mitigated
T87 Spec author → every app's ingress DoS An app lists as a custom domain a host Armada already gives an app under the root domain (another app's <app>.<root> or <label>--<app>.<root>, or its own), and once the domain verifies, two rules name one host and path, so the proxy configuration of the whole cluster fails validation and every node keeps the last good one: later rollouts, scaling, and moves no longer reach the proxies Hosts under the root domain belong to the app or labelled revision they name, whatever order apps come in (rootHosts); a custom domain naming the app's own host is routed once (appHosts); any public host and path a second backend claims is left out of that backend (publicRoutes) and its app records a RouteConflict event (a warning for alert channels) instead of failing the configuration; no certificate is requested for a host the app does not serve Mitigated
T88 Deployer / control plane → DNS zones Elevation / tampering An app's DNS token for its custom domains (ingress.dns, ADR 0047) lets whoever holds it change the zone's records, not only obtain certificates: a deployer could try to make the control plane use its token for names of other spaces, or read another app's token; a token could leak through logs, events, the audit log, or the app's own environment The token is an app secret like any other: set from a file or stdin, sealed per app and version, pinned by the revision, and read or set only by those who may deploy to the app's space; the leader opens it only to hand it to the certificate manager (certs.DNS redacts itself), never into assignments, events, logs, or envFrom; it is used only for the app's custom domains whose TXT ownership record verified (a domain belongs to one space), never for <app>.<root domain> or another app's names; a token the leader cannot open is a DNSTokenUnavailable condition and the domains keep HTTP-01 Mitigated (a compromised leader can still use every app's token on its zone, as with any app secret; scope tokens to their zone's DNS records)
T89 Client config, SSH host → client Spoofing / tampering / EoP A config that reaches the servers through SSH (ssh, ADR 0048) has the client run a program: a crafted destination could be read as an ssh option (-oProxyCommand=...) and run commands; the SSH host relays the control plane traffic and could read or change it The client runs the user's own ssh with fixed arguments and no shell; CheckSSH refuses a destination that starts with - or holds spaces or shell characters (on load and on --ssh), -- ends ssh's options, and the forwarded address is the value of -W (one starting with - is refused). mTLS runs end to end inside the SSH connection, so the SSH host sees only ciphertext and cannot pose as a server (the client checks the cluster CA); authentication to it is ssh's own (keys, agent, known_hosts), and Armada stores no SSH secret. No port beyond SSH faces the public. Residual: the SSH host can stop or delay the traffic; a config file is trusted as much as the person who wrote it (it already holds their key) Mitigated (ADR 0048)
T90 Internet → mesh nodes (ADR 0049) Spoofing / DoS / info disclosure A mesh cluster has no private network: its WireGuard port faces the public, the leader's join listener faces it while a join token is open, and anyone on the provider's network could route the mesh range to a node to reach its API, etcd, or HTTP-01 forwarding, or forge a peer's mesh address; a node's endpoint, announced by its agent, could point peers' packets anywhere WireGuard answers nothing that does not authenticate, and allows each peer only its mesh address and subnet as sources; the raw chain drops the mesh range arriving anywhere but the tunnel (loopback aside), and ARMADA-NET-MESH accepts only the mesh range on the node's own mesh address; internal services bind only to the mesh address; the join listener opens only while a token is unused and unexpired (10 minutes by default, at most an hour, single use; closed as soon as the last one is redeemed), keeps every protection of T5 and T29 (server-only TLS with the CA pinned by hash, JoinService only, rate limits, bounded connections before TLS), and with join-token --from closes connections from other addresses before TLS; endpoints are unicast IPv4 addresses checked by the leader and again by every node, outside the cluster's ranges, and a roamed endpoint is kept only while its handshake is fresh. Residual: the WireGuard port and the open join port can be found by scanning; a node's traffic to peers crosses the internet (encrypted) The same join listener protections hold where a plain or wireguard leader opens its join listener on its endpoint for a machine of another site (ADR 0050): only tokens for a site other than the leader's open it and may be used there (others are refused before redeeming); a server token for another site hands the CA and master keys over that listener, with every protection of T38 and the CA pinned by hash, so the keys only reach the real cluster. A machine of another site with no private network gets a mesh address in a plain or wireguard cluster: every node drops the cluster's mesh range arriving anywhere but its WireGuard interface (loopback aside), as in the mesh mode, and a private IP inside the recorded range is refused at join. Mitigated (ADR 0049, ADR 0050)
T91 Cluster admin / node agent → network switch (ADR 0050) Tampering / DoS / info disclosure A switch between plain and wireguard runs two tunnels on every node at once: a second port, a second interface, and routes that move; a switch to plain stops encrypting traffic between nodes; an agent's report says when its node is ready, and a false one could move the routes before the other nodes reach it Only cluster-admin starts or gives up a switch (cluster update --network), audited, with an event for every step; while it runs, both ports accept only the peers' private IPs on the node's own, both interfaces get the same forward, spoofing, and container rules (containers reach neither peer port), and a peer's WireGuard key and subnet are checked as in the wireguard mode; reports are bounded (known modes and steps, node names, a cleaned message) and only time the steps; a node counts a peer as reached only when its reply comes in through the new interface. A mesh never switches. Residual: a false report from a compromised node can cut traffic between nodes until the switch finishes, which a compromised node can do anyway Mitigated (ADR 0050)
T92 Compromised node / cluster admin → nodes behind NAT and relays (ADR 0050) Information disclosure / tampering / DoS Nodes behind NAT in two sites reach each other only through a relay, a node with an endpoint that decrypts and encrypts again what it forwards: a compromised relay sees, changes, or drops that traffic; a node could claim to be behind NAT, or a relay, to pull traffic towards it; an agent's peer list could make a node forward between arbitrary addresses The leader alone picks relays: nodes with a key and an endpoint, not behind NAT, ready, and not cordoned; only cluster-admin pins one (armada node relay, audited, with an event), and a node behind NAT cannot be one. Every node checks the peer list again (a node behind NAT that claims to relay is refused, an endpoint sent for one is ignored) and works the relay of each pair out itself, so both directions cross one relay. A relay forwards only between the subnets of the pairs it relays for, in both directions, and nothing else from a tunnel; WireGuard still allows each peer only its own addresses, and a node behind NAT allows the subnets of the nodes it reaches through a relay only on that relay's entry; the relay's isolation rules apply to what it forwards. Node addresses are never relayed (nodes behind NAT are not servers). Residual: a relay is a member holding a node certificate and sees the traffic between the pairs it relays for in plain text, as any node sees its own; it can stop or delay it, which doctor's handshake and relay checks show; choose relays as carefully as servers Mitigated (ADR 0050)
T93 Bucket holder / cluster-admin / network → control plane backups (ADR 0051) Info disclosure / tampering / DoS The leader writes the control plane, including the CA key and the master key, to a bucket: whoever reads the bucket could take the cluster; a changed, swapped, or crafted snapshot restored by an operator could plant state, keys, or files; the bucket URL's credentials or the key could reach logs, events, the audit log, cluster show, or shell history; an http:// URL sends them in clear; a name or address could point the leader at its own services; a failed or endless backup could delete the good ones Every object but the repository's ID and key check is sealed with AES-256-GCM under keys derived (HKDF-SHA-256) from the operator's key (at least 32 bytes), with the object's name as additional data, and chunk names are keyed HMACs, as for volumes (T72): the bucket shows counts, sizes, and times only. The URL and key are read by the client only from a 0600 file or stdin, never from arguments, sealed with the master key in the state (scope cluster, versioned), opened only by the leader to back up, and never logged, audited (the audit entry says url/key), put in events or errors (ParseTarget and bucket errors never repeat the URL), or returned (cluster show names only the endpoint, bucket, and prefix); only cluster-admin sets them. http:// is accepted only for private IP addresses; loopback, link-local, unspecified, and multicast addresses are refused in the URL and at dial time, and the bucket is dialled directly, never through an environment proxy. The repository lives at <prefix>_control-plane/, a root no app ID can take, so no volume's pruning reaches it. A restore from the bucket goes through the same checks as an archive (only known names and regular files, size limits, every file against the manifest's checksums, the snapshot opened as state of a known schema, the CA, the server certificate, and the master key against the state) plus every chunk against its name, into a new directory that replaces nothing. Old backups are deleted only after a new backup was written, never on a partial read of the kept ones, and chunks younger than an hour never; a backup cut short by a lost leadership records and deletes nothing. Residual: whoever holds both the bucket and the key controls the cluster (documented: keep the key apart, limit the access key to the bucket, use versioning or object lock); the bucket's holder can delete or withhold backups, which ControlPlaneBackupFailed and the backup metrics make visible Mitigated (ADR 0051)
T94 Deployer of a space / compromised app / older server → secrets Info disclosure / tampering Secrets owned by a space (ADR 0052): an app reads a secret of another space; a container receives keys its spec does not name; deleting or rotating a shared secret breaks or silently changes another app; an older leader, during a rolling upgrade, reads SECRET/KEY as a key of the app's own secret and sends the wrong value A reference names a secret only within the app's own space (spec.SecretSource: the space comes from the app's ID, never from the spec), so no spec reaches another space's keys, and the secret RPCs check the space of the secret's ID like an app's. A replica receives only the keys its revision pins; the leader opens them only into that node's assignment, as before (T28). Whoever may deploy in a space may set, use, and delete its secrets, as they could deploy or exec into any of its apps (T47); a space is the boundary, and values that must stay apart belong in another space. A key or secret an app's active or target revision uses cannot be deleted, versions any kept revision pins are kept, secret show/list name the revisions and apps that use each, and a change rolls out only the apps that use the key (or none with --no-rollout). A spec that names another secret needs every server on schema 15, so an older server can no longer lead once one is deployed; the layout of every app's own secret is unchanged, and a test compares a replica's spec with what the release before sent Mitigated