Skip to content

Stale node identities linger in cluster state after a node exits, blocking MlxJaccl (RDMA) placement #2194

Description

@batmunkh0612

Environment

  • exo official app 1.0.71, 5 nodes: 4× Mac mini M4 (macOS 26.5.1) cabled Thunderbolt (full RDMA ring visible in topology) + 1 Ethernet-only node

Symptom

When a node leaves the cluster (app quit or its exo process killed), its identity stays in GET /state → nodeIdentities for a long time (well over 10 minutes in our tests — it survived repeated checks and individual node restarts). Because MlxJaccl placement appears to require all current cluster members to be part of the RDMA ring, the lingering identity blocks RDMA placement even though every live node is RDMA-connected:

GET /instance/previews?model_id=...
→ "Requested RDMA (MlxJaccl) but no RDMA-connected cycles available"

GET /state
→ nodeIdentities: 5 entries          # one is the departed node
→ topology RDMA links: 4/4 live nodes each with degree 2 (complete ring)

Restarting individual nodes makes it worse: each restart mints a new node ID and the old one lingers too, so the ghost count grows. The only reliable way we found to clear ghosts was a coordinated stop of every node followed by a fresh start (then nodeIdentities returns to exactly the live set).

Repro

  1. Form a cluster of N nodes (N−1 of them RDMA/Thunderbolt-connected, 1 Ethernet-only).
  2. Quit/kill exo on the Ethernet-only node.
  3. Wait — nodeIdentities keeps reporting N entries long after the node is gone.
  4. GET /instance/previews for any model keeps reporting MlxJaccl unavailable ("no RDMA-connected cycles") despite the live nodes forming a complete RDMA ring in topology.

Expected

Departed node identities should expire from cluster state within a bounded, short interval (or on liveliness loss), and placement should consider only live members — otherwise a single flaky/departed device disables RDMA placement for the whole cluster until a full-cluster restart.

(Separate but related observation while testing: on a clean 4-node cluster where all members are in the RDMA ring, MlxJaccl still reported no cycles with links negotiated at 20 Gb/s over plain USB-C cabling — if certified 40 Gb/s Thunderbolt cables are a hard requirement for JACCL, it would be great to surface that in the error message instead of the generic "no RDMA-connected cycles available".)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions