Sub-issue of #601.
Summary
Add an optional coordinator that designates exactly one RESTHeart node as the MQTT→MongoDB writer in a multi-node cluster, using a distributed lock backed by MongoDB.
Motivation
The v1 module (#601) handles multi-node deduplication via a deterministic _id (hash of topic + receivedAt + payload), turning every insert into an idempotent upsert. This is correct and requires no coordination, but it does mean every node writes to MongoDB independently, multiplying write load by the number of nodes.
Single-writer mode avoids this: only the elected leader writes to MongoDB. The other nodes subscribe to MQTT only for their own SSE clients.
Proposed design
- A
MqttLeaderElection component uses a MongoDB TTL-based lock document in restheart.mqtt-leader (same pattern used by other distributed systems on MongoDB).
- The lock is refreshed every
heartbeat-interval-ms by the leader.
- If the leader fails to refresh (node crash, network partition), the lock expires after
lock-ttl-ms and another node acquires it.
- Non-leader nodes disable
MqttMongoWriter; leader nodes enable it.
- Leader transitions are logged at WARN level; a brief window of duplicate writes during transition is acceptable (the deterministic
_id upsert still prevents actual duplicates in MongoDB).
plugins-args:
mqtt-mongo-writer:
enabled: true
single-writer:
enabled: false # opt-in; default is all-nodes-write
lock-ttl-ms: 10000
heartbeat-interval-ms: 3000Tradeoffs
The deterministic _id approach in v1 already eliminates duplicate documents — single-writer mode only reduces write amplification. For most deployments the extra writes are negligible compared to the operational complexity of leader election. This feature is only worth implementing at scale (10+ nodes) or when MongoDB write throughput is a genuine bottleneck.
Dependencies
Sub-issue of #601.
Summary
Add an optional coordinator that designates exactly one RESTHeart node as the MQTT→MongoDB writer in a multi-node cluster, using a distributed lock backed by MongoDB.
Motivation
The v1 module (#601) handles multi-node deduplication via a deterministic
_id(hash of topic + receivedAt + payload), turning every insert into an idempotent upsert. This is correct and requires no coordination, but it does mean every node writes to MongoDB independently, multiplying write load by the number of nodes.Single-writer mode avoids this: only the elected leader writes to MongoDB. The other nodes subscribe to MQTT only for their own SSE clients.
Proposed design
MqttLeaderElectioncomponent uses a MongoDB TTL-based lock document inrestheart.mqtt-leader(same pattern used by other distributed systems on MongoDB).heartbeat-interval-msby the leader.lock-ttl-msand another node acquires it.MqttMongoWriter; leader nodes enable it._idupsert still prevents actual duplicates in MongoDB).Tradeoffs
The deterministic
_idapproach in v1 already eliminates duplicate documents — single-writer mode only reduces write amplification. For most deployments the extra writes are negligible compared to the operational complexity of leader election. This feature is only worth implementing at scale (10+ nodes) or when MongoDB write throughput is a genuine bottleneck.Dependencies