Skip to content

Commit 424e6fe

Browse files
robertsLandoclaude
andauthored
fix: add unique index on retained.topic to prevent COLLSCAN stalls (#91)
Without an index on `retained.topic`, every storeRetained upsert and createRetainedStreamCombi lookup falls back to a collection scan. Under load, the ordered bulkWrite in storeRetained serializes the whole publish pipeline behind those scans, stalling broker.publish callbacks by tens of seconds on deployments with thousands of retained topics (observed: ~60s uniform lag with 15k retained + 200 msg/s). The index is declared `unique` to match the {topic}-filtered upsert semantics. Also extends the index factory to honor the `unique` flag. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent fd20510 commit 424e6fe

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

asyncPersistence.js

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,9 @@ class AsyncMongoPersistence {
165165
if (typeof idx.expireAfterSeconds === 'number') {
166166
indexOpts.expireAfterSeconds = idx.expireAfterSeconds
167167
}
168+
if (idx.unique) {
169+
indexOpts.unique = true
170+
}
168171
await this.#cl[idx.collection].createIndex(idx.key, indexOpts)
169172
}
170173

@@ -188,6 +191,19 @@ class AsyncMongoPersistence {
188191
collection: 'incoming',
189192
key: { clientId: 1, 'packet.messageId': 1 },
190193
name: 'query_clientId_messageId'
194+
},
195+
{
196+
// storeRetained / createRetainedStreamCombi key off `topic`. Without
197+
// this index every operation falls back to COLLSCAN; under load
198+
// storeRetained's ordered bulkWrite serializes the whole publish
199+
// pipeline behind it, stalling broker.publish callbacks by tens of
200+
// seconds on deployments with more than a few thousand retained
201+
// topics (observed: ~60s uniform lag with 15k retained + 200 msg/s).
202+
// Unique matches the {topic}-filtered upsert semantics.
203+
collection: 'retained',
204+
key: { topic: 1 },
205+
name: 'query_topic',
206+
unique: true
191207
}
192208
]
193209

0 commit comments

Comments
 (0)