Use case
High-speed, low-cardinality counter updates — e.g. token-quota accounting where many concurrent requests increment a property on the same record. With optimistic transactions, every concurrent writer to a hot key conflicts and retries; even with coordinated retry (verification-table parking), throughput on a single hot record is fundamentally serialized read-modify-write.
Idea
RocksDB's native Merge operator is built for exactly this: writers append merge operands (deltas) instead of reading + rewriting the value, so concurrent increments are conflict-free by construction. The merge function applies the operand chain at read/compaction time.
For rocksdb-js this would mean:
- Exposing a
merge(key, operand) write on stores/transactions.
- A msgpackr-aware merge operator in native code that can apply field-level numeric deltas to an encoded record — or, more simply, a dedicated column family for counters with plain numeric encoding and a stock int64-add operator.
Costs / open questions (why this is exploratory)
- Read path must apply the merge-operand chain (get latency grows with unmerged operands until compaction).
- Harper's replication and audit log currently assume value-images produced at commit time; delta operands would need first-class support through the transaction log — although op-based increments would actually replicate commutatively across nodes, which is very attractive for multi-node quota semantics.
- Conflict semantics between merge operands and regular transactional writes to the same key need careful definition.
Relation to the shallow alternative
A JS-side intent-based increment coalescer (filed in the harper repo) gets most of the single-node win with a fraction of the effort and no storage-format implications. This issue tracks the deeper storage-engine option, gated on the shallow tier proving insufficient (e.g. multi-node commutative replication becoming a requirement).
Filed by KrAIs (Claude Opus 4.8) on behalf of @kriszyp, from the commit-batching feasibility research (2026-07-08).
Use case
High-speed, low-cardinality counter updates — e.g. token-quota accounting where many concurrent requests increment a property on the same record. With optimistic transactions, every concurrent writer to a hot key conflicts and retries; even with coordinated retry (verification-table parking), throughput on a single hot record is fundamentally serialized read-modify-write.
Idea
RocksDB's native Merge operator is built for exactly this: writers append merge operands (deltas) instead of reading + rewriting the value, so concurrent increments are conflict-free by construction. The merge function applies the operand chain at read/compaction time.
For rocksdb-js this would mean:
merge(key, operand)write on stores/transactions.Costs / open questions (why this is exploratory)
Relation to the shallow alternative
A JS-side intent-based increment coalescer (filed in the harper repo) gets most of the single-node win with a fraction of the effort and no storage-format implications. This issue tracks the deeper storage-engine option, gated on the shallow tier proving insufficient (e.g. multi-node commutative replication becoming a requirement).
Filed by KrAIs (Claude Opus 4.8) on behalf of @kriszyp, from the commit-batching feasibility research (2026-07-08).