Skip to content

perf(db): pre-size memtable slice in getMemTables#2289

Open
shaunpatterson wants to merge 1 commit into
dgraph-io:mainfrom
shaunpatterson:perf/db-memtables-presize
Open

perf(db): pre-size memtable slice in getMemTables#2289
shaunpatterson wants to merge 1 commit into
dgraph-io:mainfrom
shaunpatterson:perf/db-memtables-presize

Conversation

@shaunpatterson
Copy link
Copy Markdown

Summary

The memtable slice returned by getMemTables was previously grown via append-from-nil, triggering a growslice on every call (and additional growslices for each cap doubling boundary crossed). The final size is known up front: 1 mutable (unless ReadOnly) + len(db.imm) immutables. Pre-sizing collapses all growslice events into a single make().

Motivation

Profiled in BenchmarkRollupKeyIterator: the original append-from-nil allocation (~1.3 allocs/op from growslice) is eliminated; in its place is a single make() matching the final size.

In real dgraph workloads with 6 memtables (1 mutable + 5 immutable), the original code grew the slice through 4 cap boundaries (0→1→2→4→8), so this saves 4 allocs per iterator construction. Each rollup cache miss constructs one iterator.

Behavior

Preserved. The only observable difference is that an empty result returns a non-nil zero-length slice rather than nil. All three callers (db.get, txn.NewIterator, stream_writer) use len() or range, neither of which distinguishes the two.

Test plan

  • go test ./... passes
  • Existing memtable/iterator tests cover the path

🤖 Generated with Claude Code

The slice was previously grown via append-from-nil, triggering a
growslice on every call (and additional growslices for each cap
doubling boundary crossed). The final size is known up front: 1
mutable (unless ReadOnly) + len(db.imm) immutables. Pre-sizing
collapses all growslice events into a single make().

Profiled in BenchmarkRollupKeyIterator: line 730's append-from-nil
allocation (5.18M objects / ~4M iterations ≈ 1.3 allocs/op from
growslice) is eliminated; in its place is a single make()
allocation matching the final size.

In real dgraph workloads with 6 memtables (1 mutable + 5 immutable),
the original code grew the slice through 4 cap boundaries (0→1→2→4→8),
so this saves 4 allocs per iterator construction. Each rollup cache
miss constructs one iterator.

Behavior is preserved: the only observable difference is that an
empty result returns a non-nil zero-length slice rather than nil.
All three callers (db.get, txn.NewIterator, stream_writer) use
len() or range, neither of which distinguishes the two.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@shaunpatterson shaunpatterson requested a review from a team as a code owner May 26, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant