mmkv-go — a cgo-free, read-only MMKV reader in pure Go (zero-alloc, ~10–75× faster reads) #1670
catundercar
started this conversation in
Show and tell
Replies: 1 comment
-
|
That benchmark looks impressive. Good for you. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
mmkv-go is a small, cgo-free, read-only decoder for MMKV files, written in pure Go. It parses MMKV's on-disk format directly, so the read path never crosses the cgo boundary.
Why
In a Go service with one writer + many reader processes, every read through the official Go binding pays the cgo boundary cost (~65 ns/call here) plus a C→Go copy. For read-heavy, multi-process workloads that adds up. mmkv-go keeps writes on the official library and accelerates only the reads — entirely in Go, with no C toolchain needed on the reader side.
What it does
.crcand, on each read, compares the writer's change canary (crcDigest / actualSize / sequence), reloading under a sharedflockonly when something changed. Safe for single-writer (cgo,MMKV_MULTI_PROCESS) + multi-reader (pure Go).GetBytes/GetStringreturn views into the parsed buffer (0 alloc);*Copyvariants when you need an independent slice/string.BackupOne.Performance
Reading the same file in one process (Apple Silicon, arm64):
int32bytes4 KB (copy)bytes4 KB (zero-copy buffer)stringThe file is parsed once into a map at
Open; each read is then a lock-free snapshot lookup. The gap is the cgo boundary tax + the Go-heap copy — both gone. (Relative ratios are the point; absolutes vary by machine.)How it stays correct
It's a clean-room reader of MMKV's on-disk format, so the format is the contract — and CI guards it hard:
cgo.Get(k) == purego.Get(k)differential runs on files written by the official library, across MMKV's major release lines (v1.2.16 … v2.4.0) × {amd64, arm64} on native runners. Any format change that breaks the pure-Go reader turns the build red.Scope & honesty
*Copyvariants otherwise.Huge thanks to the MMKV team — it's a pleasure of a format to read. Repo: mmkv-go.
Feedback very welcome, especially on any on-disk format edge cases I may have missed.
Beta Was this translation helpful? Give feedback.
All reactions