fix(storage): translate the LMDB compression descriptor for RocksDB stores

[kriszyp]

What broke

rocksdb-js 2.6.0 gave the compression store option a real meaning — a RocksDB block/blob algorithm — and validates it. Harper has been passing an unrelated, LMDB-era compression value through the same options object: getDefaultCompression() returns msgpackr's { startingOffset, threshold, dictionary } descriptor, or the raw config value when falsy. 2.5.0 never read options.compression at all, so it was silently ignored. Now it's parsed.

On 2.6.1 an unset storage.compression resolves to '' and throws:

Error: Unsupported compression algorithm "". This build supports: none, snappy, zlib, bzip2, lz4, lz4hc, zstd
    at normalizeCompression (node_modules/@harperfast/rocksdb-js/src/store.ts:166:9)
    at openRocksDatabase (resources/databases.ts:214:48)

Every RocksDB store open throws, which takes component loading and the status table down with it — visible as the three componentLoader.test.js failures on #2036.

Why this is a main-branch bug, not a bump-branch bug

main pinned ^2.5.0, which already satisfies 2.6.1 (published 2026-07-31). CI is green only because npm ci uses the lockfile.

The published shrinkwrap does protect registry installs — npm-shrinkwrap.json ships in the tarball and npm honors it via _hasShrinkwrap. But the Docker image does not go through that path: Dockerfile:55 runs npm install --ignore-scripts --global harper-*.tgz, and a tarball install has no packument, so npm never learns the shrinkwrap exists and resolves fresh from package.json. That gap is already documented in dependencies.md. So a release cut from main today would ship an image resolving ^2.5.0 → 2.6.1 → broken on every table open.

~2.6.1 closes the range as well as fixing the incompatibility.

The fix

Translate at openRocksDatabase, the single point every RocksDB store opens — so re-opens of existing tables, whose persisted descriptor is LMDB-shaped, are covered along with new ones.

input	resolves to	effect
'' (unset config), true, LMDB descriptor	unset	rocksdb-js default (lz4)
false	'none'	no compression
'zstd', { algorithm, level }	pass through	as given

Compression-on leaves the option unset rather than naming lz4, because an explicit lz4 would throw on a build without it.

Behavior change worth a release note

The 2.6 binaries link lz4/zstd/bzip2 for the first time. Verified against the shipped natives:

symbol	2.5.0	2.6.1
LZ4_compress	ABSENT	PRESENT
ZSTD_compress	ABSENT	PRESENT
BZ2_bzCompress	ABSENT	PRESENT

2.5.0 also exported no supportedCompression. So RocksDB stores were uncompressed until now. That means storage.compression: false preserves today's behavior, and the notable change is the default turning compression on (lz4) — a disk win, but it changes on-disk format and CPU profile for new/compacted files. Existing SST/blob data stays readable.

Testing

• test:unit:main — 4221 passing
• test:unit:resources — 1354 passing
• test:integration:all — clean (exit 0)
• Direct resolveRocksCompression cases for '', false, true, the LMDB descriptor, {algorithm, level}, and a never-throws sweep

The table-level test alone was not sufficient and this is worth knowing when reading the diff: 2.6.1 accepts an object with no algorithm as "unset", so the default config (which yields the descriptor) exercises the one input that never failed — that test passes with or without the fix. The inputs that actually throw are '' and the booleans, and only a direct test reaches them. Caught by the cross-model review; the resolver is exported for that reason.

Open items for the reviewer

1. storage.compression: zstd is silently downgraded to lz4. getDefaultCompression() returns LMDB_COMPRESSION && LMDB_COMPRESSION_OPTS, so any truthy value — including an algorithm name — is discarded in favor of the LMDB options object. The string pass-through branch here is therefore unreachable from config today. Wiring it through is a new user-facing capability (needs docs + a release note), so I left it out of a fix PR. Worth a follow-up.
2. harper-pro still pins ^2.5.0 (both the root and its core submodule). Both resolve to 2.6.1 today so it's dormant, but once 2.7.0 ships npm would hoist 2.7.x for harper-pro and nest 2.6.x under core — two copies of a binding whose registry is process-global. Wants a coordinated bump.

Provenance

Root-caused and written by Claude Opus 5. Reviewed pre-push by Codex, Gemini, Grok, and the Harper domain pass; the test-effectiveness finding above came from that review and is fixed here. One reviewer finding was dropped as disproven — a claimed crash from injecting threshold into a RocksDB descriptor: normalizeCompression reads only algorithm/level and ignores extra keys (verified by opening a DB with {algorithm:'lz4', level:3, threshold:4036}), and the branch is unreachable because primaryAttribute.compression only ever originates from getDefaultCompression's LMDB shape.

Unblocks #2036.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@​harperfast/​rocksdb-js@​2.5.0 ⏵ 2.6.1				+1

View full report

[gemini-code-assist]

Code Review

This pull request updates the @harperfast/rocksdb-js dependency to ~2.6.1 and introduces the resolveRocksCompression helper to normalize and translate compression configurations (such as LMDB-shaped descriptors, booleans, and empty strings) before they reach the RocksDB store. It also fixes a potential strict-mode error in initStores when assigning properties to string primitives. The feedback suggests simplifying string trimming in resolveRocksCompression, removing redundant as any type casts now that the options are fully typed, and refining the object checks in initStores to be more defensive against RocksDB descriptors.

[gemini-code-assist]

We can simplify the string handling by trimming the string once. This not only combines the empty-string check and the pass-through check, but also robustly handles any leading/trailing whitespace in valid algorithm names (e.g., ' lz4 ' becomes 'lz4').

Suggested change

	if (typeof compression === 'string' && compression.trim() === '') return undefined;
	// already RocksDB-shaped ('lz4', { algorithm: 'zstd', level: 3 }, ...): pass through
	if (typeof compression === 'string') return compression;
	if (typeof compression === 'string') {
	const trimmed = compression.trim();
	return trimmed === '' ? undefined : trimmed;
	}

[gemini-code-assist]

Since @harperfast/rocksdb-js has been upgraded to ~2.6.1, the compression option is now officially supported and typed in RocksDatabaseOptions. We can remove the redundant as any type casts here to improve type safety.

Suggested change

	(options as any).compression = resolveRocksCompression((options as any).compression);
	options.compression = resolveRocksCompression(options.compression);

References

1. Avoid redundant type casting (such as casting 'this' to 'any') when accessing static properties that are already declared on the class, as they are fully typed within static methods. Only use type casting when accessing properties that are not declared on the class.

[gemini-code-assist]

While dbiInit.compression is currently expected to be an LMDB descriptor, once RocksDB compression descriptors (e.g., { algorithm: 'zstd', level: 3 }) are fully wired through, they will also be objects. To prevent accidentally injecting the LMDB-specific threshold property into a RocksDB descriptor object, we should explicitly guard this check by ensuring the object does not contain an algorithm property. Additionally, use an explicit != null check instead of a simple truthiness check to prevent falsiness bugs.

Suggested change

	if (dbiInit.compression && typeof dbiInit.compression === 'object') {
	if (dbiInit.compression != null && typeof dbiInit.compression === 'object' && !('algorithm' in dbiInit.compression)) {

References

1. Use explicit != null checks instead of simple truthiness checks to prevent falsiness bugs (e.g., when a numeric value can be 0) and to make the code clearer and more defensive for direct callers.

[claude]

Reviewed; no blockers found.