Skip to content

cherry-pick: Make RocksDB memory config configurable (#780) → v5.0 #1133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kriszyp merged 3 commits into from
Jun 5, 2026
Merged

cherry-pick: Make RocksDB memory config configurable (#780) → v5.0 #1133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f769c8d
Expose RocksDB WriteBufferManager config params
kriszyp May 26, 2026
ea65fc1
Coerce envGet values to native types before passing to RocksDB
kriszyp May 26, 2026
0f75049
Enforce types from envGet rather than coerce
kriszyp May 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions resources/databases.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,40 @@ export const databaseEventsEmitter = new EventEmitter<DatabaseWatcherEventMap>()
export const tables: Tables = Object.create(null);
export const databases: Databases = Object.create(null);

const MEMORY_FOR_ROCKS_DB = Math.min(process.constrainedMemory?.() ?? Infinity, totalmem()) * 0.25; // 25% of available memory

function openRocksDatabase(path: string, options: RocksDatabaseOptions & { dupSort?: boolean }) {
options.disableWAL ??= true;
RocksDatabase.config({ blockCacheSize: MEMORY_FOR_ROCKS_DB });
// Read RocksDB memory config lazily so env/CLI overrides applied after module load are
// respected. The block cache falls back to 25% of constrained (cgroup) memory when not
// configured; the WriteBufferManager is opt-in (0 disables).
//
// We enforce types rather than coerce — values from YAML config and env vars flow
// through configUtils.castConfigValue which produces proper numbers/booleans/null,
// so anything else is misconfiguration and should fall through to the default.
//
// Note: writeBufferManagerCostToCache and writeBufferManagerAllowStall are fixed at WBM
// creation time inside rocksdb-js (the underlying RocksDB API doesn't support changing
// costToCache on a live manager, and allowStall is only re-applied when explicitly changed).
// In practice that's fine — these come from process-level config that doesn't change.
const configuredBlockCacheSize = envGet(CONFIG_PARAMS.STORAGE_ROCKS_BLOCKCACHESIZE);
const blockCacheSize =
typeof configuredBlockCacheSize === 'number' && configuredBlockCacheSize > 0
? configuredBlockCacheSize
: Math.min(process.constrainedMemory?.() ?? Infinity, totalmem()) * 0.25;
const writeBufferManagerSize = envGet(CONFIG_PARAMS.STORAGE_ROCKS_WRITEBUFFERMANAGERSIZE);
const writeBufferManagerCostToCache = envGet(CONFIG_PARAMS.STORAGE_ROCKS_WRITEBUFFERMANAGERCOSTTOCACHE);
const writeBufferManagerAllowStall = envGet(CONFIG_PARAMS.STORAGE_ROCKS_WRITEBUFFERMANAGERALLOWSTALL);
RocksDatabase.config({
blockCacheSize,
...(typeof writeBufferManagerSize === 'number' && writeBufferManagerSize > 0
? { writeBufferManagerSize }
: {}),
...(typeof writeBufferManagerCostToCache === 'boolean'
? { writeBufferManagerCostToCache }
: {}),
...(typeof writeBufferManagerAllowStall === 'boolean'
? { writeBufferManagerAllowStall }
: {}),
});
Comment on lines +127 to +146

@gemini-code-assist gemini-code-assist Bot Jun 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Calling RocksDatabase.config on every invocation of openRocksDatabase is highly inefficient and potentially dangerous. Since openRocksDatabase is called for every table, index, and internal DBI, this repeatedly re-reads environment variables and re-applies the global RocksDB configuration. Depending on the underlying @harperfast/rocksdb-js implementation, repeatedly calling config could recreate the shared block cache and write buffer manager, leading to memory leaks, multiple independent caches, or invalidation of existing cached blocks.

Since these process-level configurations do not change after startup, we should ensure RocksDatabase.config is only called once. We can guard this initialization using a property on the openRocksDatabase function itself. Additionally, because this is a non-critical optimization executed during a critical startup path, we should wrap it in a try-catch block to ensure that any unexpected failure does not block the critical database opening process.

if (!(openRocksDatabase as any).configured) {
	(openRocksDatabase as any).configured = true;
	try {
		const configuredBlockCacheSize = envGet(CONFIG_PARAMS.STORAGE_ROCKS_BLOCKCACHESIZE);
		const blockCacheSize =
			typeof configuredBlockCacheSize === 'number' && configuredBlockCacheSize > 0
				? configuredBlockCacheSize
				: Math.min(process.constrainedMemory?.() ?? Infinity, totalmem()) * 0.25;
		const writeBufferManagerSize = envGet(CONFIG_PARAMS.STORAGE_ROCKS_WRITEBUFFERMANAGERSIZE);
		const writeBufferManagerCostToCache = envGet(CONFIG_PARAMS.STORAGE_ROCKS_WRITEBUFFERMANAGERCOSTTOCACHE);
		const writeBufferManagerAllowStall = envGet(CONFIG_PARAMS.STORAGE_ROCKS_WRITEBUFFERMANAGERALLOWSTALL);
		RocksDatabase.config({
			blockCacheSize,
			...(typeof writeBufferManagerSize === 'number' && writeBufferManagerSize > 0
				? { writeBufferManagerSize }
				: {}),
			...(typeof writeBufferManagerCostToCache === 'boolean'
				? { writeBufferManagerCostToCache }
				: {}),
			...(typeof writeBufferManagerAllowStall === 'boolean'
				? { writeBufferManagerAllowStall }
				: {}),
		});
	} catch (error) {
		logger.error('Failed to apply RocksDB configuration', error);
	}
}
References
  1. Ensure that non-critical cleanup or optimization operations executed during critical startup or recovery paths are wrapped in try-catch blocks so that their failure does not block the critical process.

if (!existsSync(path)) {
mkdirSync(path, { recursive: true });
}
Expand Down
4 changes: 4 additions & 0 deletions utility/hdbTerms.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -577,6 +577,10 @@ export const CONFIG_PARAMS = {
STORAGE_RECLAMATION_INTERVAL: 'storage_reclamation_interval',
STORAGE_RECLAMATION_EVICTIONFACTOR: 'storage_reclamation_evictionFactor',
STORAGE_ENGINE: 'storage_engine',
STORAGE_ROCKS_BLOCKCACHESIZE: 'storage_rocks_blockCacheSize',
STORAGE_ROCKS_WRITEBUFFERMANAGERSIZE: 'storage_rocks_writeBufferManagerSize',
STORAGE_ROCKS_WRITEBUFFERMANAGERCOSTTOCACHE: 'storage_rocks_writeBufferManagerCostToCache',
STORAGE_ROCKS_WRITEBUFFERMANAGERALLOWSTALL: 'storage_rocks_writeBufferManagerAllowStall',
DATABASES: 'databases',
IGNORE_SCRIPTS: 'ignoreScripts',
MQTT_NETWORK_PORT: 'mqtt_network_port',
Expand Down
Loading