Skip to content

Commit c93a726

Browse files
kriszypclaude
andcommitted
fix: cap the typed-structure dictionary in RecordEncoder
The per-encoder typedStructs dictionary (structon's random-access typed structs) is append-only and pinned on the long-lived primary store. A wide/sparse schema whose records vary by per-field value width mints a distinct structure per shape×width combination and grows the dictionary unbounded, exhausting memory (observed on a bulk-loaded table under replication catch-up). Default maxOwnStructures to 256 (caller-overridable), bounding each dictionary. Bumps structon to ^1.0.6, which adds the cap (uncapped by default). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent f7a6264 commit c93a726

3 files changed

Lines changed: 9 additions & 5 deletions

File tree

package-lock.json

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,7 +232,7 @@
232232
"ses": "^1.15.0",
233233
"stream-chain": "2.2.5",
234234
"stream-json": "1.9.1",
235-
"structon": "^1.0.5",
235+
"structon": "^1.0.6",
236236
"systeminformation": "^5.31.4",
237237
"tar-fs": "^3.1.2",
238238
"ulidx": "0.5.0",

resources/RecordEncoder.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,10 @@ export class RecordEncoder extends StructonEncoder {
9696
name: string;
9797
constructor(options) {
9898
options.useBigIntExtension = true;
99+
// Bound the per-encoder typed-structure dictionary. It is append-only and pinned on the
100+
// long-lived primary store, so a wide/sparse schema (whose records vary by per-field value
101+
// width) can grow it unbounded and exhaust memory. Caller-overridable; default caps it.
102+
options.maxOwnStructures ??= 256;
99103
/**
100104
* The base class for records that provides the read-only methods for accessing
101105
* metadata and will be assigned computed property getters. On its own, these instances

0 commit comments

Comments
 (0)