Conversation
Represent hardlinks as additional dirents pointing at the same file inode. Add a storage-level link helper and expose it through the SQLiteWorkspaceProvider sync and async APIs. Unlink now removes only the requested dirent and reaps file chunks and the inode only after the final link disappears. Rename displacement uses the same final-link rule, and renaming one hardlink onto another removes only the source name. Provider stats now report the actual link count for inode-backed entries. Tests cover shared inode identity, nlink reporting, writes through one name being visible through the other, unlink preservation, hardlink rename behavior, and common error paths.
Implement FUSE link(2) by forwarding to the DOFS provider hardlink primitive. Dirty or pending source buffers are flushed before linking so the provider has a real inode to reference, and the destination shares the same in-memory file entry when the source was already tracked by the driver. Expose provider nlink through getattr and stop treating link as an unimplemented FUSE operation. Tests cover same-inode linking, writes through a linked path, pending-create sources, and POSIX error translation for missing sources and existing destinations.
Add an integration-test case for writing through one hardlink and reading the bytes through the other name. This pins the behavior npm relies on when package-manager installs link cached files into a workspace and later consumers access the original path.
Add inline_data to vfs_nodes and use it for synchronous writes up to 16 KiB. The synchronous provider path is the FUSE hot path, so tiny files no longer need chunk rows, blob rows, or manifest rows before they can be read back locally. Keep the async writeFile path chunk-backed for now so the existing sync wire format continues to carry small files without a protocol change. Read, stat, provider readFileSync, and fd splice helpers now understand both inline and chunk-backed files. The migration is idempotent for partially-staged old schemas that get the latest baseline table before migrations run.
Track open regular-file handles by path and drop clean FileEntry buffers after the final release. Once flush has persisted data into the backing VFS, keeping the per-file Buffer around only duplicates memory already owned by the store and can mask later VFS-side writes. The eviction also handles hardlink aliases that share the same FileEntry object: clean aliases with no open handles are removed alongside the released path. A regression test writes a file through FUSE, flushes and releases it, mutates the backing VFS directly, and then verifies the next FUSE read hydrates the fresh VFS bytes instead of serving the stale clean buffer.
Expose a driver-level getBufferStats snapshot for the FUSE write buffer cache. The stats include resident entry count, dirty and pending-create counts, logical and capacity bytes, dirty logical bytes, and open handle/path counts. The snapshot is intentionally stripped before handing the operation object to fuse-native, so it is available to tests and future daemon diagnostics without registering a non-FUSE callback with libfuse. Tests cover stats rising while a write buffer is resident and dropping back to zero after the clean-release eviction path.
Add local direct-write helpers for create, byte-range writes, and truncate. The helpers update inline data or affected chunk rows by inode, so hardlinks share the mutation and unchanged chunks can keep their existing hashes. DOFS can now update its source of truth incrementally without requiring a FUSE-owned whole-file buffer. Tests cover inline writes, sparse zero-fill, partial chunk updates, hardlink write-through, and chunk-backed truncate.
Route provider writeSync and truncateSync through the direct range and truncate primitives instead of materializing and rewriting the whole file. This keeps positional writes on the direct-write path for local provider callers and preserves untouched chunk hashes for large files. The provider fd tests now assert that a write into the middle chunk of a chunk-backed file reuses the surrounding chunk rows.
Expose createFileSync, writeRangeSync, and truncateFileSync on the SQLite workspace provider so local callers can use the direct incremental write path without going through file descriptors. The provider tests cover direct create, range overwrite, readback, and truncate through the new surface.
Expose the DOFS direct-write methods through the wsd VFS wrapper and use them from the FUSE driver when available. Direct-mode create, write, truncate, and chmod update the backing provider immediately, so reads through the VFS see FUSE-written bytes before release and normal writes no longer allocate FileEntry buffers. Keep the previous staged-buffer path as a fallback for providers that do not expose the direct-write methods. Fallback-specific tests disable direct writes to keep covering flush, ranged spill, and buffer stats behavior.
In direct-write mode, serve FUSE reads from the backing VFS without hydrating a FileEntry. This keeps a read-before-write sequence on the direct DOFS path instead of accidentally switching later writes to the buffered fallback. The regression test writes, reads, writes again, and asserts that the backing VFS sees the second write immediately while FUSE buffer stats remain empty.
Teach change materialization to represent inline_data as a wire chunk and stage inline bytes in the blob store when direct writes land small files inline. Without this, direct-written inline files appeared on the sync wire as zero-byte files because they had no vfs_chunks rows. The fetch tests now cover a direct inline write through fetchChanges and fetchObjects so the change entry size, chunk hash, and object bytes stay consistent.
Add a positional read helper that slices vfs_nodes.inline_data for inline files and walks only the vfs_chunks rows overlapping the requested byte range for chunk-backed files. Direct-mode FUSE reads can now serve a kernel read without materializing the whole file on every syscall. Tests cover non-zero inline offsets, inline clamp past EOF, single chunk windows, reads crossing a chunk boundary, and past-EOF reads on chunk-backed files.
Drop the SQLiteVirtualProvider wrapper class and its FORWARDED_METHODS dispatch table. Splice VirtualProvider onto SQLiteWorkspaceProvider's prototype chain at the wsd boundary instead, so @platformatic/vfs's instanceof guard accepts the dofs provider directly. The seam used to keep two parallel surfaces in sync by hand: a forwarded-method list on the wrapper class and a separate post-create Object.defineProperty block for methods @platformatic/vfs does not expose on VirtualFileSystem. Adding readRangeSync to one path but not the other caused FUSE reads to throw EIO at runtime. After the splice, @platformatic/vfs's create() returns a VirtualFileSystem whose VirtualProvider methods reach the dofs class directly. Only the dofs-specific extensions (linkSync, createFileSync, writeRangeSync, truncateFileSync, chmodSync, readRangeSync) need a post-create attachment, and they bind straight to the provider with no indirection. Wire readRangeSync through the provider's readSync and through the FUSE driver's direct-mode read so a kernel read no longer materializes the whole file on every syscall. The provider's old whole-file readFileBytesSync helper is gone with its last caller.
Replace the chunk-backed writeRangeSync and truncateFileSync update loop so it only touches the vfs_chunks rows whose contents or size actually changed. Untouched chunk rows keep their rowids and the manifest hash is invalidated rather than recomputed, so a tiny edit into the middle of a large file stops scaling with the total chunk count. Tests cover stable rowids across a small range write and manifest invalidation after a direct range write.
Add a JSON stats endpoint that reports DOFS table row counts, total inline and blob byte sizes, the orphan-blob subset, and process memory. Useful for watching how the store grows under load without attaching a debugger. Used to confirm that direct-mode FUSE writes accumulate per-write intermediate blobs that no chunk row references: an npm install of the sandbox-sdk repo reaches 4.7 GB of blob bytes of which 4.5 GB are orphaned, dominating the wsd resident set.
Add an in-memory write buffer keyed by inode that the FUSE driver opens on create/open and releases on the matching release. While a buffer is open, writeRangeSync, truncateFileSync, and readRangeSync operate on the buffer rather than committing to vfs_chunks per syscall. Release commits the buffered bytes once and drops the entry. Bytes have one owner (DOFS), the commit boundary is per file, and intermediate write states no longer accumulate orphan blob rows. Drop the vfs_nodes.inline_data column. The buffered model removes the per-write cost that motivated inline storage in the first place, and inline contributed two storage shapes that read, write, stat, and sync all had to special-case. Without it, the orphan-vs-live blob accounting and the sync wire stay one path. Schema version goes back to 2; the v2 -> v3 migration is gone. Nothing has shipped that depends on inline_data so no data migration is needed. Tests cover buffered multi-write convergence to a single blob, buffered reads before release, truncate through the buffer, and hardlink sharing by inode.
Open a DOFS write buffer on FUSE create and open, release it on the matching FUSE release. While the buffer is open, FUSE writes, truncates, and reads operate on the buffer; release commits the final bytes to vfs_chunks in one shot. Also teach the synchronous read paths (provider.readFileSync, fs/readFile streaming, stat, provider.fileSize) to consult the buffer when one is open. Without that, an RPC or test reading through the VFS while a FUSE file is still being written would see stale chunk-store bytes.
Every FS mutation pays the rev counter; tiny-file create also reads back last_insert_rowid. SQLite's RETURNING lets us fold both reads into the same statement, cutting one round-trip off each. incrementRev now does a single UPDATE ... RETURNING v instead of an UPDATE followed by a SELECT. The new file/dir/symlink inode inserts do INSERT ... RETURNING inode, so the bare last_insert_rowid lookup goes away. createFileSync also reorders so rev is computed up front and the node row lands with its final stamp in one INSERT, removing the post-insert UPDATE entirely. Cuts createFileSync from 7 SQL statements down to 5 per file (the existing-file overwrite branch already had the optimal shape).
Add vfs_nodes.size to denormalise the chunk-sum file size. stat and the provider's fileSize helper now read it directly from the node row that resolveInode already loaded, instead of running a separate COALESCE(SUM(size), 0) FROM vfs_chunks aggregate on every call. Every write path that lands chunks (writeFile streaming, writeFileSync, writeFileRangesSync, applyChunkedInodeUpdate, releaseWriteBufferSync, truncateFileSync) stamps the new size in the same UPDATE that bumps rev. resolveInode carries the cached value through to its callers so stat, lstat, and readRangeSync don't have to re-query. Schema bumps to v3 with a v2 -> v3 migration that backfills the column from the existing chunk rows. Open buffers still override the cached size for in-flight writes.
createFileSync used to commit an empty inode immediately, then\nopenWriteBufferSync attached a buffer that the eventual release\nturned into chunk rows. Two transactions per tiny file.\n\nAdd openWriteBufferForCreateSync that stashes a pending-create\nentry in the write-buffer cache without touching SQL. Release\ncommits the INSERT, dirent, and chunk rows in a single transaction.\nFor an open-write-close cycle on a fresh file this collapses two\ntransactions into one.\n\nThe pending entry is keyed by path (no inode exists yet) and\nbridged into the rest of the FS layer:\n\n- writeRangeSync, truncateFileSync, readRangeSync, readFile, and\n stat consult the path-keyed pending cache before falling back to\n resolveInode.\n- provider.existsSync, lstatSync, readFileSync, chmodSync see\n pending entries.\n- readdir merges pending leaves into the directory listing so an\n open-before-release readdir still surfaces the new file.\n- link, rename, and unlink call flushPendingByPath on each\n candidate path so the dirent operation always sees a real inode.\n The buffer's open handles continue addressing bytes through the\n inode-keyed cache after commit.\n\nThe FUSE driver routes FUSE create through the deferred path when\nthe provider advertises openWriteBufferForCreateSync. Existing\ntests for direct mode pass unchanged; the deferred path is the new\nfast lane.
Default max_read and max_write were 128 KiB. Sequential reads of a\nchunk-backed file used to issue four FUSE reads per 512 KiB chunk\nand four SQL fetches of the same blob bytes. Raising both to 524288\nmatches CHUNK_SIZE so a single FUSE read maps to a single chunk\nfetch.\n\nThe historical 128 KiB sizing predates readRangeSync, when reads\nstill materialised the whole file per syscall and the option didn't\nshow up in the numbers. With the per-range read path in place the\nFUSE syscall count for a sequential 64 MiB read drops 4x.
Reads against chunk-backed files used to fetch the same blob bytes\nrepeatedly: FUSE issues smaller reads than CHUNK_SIZE, and dedup'd\nblobs (e.g. a file of zeroes) collapse to a single row that every\nchunk's hash points at. The previous deployed bench measured pure\nread 64 MiB at 76x of tmpfs largely because of these repeats.\n\nAdd a small per-Database LRU cache keyed by hash. vfs_blob_bytes is\ncontent-addressed and immutable, so a cached payload stays valid\nfor the life of the database; new writes produce new hashes rather\nthan overwriting an existing entry.\n\nWire the cache into the hot read sites: readFile (streaming and\nstring), readRangeSync, provider.readFileSync, and the\nreadChunkBytes helper that powers write-path read-modify-write of\npartial chunks. Sync apply/push paths are left alone because they\nare not in the read hot loop.
Capture the latest fs-bench and full sandbox-sdk npm install numbers\nfrom the wsd-container example on a standard-2 Cloudflare Container\n(1 vCPU, 6 GiB memory, 12 GB disk). Compare wsd against an in-memory\ntmpfs and against the container's ext4 disk so readers see the\nrealistic baseline for general usage, not just the tmpfs ratio. Call\nout which scenarios beat real disk (metadata-heavy work) and which\nstill lag (large sequential I/O, where chunk hashing dominates).
Drop the stale FUSE buffer flushing section from packages/wsd/README.md\nthat described the per-file FileEntry staging buffer with release/flush/\nfsync spills. Replace it with the current model: the FUSE driver opens\na DOFS write buffer on create/open, mutates it through writes and\ntruncates, and commits chunk rows in one transaction at release. Add\nthe /__wsd/stats endpoint to the endpoint list with a short note on\nwhen to reach for it.\n\nUpdate packages/dofs/README.md to enumerate the buffered-write\nsurface (openWriteBufferForCreateSync, openWriteBufferSync,\nreleaseWriteBufferSync, writeRangeSync, truncateFileSync,\ncreateFileSync, readRangeSync) and the content-addressed blob cache,\nand drop a stale pointer to a no-longer-relevant document.\n\nAdd the cached vfs_nodes.size column to docs/03_filesystem_schema.md\nso the DDL block matches the shipped schema, and explain why it is\ndenormalised onto the node row.
Cover the gaps that the recent storage and lifecycle changes left\nuncovered:\n\n- The v2 -> v3 schema migration backfills vfs_nodes.size from chunk\n row sums for each live file, leaves directories and empty files at\n zero, and reports the bumped schema_version.\n- The synchronous provider's stat reads the size column directly\n rather than re-summing vfs_chunks; the column is stamped on every\n writeFileSync.\n- openWriteBufferForCreateSync holds a new file in memory until\n release commits one transaction. The path-keyed pending cache is\n visible through stat, readRangeSync, and readdir but resolveInode\n returns null until release.\n- A second openWriteBufferForCreateSync against the same path throws\n EEXIST.\n- Provider linkSync, renameSync, and unlinkSync each commit a\n pending-create source first so the dirent operation always sees a\n real inode.
Three dirent-mutating provider paths only half-bridged the\nwrite-buffer cache, allowing real corruption modes.\n\n- linkSync flushed a pending-create source but ignored a\n pending-create destination. link's dirent check sees no row at\n newPath, the link succeeds, and the release on the displaced\n pending buffer re-checks the dirent in commitPendingBuffer,\n throws EEXIST, drops the entry, and silently loses the user's\n bytes. Flush newPath too so EEXIST surfaces immediately and the\n pending bytes land as a real file.\n- renameSync's overwrite branch deleted the displaced inode's\n chunks and node row but left any open write buffer pointing at\n the now-dead inode. The eventual release committed chunks against\n a missing row (0-row UPDATE) and the user lost the bytes\n silently. Drop the buffer for the displaced inode when its last\n link disappears.\n- unlinkSync flushed pending state then ran rm but left the\n inode-keyed buffer cache entry dangling on a freshly-deleted\n inode. Snapshot the target inode before rm, and if rm removed\n the last link drop the buffer too. Hardlinks that keep the inode\n alive keep the buffer alive.\n\nTests pin all three contracts: link into a pending destination,\nrename overwrite of an open destination, unlink of the last link,\nand unlink of a hardlinked file.
The schema invariants block in docs/03_filesystem_schema.md still\ndescribed the pre-buffered-write contract: 'file row has either a\nlazy stub or manifest_hash NOT NULL plus chunks'. The buffered and\ndirect-write paths now commit chunks with manifest_hash = NULL and\nrely on sync walking vfs_chunks directly when no manifest is\npresent. Restate the invariant in two shapes (lazy stub vs\ncommitted file) and note that the manifest is opportunistic. Add a\nsize-column invariant covering the denormalisation that stat,\nlstat, and readRangeSync rely on.\n\nList chmodSync in the dofs README's enumeration of the\nbuffered-write provider surface so the public-surface bullet is\ncomplete.\n\nMark flushPendingByPath @internal: it exists only so provider\nlink/rename/unlink can bridge a pending-create buffer into the SQL\nworld ahead of a dirent operation, never as a consumer entry point.\n\nDrop 'inline' from a handful of test descriptions and source\ncomments that survived the inline_data column removal.
…sd/stats Round one of review surfaced a set of contracts that weren't yet\npinned by tests. None pointed at live bugs, but the surfaces are\nthe ones a future change is most likely to break silently.\n\nblobCache: two tests around the 16-entry LRU. The first inserts 17\ndistinct hashes and confirms the least-recently-used entry is\nevicted while the most recent stays cached. The second touches the\nLRU entry before adding the 17th hash, then asserts the touched\nentry survives and the new-LRU is evicted instead. Together they\npin both the bound and the recency update.\n\nwriteBuffer: a multi-open test that opens the same path twice,\nwrites dirty bytes, releases once (must not commit), and releases\nagain (must commit). It then re-opens with no writes and confirms\nthe content is unchanged. A second pair covers flushPendingByPath\ndirectly: promoting a still-open pending-create commits chunks but\nleaves the open count intact, so a later release commits the\nfinal bytes through the inode-keyed cache. A third returns false\nfor a path with no pending entry.\n\nprovider: the cached-size invariant gets five entry points instead\nof one - writeFileSync (first write and overwrite), writeRangeSync\nwith a sparse extend, truncateFileSync grow and shrink, buffered\nrelease, and async writeFile. Each one reads vfs_nodes.size\ndirectly to catch a path that updates content but forgets to\nstamp the column.\n\nwsd: a smoke test for the /__wsd/stats endpoint. It only asserts\nthe shape of the JSON body (table counts, orphan totals, process\nmemory) because the values themselves depend on the underlying\nstorage backend; the contract is that every field is present and\nnumeric.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
npm installof a real-world repo (cloudflare/sandbox-sdk) was painfully slow over awsdmount, and on a big enough install the daemon ran itself out of memory at around 5.5 GB. Most of that memory turned out to be garbage: every small write produced a fresh chunk in SQLite that nothing pointed at, and we never collected them.The root cause was where the bytes lived. The FUSE driver kept its own per-file buffer in user space, and every flush rebuilt the file in
dofsfrom scratch. Editing one byte of a 64 MiB file paid for re-chunking and re-hashing all of it.The fix is to move the buffer into
dofsitself, one per open file. The FUSE driver opens it oncreate/open, writes go into it, and on close we commit the whole thing in a single transaction. One file, one set of chunks, no garbage. The same install now finishes without the daemon ballooning, and the focused benchmarks dropped proportionally (write 64 MiB went from 438 ms to 230 ms on the deployed container (standard-2), and pure reads stopped multiplying their SQL fetches).A few other small things helped: deferring the
INSERTfor new files until close, caching file size on the row, an in-memory cache for blob bytes, and aligning the FUSE block size to our chunk size. Hardlinks now work too, whichnpm's package cache relies on.A few things we tried and dropped along the way. Direct per-syscall writes (skipping the buffer entirely) were what produced the orphan blobs in the first place. A general stat cache with a short TTL regressed
findandgit status. Inline storage for tiny files, keeping bytes in thevfs_nodesrow instead of a chunk, was dropped in favor of the buffer model.writeback_cache,direct_io, andhard_removemount options either didn't help or actively broke things.The README now has a Performance section with the numbers from a deployed Cloudflare Container so you can compare against
tmpfsand the container's real disk.Testing is mostly unit tests in the two packages that changed.
@cloudflare/dofscovers every new primitive, the schema migration, the LRU cache, the multi-open buffer semantics, and thelink/rename/unlinkpaths against the in-flight buffer.@cloudflare/workspace-wsdcovers the FUSE driver against the new buffer surface. There's also a hardlink case in the integration script. Round numbers: 331 dofs tests, 126 wsd tests.