Skip to content

perf(ext/web): add write buffering for FsFile.writable streams#32676

Open
bartlomieju wants to merge 1 commit intomainfrom
perf/fsfile-writable-buffering
Open

perf(ext/web): add write buffering for FsFile.writable streams#32676
bartlomieju wants to merge 1 commit intomainfrom
perf/fsfile-writable-buffering

Conversation

@bartlomieju
Copy link
Member

Summary

  • Add opt-in write buffering to writableStreamForRid() via a new bufferSize option
  • Enable 64KB buffering for FsFile.writable streams only — network sockets, stdin, QUIC, and WebTransport remain unbuffered
  • Small chunks are accumulated in a JS-side buffer and flushed when full or on close(); large chunks (>= 64KB) bypass the buffer

Motivation

Closes #16667

Writing small buffers (128 bytes) via file.writable.getWriter().write() is ~100x slower than Node.js. The root cause is that every small write goes through: JS promise machinery → async op dispatch → task queue permit → spawn_blocking → syscall, with zero buffering.

With this change, 100K writes of 128 bytes drops from ~1.6s to ~0.06s (debug build).

Design

  • Buffering is opt-in via writableStreamForRid(rid, autoClose, cfn, { bufferSize }) — only FsFile.writable passes it
  • When bufferSize > 0, highWaterMark switches to byte-based sizing instead of count-based
  • Buffer is lazily allocated on first small write
  • close() flushes remaining buffer, abort() discards it

Test plan

  • Existing spec tests pass
  • cargo test -p deno_runtime
  • Manual benchmark: 1M x 128-byte writes via file.writable.getWriter().write()
  • Verify large writes (> 64KB) still work correctly
  • Verify close() flushes remaining buffered data
  • Verify network socket writable streams are unaffected

🤖 Generated with Claude Code

Small buffer writes (e.g. 128 bytes) via `file.writable.getWriter().write()`
were ~100x slower than Node.js because each write immediately dispatched a
syscall through the async op machinery with zero buffering.

Add an opt-in `bufferSize` option to `writableStreamForRid()` and enable it
with a 64KB buffer for `FsFile.writable`. Small chunks are accumulated in a
JS-side buffer and flushed when full or on close. Large chunks (>= bufferSize)
bypass the buffer entirely. The highWaterMark and sizeAlgorithm are also
adjusted to operate on byte length when buffering is enabled.

Only FsFile writable streams are affected — network sockets, stdin, QUIC,
and WebTransport streams remain unbuffered.

Ref: #16667

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fsFile writer slow

1 participant