Skip to content

v2.1.4

Latest

Choose a tag to compare

@inureyes inureyes released this 10 May 12:09
· 4 commits to main since this release

SFTP transfer performance release. Streams uploads/downloads in 255 KiB chunks instead of buffering whole files, pipelines up to 64 concurrent SFTP requests, and raises the server-side MAX_READ_SIZE to the 255 KiB SFTP standard. On a 1 GiB transfer over loopback this drops upload peak RSS from ~3.23 GB to ~20 MB and wall time from 38.6 s to 3.5 s; multi-GB transfers no longer OOM the client.

New Features

None.

Improvements

  • Stream SFTP uploads/downloads instead of buffering whole files in memory (#195). upload_file/upload_dir_recursive previously loaded the entire local file into a Vec<u8> via tokio::fs::read before calling write_all, and download_file/download_dir_recursive called read_to_end into a pooled buffer plus a clone() to a separate Vec before writing locally — so multi-GB transfers had peak RSS that scaled with file size and large files OOM'd the client. Each path now uses a stream_copy() helper looping on 255 KiB reads/writes through the existing AsyncRead/AsyncWrite impls on tokio::fs::File and russh_sftp::client::fs::File. Buffer size matches MAX_WRITE_LENGTH so each chunk maps to one SFTP packet without further fragmentation.

    Verified locally on macOS arm64 against bssh-server v2.1.3 over loopback with a 1 GiB file:

    Op Build real RSS
    upload unpatched 38.65s 3.23 GB
    upload streaming 3.47s 20 MB
    download unpatched 3.93s 2.17 GB
    download streaming 3.41s 16 MB
  • Pipeline up to 64 concurrent SFTP requests for upload and download (#196). Bounded pipelined SFTP upload/download helpers replace the previous strictly-sequential request/response loop. Review follow-ups also: cap server-advertised read/write lengths against local maxima to avoid oversized allocations from untrusted SFTP metadata, bound the download reorder queue across both in-flight and pending out-of-order responses, use fstat size info where available to avoid reads past EOF and validate unexpected short reads, and preserve remote download handle shutdown after syncing with main.

  • Raise bssh-server SFTP MAX_READ_SIZE from 64 KiB to the 255 KiB SFTP standard (#197). The server previously hard-capped every SFTP READ reply at 64 KiB regardless of what the client requested, while bssh-russh-sftp and OpenSSH's sftp-server both use MAX_READ_LENGTH = 261120 (255 KiB). A client asking for a 256 KiB chunk only ever got 64 KiB back, forcing four extra requests per byte stream. Bumped to 261120 so server replies match the chunk size used by the rest of the stack — combined with client-side pipelining (#196), this cuts the per-MiB request count on downloads from 16 to 4. Memory exposure stays bounded: handles are still capped at MAX_HANDLES = 1000 per session and each in-flight read still uses a single per-request buffer of this size.

Bug Fixes

None.

CI/CD Improvements

None.

Technical Details

  • Streaming buffer size (255 KiB) is aligned with the russh-sftp packet limit so each chunk stays within one SFTP read/write request without further fragmentation.
  • Downloaded remote handles are now explicitly closed after successful copies; chunk-size capping and in-memory pipelined upload/download behavior are covered by new SFTP crate tests.

Dependencies

None — Cargo.lock only changed because the workspace package bssh itself bumped from 2.1.3 to 2.1.4.

Breaking Changes

None.

Known Issues

None at release time. The 64-way SFTP pipelining is a fixed concurrency cap (not user-configurable in this release); workloads that need a different parallelism point should track follow-up issues on the repository.

What's Changed

  • perf: stream SFTP uploads/downloads instead of buffering whole file by @Yaminyam in #195
  • perf: pipeline SFTP requests for upload/download (~2-3x speedup) by @Yaminyam in #196
  • perf: raise server MAX_READ_SIZE to SFTP standard 255 KiB by @Yaminyam in #197

Full Changelog: v2.1.3...v2.1.4