Skip to content

Commit 487301c

Browse files
authored
perf: pipeline SFTP requests for upload/download
Implements bounded pipelined SFTP upload/download helpers and wires file transfers to use up to 64 concurrent SFTP requests. Review follow-up included: - cap server-advertised read/write lengths against local maxima to avoid oversized allocations from untrusted SFTP metadata - bound the download reorder queue across both in-flight and pending out-of-order responses - use fstat size information where available to avoid reads past EOF and validate unexpected short reads - preserve remote download handle shutdown behavior after syncing with main - add SFTP crate tests for chunk-size capping and in-memory pipelined upload/download behavior Validation: - cargo fmt --check - cargo clippy -- -D warnings - cargo test -p bssh-russh-sftp - cargo test --lib --verbose (1222 passed, 9 ignored) - cargo test --tests --verbose -- --skip integration_test - GitHub Actions CI and CLA checks passed on 9ac61d2
1 parent 0a9f647 commit 487301c

6 files changed

Lines changed: 428 additions & 99 deletions

File tree

ARCHITECTURE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ bssh (Backend.AI SSH / Broadcast SSH) is a high-performance parallel SSH command
1212
- SSH jump host support (-J)
1313
- SSH configuration file parsing (-F)
1414
- Interactive PTY sessions with single/multiplex modes
15-
- SFTP file transfers (upload/download) with chunked streaming
15+
- SFTP file transfers (upload/download) with bounded pipelined streaming
1616
- Backend.AI cluster auto-detection
1717
- pdsh compatibility mode
1818

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

crates/bssh-russh-sftp/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ tokio = { version = "1", default-features = false, features = [
2020
"macros",
2121
] }
2222
tokio-util = "0.7"
23+
futures = { version = "0.3", default-features = false, features = ["std", "async-await"] }
2324
serde = { version = "1.0", features = ["derive"] }
2425
serde_bytes = "0.11"
2526
bitflags = { version = "2.9", features = ["serde"] }

0 commit comments

Comments
 (0)