fix(transfer): close the gaps the live DAG engine review found against rclone

CHANGELOG.md

@@ -8,9 +8,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 #### Changed
+- **`put -r --skip-restricted` uploads a tree whose names the destination partly forbids, setting aside and reporting the offending entries instead of refusing the whole batch.** The default is unchanged and deliberate: one restricted name (a tab in a file name bound for S3, for instance) still stops the batch before anything is created. The review battery of 2026-09-05 measured the two policies side by side on the same 20-file tree: AeroFTP uploaded 0 files, rclone 19 and reported the one it could not. With the flag a folder with a restricted name takes its children along, every skip is listed once, and the run ends `partial` with exit 4 because not everything requested landed.
+- **Single-file downloads are multi-threaded by default: files of 250 MiB or more are fetched with 4 concurrent range streams, the same stream count as rclone's default (4 streams above 256Mi; AeroFTP's cutoff stays 250 MiB, so files between the two thresholds split here and not there).** `--multi-thread-streams` used to default to `1`, so the DAG range scheduler, the strict 206 probe and the single-stream fallback all existed but only ran when a user found the flag; the May 2026 live matrix showed rclone ahead on every 1 GiB download because of that default alone, while the shared harness measured the two engines within noise once the streams were equal. The gate is unchanged: only backends that prove Range honesty (S3, Azure, SFTP as independent connections, WebDAV and Koofr after a live probe) split the file, everything else and every file below `--multi-thread-cutoff` stays single-stream, and `--multi-thread-streams 1` or `AEROFTP_MULTI_THREAD_STREAMS=1` restores the old behaviour. `pget` is unaffected.
 - **An S3 endpoint that is plain HTTP and not on this machine now needs the profile's explicit consent, and a connection to one is refused until it is given.** Signing a request authenticates it, it does not encrypt it: over cleartext the `Authorization` header, the STS session token when a role is assumed, and every file byte are readable by anything on the path. The other two ways in were already closed, redirects are refused outright and no preset ships a cleartext remote endpoint, so what remained was an endpoint typed by hand or carried in from another tool's configuration file. Loopback is exempt and unaffected, because those bytes never reach a network: local bridges such as the Filen Desktop gateway keep working untouched. What changes for someone pointing at MinIO or a gateway over `http://` on a LAN is one checkbox, offered in the connection form next to the endpoint precisely when it applies, and readable as `allow_cleartext_endpoint` on the profile for the CLI, the MCP pool and schedulers that never see a form. The import listings now mark such a profile as it arrives, which is where the endpoint is still legible, rather than leaving the surprise for a connection weeks later. This mirrors the consent Swift already asks for the same reason, and the two predicates that decide "local" are deliberately kept apart: the one that trusts a self-signed certificate accepts `.local` mDNS names, and the one that decides whether bytes stay off the wire does not.
 
 #### Fixed
+- **A warm transfer connection is retired after 128 files.** Multi-file transfers reuse a clone worker's connection across files (FTP already did, SFTP since this release). A session that lives across thousands of files accumulates state no single transfer sees: the review battery of 2026-09-06 hit the SFTP client's open-handle counter on 4 of 5000 downloads with unbounded reuse, even with every read handle closed and awaited. A worker that has served 128 files is now dropped and the next file dials afresh, the way rclone's pools retire connections; the handshake stays amortised to well under one percent per file on a WAN link.
+- **One-way `sync` no longer re-transfers every unchanged file on backends that do not preserve modification time.** The CLI planner skipped a file only when size and mtime were exactly equal; S3-compatible stores report the upload time as the object's mtime, so equality never held and a second `sync --direction upload` re-uploaded the whole tree (the DAG engine review battery of 2026-09-05 measured 5000 files re-uploaded in 84 s on a run that should have moved nothing; rclone's no-op took 34 s). In one-way mode a destination of the same size now counts as current when it is at least as new as the source, within a 2 s tolerance for FAT and FTP `MDTM` granularity: a copy written after the source last changed already holds it, and a source edited later is still transferred. Bidirectional sync keeps exact equality, where a difference is a conflict to resolve. `--skip-matching` is unchanged.
+- **Multi-file SFTP transfers reuse their SSH connections across files instead of dialling a new one per file.** The shared batch executor recycles a warm worker only for providers that opt in; FTP did, SFTP did not, so `put -r`, `get -r` and `sync` on SFTP paid a full key exchange and authentication for every file. The DAG engine review battery of 2026-09-05 measured it on 5000 small files over a 53 ms link: about 3 files per second with four leases, most of it handshake. SFTP now opts in on the same terms as FTP (a worker is parked only after a successful transfer; paths are absolute; a dead channel fails its next file loudly), so one batch pays one handshake per lease.
+- **A single-file `put` into a remote folder that does not exist yet creates the folder on SFTP, FTP/FTPS and WebDAV instead of failing with "Failed to create remote file: No such file".** `put -r` already created every missing ancestor and rclone's `copyto` creates parents as a matter of course; the single-file path did neither. The parent is now checked once and, only when missing, created top-down with the same ladder the recursive path uses; object stores, which have no folders to create, are untouched. Found by the DAG engine review harness of 2026-09-05 on the Hetzner lab SFTP.
+- **`export rclone` keeps an SFTP profile's host-key trust and every path-rooted profile's starting folder.** The exported sftp remote carried no `known_hosts_file`, so rclone performed no host-key validation and warned about it on every run, while AeroFTP checks the same server against `~/.ssh/known_hosts`; the remote now points at that file. And a profile that opens in a sub-folder (sftp, ftp, ftps, webdav) exported to the account root, because those rclone backends have no start-folder key; the folder is now carried as an `alias` remote (`<name>-path`, `remote = <name>:<folder>`), the same shape the exporter already uses for a pinned S3 bucket, absolute for sftp and relative to the URL root for ftp and webdav. Found while exporting the review's nine test profiles to rclone: all nine listed, two landed in the wrong folder and three warned.
+- **`get --partial` over a download that had already finished is a no-op again instead of failing with "Range not satisfiable: file may have changed on server" (exit 4).** The resume path took the bytes already on disk as the offset and asked every HTTP backend for `Range: bytes=<size>-`, which a complete object answers with 416; the same happened when the local file was longer than the remote one. The CLI now compares the local byte count with the remote size before asking for a range and decides the way rclone does: equal is complete (a leftover `.aerotmp` is finalised in place), longer is stale and restarts from scratch, shorter resumes, unknown remote size still asks for the tail. Reproduced against MinIO and a local S3 endpoint during the DAG engine review of 2026-09-05; the provider 416 arms are unchanged as the second line of defence.
+- **`--limit-rate` and `--bwlimit` now cap S3, WebDAV, FTP/FTPS, Azure, Backblaze B2 and every HTTP cloud provider, not only SFTP and MEGA.** The flag used to call the provider's own `set_speed_limit`, which only SFTP and MEGA implement; the others answered NotSupported and the CLI discarded the answer, so a 20 MiB S3 download asked to run at 2 MiB/s ran at line speed (measured at 9.2 MB/s in the DAG engine review of 2026-09-05). The limit is now armed once on the process-global transfer governor as a pair of directional caps, and the shared byte loops charge them where the bytes move: the resumable HTTP download stream, the S3, WebDAV and Azure download loops, the segmented range writer, the S3 request bodies, B2 part bodies, every multipart `PartBody`, the `ReaderStream` whole-file uploads and the four FTP data-channel loops; the SFTP loops charge the same caps alongside their own pacing. Concurrent files and parts share one budget. The GUI speed-limit setting lands on the same governor, so a provider without its own pacing no longer reports "Failed to set speed limit". Providers whose download or upload loop does not go through these shared paths are listed in the review report as the remaining siblings.
 - **The `--tui` menu of `profiles`, `groups` and `users` offers every verb the `-i` prompt has.** It used to be a second, hand-kept list that had fallen behind the action bar: Help, refresh, New, Groups, Users, Add and remove were reachable at the prompt but not from the menu, so `-i` was more capable than `--tui`. The menu is now derived from the same verb table, in the same order and with the same keys (`?` and `0` included), and a test pins that the two cannot drift apart again. Verbs that take no selector send the bare command, so Help prints and the menu reopens; membership and add/remove get the prompt their line-mode syntax needs. Alongside, the action bar capitalises a label only when the verb is invoked by its first letter: `refresh(.)` and `remove(X)` now read like `re-index(#)`, and that rule is a test rather than a convention. Raised in [#347](https://github.com/axpdev-lab/aeroftp/discussions/347)
 - **Jottacloud folder restore from the recycle bin works, because it stopped asking for a verb that does not exist.** Probed against a real directory tombstone, JFS has no command that restores a directory: `?restore=true` is rejected as an unknown command on both the Trash view and the original path, and `?mv=`/`?mvDir=` out of Trash 404 because the Trash listing is a virtual view whose entries no verb can address. The restore is now composed from the primitives that already carry file restore: the tombstone at the original path still lists the deleted tree with every child's `deleted` stamp and revision, each tombstoned file is revived with `?cphash=true` against its own path — which revives the ancestor folders along with it — and only the folders no file could bring back, the empty ones, are recreated with `?mkDir=true`. The reported result counts only what the server confirmed: children found already live are listed as already present rather than claimed, and a restore that stops partway is an error carrying the confirmed counts and the paths that failed, never a quiet success. Re-running a restore restores nothing twice. (#397)
 

docs/CLI-GUIDE.md

@@ -432,6 +432,8 @@ aeroftp-cli pget --profile "AWS S3" /backup/big.tar.gz --json
 
 Alias of `get` with a parallel-segments preset. Splits a single file into N byte ranges and downloads them concurrently, then stitches them back together. Useful when latency or per-connection throughput is the bottleneck (large `.tar.gz` archives, S3 buckets from far regions). Falls back to a single sequential stream when the provider does not advertise range-request support.
 
+Plain `get` is multi-threaded by default as well, on the same terms as rclone: files at or above `--multi-thread-cutoff` (default `250M`) are fetched with `--multi-thread-streams` concurrent range streams (default `4`) when the backend proves Range honesty (S3, Azure, SFTP as independent connections, WebDAV and Koofr after a strict 206 probe); smaller files and backends without strict ranges use one stream. `--multi-thread-streams 1` restores the single-stream behaviour, and `AEROFTP_MULTI_THREAD_STREAMS` sets the default for a shell.
+
 ### put - Upload Files
 
 ```bash
@@ -821,7 +823,9 @@ aeroftp-cli sync --profile "Backup" ./photos/ /backup/photos/ --direction upload
 aeroftp-cli sync --profile "Backup" ./photos/ /backup/photos/ --direction upload --ec=quartile
 ```
 
-Sync options: `--direction` (upload/download/both), `--dry-run`, `--delete`, `--exclude`, `--error-correction[=LEVEL]` / `--ec[=LEVEL]`, `--max-delete`, `--backup-dir`, `--backup-suffix`, `--track-renames`, `--bwlimit`, `--conflict-mode`, `--resync`.
+How an unchanged file is recognised: in one-way sync (`--direction upload` or `download`) a file is skipped when the sizes match and the destination copy is not older than the source (2 s tolerance). This is what lets `sync` converge on backends that report the upload time as the object's mtime (S3 and compatible stores, where exact equality never holds). The trade-off is the same as rclone's `--update`: a destination edited later than the source with content of the same size is treated as current; when that can happen, run `sync --direction both` (exact mtime equality, conflicts surfaced) or `check` afterwards. `--skip-matching` skips on size alone.
+
+Sync options: `--direction` (upload/download/both), `--dry-run`, `--delete` (in a non-interactive shell, a pipeline or an agent run, `--delete` is refused unless `--max-delete N` or `N%` caps it or `--dry-run` previews it: rclone deletes by default, AeroFTP asks for the ceiling first), `--exclude`, `--error-correction[=LEVEL]` / `--ec[=LEVEL]`, `--max-delete`, `--backup-dir`, `--backup-suffix`, `--track-renames`, `--bwlimit`, `--conflict-mode`, `--resync`.
 
 `--error-correction` is opt-in for CLI sync and protects uploaded remote files at rest by writing a sibling `<remote>.aerorec` sidecar after each successful upload. If no level is supplied the CLI uses `medium` (15% target overhead). When enabled, sync automatically excludes `*.aerorec` from comparisons so parity sidecars are not mirrored back as user data or deleted as orphans. Remote deletes best-effort remove the protected file's companion sidecar after the primary delete succeeds; missing sidecars are ignored and sidecar delete failures do not fail the file delete. Phase 1 sidecar generation is capped at 256 MiB per source file; larger files are uploaded normally and counted as `ec_skipped_too_large`. JSON sync reports include `ec_generated`, `ec_skipped_too_large`, `ec_generate_failed`, `ec_sidecar_deleted`, and `ec_sidecar_delete_failed` when EC is enabled. Local-to-local sync ignores this flag.
 
@@ -1927,6 +1931,7 @@ It also emits the transfer-scheduler surface: a `protocol_transfer_capabilities`
 | `--files-from <file>` | Transfer only files listed in file (one per line, `#` comments). Works with get -r, put -r, sync |
 | `--files-from-raw <file>` | Like `--files-from` but preserves whitespace and empty lines |
 | `--immutable` | Skip a destination the pre-write check finds already there (append-only mode, best effort). Honoured by `put`, `cp`, `mv`, `get` and `sync`: an existing target is skipped with exit 9 and left untouched, and a target whose existence cannot be checked (timeout, permission error, unsupported `stat`) is refused with the provider's error code rather than written. The check is a stat before the write, not a server-side precondition, so two writers racing for the same target can still both pass it |
+| `--skip-restricted` | Recursive `put`: skip files or folders whose name the destination forbids, upload the rest, report each skip; the run ends `partial` (exit 4) like rclone does with failed items. Default refuses the whole batch before uploading anything |
 | `--no-check-dest` | Skip remote directory listing during sync (assume destination is empty) |
 | `--max-depth <n>` | Maximum recursion depth for ls -R, find, sync, get -r, put -r |
 | `--default-time <ts>` | Fallback mtime when backend returns None. Accepts ISO 8601, RFC 3339, or `now` |
@@ -2467,6 +2472,8 @@ aeroftp-cli get sftp://user@host /large-file.iso --limit-rate 5M
 aeroftp-cli get sftp://user@host /large-file.iso --bwlimit "08:00,512k 18:00,off"
 ```
 
+The cap is process-wide and protocol-independent: it is enforced by the transfer governor where the bytes move (the HTTP download and upload streams, every multipart part, the FTP data channel, the SFTP loops), so it holds on S3, WebDAV, FTP/FTPS, Azure, Backblaze B2 and the HTTP cloud providers as well as on SFTP and MEGA. Concurrent files and parts share the one budget: `--parallel 8 --limit-rate 5M` moves at most 5 MiB/s in total, not per file. The `--bwlimit` schedule is resolved once at startup. Owned request bodies (S3 signed parts, B2 parts) are paced in 256 KiB slices, so a low cap still produces a smooth rate rather than one burst per part.
+
 ### Encoding Issues
 
 The CLI sanitizes filenames with ANSI escape sequences. If filenames appear truncated, the server is sending control characters in directory listings.

src-tauri/Cargo.toml

@@ -39,6 +39,7 @@ serde_json = "1.0"
 serde = { version = "1.0", features = ["derive"] }
 toml = "1.1"
 log = "0.4"
+bytes = "1"
 futures-util = "0.3"
 # DO NOT BUMP. Dependabot raised this to =2.11.1 (PR #201) and broke EVERY
 # Linux build: every custom command started returning "not allowed by ACL"