Problem
The only disk-space protection on download/restore_remote is CheckDisksUsage (pkg/backup/backuper.go:613, added for #878), which compares free space against the whole backup size. Two gaps:
- No headroom guarantee. A download can legally fill a disk to 100%, taking ClickHouse down with it. Operators typically want "never push the data disk past N%".
- Whole-backup size vs filtered download. When downloading with
--tables db.one_table from a multi-TiB backup, the space check still uses the full backup size, refusing downloads that would actually only fetch a few GiB.
Proposal
- Add
--disk-limit <pct> (1–100) to download and restore_remote CLI commands + the corresponding server API query parameter. 0/absent = disabled (default, current behavior unchanged).
CheckDiskLimit computes projected usage per data disk: (used + downloadSize) / total * 100, and refuses the download with a clear error naming the disk, projected pct, and limit.
estimateFilteredDownloadSize estimates the size actually downloaded honoring --tables/--partitions filtering (per-table metadata sizes for the tables that match), so single-table downloads from huge backups aren't blocked.
- Refactor
CheckDisksUsage into CheckDisksUsageFiltered(backup, disks, isResumeExists, downloadSize) so the free-space check also benefits from the filtered estimate (the old signature stays as a thin wrapper).
Extracted from a production fork used to back up multi-TiB ClickHouse clusters, where an unguarded restore once filled a data disk.
I have a PR ready to submit.
Problem
The only disk-space protection on
download/restore_remoteisCheckDisksUsage(pkg/backup/backuper.go:613, added for #878), which compares free space against the whole backup size. Two gaps:--tables db.one_tablefrom a multi-TiB backup, the space check still uses the full backup size, refusing downloads that would actually only fetch a few GiB.Proposal
--disk-limit <pct>(1–100) todownloadandrestore_remoteCLI commands + the corresponding server API query parameter.0/absent = disabled (default, current behavior unchanged).CheckDiskLimitcomputes projected usage per data disk:(used + downloadSize) / total * 100, and refuses the download with a clear error naming the disk, projected pct, and limit.estimateFilteredDownloadSizeestimates the size actually downloaded honoring--tables/--partitionsfiltering (per-table metadata sizes for the tables that match), so single-table downloads from huge backups aren't blocked.CheckDisksUsageintoCheckDisksUsageFiltered(backup, disks, isResumeExists, downloadSize)so the free-space check also benefits from the filtered estimate (the old signature stays as a thin wrapper).Extracted from a production fork used to back up multi-TiB ClickHouse clusters, where an unguarded restore once filled a data disk.
I have a PR ready to submit.