Skip to content

Add non-zero check for incremental upload buffer sizes #1543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mansi153 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add non-zero check for incremental upload buffer sizes #1543

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
88adfdf
Add non-zero check for incremental upload buffer sizes
Jul 28, 2025
e2bb2e6
Address comments
Jul 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions mountpoint-s3-fs/src/upload.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,9 +149,10 @@
&self,
bucket: String,
key: String,
initial_offset: u64,

Check warning on line 152 in mountpoint-s3-fs/src/upload.rs

View workflow job for this annotation

GitHub Actions / Formatting 

Diff in /home/runner/work/mountpoint-s3/mountpoint-s3/mountpoint-s3-fs/src/upload.rs
initial_etag: Option<ETag>,
) -> AppendUploadRequest<Client> {
assert!(self.buffer_size > 0, "write-part-size should be greater than 0 for incremental uploads");
Copy link
Contributor

@passaro passaro Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems the wrong place to check. Why not UploaderConfig::new (where buffer_size is passed)?

Also, the message here should be related to this type, rather than user-facing. --write-part-size is a CLI argument and should be validated elsewhere.

Copy link
Contributor

@c-hagem c-hagem Jul 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should consider making the size (NonZero)[https://doc.rust-lang.org/std/num/type.NonZeroU32.html]?

// Limit the queue capacity to hold buffers for a total of at most
// MAX_BYTES_IN_QUEUE, but ensure it allows at least 1 buffer.
let capacity = (MAX_BYTES_IN_QUEUE / self.buffer_size).max(1);
Expand Down
Loading