Skip to content

Conversation

@nbari
Copy link

@nbari nbari commented Dec 25, 2025

Review the Contributing Guidelines

Before submitting a pull request, verify it meets all requirements in the Contributing Guidelines.

Describe Why This Is Awesome

Why is this awesome?

The aim of s3m apart from trying to consume fewer resources is to make as much as possible "fault-tolerant" the storage procedure of the incoming stream so that even if the server lost network connectivity, the stream can be resumed and continue the upload process where it was left without the need to start all over again (no when input is from STDIN/pipe).

Like this pull request? Vote for it by adding a 👍

@desingh9
Copy link

Need more detailed descriptions in that context.

@nbari
Copy link
Author

nbari commented Jan 14, 2026

Hi @desingh9, maybe this helps:

s3m is designed for streaming uploads to S3 from stdin/pipes (e.g. database backups) using resumable multipart uploads. This allows large or hard-to-reproduce streams to continue after network interruptions instead of restarting the entire pipeline. It is optimized for low memory usage and supports optional client-side encryption, zstd compression, throttling, and retries, making it suitable for production backup workflows and constrained hosts.

s3m is designed to run in very constrained environments with minimal resource usage:

Disk Space: 512MB when streaming from STDIN (for buffering multipart uploads)
Memory: Minimal - uses streaming to avoid loading data into RAM
Network: Internet connection for S3 uploads (supports resumable uploads on disconnection)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants