Describe the feature
Add opt-in, serializable multipart upload resume support to
feature/s3/transfermanager.
When resumable behavior is enabled, a failed or canceled multipart upload would preserve completed parts and return a resume token containing the upload ID, part size, object size, and completed parts.
A later process could pass the token and the original seekable source to a resume API. The transfer manager would verify the state using ListParts, upload only missing parts, and complete the multipart upload.
The existing abort-on-failure behavior should remain the default to avoid unexpected storage costs.
This is related to #113, but specifically proposes an implementation for the new feature/s3/transfermanager. Similar functionality is available in the Java v2 SDK through ResumableFileUpload.
Use Case
Large uploads over unstable networks currently restart from the beginning after the multipart upload is aborted. Applications should be able to preserve completed parts and continue the upload in another process.
Proposed Solution
Introduce a serializable resume token and a method such as ResumeUploadObject. The exact API is open for maintainer feedback.
I would like to work on the implementation if the maintainers agree with the direction.
Other Information
No response
Acknowledgements
AWS Go SDK V2 Module Versions Used
github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.7
Go version used
go version go1.22.12 darwin/arm64
Describe the feature
Add opt-in, serializable multipart upload resume support to
feature/s3/transfermanager.When resumable behavior is enabled, a failed or canceled multipart upload would preserve completed parts and return a resume token containing the upload ID, part size, object size, and completed parts.
A later process could pass the token and the original seekable source to a resume API. The transfer manager would verify the state using
ListParts, upload only missing parts, and complete the multipart upload.The existing abort-on-failure behavior should remain the default to avoid unexpected storage costs.
This is related to #113, but specifically proposes an implementation for the new
feature/s3/transfermanager. Similar functionality is available in the Java v2 SDK throughResumableFileUpload.Use Case
Large uploads over unstable networks currently restart from the beginning after the multipart upload is aborted. Applications should be able to preserve completed parts and continue the upload in another process.
Proposed Solution
Introduce a serializable resume token and a method such as
ResumeUploadObject. The exact API is open for maintainer feedback.I would like to work on the implementation if the maintainers agree with the direction.
Other Information
No response
Acknowledgements
AWS Go SDK V2 Module Versions Used
github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.7
Go version used
go version go1.22.12 darwin/arm64