-
Notifications
You must be signed in to change notification settings - Fork 10
Description
We've been getting reported issues from users uploading large files to an origin with a S3 backend. It appears the CompleteMultipartUpload request takes long enough to complete when we invoke Close() that the client talking to the origin times out.
Indeed, from AWS's documentation:
The processing of a CompleteMultipartUpload request could take several minutes to finalize. After Amazon S3 begins processing the request, it sends an HTTP response header that specifies a 200 OK response. While processing is in progress, Amazon S3 periodically sends white space characters to keep the connection from timing out. A request could fail after the initial 200 OK response has been sent.
The Pelican client has a 10s timeout by default which is fairly incompatible with a "several minute" delay.
Luckily, this is mostly not necessary. XRootD now informs the OSS layer of the size of the object that will be uploaded; in that case, we could just use pure streaming with a single S3 interaction. We don't do this currently to avoid having too many variant code paths; however, it now seems like a necessity.