S3: Content length error on put object request

[toon-borgers-hs]

Describe the bug

Since version 2.30.1 , I'm getting an error when I do a put request using multipart upload and the transfer manager. If I pass the contentLength parameter in the put object request, I get an error like Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.

I don't get this if I leave the content length parameter off of the put object request.

I'm also setting it on the request body, but there it's not causing an issue.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The upload should still work as before 2.30.1

Current Behavior

I get an error:

software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
	at java.base/java.util.concurrent.CompletableFuture.wrapInCompletionException(CompletableFuture.java:323)
	at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:457)
	at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2162)
	at com.sparkcentral.aws.s3.SyncS3ClientUploadIntegrationTest.upload(SyncS3ClientUploadIntegrationTest.java:150)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:227)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:222)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:151)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:113)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:978)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:955)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:526)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
	Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Failed to send multipart upload requests
		at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
		at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
		at software.amazon.awssdk.services.s3.internal.multipart.GenericMultipartHelper.handleException(GenericMultipartHelper.java:66)
		at software.amazon.awssdk.services.s3.internal.multipart.MultipartUploadHelper.failRequestsElegantly(MultipartUploadHelper.java:126)
		at software.amazon.awssdk.services.s3.internal.multipart.KnownContentLengthAsyncRequestBodySubscriber.lambda$onNext$4(KnownContentLengthAsyncRequestBodySubscriber.java:172)
		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallMetricCollectionStage.lambda$execute$0(AsyncApiCallMetricCollectionStage.java:56)
		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.lambda$execute$2(AsyncApiCallTimeoutTrackingStage.java:67)
		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:135)
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:152)
		... 23 more
Caused by: java.lang.IllegalStateException: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor$StreamedRequest$1.onComplete(NettyRequestExecutor.java:477)
	at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody$ChecksumCalculatingSubscriber.onComplete(ChecksumCalculatingAsyncRequestBody.java:265)
	at software.amazon.awssdk.utils.async.BaseSubscriberAdapter.handleOnCompleteState(BaseSubscriberAdapter.java:254)
	at software.amazon.awssdk.utils.async.BaseSubscriberAdapter.handleStateUpdate(BaseSubscriberAdapter.java:181)
	at software.amazon.awssdk.utils.async.BaseSubscriberAdapter$1.request(BaseSubscriberAdapter.java:109)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerSubscriber.maybeRequestMore(HandlerSubscriber.java:303)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerSubscriber.channelWritabilityChanged(HandlerSubscriber.java:157)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:526)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:498)
	at io.netty.handler.logging.LoggingHandler.channelWritabilityChanged(LoggingHandler.java:296)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:526)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:498)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelWritabilityChanged(CombinedChannelDuplexHandler.java:448)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:129)
	at io.netty.channel.CombinedChannelDuplexHandler.channelWritabilityChanged(CombinedChannelDuplexHandler.java:271)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:526)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:498)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1380)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:524)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:880)
	at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:659)
	at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:625)
	at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:207)
	at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:304)
	at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:383)
	at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:437)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:929)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.forceFlush(AbstractNioChannel.java:366)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:782)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	... 1 more


Unable to execute HTTP request: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
	at app//software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at app//software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:227)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:222)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:151)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:113)
	at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at app//software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
	at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108)
	at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255)
	at app//software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167)
	at java.base@23/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:978)
	at java.base@23/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:955)
	at java.base@23/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:526)
	at java.base@23/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base@23/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base@23/java.lang.Thread.run(Thread.java:1575)
	Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Failed to send multipart upload requests
		at app//software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
		at app//software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
		at app//software.amazon.awssdk.services.s3.internal.multipart.GenericMultipartHelper.handleException(GenericMultipartHelper.java:66)
		at app//software.amazon.awssdk.services.s3.internal.multipart.MultipartUploadHelper.failRequestsElegantly(MultipartUploadHelper.java:126)
		at app//software.amazon.awssdk.services.s3.internal.multipart.KnownContentLengthAsyncRequestBodySubscriber.lambda$onNext$4(KnownContentLengthAsyncRequestBodySubscriber.java:172)
		at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at app//software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
		at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at app//software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallMetricCollectionStage.lambda$execute$0(AsyncApiCallMetricCollectionStage.java:56)
		at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at app//software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.lambda$execute$2(AsyncApiCallTimeoutTrackingStage.java:67)
		at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at app//software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
		at java.base@23/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
		at java.base@23/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
		at java.base@23/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
		at java.base@23/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
		at app//software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:135)
		at app//software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:152)
		... 23 more
Caused by: java.lang.IllegalStateException: Request content was only 4196352 bytes, but the specified content-length was 20981796 bytes.
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor$StreamedRequest$1.onComplete(NettyRequestExecutor.java:477)
	at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody$ChecksumCalculatingSubscriber.onComplete(ChecksumCalculatingAsyncRequestBody.java:265)
	at software.amazon.awssdk.utils.async.BaseSubscriberAdapter.handleOnCompleteState(BaseSubscriberAdapter.java:254)
	at software.amazon.awssdk.utils.async.BaseSubscriberAdapter.handleStateUpdate(BaseSubscriberAdapter.java:181)
	at software.amazon.awssdk.utils.async.BaseSubscriberAdapter$1.request(BaseSubscriberAdapter.java:109)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerSubscriber.maybeRequestMore(HandlerSubscriber.java:303)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerSubscriber.channelWritabilityChanged(HandlerSubscriber.java:157)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:526)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:498)
	at io.netty.handler.logging.LoggingHandler.channelWritabilityChanged(LoggingHandler.java:296)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:526)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:498)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelWritabilityChanged(CombinedChannelDuplexHandler.java:448)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:129)
	at io.netty.channel.CombinedChannelDuplexHandler.channelWritabilityChanged(CombinedChannelDuplexHandler.java:271)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:526)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:498)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1380)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:524)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:505)
	at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:880)
	at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:659)
	at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:625)
	at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:207)
	at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:304)
	at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:383)
	at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:437)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:929)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.forceFlush(AbstractNioChannel.java:366)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:782)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	... 1 more

Reproduction Steps

    @Test
    void upload() {
        var bucket = "com.sparkcentral.dev.conv-images";

        var contentLength = 20 * 1024 * 1024;
        var random = new Random();
        var randomData = new byte[contentLength];
        random.nextBytes(randomData);

        var client =
                S3AsyncClient.builder()
                        .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("key", "secret")))
                        .region(Region.of("us-east-1"))
                        .endpointOverride(URI.create("http://s3.localhost.localstack.cloud:4566"))
                        .multipartEnabled(true)
                        .build();

        var transferManager = S3TransferManager.builder().s3Client(client).build();

        try (var stream = new ByteArrayInputStream(randomData)) {

            var uploadRequest = UploadRequest.builder()
                    .putObjectRequest(
                            b ->
                                    b.bucket(bucket)
                                            .key("key")
                                            .contentType("text/plain")
                                            .contentLength((long) contentLength))
                    .requestBody(
                            AsyncRequestBody.fromInputStream(
                                    b ->
                                            b.inputStream(stream)
                                                    .contentLength((long) contentLength)
                                                    .executor(newVirtualThreadPerTaskExecutor())))
                    .build();

            transferManager
                    .upload(uploadRequest)
                    .completionFuture()
                    .join();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

    }

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.30.1

JDK version used

openjdk 23 2024-09-17 OpenJDK Runtime Environment (build 23+37-2369) OpenJDK 64-Bit Server VM (build 23+37-2369, mixed mode, sharing)

Operating System and version

MacOS 15.0.1 (24A348)

[bhoradc]

Hi @toon-borgers-hs,

Thank you for reporting the issue. I am able to reproduce the exception with AWS S3 endpoint as well with version 2.30.0 and above.

This looks related to S3 default integrity protection change that went in v2.30.0.

Minimal reproducible code snippet

    public static void upload() {
        var bucket = "<<bucketname";
        var contentLength = 20 * 1024 * 1024;
        var random = new Random();
        var randomData = new byte[contentLength];
        random.nextBytes(randomData);

        var client =
                S3AsyncClient.builder()                 
                        .region(Region.of("us-east-1"))                    
                        .multipartEnabled(true)
                        .build();

        var transferManager = S3TransferManager.builder().s3Client(client).build();

        ExecutorService executor = Executors.newSingleThreadExecutor();
        try (var stream = new ByteArrayInputStream(randomData)) {

            var uploadRequest = UploadRequest.builder()
                    .putObjectRequest(
                            b ->
                                    b.bucket(bucket)
                                            .key("key")
                                            .contentType("text/plain")
                                            .contentLength((long) contentLength))
                    .requestBody(
                            AsyncRequestBody.fromInputStream(
                                    b ->
                                            b.inputStream(stream)
                                                    .contentLength((long) contentLength)
                                                    .executor(executor)))
                    .build();
            transferManager
                    .upload(uploadRequest)
                    .completionFuture()
                    .join();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        finally {
            executor.shutdown();
            try {
                executor.awaitTermination(1, TimeUnit.MINUTES);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
            transferManager.close();
        }
    }

We are looking further into this.

Regards,
Chaitanya

[zoewangg]

Hi @toon-borgers-hs we released a fix in 2.30.2, could you try with the latest version?

[toon-borgers-hs]

@zoewangg @bhoradc

I can confirm that the fix in 2.30.2 fixes the issue for me too.

Thank you so much for the quick action and resolution!

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.