Upload not possible

[TrueKillRob]

Hi,

I'm using the s3proxy in Azure. It runs in a container instance. The instance listens on port 8080 and 443.
The certificate is created by the startup script using the commands you provided in your wiki.
The storage is an Azure StorageAccount (BlobStorage).

I'm using curl and S3Browser (https://s3browser.com/) to test the proxy.

• It works to create buckets and directories.
• It works to delete the buckets and directories.
• It works to download a file uploaded directly to the StorageAccount (without the proxy via the Azure portal).
• I can't upload any files. It doesn't matter if authentication is enabled or not and if I use https or http.

I have enabled the proxy's debug mode. From the many lines it generates, I found that this might be the error:

[s3proxy] W 11-25 14:11:47.970 S3Proxy-Jetty-20 o.g.s.o.e.j.server.HttpChannel:793 |::] handleException /test2/local-identity.txt java.io.IOException: com.azure.storage.blob.models.BlobStorageException: Status Code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidMetadata</Code><Message>The metadata provided is invalid. It contains characters that are not allowed.

My Configuration:

S3PROXY_AUTHORIZATION=none
S3PROXY_CORS_ALLOW_ALL=false
S3PROXY_CORS_ALLOW_CREDENTIAL=
S3PROXY_CORS_ALLOW_HEADERS=
S3PROXY_CORS_ALLOW_METHODS=
S3PROXY_CORS_ALLOW_ORIGINS=
S3PROXY_CREDENTIAL=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX X
S3PROXY_ENCRYPTED_BLOBSTORE=
S3PROXY_ENCRYPTED_BLOBSTORE_PASSWORD=
S3PROXY_ENCRYPTED_BLOBSTORE_SALT=
S3PROXY_ENDPOINT=http://0.0.0.0:8080
S3PROXY_IDENTITY=06291a9a-XXXX-XXXX-XXXX-140160e6a711
S3PROXY_IGNORE_UNKNOWN_HEADERS=false
S3PROXY_KEYSTORE_P ASSWORD=XXXXXXXXXX
S3PROXY_KEYSTORE_PATH=/opt/s3proxy/keystore.p12
S3PROXY_READ_ONLY_BLOBSTORE=false
S3PROXY_VIRTUALHOST=
JCLOUDS_AZUREBLOB_AUTH=azureKey
JCLOUDS_CREDENTIAL=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
JCLOUDS_ENDPOINT=https://XXXXXXXXX XXX.blob.core.windows.net/
JCLOUDS_FILESYSTEM_BASEDIR=/data
JCLOUDS_IDENTITY=XXXXXXXXXX
JCLOUDS_KEYSTONE_PROJECT_DOMAIN_NAME=
JCLOUDS_KEYSTONE_SCOPE=
JCLOUDS_KEYSTONE_VERSION=
JCLOUDS_PROVIDER=azureblob-sdk
JCLOUDS_REGION=
JCLOUDS_REGIONS=us-east-1

Thanks for any help
Robert

[gaul]

It appears that the S3 client added metadata with characters that Azure does not support. Consider remapping them via https://github.com/gaul/s3proxy/wiki/Middleware-user-metadata-replacer.

[TrueKillRob]

Thank you for your reply.
I have added the parameters from the link you gave me. But nothing changes :-(
It could be that the tool I am using is faulty.
CURL works now (without authentication). I am looking for another tool.
Can you recommend an S3 tool for Windows?

[gaul]

Can you check which characters your client is sending? You can do this by running S3Proxy with more logging:

java -DLOG_LEVEL=trace -jar /path/to/s3proxy

[TrueKillRob]

I set it to trace but I'm to blind to se the corret line. Can you find the character?
Alternative: Should I disable SSL and use WireShark on the client?

Caused by: com.azure.storage.blob.models.BlobStorageException: Status code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidMetadata</Code><Message>The metadata specified is invalid. It has characters that are not permitted.
RequestId:a41d3f3c-101e-0001-12a3-409dff000000
Time:2024-11-27T08:11:11.9891638Z</Message></Error>"
	at com.azure.storage.blob.implementation.util.ModelHelper.mapToBlobStorageException(ModelHelper.java:483)
	at reactor.core.publisher.Mono.lambda$onErrorMap$30(Mono.java:3797)
	at reactor.core.publisher.Mono.lambda$onErrorResume$32(Mono.java:3887)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
	... 82 common frames omitted
[s3proxy] W 11-27 08:11:11.999 S3Proxy-Jetty-18 o.g.s.o.e.j.server.HttpChannel:793 |::] handleException /test2/localidentity.txt java.io.IOException: com.azure.storage.blob.models.BlobStorageException: Status code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidMetadata</Code><Message>The metadata specified is invalid. It has characters that are not permitted.
RequestId:a41d3f3c-101e-0001-12a3-409dff000000
Time:2024-11-27T08:11:11.9891638Z</Message></Error>"
[s3proxy] D 11-27 08:11:12.003 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [response_completed])
[s3proxy] D 11-27 08:11:12.004 reactor-http-epoll-1 reactor.netty.ReactorNetty:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Removed handler: azureSdkHandler, pipeline: DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[s3proxy] D 11-27 08:11:12.007 reactor-http-epoll-1 reactor.netty.ReactorNetty:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Non Removed handler: azureSdkHandler, context: null, pipeline: DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[s3proxy] D 11-27 08:11:12.007 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [disconnecting])
[s3proxy] D 11-27 08:11:12.008 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:254 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Releasing channel
[s3proxy] D 11-27 08:11:12.008 reactor-http-epoll-1 r.n.r.PooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Channel cleaned, now: 0 active connections, 1 inactive connections and 0 pending acquire requests.
[s3proxy] D 11-27 08:11:13.061 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:302 |::] request: Request(PUT https://s3proxy.westeurope.azurecontainer.io/test2/localidentity.txt)@737f3ad8
[s3proxy] T 11-27 08:11:13.062 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Authorization: AWS4-HMAC-SHA256 Credential=0995ee04-0c58-4d83-ad56-de3d75dbc3ad/20241127/us-east-1/s3/aws4_request,SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3b-last-modified;x-amz-meta-sha256, Signature=f1e17baa4a76707ca506611d36e471918ca994f88fac9b444fc718ab6cfb1762
[s3proxy] T 11-27 08:11:13.062 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-content-sha256: b8e8590bad590f7fc37152f54faabbcf49c884b8ef84c8d7ed37636afe636841
[s3proxy] T 11-27 08:11:13.062 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-meta-sha256: b8e8590bad590f7fc37152f54faabbcf49c884b8ef84c8d7ed37636afe636841
[s3proxy] T 11-27 08:11:13.063 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-date: 20241127T081113Z
[s3proxy] T 11-27 08:11:13.063 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-meta-s3b-last-modified: 20241125T105436Z
[s3proxy] T 11-27 08:11:13.064 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: User-Agent: S3 Browser/12.1.5 (https://s3browser.com)
[s3proxy] T 11-27 08:11:13.065 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Host: s3proxy.westeurope.azurecontainer.io
[s3proxy] T 11-27 08:11:13.065 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Content-Length: 34
[s3proxy] T 11-27 08:11:13.066 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Content-Type: text/plain
[s3proxy] D 11-27 08:11:13.085 reactor-http-epoll-1 r.n.r.PooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Channel acquired, now: 1 active connections, 0 inactive connections and 0 pending acquire requests.
[s3proxy] D 11-27 08:11:13.085 reactor-http-epoll-1 r.n.h.client.HttpClientConnect:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Handler is being applied: {uri=https://blobstoragename.blob.core.windows.net/test2/localidentity.txt, method=PUT}
[s3proxy] D 11-27 08:11:13.085 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [request_prepared])
[s3proxy] D 11-27 08:11:13.086 reactor-http-epoll-1 reactor.netty.ReactorNetty:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Added decoder [azureSdkHandler] at the end of the user pipeline, full pipeline: [reactor.left.sslHandler, reactor.left.httpCodec, azureSdkHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
[s3proxy] D 11-27 08:11:13.087 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [request_sent])
[s3proxy] D 11-27 08:11:13.101 reactor-http-epoll-1 r.n.h.c.HttpClientOperations:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Received response (auto-read:false) : RESPONSE(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 400 The metadata specified is invalid. It has characters that are not permitted.
Content-Length: <filtered>
Content-Type: <filtered>
Server: <filtered>
x-ms-request-id: <filtered>
x-ms-client-request-id: <filtered>
x-ms-version: <filtered>
x-ms-error-code: <filtered>
Date: <filtered>
[s3proxy] D 11-27 08:11:13.108 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [response_received])
[s3proxy] D 11-27 08:11:13.112 reactor-http-epoll-1 r.netty.channel.FluxReceive:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] [terminated=false, cancelled=false, pending=0, error=null]: subscribing inbound receiver
[s3proxy] D 11-27 08:11:13.113 reactor-http-epoll-1 r.n.h.c.HttpClientOperations:254 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Received last HTTP packet
[s3proxy] T 11-27 08:11:13.113 reactor-http-epoll-1 r.n.channel.ChannelOperations:244 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation terminal stack
	at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:485)
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:789)
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:224)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1349)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1389)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:799)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Unknown Source)
[s3proxy] D 11-27 08:11:13.115 S3Proxy-Jetty-19 o.g.s.S3ProxyHandlerJetty:164 |::] Unknown exception:

[TrueKillRob]

I disabled SSL and used WireShark to capture the upload.
Since all the sesitive data is in the file, I deleted the container and the StorageAccount and created a new one with a different IP and credentials...
I hope you can work with pcap files. If you want a different format, let me know.

I have also used another commercial product. It works with the same client application.
Please don't get me wrong: I want to use your application. It was just a test and the commercial application uses a Maria-SQL and a WEB-GUI and Automation is not possible...

I hope you can help!
Upload-Error.zip