[file system] Add configuration option to enable/disable token delegation process for S3 file system (allows support for MinIO and other S3-compatible object stores)

[michaelkoepf]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Currently, the Fluss token delegation process cannot be disabled. This limits file systems to use Fluss-internal credential providers. By adding the configuration option, also other credential providers in the provider chain can be used.

Solution

• Add configuration option to disable provider chain per file system (fs.s3.enable-token-delegation)
• Decide based on fs.s3.enable-token-delegation whether to obtain a token via STS or return a dummy token. The dummy token-approach is similar to the implementation of the HDFS file system and stops the token renewal process in com.alibaba.fluss.client.token.DefaultSecurityTokenManager.java after the initial iteration.
• Add com.alibaba.fluss.fs.s3.token.DynamicTemporaryAWSCredentialsProvider.java to provider chain by default and let the library figure out which credential provider in the chain is valid automatically.

Anything else?

No response

Willingness to contribute

• I'm willing to submit a PR!