-
Notifications
You must be signed in to change notification settings - Fork 23
Description
Describe the bug
When creating a delta lake sink and using the SimpleAWSCredentialProvider, the entire config is printed to stdout: including the AWS access and secret keys. This is quite undesirable. This makes using the SimpleAWSCredentialProvider not an option for us. However its very difficult to set environment variables in the sink when using the Kubernetes runtime (only setting of secrets seems to be an option). Therefore other credential providers like the EnvironmentCredentialProvider are not implementable.
To Reproduce
Steps to reproduce the behavior:
- configure the delta lake sink with a SimpleAWSCredentialProvider
- create the sink
- output shows the entire config: including secrets
Expected behavior
Either to not have the secrets being printed to the console/stdout or have it anonymised with ******
Or, an way to pass environment variables to the kubernetes runtime in the sink create statement (via sink-config), so the EnvironmentCredentialProvider can be used for kubernetes runtimes.
Environment
- Pulsar version: 3.0.10
- Deployment: standalone and cluster
- Connector: pulsar-io-lakehouse-3.0.7.5-cloud.nar