Skip to content

[Bug] Printing of secrets to console when creating a delta lake sink #840

@szindel

Description

@szindel

Describe the bug
When creating a delta lake sink and using the SimpleAWSCredentialProvider, the entire config is printed to stdout: including the AWS access and secret keys. This is quite undesirable. This makes using the SimpleAWSCredentialProvider not an option for us. However its very difficult to set environment variables in the sink when using the Kubernetes runtime (only setting of secrets seems to be an option). Therefore other credential providers like the EnvironmentCredentialProvider are not implementable.

To Reproduce
Steps to reproduce the behavior:

  1. configure the delta lake sink with a SimpleAWSCredentialProvider
  2. create the sink
  3. output shows the entire config: including secrets

Expected behavior
Either to not have the secrets being printed to the console/stdout or have it anonymised with ******
Or, an way to pass environment variables to the kubernetes runtime in the sink create statement (via sink-config), so the EnvironmentCredentialProvider can be used for kubernetes runtimes.

Environment

  • Pulsar version: 3.0.10
  • Deployment: standalone and cluster
  • Connector: pulsar-io-lakehouse-3.0.7.5-cloud.nar

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions