Problem
DataHub's Snowflake ingestion source currently supports DEFAULT_AUTHENTICATOR , EXTERNAL_BROWSER_AUTHENTICATOR , KEY_PAIR_AUTHENTICATOR , and OAUTH_AUTHENTICATOR via the authentication_type config. It does not support Snowflake Workload Identity Federation, which allows services running on cloud providers (GCP, AWS, Azure) to authenticate to Snowflake using their cloud identity - with no secrets, keys, or passwords to manage or rotate.
We are currently forced to use key pair authentication, which requires generating and distributing key pairs across environments and handling key rotation manually.
Suggested Solution
Add a new authentication_type option (e.g. WORKLOAD_IDENTITY) that passes the required parameters to snowflake-connector-python 's connect() method for workload identity federation. The Snowflake Python connector already supports this — see snowflakedb/snowflake-connector-python#2578 and the dbt-adapters PR for reference implementations.
Alternatives we've considered
- Key pair authentication (
KEY_PAIR_AUTHENTICATOR) - works but requires key generation, secure distribution, and periodic rotation across all environments.
- OAuth - requires standing up and maintaining an OAuth provider integration.
- Basic auth - not viable for service accounts in many Snowflake configurations.
Problem
DataHub's Snowflake ingestion source currently supports
DEFAULT_AUTHENTICATOR,EXTERNAL_BROWSER_AUTHENTICATOR,KEY_PAIR_AUTHENTICATOR, andOAUTH_AUTHENTICATORvia theauthentication_typeconfig. It does not support Snowflake Workload Identity Federation, which allows services running on cloud providers (GCP, AWS, Azure) to authenticate to Snowflake using their cloud identity - with no secrets, keys, or passwords to manage or rotate.We are currently forced to use key pair authentication, which requires generating and distributing key pairs across environments and handling key rotation manually.
Suggested Solution
Add a new
authentication_typeoption (e.g.WORKLOAD_IDENTITY) that passes the required parameters to snowflake-connector-python 's connect() method for workload identity federation. The Snowflake Python connector already supports this — see snowflakedb/snowflake-connector-python#2578 and the dbt-adapters PR for reference implementations.Alternatives we've considered
KEY_PAIR_AUTHENTICATOR) - works but requires key generation, secure distribution, and periodic rotation across all environments.