Skip to content

AWS secret manager error to connect #67588

@Herycklys

Description

@Herycklys

Helm Chart Version

2.0.17

What step the error happened?

During the Sync

Relevant information

After upgrading to chart v2, all connectors seem to be ignoring my configuration for the secret manager. More specifically aws-secret-manager-access-key-id and aws-secret-manager-secret-access-key. I have AWS_SECRET_MANAGER_ACCESS_KEY_ID and AWS_SECRET_MANAGER_SECRET_ACCESS_KEY configured as well. The pods do not start without it.

I am using abctl for deploy. The command:
sudo abctl local install --values .airbyte/values.yaml --secret .airbyte/secrets.yaml --host airbyte-internal.getadmira.com --insecure-cookies --no-browser

Here is how my values.yaml looks like:

global:
  secretsManager:
    type: AWS_SECRET_MANAGER
    secretName: "airbyte-config-secrets" # Name of your Kubernetes secret.
    awsSecretManager:
      region: us-east-2
      authenticationType: credentials ## Use "credentials" or "instanceProfile"
      tags: ## Optional - You may add tags to new secrets created by Airbyte.
        - key: app
          value: airbyte
      kms: arn:aws:kms:us-east-2****

server:
  extraEnv:
    - name: AB_COOKIE_SECURE
      value: "false"
    - name: AB_COOKIE_SAME_SITE
      value: "Lax"

I have tried more combinations for the values.yaml, including accessKeyIdSecretKey, setting jobs.connectorPod, worker.extraEnv, and a few others with the base key secret but none of it have worked.

The secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets # Or a custom name
type: Opaque # Or a specific type if applicable
stringData:
  AWS_SECRET_MANAGER_ACCESS_KEY_ID: ***
  AWS_SECRET_MANAGER_SECRET_ACCESS_KEY: '***'
  aws-secret-manager-access-key-id: ***
  aws-secret-manager-secret-access-key: '***'

Relevant log output

Encountered exception of type: class com.amazonaws.SdkClientException. Exception message: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@5403907: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@71c0b742: Unauthorized (Service: null; Status Code: 401; Error Code: null; Request ID: null; Proxy: null)].

Failure origin: airbyte_platform

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions