Assume role support has been broken since 2022 🤣 #2879
Description
kafka-delta-ingest version 0.2.0 fails to assume the specified IAM role for cross-account writes to S3, resulting in an Access Denied (403) error, the previous version of the application worked correctly under the same configuration.
error log
2024-09-04T18:23:33 [ERROR] - my.app: Ingest service exited with error Writer { source: Storage { source: Generic { store: "S3", source: Client { status: 403, body: Some("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>CAE9GFWZPREMDXXX</RequestId><HostId>fsdfsdfsdfAaXXXXXqUenLk4JxzSkdGI/msdfsfsfdHyg=</HostId></Error>") } } } }
Current setup:
• AWS Account A: kafka-delta-ingest running as an ECS task.
• AWS Account B: S3 bucket for storage and DynamoDB for lock table.
Steps Taken:
1. With AWS_S3_ASSUME_ROLE_ARN set: The error still occurred with the same 403 Access Denied response.
2. Without AWS_S3_ASSUME_ROLE_ARN: The error persisted, Same result.
3. With a dummy IAM role in AWS_S3_ASSUME_ROLE_ARN: Same result.
4. With S3 in the same account (Account A): Everything worked perfectly, confirming the issue is specific to cross-account role assumption.
5. Using the old version: Previous versions of kafka-delta-ingest (before v0.2.0) work fine with the exact same IAM role assumption setup.
Expected Behavior:
The application should properly assume the IAM role specified in the AWS_S3_ASSUME_ROLE_ARN environment variable and perform cross-account writes to S3 in Account B.