KMS-encrypted S3 Transfers with S3 Events enabled not possible with IAM user

[kaplanan]

Use case: Using DTH to transfer any changes made to the source S3 bucket into the corresponding destination bucket in cn-north-1. The source S3 bucket is in the same account as the DTH, however, it is KMS encrypted. That is, the vanilla S3 Transfer Task for Source Buckets in the current region will not work for this scenario. One possible solution to this is described in Issue #73

The approach is to create an IAM User in the source bucket account and pass the credentials for it to the DTH S3 Transfer Task. For doing so, one must select "Is bucket in this account?" with "No" in the Source Settings. This will disable the option to listen to S3 events from the source bucket as "Enable S3 Event?" will not be available when selecting that the source bucket is not in the current account.

Expected behavior

We should make sure that either:

• One may add credentials from ASM even if the source S3 bucket is in the current account or
• The "Enable S3 Event" is still an available option for source buckets "outside" of the current account

---

• Version: v2.4.0
• Region (source): eu-central-1
• Region (destination): cn-north-1

[evalzy]

Hi Kaplana

Thanks for the feedback and contribute the PR meanwhile. I have discussed with team and right now for the KMS-encrypted S3 we suggest to use https://github.com/awslabs/data-transfer-hub/blob/main/docs/S3-SSE-KMS-Policy.md approach, and we are going to enhance UI to provide better user experience.

For the suggestion that use AK/SK to access all the S3 bucket including in/ out same account, this might impact too many customers, especially that for those who didn't initiate KMS in source S3, might leading them addition effort to create AK/ SK compare with existing feature.

Let me know if you have other thoughts.

Best Regard/ Eva
DTH Product Manager

[kaplanan]

Hi @evalzy , thanks for your reply. I see that but there must be a way in which one could do it via UI instead of manual modification within the Cloudformation stack after creating said S3 transfer task, right? Did I understand this correctly that there are UI features in plan which will incorporate this? Or even select from available KMS keys of that account (just like the ASM secrets) would be nice!

I think if this is what is coming up, that's totally fine!

Best regards
Ayhan

[evalzy]

We add this into UI and feature design backlog. Will release within next 2 version.

[kaplanan]

We're looking forward to it. Thank you!

[bassemwanis]

Hi @kaplanan,

After reviewing the documentation and the available workarounds mentioned in #73, I believe it would be more appropriate to classify this as an enhancement rather than a bug. The reason for this change is to ensure that our labels are accurate, which will help us prioritize backlog items more effectively.

Thank you
Bassem