[Feature] Add S3 Tables support to dbt-athena adapter

[chadmyers]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Currently, you can't use S3 Tables with the dbt-athena adapter because AWS requires the special catalog s3tablescatalog/<bucketname> as the Catalog.

Normally, Catalog is something like "AwsDataCatalog" or empty (defaults to AwsDataCatalog) or a Federated Query connector like "mysql-db" or a DynamoDB table.

But, in the API, SDKs (boto3, etc), and CLI, if you set catalog to s3tablescatalog/<bucketname> then it will assume the actual catalog is AwsDataCatalog and then use the new sub-catalog of s3tablescatalog.... This appears to be a way they shoe-horned querying S3 Table buckets in Athena without having to change the API.

However, dbt-athena currently checks to see if the catalog is AwsDataCatalog and, if not, does special logic like call GetDataCatalog (see: impl.py lines 676-691). If you try to set the DBT database to s3tablescatalog/... DBT will throw an error and say that the call to GetDataCatalog API failed.

Describe alternatives you've considered

It's possible to query Athena against AwsDataCatalog and using the database/schema identifier of "s3tablescatalog/bucketname"."schemaname"."tablename" in the query. But since DBT quotes all identifiers, it ends up trying to query ""s3tablescatalog/bucketname"."schemaname"."tablename"" and doesn't work.

Who will this benefit?

Anyone using Athena and the new S3 Tables product in AWS

Are you interested in contributing this feature?

Yes, but there are many other calls to GetDataCatalog so I'm afraid of breaking other code paths

Anything else?

No response

[grhaonan]

+1

[kevin-veerasammy-ad]

+1 we need this

[valdasg]

The workaround is to create an alias for "s3tablescatalog/bucketname" to something like "s3tablescatalog", however in this way you can only have target to the same catalog, meaning you can not write to awsgluecatalog.

[rutvik-weather]

+1

[NurFaizin]

+1

[villekr]

+1

[amaingot]

+1

[MaxxBusch]

+1

[niklasgranqvist]

+1

[pmalmirae]

+1