[AWS] VPC endpoint creation failing for AWS us-west-1 region

[jpgianfaldoni]

WARNING: Do not disclose any secrets or credentials while filing bug reports.

Describe the bug
The rest api and scc relay vpc endpoints creation are failing with the following error when the chosen region on AWS is us-west-1:

│ Error: creating EC2 VPC Endpoint (com.amazonaws.vpce.us-west-1.vpce-svc-09bb6ca26208063f2): operation error EC2: CreateVpcEndpoint, https response error StatusCode: 400, RequestID: 5d11dda2-377d-4920-95ff-7e88287c144f, api error DuplicateSubnetsInSameZone: Found another VPC endpoint subnet in the availability zone of subnet-023a0fce1c73791e8. VPC endpoint subnets should be in different availability zones supported by the VPC endpoint service.
│
│  with aws_vpc_endpoint.backend_rest[0],
│  on privatelink.tf line 356, in resource "aws_vpc_endpoint" "backend_rest":
│ 356: resource "aws_vpc_endpoint" "backend_rest" {
│
│ Error: creating EC2 VPC Endpoint (com.amazonaws.vpce.us-west-1.vpce-svc-04cb91f9372b792fe): operation error EC2: CreateVpcEndpoint, https response error StatusCode: 400, RequestID: bf92ba16-22ba-4a04-b87e-2ed5e2a2e359, api error DuplicateSubnetsInSameZone: Found another VPC endpoint subnet in the availability zone of subnet-023a0fce1c73791e8. VPC endpoint subnets should be in different availability zones supported by the VPC endpoint service.
│
│  with aws_vpc_endpoint.backend_relay[0],
│  on privatelink.tf line 372, in resource "aws_vpc_endpoint" "backend_relay":
│ 372: resource "aws_vpc_endpoint" "backend_relay" {

To Reproduce
Steps to reproduce the behavior:

1. Clone the repo
2. Choose the us-west-1 region on AWS
3. Run terraform apply

Expected behavior
All the resources should be created with no errors

Versions
Terraform Version:
Cloud Provider Version:
Databricks Version:

Additional context
Add any other context about the problem here.

[jpgianfaldoni]

I think the issue is related with the template.tfvars.example values:

private_subnets_cidr = ["10.0.0.0/22", "10.0.4.0/22", "10.0.8.0/22"]
privatelink_subnets_cidr = ["10.0.28.0/26", "10.0.28.64/26", "10.0.28.128/26"]

us-west-1 has only 2 AZ, so I think it is repeating one of the AZs and breaking when it tries to create the VPC endpoints, changing the values to use only two cidr works. Maybe it's worth adding a warning message on the tfvars.example? If not I can close the issue, the code itself works fine

[JDBraun]

Thanks for raising @jpgianfaldoni!

I was taking a look at a us-west-1 deployment prior to the holidays, and this will need to be a part of a bigger initiative to see if we can actually support the region given SRA's architecture.

In particular, for classic compute clusters spun-up in us-west-1, according to the docs, are dependent on an artifact bucket in us-west-2, which wouldn't be accessible via a regional S3 gateway endpoint.

So, I'll take a more holistic look and if we can support the region with SRA first, then I'll fix this bug.