18.2.0 breaks plan due to Service Account not existing #1022
Description
TL;DR
18.2.0 added a undocumented breaking change. The required Service Agent may not exist which will break the plan.
Expected behavior
Adding the network.user role to the aiplatform service agent should only be done in case the SA exists.
Observed behavior
When planning we get an error as the service account which should get a role granted does not exist:
running 'sh -c' 'terragrunt apply -no-color $PLANFILE' in '/atlantis/XXXXXX/redis': exit status 1: running "terragrunt apply -no-color $PLANFILE" in "/atlantis/XXXXXX/redis":
10:07:20.064 STDOUT terraform1.13.5: module.project.module.project.module.project.module.shared_vpc_access.google_project_iam_member.service_shared_vpc_user["aiplatform.googleapis.com"]: Creating...
10:07:24.599 STDERR terraform1.13.5: Error: Request `Create IAM Members roles/compute.networkUser serviceAccount:service-12340815@gcp-sa-aiplatform.iam.gserviceaccount.com for project "XXXXXX"` returned error: Error applying IAM policy for project "XXXXXX": Error setting IAM policy for project "XXXXXX": googleapi: Error 400: Service account service-12340815@gcp-sa-aiplatform.iam.gserviceaccount.com does not exist., badRequest
10:07:24.600 STDERR terraform1.13.5: with module.project.module.project.module.project.module.shared_vpc_access.google_project_iam_member.service_shared_vpc_user["aiplatform.googleapis.com"],
10:07:24.600 STDERR terraform1.13.5: on .terraform/modules/project.project.project/modules/shared_vpc_access/main.tf line 159, in resource "google_project_iam_member" "service_shared_vpc_user":
10:07:24.600 STDERR terraform1.13.5: 159: resource "google_project_iam_member" "service_shared_vpc_user" {
10:07:24.747 ERROR error occurred:
* Failed to execute "terraform1.13.5 apply ./gcp::projects::redis-redis.tfplan -no-color" in .
Error: Request `Create IAM Members roles/compute.networkUser serviceAccount:service-12340815@gcp-sa-aiplatform.iam.gserviceaccount.com for project "XXXXXX"` returned error: Error applying IAM policy for project "XXXXXX": Error setting IAM policy for project "XXXXXX": googleapi: Error 400: Service account service-12340815@gcp-sa-aiplatform.iam.gserviceaccount.com does not exist., badRequest
with module.project.module.project.module.project.module.shared_vpc_access.google_project_iam_member.service_shared_vpc_user["aiplatform.googleapis.com"],
on .terraform/modules/project.project.project/modules/shared_vpc_access/main.tf line 159, in resource "google_project_iam_member" "service_shared_vpc_user":
159: resource "google_project_iam_member" "service_shared_vpc_user" {
exit status 1
Additional information
The SA not existing is even considered in the official documentation:
If the Vertex AI Service Agent doesn't exist in the host project, create it using the following command:
gcloud beta services identity create --service=aiplatform.googleapis.com --project=PROJECT_ID