fix: add cluster creator access entry

[muse-sisay]

Description

PR creates KubernetesAdmin role and an access entry with AmazonEKSClusterAdminPolicy policy for that role in the management cluster and any clusters created in the management cluster. I have removed ArgoCD and Atlantis's Access entry as they only AmazonEKSViewPolicy in the default namespace.

Related Issue(s)

Access entry is the current recommended way of granting IAM principals access to an EKS cluster. It decouples cluster access policy from the principals permission policy and the account. Permissions are defined in Access Polcies and principals are granted access by creating access entries.

Our Terraform code for management clusters adds two access entries for two principals ( ArgoCD and Atlantis service Account) with AmazonEKSViewPolicy to the default namespace for all resources. Additional the upstream EKS module we utilize has enable_cluster_creator_admin_permissions variable which creates an access entry to the principal who creates the cluster. Unfortunately the way it is implemented, the principal ARN for the access entries get updates to the principal running the the Terraform code. In Kubefirst there are two principals running the Terraform code for the management cluster

1. Role or User who run kubefirst aws create command and
2. Atlantis

When the responsibility of managing the Terraform code is transferred to Atlantis, terraform apply fails with the following error

╷
│ Error: creating EKS Access Entry (ci-k1-8ddf1835-aws-gh-cf:arn:aws:iam::126827061464:role/atlantis-ci-k1-8ddf1835-aws-gh-cf): operation error EKS: CreateAccessEntry, https response error StatusCode: 409, RequestID: 2b151f30-bccf-419d-bf02-739a3bfd57dc, ResourceInUseException: The specified access entry resource is already in use on this cluster.
│ 
│   with module.eks.module.eks.aws_eks_access_entry.this["cluster_creator"],
│   on .terraform/modules/eks.eks/main.tf line 185, in resource "aws_eks_access_entry" "this":
│  185: resource "aws_eks_access_entry" "this" {
│ 

This can be fixed by removing Atlantis’s access entry. But does Atlantis need AmazonEKSViewPolicy access to the cluster? No. This PR removes access entry for both ArgoCD and Atlantis.

How to test

1. Create a cluster using --gitops-template-branch "muse-sisay/aws-bootstrap-eks-admin" flag

kubefirst aws create --git-provider github --cluster-name aentry-04 \ 
  --domain-name kubesecond.com --subdomain aentry-04 --alerts-email \
  [email protected] --github-org k1-aws --dns-provider \
  cloudflare --gitops-template-branch "muse-sisay/aws-bootstrap-eks-admin"

2. Create a downstream cluster in the default cloud account
3. Step 1 creates a role called aentry-04-KuberenetesAdmin. Add the following to your aws config

[profile starter-kubernetes-admin]
role_arn = arn:aws:iam::126827061464:role/aentry-04-KubernetesAdmin
source_profile = starter # The profile you used to create the management cluster

4. Update your kubeconfig file

aws eks update-kubeconfig --name aentry-04  --region us-east-1 --kubeconfig ignore-cluster-01.yaml
export KUBECONFIG=ignore-cluster-01.yaml
# if you are using fish shell
# set -x KUBECONFIG ignore-cluster-01.yaml

5. Run if you are able to list resources.

---

Important

I am not a fan of a single role in the downstream account used by Kubefirst, Crossplane and users.