Deploy EKS Resources

[aliziel]

See: hotosm/k8s-infra#14

Control plane: EKS v1.32, Extended Support disabled (upgrade_policy)
Worker nodes: Start with managed to unblock STAC, with possible Karpenter, hybrid, or self-managed alternative to follow
Networking: Starting with dedicated resources pending further input. Prefer HOT OSM module https://github.com/hotosm/terraform-aws-vpc for consistency
Ingress: ingress-nginx controller + NLB
Storage: EBS

Noted in #177 (comment) that cluster infra should perhaps be pushed alongside project code given the temporary nature of this setup (typically separate). I'd like to avoid limiting the HOT team on the long-term, global solution, but if other apps/projects may end up deployed on the cluster in the foreseeable future, we may want to reassess.

[aliziel]

@spwoodcock You had mentioned in hotosm/k8s-infra#14:

We have two subnets in different availability zones for HA

Can I assume that means we want to use existing VPCs and subnets ? Will add those as lookups or inputs if so.

[spwoodcock]

Glad to see this moving 🎉

• The config you propose sounds sensible.
• I will defer to @dakotabenjamin on if we want to use existing subnets, but my hunch is no, we want to create new VPCs and subnets to be used by the cluster. The comment I left would be better written as We **should** have two subnets in different availability zones for HA.
• Would it make sense to push the cluster infra setup in hotosm/k8s-infra instead? Is there are downside / does it make things more difficult for you? I only mention this, because although OAM is definitely the primary candidate for the cluster to start, I can't discount the possibility of finding time to dabble and migrate a few services into the cluster (e.g. osm-sandbox, or odk-central).

[aliziel]

@spwoodcock Yeah, I'm down with hotosm/k8s-infra if you are. As mentioned, I was mostly just concerned about getting in the way of preexisting/long-term plans. I'll fork and open up a PR shortly once I add in those networking definitions 👍

I'll start with 2 public + 2 private subnets as referenced in the proposal ticket and we can modify as needed.

[aliziel]

Following up on the Terraform license conversation that came up during the stakeholder meeting earlier:

My understanding is that the recent change to the Business Source License (BSL) largely applies to the use of Terraform to provide a competitive offering. It's not exactly clear cut, but it seems like money needs to be involved which would exclude HOT (same disclaimer as earlier though: I am definitely not a lawyer). I'll reach out at DS to see if other teams we've worked with have faced similar concerns.

[spwoodcock]

Following up on the Terraform license conversation that came up during the stakeholder meeting earlier:

My understanding is that the recent change to the Business Source License (BSL) largely applies to the use of Terraform to provide a competitive offering. It's not exactly clear cut, but it seems like money needs to be involved which would exclude HOT (same disclaimer as earlier though: I am definitely not a lawyer). I'll reach out at DS to see if other teams we've worked with have faced similar concerns.

Thanks @aliziel!

I think that specifically applies to modifying Terraform source code and offering a paid equivalent. As as you say, either way I don't think HOT is implicated. So legally I think we are fine.

The other part of the discussion is around the project governance. Hashicorp was acquired by IBM, resulting in the license change. At any point they could decide to change to a more restrictive license. It's no longer an open-source project with input from the communty, but rather a commercial product that has open code (for now).

So in summary, we can use Terraform today without issues, but need to be wary of the potential to have to change in future 😄

[aliziel]

hotosm/k8s-infra#16 includes this build and some scripts to install eoAPI manually (CI workflow in hotosm/k8s-infra#17). The GitHub deployer permission set will need to be adjusted for successful provisioning.

NOTE:

Right now the GitHub deployer is the only principal set up to access the cluster. If direct access for devs is required, a TF variable has been defined to create access entries for cluster admins. A list of principals can be passed to the build:

- arg-var: cluster_ci_access_role_arn=${{ secrets.AWS_OIDC_ROLE }}
+ arg-var: cluster_ci_access_role_arn=${{ secrets.AWS_OIDC_ROLE }},cluster_admin_access_role_arns={{ var.ADMIN_ROLES }}

Tip

The var accepts multiple principals but, if its needed, its recommended to only add one assumable role and manage who can assume via IAM strategies like User Groups in order to minimize surface area of auth controls.

• e.g. someone moves teams within the same account: their policies may be adjusted in IAM, but their base profile can still access the cluster.

Important

The current variant is geared toward cluster owners. As more apps are added and more contributors need access, there should be less-privileged variants added scoped to namespace.