infra(openchallenges): deploy VPC, NAT, ALB, ECS cluster and nginx as a web app for testing

[tschaffter]

Description

This PR implements the core infrastructure components for deploying OpenChallenges applications using AWS CDK, establishing a production-ready foundation for container-based deployments. The implementation includes a complete VPC setup with cost-optimized NAT Gateway configuration, Application Load Balancer integration, ECS cluster management, and Fargate service orchestration. This work builds upon the initial CDK setup and represents a significant milestone in migrating OpenChallenges infrastructure to Infrastructure as Code using AWS CDK.

TODO

• Identify a strategy to load config from env vars and possibly context variables defined in cdk.json

Related Issue

Related to OpenChallenges CDK migration efforts

Changelog

• Add VPC construct with configurable NAT Gateway deployment strategy
• Add Application Load Balancer construct with target group and listener management
• Add ECS cluster construct for container orchestration
• Add Fargate service construct for deploying containerized applications
• Add VPC stack with environment-specific configuration support
• Add ALB stack with HTTPS listener and target group routing
• Add ECS cluster stack for managing container runtime environment
• Add application service stack for deploying Fargate services
• Configure NAT Gateway strategy (1 NAT for dev, num_az NATs for stage/prod) to optimize costs
• Add GuardDuty VPC endpoint management with CDK
• Add documentation for GuardDuty VPC endpoint deletion procedures
• Update environment configuration files with ECS and ALB settings
• Add unit tests for VPC stack validation
• Add unit tests for ALB stack configuration
• Configure edge container images for deployment
• Deploy NGINX as temporary solution for web application initialization issues
• Add missing environment variables to application service stack

Testing

Login into an AWS profile that provides the Developer role for the org-sagebase-openchallenges-dev AWS account.

aws sso login --profile openchallenges-dev-Developer

Deploy the dev stack:

nx deploy openchallenges-infra-cdk

Copy the value of the CDK app output openchallenges-dev-{developer}-app-service.AppUrl and enter it into a browser.

Destroy the stack:

nx destroy openchallenges-infra-cdk

Preview

Features

The OpenChallenges infrastructure now has:

• Complete VPC infrastructure with multi-AZ support and configurable NAT Gateway deployment
• Application Load Balancer with HTTPS support and target group routing
• ECS cluster and Fargate service orchestration for containerized deployments
• Environment-specific configuration for dev, stage, and prod environments
• Cost-optimized NAT Gateway strategy reducing expenses in development environment
• GuardDuty VPC endpoint integration for enhanced security monitoring
• Comprehensive documentation for infrastructure management and troubleshooting
• Unit test coverage for critical infrastructure components
• Reusable CDK constructs for VPC, ALB, ECS cluster, and Fargate services

[zaro0508]

Did created all of those contructs yourself? or did you get them from a vetted source? if it's from another source then it would be good to reference the sources. At this point I'm not sure whether i should even review those constructs.

[tschaffter]

Did created all of those contructs yourself? or did you get them from a vetted source?

@zaro0508 I created them my self. This change aligns with AWS CDK's recommended patterns where Constructs are the building blocks and Stacks (the approach taken in the current OC CDK app) are the deployment containers.

[zaro0508]

@zaro0508 I created them my self.

I’m curious, was there a reason you chose to implement all of them yourself instead of looking for equivalent constructs on the AWS Construct Hub? Using publicly available constructs, especially ones from AWS, can often be a better option since they’ve already been tested and vetted by other users.

Also, I wanted to clarify your intention regarding these constructs — are they meant to be shared across multiple projects beyond openchallenges? There doesn't seem to be any versioning in place for these constructs, so if multiple projects start using them, changes to the constructs specific for openchallenges could unintentionally break other projects. It would be worth thinking about a versioning strategy if these are intended to be shared.