|
| 1 | +# K-Stack: A Production-Grade 3-Tier Application on AWS EKS |
1 | 2 |
|
2 | | -xyz |
3 | | -# 3-Tier EKS Application on AWS |
| 3 | +This repository documents the end-to-end deployment of K-Stack, a scalable and observable 3-tier web application on a managed Kubernetes (EKS) cluster. The entire infrastructure is provisioned as code using Terraform, with a fully automated CI/CD pipeline for container image delivery. |
4 | 4 |
|
5 | | -This project deploys a complete 3-tier web application onto a managed Kubernetes (EKS) cluster in AWS. The entire infrastructure is provisioned using Terraform. |
| 5 | +## Live Project Showcase |
6 | 6 |
|
7 | | -## Architecture Overview |
| 7 | +The final deployment results in a fully functional, observable, and scalable system. |
8 | 8 |
|
9 | | - * **Infrastructure:** AWS (VPC, EKS, RDS) |
10 | | - * **Orchestration:** Terraform |
11 | | - * **Application:** React Frontend, Node.js Backend |
12 | | - * **Database:** PostgreSQL (AWS RDS) |
13 | | - * **CI/CD:** GitHub Actions for building Docker images |
14 | | - * **Access:** AWS Load Balancer |
| 9 | +**Live Application - Full Stack Operational:** |
15 | 10 |
|
16 | | -## Prerequisites |
| 11 | + |
| 12 | + |
17 | 13 |
|
18 | | -1. **AWS Account:** With appropriate IAM permissions. |
19 | | -2. **AWS CLI:** Installed and configured (`aws configure`). |
20 | | -3. **Terraform:** Installed locally. |
21 | | -4. **kubectl:** Installed locally. |
22 | | -5. **GitHub Repository:** A personal GitHub repo with the project code. |
| 14 | +*(The frontend, backend, and database are fully operational and connected, demonstrating a successful full-stack deployment.)* |
| 15 | +## Architecture & Design |
23 | 16 |
|
24 | | -## Deployment Steps |
| 17 | +This project implements a robust three-tier architecture to ensure separation of concerns, scalability, and security between the presentation (frontend), logic (backend), and data (database) layers. |
25 | 18 |
|
26 | | -### Step 1: Provision the Infrastructure |
| 19 | + * **Infrastructure Layer:** A secure and isolated network foundation is established using a custom AWS VPC with public/private subnets, NAT Gateways, and granular routing rules. |
| 20 | + * **Orchestration Layer:** A managed Amazon EKS cluster serves as a resilient and scalable platform for running containerized workloads, abstracting away the underlying node management. |
| 21 | + * **Data Layer:** A managed AWS RDS for PostgreSQL instance provides a robust, highly available database backend, decoupled from the application for enhanced security and performance. |
27 | 22 |
|
28 | | -Navigate to the `terraform/` directory and run the following commands. This will build the VPC, EKS cluster, and RDS database. (This step takes \~20 minutes). |
| 23 | +## Key Features & Business Impact |
| 24 | + |
| 25 | +This project is an exercise in production-readiness, demonstrating mastery of core DevOps principles to deliver a system that is automated, consistent, and optimized for cost. |
| 26 | + |
| 27 | +### 100% Reproducible Environments via IaC |
| 28 | + |
| 29 | +The entire cloud infrastructure is defined declaratively using **Terraform**. This Infrastructure as Code (IaC) approach eliminates configuration drift, ensures environment consistency between deployments, and drastically reduces the time and risk associated with manual setup. |
| 30 | + |
| 31 | +### Automated CI/CD for Zero-Touch Builds |
| 32 | + |
| 33 | +A robust CI/CD pipeline, powered by **GitHub Actions**, automates the entire build lifecycle. On every push to the `main` branch, the pipeline builds versioned, production-ready Docker images for the React and Node.js applications and pushes them to a centralized **Amazon ECR** registry. |
| 34 | + |
| 35 | +### Data-Driven Cost Optimization & Observability |
| 36 | + |
| 37 | +A key deliverable of this project was the implementation of a comprehensive observability stack using **Prometheus and Grafana**. By collecting and visualizing real-time metrics on CPU and memory utilization across the cluster, we established a clear baseline for resource consumption. **This critical data provides the foundation for data-driven optimization strategies, such as rightsizing nodes and setting appropriate resource requests/limits, paving the way for potential infrastructure cost reductions of up to 40%.** |
| 38 | + |
| 39 | +*Detailed cluster compute resource dashboards provide immediate visibility into performance bottlenecks and overallocation.* |
| 40 | + |
| 41 | + |
| 42 | +*Granular, pod-level metrics allow for precise application performance tuning and troubleshooting.* |
| 43 | + |
| 44 | + |
| 45 | +*A complete, real-time command center for monitoring the health and performance of the entire kingdom.* |
| 46 | + |
| 47 | +## Technology Stack |
| 48 | + |
| 49 | + * **Cloud:** AWS (EKS, RDS, ECR, VPC, EC2, ALB) |
| 50 | + * **Infrastructure as Code:** Terraform |
| 51 | + * **CI/CD:** GitHub Actions |
| 52 | + * **Containerization:** Docker |
| 53 | + * **Orchestration:** Kubernetes |
| 54 | + * **Observability:** Prometheus, Grafana |
| 55 | + * **Application:** React.js (Frontend), Node.js (Backend), PostgreSQL (Database) |
| 56 | + |
| 57 | +## Deployment Instructions |
| 58 | + |
| 59 | +To replicate this environment, follow these steps: |
| 60 | + |
| 61 | +### 1\. Prerequisites |
| 62 | + |
| 63 | + * An AWS account with appropriate IAM permissions. |
| 64 | + * AWS CLI, Terraform, and `kubectl` installed and configured locally. |
| 65 | + |
| 66 | +### 2\. Provision Infrastructure |
| 67 | + |
| 68 | +Navigate to the `terraform/` directory. Initialize and apply the Terraform configuration. This process will take approximately 20-30 minutes. |
29 | 69 |
|
30 | 70 | ``` |
31 | 71 | cd terraform |
32 | 72 | terraform init |
33 | 73 | terraform apply -auto-approve |
34 | 74 | ``` |
35 | 75 |
|
36 | | -### Step 2: Configure GitHub Actions |
| 76 | +### 3\. Configure CI/CD Pipeline |
37 | 77 |
|
38 | | -The `terraform apply` command will output an IAM Role ARN. |
| 78 | +The `terraform apply` command will output an IAM Role ARN for GitHub Actions. |
39 | 79 |
|
40 | | -1. Go to your GitHub repository's `Settings` \> `Secrets and variables` \> `Actions`. |
41 | | -2. Create a new repository secret named `AWS_IAM_ROLE_ARN` and paste the ARN value. |
42 | | -3. Go to the AWS IAM Console, find the new `github-actions-role`, and update its "Trust Policy" to allow your GitHub repository to use it. |
| 80 | +1. In your GitHub repository settings, create a secret named `AWS_IAM_ROLE_ARN` and provide the ARN. |
| 81 | +2. In the AWS IAM Console, update the "Trust Policy" of the created `github-actions-role` to allow your specific GitHub repository. |
43 | 82 |
|
44 | | -### Step 3: Run the CI Pipeline |
| 83 | +### 4\. Build Container Images |
45 | 84 |
|
46 | | -Make a small change to the application code (e.g., in `app/frontend/src/App.js`) and push the commit to your `main` branch. |
| 85 | +Trigger the GitHub Actions workflow by pushing a commit to the `main` branch. The pipeline will build and push the Docker images to ECR and output the new image tags in the logs. |
47 | 86 |
|
48 | | -``` |
49 | | -git add . |
50 | | -git commit -m "feat: trigger initial build" |
51 | | -git push |
52 | | -``` |
53 | | - |
54 | | -This will build your Docker images and push them to ECR. |
55 | | - |
56 | | -### Step 4: Deploy the Application |
| 87 | +### 5\. Deploy the Application |
57 | 88 |
|
58 | | -1. **Configure kubectl:** Connect your terminal to your new EKS cluster. |
| 89 | +1. **Configure `kubectl`:** |
59 | 90 | ``` |
60 | | - aws eks update-kubeconfig --region ap-south-1 --name titan-cluster |
| 91 | + aws eks update-kubeconfig --region <your-region> --name <your-cluster-name> |
61 | 92 | ``` |
62 | | -2. **Get the new image tags:** Go to the successful GitHub Actions run and find the final image URLs from the "Print New Image Tags" step. |
63 | | -3. **Update your manifests:** Manually update the `image:` tags in `manifests/02-backend.yaml` and `manifests/03-frontend.yaml` with the new URLs. |
64 | | -4. **Create the Database Secret:** Get your new database password from Terraform and create the Kubernetes secret. |
| 93 | +2. **Update Manifests:** Replace the `image:` placeholders in `manifests/02-backend.yaml` and `manifests/03-frontend.yaml` with the new tags from the pipeline. |
| 94 | +3. **Create Database Secret:** Retrieve the database password from Terraform outputs and create the Kubernetes secret. |
65 | 95 | ``` |
66 | | - # Get the password |
| 96 | + # Get password |
67 | 97 | terraform output -raw db_password |
68 | 98 |
|
69 | | - # Create the secret (paste the new password) |
| 99 | + # Create secret |
70 | 100 | kubectl create secret generic db-credentials \ |
71 | | - --from-literal=DB_HOST='...' \ |
| 101 | + --from-literal=DB_HOST='<PASTE_DB_ADDRESS_HERE>' \ |
72 | 102 | --from-literal=DB_USER='dbadmin' \ |
73 | | - --from-literal=DB_PASSWORD='PASTE_YOUR_NEW_PASSWORD_HERE' \ |
| 103 | + --from-literal=DB_PASSWORD='<PASTE_PASSWORD_HERE>' \ |
74 | 104 | --from-literal=DB_NAME=appdb \ |
75 | 105 | -n app |
76 | 106 | ``` |
77 | | -5. **Launch the Application:** |
| 107 | +4. **Launch:** |
78 | 108 | ``` |
79 | 109 | kubectl apply -f manifests/ |
80 | 110 | ``` |
81 | 111 |
|
82 | | -### Step 5: Access Your Application |
| 112 | +## Accessing the Grafana Dashboard |
83 | 113 |
|
84 | | -Find the public URL of your application's load balancer. (This can take 2-5 minutes to become available). |
85 | | -
|
86 | | -``` |
87 | | -kubectl get ingress/app-ingress -n app |
88 | | -``` |
| 114 | +1. **Retrieve Admin Password:** |
| 115 | + ``` |
| 116 | + kubectl get secret -n monitoring prometheus-grafana -o jsonpath="{.data.admin-password}" | base64 --decode |
| 117 | + ``` |
| 118 | +2. **Open Secure Tunnel:** |
| 119 | + ``` |
| 120 | + kubectl port-forward -n monitoring svc/prometheus-grafana 8080:80 |
| 121 | + ``` |
| 122 | +3. Navigate to `http://localhost:8080` and log in with username `admin` and the retrieved password. |
89 | 123 |
|
90 | 124 | ## Cleanup |
91 | 125 |
|
92 | | -To destroy all the AWS resources created by this project and avoid charges, run the following command from the `terraform/` directory: |
| 126 | +To prevent ongoing charges, destroy all created resources using Terraform: |
93 | 127 |
|
94 | 128 | ``` |
| 129 | +cd terraform |
95 | 130 | terraform destroy -auto-approve |
96 | 131 | ``` |
| 132 | +
|
| 133 | +
|
| 134 | +## Future Improvements & Scalability |
| 135 | +
|
| 136 | +While K-Stack is a complete and functional project, a real-world production system would include several further enhancements. The current architecture serves as the perfect foundation for: |
| 137 | +
|
| 138 | +* **Implementing True GitOps:** Integrating a tool like **ArgoCD** to fully automate the final deployment step, creating a "hands-off" CI/CD pipeline where a `git push` to `main` is the only manual action required. |
| 139 | +* **Enforcing Zero-Trust Networking:** Adding a CNI plugin like **Cilium** to implement Kubernetes Network Policies. This would create granular firewall rules, ensuring that frontend pods can only talk to the backend, and only the backend can talk to the database. |
| 140 | +* **Centralized Log Aggregation:** Deploying a logging stack like **Loki** or the **ELK Stack** to aggregate logs from all pods into a single, searchable database, dramatically improving debugging and auditing capabilities. |
| 141 | +
|
| 142 | +
|
| 143 | +This repository is the result of many hours of building, debugging, and documenting. If it helped you understand a complex topic or saved you time on your own projects, the best way to say thanks is to leave a star ⭐ on the repo. |
0 commit comments