Feat: Setup ELK Stack for Honeypot Log Collection

README.md

@@ -1,2 +1,36 @@
-# honeynet
-Develop a scalable, cloud-native honeypot deployment framework that leverages Terraform to provision and manage honeypot instances across multiple geographic regions. 
+# Cloud-Native Honeynet Platform
+
+A distributed, adaptive honeynet platform with centralized control, dynamic deception, and cloud-native deployment.
+
+## Quick Start
+
+### Prerequisites
+- Terraform >= 1.0
+- AWS CLI configured with credentials
+- SSH key pair
+
+### Initial Setup
+```bash
+cd infrastructure/terraform
+terraform init
+terraform plan
+terraform apply
+```
+
+### Verification
+```bash
+# Get the public IP
+terraform output public_ip
+
+# SSH into the VM
+ssh -i ~/.ssh/your-key.pem ec2-user@$(terraform output public_ip)
+```
+
+## Architecture
+This is the foundational setup for a distributed honeynet system. Current implementation provisions a single VM as the building block for future multi-region deployments.
+
+## Next Steps
+- Honeypot installation and configuration
+- Multi-region deployment
+- Logging and monitoring setup
+- Automation and orchestration 

docs/elk-verification.md

@@ -0,0 +1,113 @@
+# ELK Stack Verification Guide
+
+## 🎯 Issue #18: Setup ELK Stack for Honeypot Log Collection
+
+### Changes Made
+
+1. **Elasticsearch Module**
+   - Single-node cluster configuration
+   - Security groups for HTTP (9200) and transport (9300)
+   - Memory-optimized settings
+   - Auto-discovery configuration
+
+2. **Logstash Module**
+   - JSON log parsing for Cowrie format
+   - GeoIP enrichment for attacker location
+   - Elasticsearch output configuration
+   - Pipeline for honeypot log processing
+
+3. **Kibana Module**
+   - Web interface configuration
+   - Elasticsearch integration
+   - Security settings for development
+   - Public access for dashboard
+
+4. **Filebeat Integration**
+   - Honeypot agent configuration
+   - JSON log shipping from Cowrie
+   - Template configuration for honeypot logs
+
+### Deployment Steps
+
+1. **Deploy ELK Infrastructure**
+   ```bash
+   cd infrastructure/terraform
+   terraform init
+   terraform apply -var="create_elk_stack=true"
+   ```
+
+2. **Verify Elasticsearch**
+   ```bash
+   # Get Elasticsearch endpoint
+   terraform output elasticsearch_endpoint
+
+   # Test connection
+   curl http://<ELASTIC_IP>:9200/_cluster/health
+   ```
+
+3. **Verify Logstash Pipeline**
+   ```bash
+   # Check Logstash status
+   ssh -i ~/.ssh/key.pem ec2-user@<LOGSTASH_IP>
+   sudo systemctl status logstash
+
+   # Test pipeline
+   echo '{"timestamp":"2024-01-01T12:00:00Z","srcip":"1.2.3.4"}' | 
+   curl -XPOST http://<LOGSTASH_IP>:5044 -H 'Content-Type: application/json'
+   ```
+
+4. **Verify Filebeat**
+   ```bash
+   # Check Filebeat on honeypot
+   ssh -i ~/.ssh/key.pem ec2-user@<HONEYPOT_IP>
+   sudo systemctl status filebeat
+
+   # Test log shipping
+   sudo tail -f /var/log/filebeat/filebeat
+   ```
+
+5. **Access Kibana Dashboard**
+   ```bash
+   # Get Kibana URL
+   terraform output kibana_endpoint
+
+   # Open in browser
+   http://<KIBANA_IP>:5601
+   ```
+
+### Expected Results
+
+- Elasticsearch cluster responds to health checks
+- Logstash processes Cowrie JSON logs correctly
+- Filebeat ships honeypot logs to Logstash
+- Kibana dashboard displays honeypot data
+- GeoIP enrichment shows attacker locations
+
+### Success Criteria
+
+- ✅ Elasticsearch cluster runs on port 9200
+- ✅ Logstash processes logs and forwards to Elasticsearch
+- ✅ Filebeat ships honeypot logs within 30 seconds
+- ✅ Kibana dashboard displays attack data
+- ✅ GeoIP enrichment shows geographic attack patterns
+
+### Dashboard Setup
+
+1. **Create Index Pattern**
+   - Pattern: `honeypot-logs-*`
+   - Time field: `@timestamp`
+
+2. **Visualizations**
+   - World Map: Attack source locations
+   - Timeline: Attack frequency over time
+   - Top IPs: Most active attackers
+   - Usernames: Most attempted credentials
+
+### Troubleshooting
+
+**Common Issues**:
+- Elasticsearch won't start: Check Java installation
+- No logs in Kibana: Verify Filebeat → Logstash → Elasticsearch flow
+- GeoIP not working: Install GeoIP database on Logstash
+
+This enables real-time attack visibility and threat intelligence extraction from honeypot logs.

docs/terraform-setup.md

@@ -0,0 +1,51 @@
+# Terraform Setup Guide
+
+## Prerequisites
+1. Install Terraform >= 1.0
+2. Configure AWS CLI with credentials
+3. Create SSH key pair in AWS
+
+## Setup Steps
+
+### 1. Initialize Terraform
+```bash
+cd infrastructure/terraform
+terraform init
+```
+
+### 2. Plan Deployment
+```bash
+terraform plan -var="key_name=your-key-name"
+```
+
+### 3. Apply Configuration
+```bash
+terraform apply -var="key_name=your-key-name" -auto-approve
+```
+
+### 4. Verify Deployment
+```bash
+# Get public IP
+terraform output public_ip
+
+# Test SSH connection
+ssh -i ~/.ssh/your-key.pem ec2-user@$(terraform output public_ip)
+```
+
+### 5. Cleanup (if needed)
+```bash
+terraform destroy -auto-approve
+```
+
+## Troubleshooting
+
+### Common Issues
+- **SSH Key Not Found**: Ensure key pair exists in AWS region
+- **Permission Denied**: Check SSH key permissions (chmod 400)
+- **Instance Not Accessible**: Verify security group allows SSH
+
+### Next Steps
+Once VM is accessible, proceed with:
+1. Honeypot installation
+2. Logging configuration
+3. Multi-region setup

infrastructure/terraform/main.tf

@@ -0,0 +1,34 @@
+terraform {
+  required_version = ">= 1.0"
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = "~> 5.0"
+    }
+  }
+
+  backend "local" {
+    path = "terraform.tfstate"
+  }
+}
+
+provider "aws" {
+  region = var.aws_region
+}
+
+# Call the VM module
+module "honeypot_vm" {
+  source = "./modules/vm"
+
+  instance_type = var.instance_type
+  ami_id       = var.ami_id
+  key_name     = var.key_name
+  subnet_id    = var.subnet_id
+
+  tags = {
+    Name        = "honeynet-platform-initial"
+    Project     = "honeynet-platform"
+    Environment = "development"
+    Purpose     = "initial-setup"
+  }
+}

infrastructure/terraform/modules/elasticsearch/main.tf

@@ -0,0 +1,49 @@
+# Elasticsearch Module
+resource "aws_instance" "elasticsearch" {
+  ami           = var.ami_id
+  instance_type = var.instance_type
+  key_name      = var.key_name
+  subnet_id     = var.subnet_id
+
+  vpc_security_group_ids = [aws_security_group.elasticsearch_sg.id]
+  associate_public_ip_address = false
+
+  tags = merge(var.tags, {
+    Name = "honeynet-elasticsearch"
+  })
+}
+
+# Security Group for Elasticsearch
+resource "aws_security_group" "elasticsearch_sg" {
+  name        = "elasticsearch-sg"
+  description = "Security group for Elasticsearch cluster"
+  vpc_id      = var.vpc_id
+
+  ingress {
+    description = "Elasticsearch HTTP"
+    from_port   = 9200
+    to_port     = 9200
+    protocol    = "tcp"
+    cidr_blocks = [var.vpc_cidr]
+  }
+
+  ingress {
+    description = "Elasticsearch transport"
+    from_port   = 9300
+    to_port     = 9300
+    protocol    = "tcp"
+    cidr_blocks = [var.vpc_cidr]
+  }
+
+  egress {
+    description = "Allow all outbound traffic"
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+
+  tags = {
+    Name = "elasticsearch-sg"
+  }
+}

infrastructure/terraform/modules/elasticsearch/outputs.tf

@@ -0,0 +1,14 @@
+output "elasticsearch_endpoint" {
+  description = "Elasticsearch endpoint URL"
+  value       = "http://${aws_instance.elasticsearch.private_ip}:9200"
+}
+
+output "elasticsearch_private_ip" {
+  description = "Private IP of Elasticsearch node"
+  value       = aws_instance.elasticsearch.private_ip
+}
+
+output "elasticsearch_id" {
+  description = "Instance ID of Elasticsearch"
+  value       = aws_instance.elasticsearch.id
+}

infrastructure/terraform/modules/elasticsearch/user_data.sh

@@ -0,0 +1,37 @@
+#!/bin/bash
+set -e
+
+# Update system
+yum update -y
+
+# Install Java (required for Elasticsearch)
+yum install -y java-1.8.0-openjdk
+
+# Install Elasticsearch
+yum install -y https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.11.1-x86_64.rpm
+
+# Configure Elasticsearch
+cat > /etc/elasticsearch/elasticsearch.yml << EOF
+cluster.name: honeynet-cluster
+node.name: ${HOSTNAME}
+network.host: 0.0.0.0
+http.port: 9200
+discovery.type: single-node
+
+# Memory settings
+bootstrap.memory_lock: true
+xpack.security.enabled: false
+
+EOF
+
+# Create data directory
+mkdir -p /var/lib/elasticsearch
+chown -R elasticsearch:elasticsearch /var/lib/elasticsearch
+
+# Start Elasticsearch
+systemctl enable elasticsearch
+systemctl start elasticsearch
+
+echo "Elasticsearch installed and configured!"
+echo "Node running on http://$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4):9200"
+echo "Cluster: honeynet-cluster"

infrastructure/terraform/modules/elasticsearch/variables.tf

@@ -0,0 +1,36 @@
+variable "ami_id" {
+  description = "AMI ID for Elasticsearch nodes"
+  type        = string
+}
+
+variable "instance_type" {
+  description = "Instance type for Elasticsearch"
+  type        = string
+  default     = "t3.medium"
+}
+
+variable "key_name" {
+  description = "SSH key name"
+  type        = string
+}
+
+variable "subnet_id" {
+  description = "Subnet ID for Elasticsearch"
+  type        = string
+}
+
+variable "vpc_id" {
+  description = "VPC ID"
+  type        = string
+}
+
+variable "vpc_cidr" {
+  description = "VPC CIDR block"
+  type        = string
+}
+
+variable "tags" {
+  description = "Tags to apply to resources"
+  type        = map(string)
+  default     = {}
+}