Monte Carlo AWS OpenTelemetry Collector Module

A Terraform module that deploys Monte Carlo's OpenTelemetry Collector Service on AWS ECS Fargate.

Architecture

This module creates:

• ECS Fargate cluster and service (optional)
• Network Load Balancer with gRPC and HTTP listeners (optional)
• Security groups and IAM roles (optional)
• CloudWatch log group (optional)
• External access role for S3 bucket access
• S3 bucket policy allowing collector writes

Prerequisites

• Terraform >= 1.9.0
• AWS CLI configured with appropriate permissions
• Existing VPC with at least 2 private subnets (only if deploying the collector)
• S3 bucket for storing telemetry data (optional, created when not provided)

Usage

Basic Example

module "otel_collector" {
  source = "monte-carlo-data/otel-collector/aws"

  deployment_name           = "my-otel-collector"
  existing_vpc_id           = "vpc-12345678"
  existing_subnet_ids       = ["subnet-12345678", "subnet-87654321"]
  telemetry_data_bucket_arn = "arn:aws:s3:::my-telemetry-bucket"
}

Storage-Only Example

module "otel_collector" {
  source = "monte-carlo-data/otel-collector/aws"

  deployment_name              = "my-data-store"
  deploy_otel_collector        = false
  mcd_otel_collector_task_role_arn = "arn:aws:iam::123456789012:role/my-collector-role"
  vpc_endpoint_id              = "vpce-1234567890abcdef"
}

Advanced Example

module "otel_collector" {
  source = "monte-carlo-data/otel-collector/aws"

  # Required variables
  deployment_name           = "production-otel-collector"
  existing_vpc_id           = "vpc-12345678"
  existing_subnet_ids       = ["subnet-12345678", "subnet-87654321"]
  telemetry_data_bucket_arn = "arn:aws:s3:::my-telemetry-bucket"

  # Optional customizations
  existing_security_group_id = "sg-12345678"
  task_desired_count         = 3
  task_cpu                   = 2048
  task_memory                = 4096

  # External access configuration
  external_id                        = "secure-random-string"
  external_access_principal          = "arn:aws:iam::123456789012:root"
  external_access_principal_type     = "AWS"
}

Migration Notes

This release refactors resources into submodules. To avoid resource recreation for existing users, the module includes moved blocks that map old resource addresses to their new module addresses. Run terraform apply as usual and Terraform will migrate state automatically.

If you cannot use moved blocks (older Terraform versions), you will need to perform terraform state mv operations that mirror the mappings in moved.tf.

Requirements

Name	Version
terraform	>= 1.0
aws	~> 5.0

Providers

Name	Version
aws	~> 5.0

Resources

Name	Type
aws_cloudwatch_log_group.log_group	resource
aws_ecs_cluster.ecs_cluster	resource
aws_ecs_service.ecs_service	resource
aws_ecs_task_definition.task_definition	resource
aws_iam_policy.external_access_s3_read_only_policy	resource
aws_iam_role.external_access_role	resource
aws_iam_role.task_execution_role	resource
aws_iam_role.task_role	resource
aws_iam_role_policy.task_role_s3_policy	resource
aws_iam_role_policy_attachment.external_access_policy_attachment	resource
aws_iam_role_policy_attachment.task_execution_role_policy	resource
aws_lb.network_load_balancer	resource
aws_lb_listener.listener_grpc	resource
aws_lb_listener.listener_http	resource
aws_lb_target_group.target_group_grpc	resource
aws_lb_target_group.target_group_http	resource
aws_security_group.security_group	resource
aws_security_group_rule.security_group_ingress	resource

Inputs

Name	Description	Type	Default	Required
deployment_name	Name of the deployment (used for naming resources)	string	n/a	yes
existing_subnet_ids	List of private subnet IDs (at least 2) for deploying the OpenTelemetry Collector.	list(string)	[]	no
existing_vpc_id	VPC ID to deploy the OpenTelemetry Collector into.	string	"N/A"	no
telemetry_data_bucket_arn	ARN of the S3 bucket to store OpenTelemetry data such as traces, metrics, and logs. If omitted, a bucket named ${deployment_name}-otel-collector is created.	string	""	no
batch_size	Batch size for sending telemetry data	number	1024	no
batch_timeout	Timeout for batch processor in seconds	string	"10s"	no
container_image	OpenTelemetry Collector container image	string	"otel/opentelemetry-collector-contrib:latest"	no
deploy_otel_collector	Whether to deploy the OpenTelemetry Collector infrastructure (ECS, NLB, IAM, etc.)	bool	true	no
existing_security_group_id	Optional additional security group ID to attach to the OpenTelemetry Collector resources.	string	"N/A"	no
external_access_principal	Principal (AWS ARN/account ID or Federated identifier) allowed to assume the external access role.	string	"N/A"	no
external_access_principal_type	Type of principal for external access role	string	"AWS"	no
external_access_role_name	Custom name of the external access role. If left empty, will use the default name.	string	"N/A"	no
external_id	External ID to access the S3 bucket. Update this value later after the stack is created.	string	"N/A"	no
grpc_port	Port for OTLP gRPC receiver	number	4317	no
http_port	Port for OTLP HTTP receiver	number	4318	no
mcd_otel_collector_task_role_arn	ARN of the role that should be granted write access to the telemetry S3 bucket.	string	""	no
memory_limit_mib	Memory limit for the collector in MiB	number	1500	no
memory_spike_limit_mib	Memory spike limit for the collector in MiB	number	512	no
task_cpu	CPU units for the task (1024 = 1 vCPU)	number	1024	no
task_desired_count	Desired number of running tasks for the OpenTelemetry Collector service	number	2	no
task_memory	Memory for the task in MB	number	2048	no
vpc_endpoint_id	Optional VPC endpoint ID to restrict S3 writes to that endpoint.	string	""	no

Outputs

Name	Description
opentelemetry_collector_external_access_role_arn	The ARN of the IAM role for external access to the OpenTelemetry S3 bucket
opentelemetry_collector_grpc_endpoint	The gRPC endpoint for the OpenTelemetry Collector (null when deploy_otel_collector is false)
opentelemetry_collector_http_endpoint	The HTTP endpoint for the OpenTelemetry Collector (null when deploy_otel_collector is false)
opentelemetry_collector_security_group_id	The ID of the security group for the OpenTelemetry Collector (null when deploy_otel_collector is false)
telemetry_data_bucket_arn	The ARN of the telemetry S3 bucket (created or provided)

Post-Deployment Configuration

After deployment, update the external access configuration:

1. Set external_id to a secure random value
2. Set external_access_principal to the appropriate AWS account or federated identity
3. Run terraform apply again to update the external access role

Releases and Development

The README and basic example in the examples/basic directory is a good starting point to familiarize yourself with using the module.

Note that all Terraform files must conform to the standards of terraform fmt and the standard module structure. CircleCI will sanity check formatting and for valid tf config files. It is also recommended you use Terraform Cloud as a backend. Otherwise, as normal, please follow Monte Carlo's code guidelines during development and review.

When ready to release simply add a new version tag, e.g. v0.0.42, and push that tag to GitHub. See additional details here.

License

See LICENSE for more information.

Security

See SECURITY for more information.