Monte Carlo Google OpenTelemetry Collector Module

A Terraform module that deploys Monte Carlo's OpenTelemetry Collector Service on Google Cloud Run.

Architecture

This module creates:

• Cloud Run service running the OpenTelemetry Collector
• VPC Access Connector for internal networking
• Service account with appropriate IAM permissions
• Pub/Sub topic and subscription for trace data export from Collector
• BigQuery integration for storing trace data (optional)

Prerequisites

• Terraform >= 1.0
• Google Cloud CLI configured with appropriate permissions
• Existing VPC network
• BigQuery dataset (optional, for storing telemetry data)

Usage

Basic Example

module "otel_collector" {
  source = "monte-carlo-data/terraform-google-otel-collector"

  project_id      = "my-gcp-project"
  deployment_name = "my-otel-collector"
  region          = "us-central1"
  vpc_network     = "projects/my-gcp-project/global/networks/my-vpc"
  vpc_subnet      = "10.8.0.0/28"
}

Advanced Example

module "otel_collector" {
  source = "monte-carlo-data/terraform-google-otel-collector"

  # Required variables
  project_id      = "my-gcp-project"
  deployment_name = "production-otel"
  region          = "us-central1"
  vpc_network     = "projects/my-gcp-project/global/networks/my-vpc"

  # Use existing VPC connector
  existing_vpc_connector = "projects/my-gcp-project/locations/us-central1/connectors/my-connector"

  # Optional customizations
  min_instances = 2
  max_instances = 20
  cpu           = "2"
  memory        = "4Gi"

  # BigQuery integration
  bigquery_table_id = "my-project.my_dataset.otel_traces"

  # OTEL configuration
  batch_timeout          = "15s"
  batch_size             = 2048
  memory_limit_mib       = 3000
  memory_spike_limit_mib = 1024
}

Requirements

Name	Version
terraform	>= 1.0
google	~> 6.0

Providers

Name	Version
google	~> 6.0

Resources

Name	Type
google_cloud_run_v2_service.otel_collector	resource
google_service_account.otel_collector	resource
google_vpc_access_connector.otel_connector	resource
google_pubsub_topic.otel_collector_traces_topic	resource
google_pubsub_subscription.otel_collector_traces_subscription	resource
google_project_iam_member.pubsub_publisher	resource
google_project_iam_member.bigquery_data_editor	resource
google_project.project	data source

Inputs

Name	Description	Type	Default	Required
deployment_name	Name prefix for all resources created by this module (max 20 chars due to VPC connector name limits)	string	n/a	yes
project_id	GCP project ID where resources will be created	string	n/a	yes
region	GCP region for Cloud Run deployment	string	n/a	yes
vpc_network	VPC network name or self-link for internal access	string	n/a	yes
batch_size	Batch processor send_batch_size	number	1024	no
batch_timeout	Batch processor timeout (e.g., '10s', '1m')	string	"10s"	no
bigquery_table_id	BigQuery table ID for storing telemetry data (format: project.dataset.table). If provided, creates a Pub/Sub subscription with BigQuery integration.	string	null	no
concurrency	Maximum number of concurrent requests per Cloud Run instance	number	80	no
container_image	Docker image for the OpenTelemetry Collector	string	"otel/opentelemetry-collector-contrib:latest"	no
cpu	CPU allocation for Cloud Run container (e.g., '1' for 1 vCPU, '2' for 2 vCPUs)	string	"1"	no
deletion_protection	Enable deletion protection for Cloud Run service (recommended for production)	bool	true	no
existing_vpc_connector	ID of an existing VPC connector to use. If not provided, a new connector will be created.	string	null	no
grpc_port	Port for OTLP gRPC receiver	number	4317	no
http_port	Port for OTLP HTTP receiver	number	4318	no
labels	Labels to apply to all resources	map(string)	{}	no
max_instances	Maximum number of Cloud Run instances	number	10	no
memory	Memory allocation for Cloud Run container (e.g., '512Mi', '2Gi', '4Gi')	string	"2Gi"	no
memory_limit_mib	Memory limiter limit in MiB	number	1500	no
memory_spike_limit_mib	Memory limiter spike limit in MiB	number	512	no
min_instances	Minimum number of Cloud Run instances (0 for scale-to-zero)	number	1	no
service_account_email	Email of an existing service account to use for Cloud Run. If not provided, a new service account will be created.	string	null	no
timeout_seconds	Request timeout in seconds (max 3600 for Cloud Run 2nd gen)	number	300	no
vpc_subnet	Subnet CIDR range for VPC connector (e.g., '10.8.0.0/28'). Required if existing_vpc_connector is not provided.	string	null	no

Outputs

Name	Description
otel_collector_grpc_endpoint	The gRPC endpoint for the OpenTelemetry Collector
otel_collector_http_endpoint	The HTTP endpoint for the OpenTelemetry Collector
otel_collector_service_account_email	Email of the service account used by the Cloud Run service
otel_collector_service_name	Name of the Cloud Run service
otel_collector_service_url	Cloud Run service URL for the OpenTelemetry Collector
otel_collector_traces_subscription_id	Full ID of the Pub/Sub subscription for traces (if created)
otel_collector_traces_subscription_name	Name of the Pub/Sub subscription for traces (if created)
otel_collector_traces_topic_id	Full ID of the Pub/Sub topic for traces
otel_collector_traces_topic_name	Name of the Pub/Sub topic for traces
otel_config	OpenTelemetry Collector configuration (YAML)
vpc_connector_id	ID of the VPC connector (if created by this module)
vpc_connector_name	Name of the VPC connector (if created by this module)

Releases and Development

The README and basic example in the examples/basic directory is a good starting point to familiarize yourself with using the module.

Note that all Terraform files must conform to the standards of terraform fmt and the standard module structure. CircleCI will sanity check formatting and for valid tf config files. It is also recommended you use Terraform Cloud as a backend. Otherwise, as normal, please follow Monte Carlo's code guidelines during development and review.

When ready to release simply add a new version tag, e.g. v0.0.42, and push that tag to GitHub. See additional details here.

License

See LICENSE for more information.

Security

See SECURITY for more information.