Run the preloader in a container in a VM. (#392)

* [preloader] initial implementation

* [preloader] run in a container

* [preloader] more preloader load

* [preloader] use ilb address output

* [preloader] add flags to configure src and start index

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h1/terragrunt.hcl

* [preloader] rename module

* [preloader] edit network tags

# Conflicts:
#	deployment/modules/gcp/vm/main.tf

* [preloader] use bigger VM

* [preloader] more load

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h1/terragrunt.hcl

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h1/terragrunt.hcl

* [preloader] break down preloader in its own module

# Conflicts:
#	deployment/modules/gcp/gce/tesseract/main.tf

# Conflicts:
#	deployment/modules/gcp/gce/tesseract/main.tf

# Conflicts:
#	deployment/modules/gcp/gce/tesseract/main.tf

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h1/terragrunt.hcl

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h2/terragrunt.hcl

# Conflicts:
#	deployment/modules/gcp/gce/tesseract/main.tf

# Conflicts:
#	deployment/modules/gcp/gce/tesseract/variables.tf

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h1/terragrunt.hcl

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h2/terragrunt.hcl

# Conflicts:
#	deployment/live/gcp/static-ct-staging/logs/arche2025h1/terragrunt.hcl
#	deployment/live/gcp/static-ct-staging/logs/arche2025h2/terragrunt.hcl
#	deployment/modules/gcp/vm/main.tf

* [preloader] fix a few things

* make machine type configurable

* [preloader] more nits

Author: phbnf

deployment/modules/gcp/gce/preloader/main.tf

@@ -0,0 +1,79 @@
+terraform {
+  required_providers {
+    google = {
+      source  = "registry.terraform.io/hashicorp/google"
+      version = "6.12.0"
+    }
+  }
+}
+
+locals {
+  cloudrun_service_account_id = var.env == "" ? "cloudrun-sa" : "cloudrun-${var.env}-sa"
+}
+
+module "gce_container_preloader" {
+  # https://github.com/terraform-google-modules/terraform-google-container-vm
+  source = "terraform-google-modules/container-vm/google"
+  version = "~> 2.0"
+
+  container = {
+    image = "us-central1-docker.pkg.dev/static-ct-staging/docker-staging/preloader@sha256:latest",
+    args = [
+      "--target_log_uri=${var.target_log_uri}",
+      "--source_log_uri=${var.source_log_uri}",
+      "--start_index=${var.start_index}",
+      "--num_workers=500", 
+      "--parallel_fetch=20", 
+      "--parallel_submit=500",
+    ]
+    tty : true # maybe remove this
+  }
+
+  restart_policy = "Always"
+}
+
+resource "google_compute_instance" "preloader" {
+  name         = "${var.base_name}-preloader"
+  machine_type = var.machine_type
+  zone         = "${var.location}-f"
+
+  tags = ["preloader-allow-group"]
+
+  boot_disk {
+    initialize_params {
+      image  = module.gce_container_preloader.source_image
+      labels = {
+        my_label = "value"
+      }
+    }
+  }
+
+  network_interface {
+    network = "default"
+  }
+
+  labels = {
+    environment = var.env
+    container-vm = module.gce_container_preloader.vm_container_label
+  }
+
+  # Come back to this: the start_index needs to be manually edited
+  # when the prelaoder restarts.
+  scheduling {
+    automatic_restart   = true
+    on_host_maintenance = "MIGRATE"
+  }
+  allow_stopping_for_update =  true
+
+  metadata = {
+    gce-container-declaration = module.gce_container_preloader.metadata_value
+    google-logging-enabled = "true"
+    google-monitoring-enabled = "true"
+  }
+
+  service_account {
+    # Google recommends custom service accounts that have cloud-platform scope and permissions granted via IAM Roles.
+    email = "${local.cloudrun_service_account_id}@${var.project_id}.iam.gserviceaccount.com" # TODO(phbnf): change this
+    scopes = ["cloud-platform"] # Allows using service accounts and OAuth.
+  }
+}

deployment/modules/gcp/gce/preloader/variables.tf

@@ -0,0 +1,50 @@
+variable "project_id" {
+  description = "GCP project ID where the log is hosted"
+  type        = string
+}
+
+variable "base_name" {
+  description = "Base name to use when naming resources"
+  type        = string
+}
+
+variable "origin_suffix" {
+  description = "Origin suffix, appended to base_name"
+  type        = string
+}
+
+variable "location" {
+  description = "Location in which to create resources"
+  type        = string
+}
+
+variable "env" {
+  description = "Unique identifier for the env, e.g. dev or ci or prod"
+  type        = string
+}
+
+variable "machine_type" {
+  description = "GCP Compute Engine machine type to run the TesseraCT container on"
+  type        = string
+  default     = "n2-standard-4"
+}
+
+variable "server_docker_image" {
+  description = "The full image URL (path & tag) for the Docker image to deploy in Cloud Run"
+  type        = string
+}
+
+variable "target_log_uri" {
+  description = "URL of the destination log to preload to."
+  type        = string
+}
+
+variable "source_log_uri" {
+  description = "URL of the source RFC6962 log."
+  type        = string
+}
+
+variable "start_index" {
+  description = "Index to start preloading from on the source log."
+  type        = number
+}