Skip to content

gcp: Cloud Run service does not set cpu_idle=false, causing cron jobs to be context-cancelled mid-run #242

Description

@robbiet480

Summary

The GCP Cloud Run service in gcp/byo-project/cloud_run.tf does not set cpu_idle = false on the container's resources block, so the deployed service runs with CPU throttling enabled (CPU only allocated during request processing). This causes Fleet's background cron goroutines — including apple_mdm_dep_profile_assigner, cleanups_then_aggregation, and others — to be terminated mid-run with context canceled errors. In the case of apple_mdm_dep_profile_assigner, this can silently drop DEP device events (see fleetdm/fleet#46235).

min_instance_count = 1 is already the default, which keeps a warm instance alive, but with cpu_idle = true (the Cloud Run v2 default when unset) that warm instance's CPU is frozen between requests, so the cron goroutines still can't run reliably.

Repro

  1. Deploy Fleet via this module with defaults.
  2. Tail the deployed service logs: gcloud --project <id> alpha run services logs tail fleet-api --region <region>.
  3. Filter for cron-related entries; observe repeated unlock failed: context canceled, update cron stats: context canceled, and pending job might still be running, wait 1m0s.
  4. With ABM ADE devices in flight, observe that some don't propagate to Fleet's host list.

Expected

The deployed Cloud Run service should run with cpu_idle = false (CPU always allocated), so background cron goroutines can run reliably between requests.

Suggested fix

Add cpu_idle = false to the resources block in gcp/byo-project/cloud_run.tf:

resources = {
  limits   = local.fleet_resources_limits
  cpu_idle = false
}

Optionally expose this as a variable defaulting to false for operators who want to override.

Fix

PR open: #243.

Related

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    🪺 Not yet

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions