Summary
The GCP Cloud Run service in gcp/byo-project/cloud_run.tf does not set cpu_idle = false on the container's resources block, so the deployed service runs with CPU throttling enabled (CPU only allocated during request processing). This causes Fleet's background cron goroutines — including apple_mdm_dep_profile_assigner, cleanups_then_aggregation, and others — to be terminated mid-run with context canceled errors. In the case of apple_mdm_dep_profile_assigner, this can silently drop DEP device events (see fleetdm/fleet#46235).
min_instance_count = 1 is already the default, which keeps a warm instance alive, but with cpu_idle = true (the Cloud Run v2 default when unset) that warm instance's CPU is frozen between requests, so the cron goroutines still can't run reliably.
Repro
- Deploy Fleet via this module with defaults.
- Tail the deployed service logs:
gcloud --project <id> alpha run services logs tail fleet-api --region <region>.
- Filter for cron-related entries; observe repeated
unlock failed: context canceled, update cron stats: context canceled, and pending job might still be running, wait 1m0s.
- With ABM ADE devices in flight, observe that some don't propagate to Fleet's host list.
Expected
The deployed Cloud Run service should run with cpu_idle = false (CPU always allocated), so background cron goroutines can run reliably between requests.
Suggested fix
Add cpu_idle = false to the resources block in gcp/byo-project/cloud_run.tf:
resources = {
limits = local.fleet_resources_limits
cpu_idle = false
}
Optionally expose this as a variable defaulting to false for operators who want to override.
Fix
PR open: #243.
Related
Summary
The GCP Cloud Run service in
gcp/byo-project/cloud_run.tfdoes not setcpu_idle = falseon the container'sresourcesblock, so the deployed service runs with CPU throttling enabled (CPU only allocated during request processing). This causes Fleet's background cron goroutines — includingapple_mdm_dep_profile_assigner,cleanups_then_aggregation, and others — to be terminated mid-run withcontext cancelederrors. In the case ofapple_mdm_dep_profile_assigner, this can silently drop DEP device events (see fleetdm/fleet#46235).min_instance_count = 1is already the default, which keeps a warm instance alive, but withcpu_idle = true(the Cloud Run v2 default when unset) that warm instance's CPU is frozen between requests, so the cron goroutines still can't run reliably.Repro
gcloud --project <id> alpha run services logs tail fleet-api --region <region>.unlock failed: context canceled,update cron stats: context canceled, andpending job might still be running, wait 1m0s.Expected
The deployed Cloud Run service should run with
cpu_idle = false(CPU always allocated), so background cron goroutines can run reliably between requests.Suggested fix
Add
cpu_idle = falseto theresourcesblock ingcp/byo-project/cloud_run.tf:Optionally expose this as a variable defaulting to
falsefor operators who want to override.Fix
PR open: #243.
Related
apple_mdm_dep_profile_assignercron silently drops device events when context-cancelled mid-run fleet#46235 — the underlying cron behavior that this exacerbates.