Skip to content

perf(mdm): Route all sync/push actions through Dagster to prevent request hangs#136

Open
Copilot wants to merge 5 commits intomainfrom
copilot/update-sync-actions-to-dagster
Open

perf(mdm): Route all sync/push actions through Dagster to prevent request hangs#136
Copilot wants to merge 5 commits intomainfrom
copilot/update-sync-actions-to-dagster

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 1, 2026

Synchronous MDM API calls in the request/response cycle (sync fleet, push device config, push policy child devices) block for orgs with many devices. Dagster was already wired up for device CSV imports in apps/mdm/ but five other call sites remained synchronous.

New Dagster infrastructure

  • sync_and_push_mdm_devices asset — accepts a fleet_pks: list[int] config and calls sync_fleet() for each specified fleet with push_config=True
  • sync_fleets_job — new job targeting the asset, used for the devices-list sync button

Updated call sites

All five sites now follow the same pattern: trigger Dagster when dagster_enabled(), fall back to synchronous behavior when it isn't (local dev / no DAGSTER_URL).

Site Async action
devices_list "Sync" button trigger_dagster_job("sync_fleets_job", {fleet_pks: [...]}) with the current org's fleet PKs → "Sync queued" message
device_update_app_user HTMX trigger_dagster_job("mdm_job", {device_pks: [pk]})
publish_mdm/import_export.py after_import trigger_dagster_job("mdm_job", {device_pks: [...]})
mdm/views.py _push_policy_to_mdm Base policy push stays synchronous; child-device pushes queued via mdm_job
mdm/admin.py PolicyAdmin.save_related Same split: base policy sync, per-device child policies queued

The sync_fleets_job is scoped to the organization whose sync button was pressed — only that org's fleet PKs are passed in the run config, matching the non-Dagster per-fleet loop behaviour.

# Pattern used consistently across all sites
if dagster_enabled():
    run_config = {"ops": {"push_mdm_device_config": {"config": {"device_pks": device_pks}}}}
    trigger_dagster_job(job_name="mdm_job", run_config=run_config)
else:
    for device in child_devices:
        active_mdm.push_device_config(device)

Tests

New test classes added for each site asserting:

  • Dagster path: job triggered with correct PKs (fleet or device), no direct MDM calls
  • Fallback path: synchronous MDM calls made, Dagster not triggered

Copilot AI changed the title [WIP] Refactor sync actions to use Dagster for MDM requests perf(mdm): Route all sync/push actions through Dagster to prevent request hangs Apr 1, 2026
Copilot AI requested a review from simonkagwi April 1, 2026 20:44
@simonkagwi
Copy link
Copy Markdown
Contributor

@copilot The sync_fleets_job Dagster job should be able to sync specific fleets, instead of always syncing all fleets. When triggered from the devices list page, it should only sync fleets related to the current organization, similar to the non-Dagster flow

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 2, 2026

@copilot The sync_fleets_job Dagster job should be able to sync specific fleets, instead of always syncing all fleets. When triggered from the devices list page, it should only sync fleets related to ...

Done in eb27cc1. The sync_and_push_mdm_devices asset now accepts a fleet_pks: list[int] config and iterates through only those specific fleets using sync_fleet() — matching the non-Dagster per-fleet loop. When triggered from the devices list page, the view now passes request.organization.fleets.values_list("pk", flat=True) in the run config so only the org's fleets are synced.

@simonkagwi simonkagwi marked this pull request as ready for review April 2, 2026 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(mdm): Route all sync/push actions through Dagster to prevent request hangs

2 participants