Skip to content

N+1 query in sync_vendor_data - batch data_source lookups and event_record inserts #492

@bartmichalak

Description

@bartmichalak

sync_vendor_data_task executes the same SELECT … FROM data_source query once per record during sync, resulting in an N+1 problem. In a single transaction the query pattern repeats ~85 times alongside individual INSERT INTO event_record and INSERT INTO event_record_detail statements.

Impact: 255 occurrences detected on our internal instance since Feb 6. Adds unnecessary DB load on every vendor data sync.

Fix: Pre-fetch/cache data_source lookups before the loop and batch event_record / event_record_detail inserts using bulk_save_objects or executemany.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions