Skip to content

Deadlock: Event::delete locks per-monitor Event_Summaries before shared Storage row #4964

Description

@connortechnology

Symptom

InnoDB deadlocks on production between two batched Event::delete transactions:

Holds Waits for
TXN 1 (SELECT * FROM Storage WHERE Id=2 FOR UPDATE) Event_Summaries MonitorId=8 Storage Id=2
TXN 2 (Month delete trigger UPDATE Event_Summaries) Storage Id=2 Event_Summaries MonitorId=8

Root cause

scripts/ZoneMinder/lib/ZoneMinder/Event.pm delete() deletes the Events row first — the delete triggers cascade into the bucket tables and X-lock the per-monitor Event_Summaries row — and only afterwards calls $storage->lock_and_load() (SELECT ... Storage FOR UPDATE).

Storage is shared across monitors, but Event_Summaries is per-monitor. In a caller-managed batch (AutoCommit off) that deletes events spanning several monitors on one storage area, the coarse Storage lock ends up held while a finer Event_Summaries lock for another monitor is requested — the reverse of a concurrent batch's order. Classic ABBA cycle.

The Storage row was never included in the documented "canonical lock-acquisition order" (Events -> buckets -> Event_Summaries); lock_and_load was bolted on at the end, inverting it.

Fix

Lock the coarse shared Storage row first, before the event/bucket/Event_Summaries deletes, giving a consistent global order Storage -> Events -> buckets -> Event_Summaries. Replace the lock_and_load + save read-modify-write with a single atomic UPDATE Storage SET DiskSpace = GREATEST(COALESCE(DiskSpace,0) - ?, 0) WHERE Id = ?. Align MoveTo similarly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions