Skip to content

improvement: Add graceful handling for disk-full conditions #2204

@tphakala

Description

@tphakala

Summary

When the database disk is full, multiple subsystems fail simultaneously with cascading errors. There is no proactive disk space check or graceful degradation.

Sentry References

  • BIRDNET-GO-SW: Alerting save_history fails (5 events)
  • BIRDNET-GO-SX: FFmpeg audio export broken pipe / no space left (33 events, escalating)
  • BIRDNET-GO-SZ: Save threshold event fails (4 events)
  • BIRDNET-GO-T0: Save threshold event fails (1 event)
  • BIRDNET-GO-T1: SQL query fails (1 event)
  • BIRDNET-GO-T2: Save hourly weather fails (1 event)
  • BIRDNET-GO-SY: Commit transaction fails (1 event)

All from the same category: database or disk is full (SQLite sqlite3.Error).

Error Details

[database] database or disk is full
[system-resource] failed to write PCM data to FFmpeg: No space left on device

Impact

When disk fills up, at least 7 different error types fire simultaneously, creating noise. The system continues to attempt writes that will fail.

Fix Suggestions

  1. Proactive monitoring: Check available disk space periodically and warn before it's critically low
  2. Circuit breaker: When a disk-full error is detected, temporarily pause non-critical writes (weather, alerting history) to reduce error noise
  3. Diskmanager integration: The diskmanager should proactively clean up old recordings before disk fills completely
  4. Single error reporting: Deduplicate the disk-full condition into a single high-priority alert rather than per-operation errors
  5. User notification: Surface a clear UI warning when disk space is critically low

Related Code

  • internal/datastore/ — all database write operations
  • internal/myaudio/ — FFmpeg audio export
  • internal/diskmanager/ — disk space management

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions