Skip to content

DB Write sanitizer - #822

Merged
NeptuneHub merged 3 commits into
mainfrom
devel
Jul 30, 2026
Merged

DB Write sanitizer#822
NeptuneHub merged 3 commits into
mainfrom
devel

Conversation

@NeptuneHub

@NeptuneHub NeptuneHub commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Strip NUL bytes from media-server metadata on all map and id write paths

Album analysis crashed with psycopg2 "A string literal cannot contain NUL (0x00) characters" when an artist tag carried a NUL byte, burning all RQ retries on the same album. The score table already sanitized tag text, but the registry map upserts bound it raw.

Route every such bind site through the existing sanitize_string_for_db:

  • registry: upsert_artist_maps / upsert_track_maps clean names, ids and paths BEFORE the dedup keys are built (cleaning after dedup could turn the NUL crash into an ON CONFLICT cardinality error), drop entries than clean to empty, and log each sanitized value loudly
  • sweep: _strip_nul now delegates to the shared sanitizer and the prune's present-id staging applies it too, so stored and staged ids always compare equal and pruning cannot churn; metadata staging keys rows by the sanitized id
  • chromaprint persist/lookup and the migration id map sanitize provider ids the same way for join and lookup parity

Covered by test/unit/test_nul_sanitization_maps.py.

PR test builds:

…itize map upserts, sweep staging, chromaprint and migration ids with the shared sanitizer
…alysis/duration lookups, keep DB-sourced ids raw, scrub legacy dirty map rows
@sonarqubecloud

Copy link
Copy Markdown

@NeptuneHub
NeptuneHub merged commit 82e64f5 into main Jul 30, 2026
17 of 27 checks passed
@NeptuneHub
NeptuneHub deleted the devel branch July 30, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant