Skip to content

Dupe check misses exact duplicates when the resolved title year differs from the tracker's (e.g. 2002 vs 2003) #325

Description

@nitrobass24

Summary

An upload that was an exact duplicate of an existing tracker torrent — identical size to the byte, same release group, same type/resolution — passed the RF dupe check with no_match and was uploaded, because the release's resolved title year (2002) differed from the year in the tracker's existing entry (2003). The dupe criteria carry TMDB/IMDB provider IDs, but exact-identity matching is defeated by a ±1-year title discrepancy even on byte-identical content.

Year-off-by-one is a common, structural case (direct-to-video/regional titles where the filename year and the TMDB canonical year disagree), not an exotic edge.

Evidence (real occurrence, RF)

Release: 101 Dalmatians II - Patchs London Adventure (2002) (1080p DSNP WEB-DL H264 SDR DDP 5.1 English - HONE).mkv (single 4,821,745,127-byte file).

At upload time, metadata resolved the canonical name with year 2002 (filename-derived; that run also logged IMDB http 403 lookup failures):

metadata: 101 Dalmatians II: Patch's London Adventure 2002 1080p DSNP WEB-DL DD+ 5.1 H.264-HONE | TMDB=13654 IMDB=324941

The RF dupe check returned no blocking result and the upload proceeded (created torrent 124722) — despite RF already having torrent 89056: 101 Dalmatians II: Patch's London Adventure 2003 1080p DSNP WEB-DL DD+ 5.1 H.264-HONE, size 4,821,745,127 — the same release, titled with year 2003.

Re-running the dupe assessment later (same file, metadata now resolving year 2003), the engine immediately reports both torrents as exact duplicates — policyId: rf/duplicate/v2:

{"id": "89056",  "name": "101 Dalmatians II: Patch's London Adventure 2003 1080p DSNP WEB-DL DD+ 5.1 H.264-HONE",
 "relation": "exact_duplicate", "reasons": ["exact_identity"], "type": "WEB-DL", "resolution": "1080p", "sizeBytes": 4821745127}
{"id": "124722", "name": "101 Dalmatians II: Patch's London Adventure 2002 1080p DSNP WEB-DL DD+ 5.1 H.264-HONE",
 "relation": "exact_duplicate", "reasons": ["exact_identity"], "type": "WEB-DL", "resolution": "1080p", "sizeBytes": 4821745127}

So the matcher demonstrably classifies these as exact-identity matches — the only reason the original check missed 89056 was the 2002-vs-2003 year in the compared names. The search itself found the candidates (search.complete: true, candidateCount: 7, scope: work_category); this is a matching/identity decision, not a search-coverage gap.

The dupe criteria at check time included resolved provider IDs:

"providerIds": [{"provider": "tmdb", "value": "13654"}, {"provider": "imdb", "value": "324941"}]

yet identity matching still keyed on the year-bearing name.

Impact

Byte-identical re-uploads pass the automated dupe gate whenever the local year resolution disagrees with the tracker's listed year by one — resulting in true duplicates on the tracker (ours had to be reported for deletion). Unattended/batch workflows are especially exposed since no human reviews the candidate list.

Suggested direction

  • Treat candidates as identity matches when provider IDs (TMDB/IMDB) agree and the year differs by ≤1, rather than requiring the year-bearing names to align; and/or
  • Weigh exact size equality (sizeBytes identical) + same group + same media class as strong duplicate evidence that at minimum downgrades no_match to a reviewable duplicate outcome instead of a clean pass.

Related context: the year flip itself came from filename-derived year vs TMDB canonical year (with IMDB scrape 403s degrading enrichment at the time) — but even with perfect metadata, the tracker's own listed year can disagree with TMDB, so tolerance belongs in the matcher.

Update: naming-precedence root cause located (current main)

The divergent year is not only a metadata-history artifact — the name builder actively prefers the filename year over matched provider metadata. resolveReleaseNameTitle (internal/metadata/naming.go:508-548):

year := meta.Release.Year                    // year parsed from the filename
...
case matchingTMDBMetadataForNaming(meta):
    title = strings.TrimSpace(tmdb.Title)    // TMDB title wins unconditionally
    if year == 0 && tmdb.Year > 0 {          // TMDB year is only a fallback
        year = tmdb.Year
    }

The asymmetry is what produced this incident: the title is always canonicalized from TMDB, but the year from the filename outranks TMDB whenever the filename contains one. In our case TMDB had listed this movie as 2002 historically and later corrected it to 2003 (the Radarr folder — frozen at import — still says 2002 while Radarr's current record and renamed file say 2003), so every filename minted under the old metadata carries 2002, and any of them uploaded through upbrr gets a canonical name that disagrees with the tracker's TMDB-current title year.

So there are two complementary fixes:

  1. Matcher (original suggestion): identity via provider IDs with ±1-year tolerance, or exact-size+group+class as reviewable duplicate evidence.
  2. Naming precedence: when TMDB metadata is matched for naming, the provider year should win over the filename-parsed year (matching the way the title is already handled) — or at minimum, a filename-vs-provider year disagreement should surface as a warning, since it predicts exactly this dupe-check blind spot.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions