Skip to content

fix(ss): tolerate ScreenScraper's invalid JSON escapes#3778

Merged
gantoine merged 2 commits into
masterfrom
posthog-code/fix-ss-invalid-json-escape
Jul 17, 2026
Merged

fix(ss): tolerate ScreenScraper's invalid JSON escapes#3778
gantoine merged 2 commits into
masterfrom
posthog-code/fix-ss-invalid-json-escape

Conversation

@gantoine

Copy link
Copy Markdown
Member

Description
Explain the changes or enhancements you are proposing with this pull request.

ScreenScraper's API occasionally returns malformed JSON: some text fields (e.g. game synopses) contain raw, unescaped backslashes, producing sequences like \e that aren't valid JSON escapes. Python's strict parser (via aiohttp's res.json()) rejects the entire response with:

[screenscraper] Error decoding JSON response from ScreenScraper: Invalid \escape: line 26 column 11

_request() caught the JSONDecodeError and returned {}, which reads as "no match". So ScreenScraper silently found nothing, even for trivially-matchable titles like Banjo-Kazooie, while IGDB/SGDB/Libretro (clean JSON) matched fine.

Fix: parse leniently. On a decode error, double any backslash that isn't a valid JSON escape and retry, instead of discarding the whole response. Valid escapes (\n, \", \uXXXX, etc.) are preserved; if the retry still fails, the existing {} fallback applies.

Applied at both request sites via aiohttp's res.json(loads=...) hook to keep the change minimal.

Fixes #3764

Note: the PlayMatch error in the same issue is tracked separately and already fixed for the next release. The issue also mentions LaunchBox, but there was no LaunchBox error in the reporter's logs to diagnose.

Checklist
Please check all that apply.

  • I've tested the changes locally
  • I've updated relevant comments
  • I've assigned reviewers for this PR
  • I've added unit tests that cover the changes

Screenshots (if applicable)

AI assistance disclosure: This PR was written with AI assistance (PostHog Code / Claude). The root-cause analysis, code change, and tests were AI-generated and reviewed by a human before submission.


Created with PostHog Code

ScreenScraper occasionally returns malformed JSON with raw, unescaped
backslashes in text fields (e.g. synopses), which the strict parser
rejects with "Invalid \escape". The whole response was then discarded as
{}, so ScreenScraper found no match even for trivially-matchable titles
while other providers matched fine.

Parse leniently: on a decode error, double any backslash that isn't a
valid JSON escape and retry, instead of dropping the entire response.

Fixes #3764

Generated-By: PostHog Code
Task-Id: bbb1e2a8-bcb0-45a3-840e-86ce39f906a9
Copilot AI review requested due to automatic review settings July 16, 2026 22:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds tolerant JSON parsing for malformed ScreenScraper responses. The main changes are:

  • Repairs invalid backslash escapes before retrying JSON parsing.
  • Uses the tolerant loader in both ScreenScraper request paths.
  • Adds tests for malformed backslashes and valid JSON escapes.

Confidence Score: 4/5

The malformed Unicode escape path needs a fix before merging.

  • A raw backslash followed by u is preserved without checking for four hexadecimal digits.
  • The retry still fails for values such as C:\users, causing the response to be discarded as {}.
  • Valid JSON and ordinary invalid escapes follow the intended path.

backend/adapters/services/screenscraper.py

Important Files Changed

Filename Overview
backend/adapters/services/screenscraper.py Adds the tolerant loader and integrates it into both request attempts, but incomplete Unicode escapes are incorrectly preserved.
backend/tests/adapters/services/test_screenscraper.py Adds direct coverage for ordinary invalid backslashes and valid escapes, but omits malformed Unicode escape prefixes.

Fix All in Claude Code

Reviews (1): Last reviewed commit: "fix(ss): tolerate ScreenScraper's invali..." | Re-trigger Greptile

Comment thread backend/adapters/services/screenscraper.py Outdated
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@gantoine
gantoine merged commit f771f11 into master Jul 17, 2026
8 of 10 checks passed
@gantoine
gantoine deleted the posthog-code/fix-ss-invalid-json-escape branch July 17, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] ScreenScraper, LaunchBox, and PlayMatch not matching/working

2 participants