fix(ss): tolerate ScreenScraper's invalid JSON escapes#3778
Merged
Conversation
ScreenScraper occasionally returns malformed JSON with raw, unescaped
backslashes in text fields (e.g. synopses), which the strict parser
rejects with "Invalid \escape". The whole response was then discarded as
{}, so ScreenScraper found no match even for trivially-matchable titles
while other providers matched fine.
Parse leniently: on a decode error, double any backslash that isn't a
valid JSON escape and retry, instead of dropping the entire response.
Fixes #3764
Generated-By: PostHog Code
Task-Id: bbb1e2a8-bcb0-45a3-840e-86ce39f906a9
Contributor
Greptile SummaryThis PR adds tolerant JSON parsing for malformed ScreenScraper responses. The main changes are:
Confidence Score: 4/5The malformed Unicode escape path needs a fix before merging.
backend/adapters/services/screenscraper.py Important Files Changed
Reviews (1): Last reviewed commit: "fix(ss): tolerate ScreenScraper's invali..." | Re-trigger Greptile |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Explain the changes or enhancements you are proposing with this pull request.
ScreenScraper's API occasionally returns malformed JSON: some text fields (e.g. game synopses) contain raw, unescaped backslashes, producing sequences like
\ethat aren't valid JSON escapes. Python's strict parser (via aiohttp'sres.json()) rejects the entire response with:_request()caught theJSONDecodeErrorand returned{}, which reads as "no match". So ScreenScraper silently found nothing, even for trivially-matchable titles like Banjo-Kazooie, while IGDB/SGDB/Libretro (clean JSON) matched fine.Fix: parse leniently. On a decode error, double any backslash that isn't a valid JSON escape and retry, instead of discarding the whole response. Valid escapes (
\n,\",\uXXXX, etc.) are preserved; if the retry still fails, the existing{}fallback applies.Applied at both request sites via aiohttp's
res.json(loads=...)hook to keep the change minimal.Fixes #3764
Checklist
Please check all that apply.
Screenshots (if applicable)
AI assistance disclosure: This PR was written with AI assistance (PostHog Code / Claude). The root-cause analysis, code change, and tests were AI-generated and reviewed by a human before submission.
Created with PostHog Code