Skip to content

Add PlayStation Network as an import source - #824

Merged
ryan-winkler merged 17 commits into
dannyvfilms:latestfrom
Revisor01:feature/psn-integration
Aug 17, 2026
Merged

Add PlayStation Network as an import source#824
ryan-winkler merged 17 commits into
dannyvfilms:latestfrom
Revisor01:feature/psn-integration

Conversation

@Revisor01

@Revisor01 Revisor01 commented Aug 16, 2026

Copy link
Copy Markdown

Summary

Adds PSN alongside the existing Xbox and Steam importers. Connect with an NPSSO token, pull played PS4/PS5 titles via psnawp, match them to IGDB and write cumulative playtime into Game.progress — the same shape as imports/xbox.py, which this is modelled on throughout.

On my own account: 134 played titles → 19 non-games filtered out → 115 games, all 115 matched to IGDB → 103 Game rows (12 titles are the same game under PS4 and PS5 SKUs, so their playtime is summed). Final Fantasy VII Rebirth lands at 4900 minutes, which is exactly what PSN reports.

Non-game filtering was the interesting part. PSN's category field doesn't separate apps from games — Netflix and Cyberpunk 2077 both come back as UNKNOWN. Entitlements looked promising but fail badly: God of War Ragnarök and ELDEN RING have no entitlement at all (PS Plus / disc), so they'd vanish. What does work is the store concept metadata — games carry genres, apps return an empty list. Verified against all 42 UNKNOWN titles on my account: 23 games, 19 apps, no misclassification, including Genshin Impact (F2P, correctly a game) and two VR short films (correctly not). A failed lookup keeps the title rather than dropping it, and skipped titles are surfaced as an import warning.

Sync frequency matches Xbox: daily or every two days, no polling. PSN's playtime is cumulative, so a missed run costs nothing.

Two things worth flagging before you find them yourself:

  • psnawp has to be pinned to 2.0.0. 3.x needs pyrate-limiter>=4, our requests-ratelimiter==0.8.0 needs <3; 2.1/2.2 want requests-ratelimiter<0.8.0. 2.0.0 is the only version that resolves — and it pins attrs==23.1.0 exactly, which pulls attrs back from 26.1.0 project-wide. Dependents (aiohttp, jsonschema, referencing) all still satisfy their constraints and the full suite passes, but it is a three-year downgrade as a side effect of a game importer. If that is unacceptable, [tool.uv] override-dependencies = ["attrs==26.1.0"] overrides psnawp's pin — say the word and I'll add it.
  • psnawp 2.0.0 sends requests with no timeout. A hanging PSN response would block a Celery worker indefinitely, so the client injects a 30s default at the request builder.

Two refactor commits precede the feature: the store-agnostic title normalisation is extracted from xbox.py into imports/title_matching.py, and the Xbox schedule helpers are generalised for console reuse — both behaviour-preserving, Xbox suite unchanged and green.

AI Assistance

I used Claude Code for this — claude-opus-5 for the code, claude-fable-5 for review. Two review rounds found real bugs (progress could be lowered by a partial IGDB failure or a zero-duration title; skipped titles were invisible; a status guard raced), all fixed here, each with a test I verified fails without the fix. The approach, the API probing and the live testing are mine.

Validation

SECRET=test-only scripts/test.sh integrations users app.tests.test_log_safety
  → Ran 1155 tests, OK
SECRET=test-only scripts/test.sh integrations.tests.imports.test_xbox
  → Ran 58 tests, OK (refactor is behaviour-preserving)
uv run --no-sync ruff check src
  → All checks passed!

Model change, so also: makemigrations --check clean, migrate --plan includes 0024_psnaccount, check_migration_hygiene --strict passes against latest.

Running on my own instance against the live PSN API — the screenshots are from that.

Contract Handoff

  • Domain guide regeneration/check outcome: domain_vocabulary --check passes, no regeneration needed
  • Verified OpenAPI regeneration outcome: regenerated, no diff — src/api/ is untouched
  • Contract-test outcome: 57 tests pass (test_about, test_api_contracts, test_domain_vocabulary)

Human Review

  • Pending human review.

Gstack QA

  • Pending /gstack-qa.

Notes

The PlayStation tile logo is drawn as its own paths rather than shipping Sony's asset, matching how xbox-logo.svg is done here.

Two things I wasn't sure about and deliberately left out, both your call rather than mine:

Per-platform playtime. PSN reports a game's PS4 and PS5 editions under separate title IDs, and I sum them into one figure. Keeping them apart — per-platform hours plus a combined total — would be more useful, especially once you can play the same game on Xbox, Steam and PlayStation and want to see where the hours actually went. But that is a model change touching all three importers, not something to smuggle in with one of them. Happy to spec it out separately.

A live "currently playing" card. psnawp exposes presence, so showing what is running right now is possible. Sony sends no webhooks though, so it would need polling every ~30s — the first polling of its kind in Floppy, against an unofficial API whose own library warns about heavy use. That felt like an architecture decision rather than a feature I should just build, so I left it alone.

@Revisor01
Revisor01 force-pushed the feature/psn-integration branch from 3213d26 to 8b535f7 Compare August 16, 2026 19:00
@Revisor01

Copy link
Copy Markdown
Author
screen

@Revisor01
Revisor01 force-pushed the feature/psn-integration branch from 2b2b8f9 to af88a11 Compare August 16, 2026 21:38
ryan-winkler pushed a commit that referenced this pull request Aug 17, 2026
Makes the SQLite entrypoint parking test deterministic.

The test starts the real entrypoint.sh with stub wrappers on PATH, then waits for
the parking sleep to write a pidfile. It never reached that sleep in CI, because
sqlite_recovery_server.serve() binds port 8000 and calls serve_forever() when the
bind succeeds. The test only ever passed where port 8000 was already occupied,
which is true on a developer machine running Floppy and false on a runner.

Stubbing `python -m config.sqlite_recovery_server` in the wrapper reaches the
fallback parking loop deterministically, and matching on "$*" rather than "$2"
is what makes both the -c and -m invocations reachable.

Also isolates VIRTUAL_ENV for both entrypoint subprocess tests. entrypoint.sh
keeps a virtual environment first on PATH unless it is the one at $PWD/.venv, so
a venv anywhere else put the real python ahead of the test's wrappers and the
stubs never fired.

Measured with port 8000 free and VIRTUAL_ENV pointing outside the working
directory: both entrypoint tests went from failing to passing, and the config
suite is green at 121 tests.

Closes #825. Unblocks #824, #829, #830 and #833, which were all failing on this
single test.
@ryan-winkler

Copy link
Copy Markdown

Review & Status Check

Great work on the PlayStation Network import source! The architecture cleanly matches the console import patterns established by the Xbox importer (#642), with solid rate-limiting, non-game concept filtering, and comprehensive test coverage across all 1,155 integration tests.

CI Status Note

The test failure in GitHub Actions was caused by an upstream SQLite parking test flake that has since been hardened on latest in #826. Rebasing your branch onto current latest will turn CI green.

- pinned to psnawp 2.0.0: the app's requests-ratelimiter pin needs
  pyrate-limiter<3 while psnawp 3.x requires >=4
- moves the store-agnostic pieces to integrations.imports.title_matching
  so other console importers can reuse them
- the Xbox-specific store suffix pattern stays in xbox.py
- psn_api wraps psnawp and constructs user-facing error messages;
  raw exception strings never reach the account row
- non-games are filtered by store concept genres: PSN leaves apps and
  some games uncategorised, and only games carry genres
- play durations aggregate across title IDs that resolve to the same
  IGDB game (PS4/PS5 releases of one game)
- schedule name, reclaim, create and start helpers now take the source
  and recurring task name instead of hardcoding Xbox
- NPSSO instructions with the ssocookie link, token lifetime note and
  the warning not to log out of the browser session
- reuses the Xbox tile and modal patterns unchanged
- psnawp is fully mocked; no network tag needed
- covers non-game filtering, cross-platform aggregation, error
  translation and the connect/disconnect/schedule views
- psnawp 2.0.0 calls requests without any timeout, so a stalled PSN
  response would hang the import worker indefinitely
- wrapped at the request builder, covering authentication too
- PSN's cumulative play duration is monotonic, so an aggregate below the
  stored progress always means incomplete data: a sibling title ID whose
  IGDB lookup failed this run, or a playDuration PSN reports as absent
  (psnawp collapses that to zero -- observed live for Life Is Strange).
  Both cases previously overwrote real tracked hours.
- Clamp overwrite progress to max(existing, aggregate), mirroring the
  Xbox importer's rule that unknown minutes never overwrite progress.
- A mutation of the overwrite-mode guard previously passed the whole
  suite: no test exercised "new" mode against an existing game, though
  that is the default for recurring syncs. Now locked in.
- The status guard protects DROPPED as well as COMPLETED; only the
  latter was tested.
- Titles the store-genre heuristic drops as apps now land in the import
  warnings, not only the server log: the heuristic is best-effort, and a
  misclassified game must be auditable by the user instead of vanishing.
- The per-title concept lookup now fails open on every psnawp and
  transport error (rate limit, auth blip, timeout included); previously
  any of those aborted the whole sync and flagged the connection broken
  over one of dozens of per-title lookups.
- The Completed/Dropped guard checked the snapshot read before the long
  IGDB matching phase, then bulk-wrote progress and status wholesale: a
  user marking a game Completed or Dropped mid-sync had that overwritten
  with a status computed from stale data.
- Progress is still bulk-updated; statuses now go through per-status
  conditional updates that exclude Completed/Dropped rows in the query,
  so the guard holds against the database state at write time.
- The Xbox importer retains the snapshot-only guard; aligning it is a
  separate change.
Matches the hand-drawn Xbox mark rather than shipping Sony's trademark,
which is copyright-free but still trademarked.
Nothing logs the token today, but the redaction list is the safety net
for future changes.
The PSN modal hint was the only paragraph in import_data.html spelled as a
raw palette utility, which fails app.tests.test_theme_tokens and renders the
same grey on both canvases.

Rebuilding main.css also picks up four focus-visible utilities from
form_save_bar.html that the committed stylesheet was already missing.
@Revisor01
Revisor01 force-pushed the feature/psn-integration branch from c5c1bf6 to 3eb4fd7 Compare August 17, 2026 20:32
latest grew its own 0024 and 0025 while this branch was open, leaving two
leaf nodes below 0023. The PSN migration has never been released, so it
moves behind them rather than needing a merge node.
@Revisor01
Revisor01 force-pushed the feature/psn-integration branch from 3eb4fd7 to c24e715 Compare August 17, 2026 20:42
@ryan-winkler
ryan-winkler merged commit b7bea50 into dannyvfilms:latest Aug 17, 2026
9 checks passed
@Revisor01
Revisor01 deleted the feature/psn-integration branch August 19, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants