Skip to content

prefer=scryfall: Scryfall's Own Printing Ordering, as Its Own Score - #920

Open
daveycodez wants to merge 2 commits into
jbylund:mainfrom
daveycodez:prefer-extended-art-sign
Open

prefer=scryfall: Scryfall's Own Printing Ordering, as Its Own Score#920
daveycodez wants to merge 2 commits into
jbylund:mainfrom
daveycodez:prefer-extended-art-sign

Conversation

@daveycodez

@daveycodez daveycodez commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Reworked per review: Scryfall's preference is now a separate score reproducing their full per-card printing ordering, not a reweighting of the curated prefer_score (which this PR no longer touches at all).

The score

scryfall_prefer_score (new real column): 0 for the printing Scryfall itself shows for the card, then −1, −2, … down the card's remaining printings — so DESC reproduces their full ordering under the same higher-wins machinery prefer_score already uses, and the two never mix.

Where the ordering comes from

The two halves come from the two places Scryfall makes its ordering observable:

  1. The top is exact by construction. The oracle_cards bulk dump is one card object per oracle_id, and that object IS the representative printing they show. Those ids sync into magic.scryfall_representatives each import — the same table and importer as prefer_score: Pin the Representative Scryfall Itself Names #921, byte-identical, so the two PRs merge cleanly in either order.
  2. The rest reproduce the prints listing every card object links as its prints_search_uri (order=released&unique=prints): release date, set code, collector number, all descending. Tie-breaks were verified against the live listing rather than assumed: same set and day orders 305 → 184 → 63 and 1638★1638 (numeric part first, text second); different sets, same day orders pfdn → fdc, mb2 → blc, and fbb → 3ed — the last rules out set name ("Foreign Black Border" < "Revised Edition") in favor of set code (f > 3).

One boundary, stated plainly

When a filter excludes their representative, Scryfall's unique=cards fallback follows unpublished editorial rules, not release order — probed: Lightning Bolt minus its representative's set falls back to clu 141 (2024) over five strictly newer printings. That fallback is not reproducible from any bulk source, so under such filters this ordering yields their prints-listing order instead of their unpublished pick. Named lookups and prints listings — what a /scryfall mount would serve — reproduce exactly.

Selectable everywhere prefers are

  • API: prefer=scryfall (PreferOrder.SCRYFALLscryfall_prefer_score DESC).
  • Engine: Prefer::Scryfall reads the stored per-printing score; the store carries the new column, so ARCHIVE_FORMAT_VERSION → 2026082701.
  • UI: a Scryfall option in the prefer dropdown.

Import order: representatives sync after art tags; backfill_scryfall_prefer_scores then ranks every printing. LEFT JOIN, so an empty label table degrades to release order rather than failing the import.

Tests

Full ordering against a live schema (pin beats newer printings; set-code tie-break; unlabelled cards still fully ranked; prefer_score untouched), engine prefer=scryfall picking the pinned-not-newest fixture printing, enum + SQL compilation coverage. pytest: 3427 passed. cargo test: 160 passed.

@github-actions github-actions Bot added api Changes to the HTTP API / request handling in api_resource.py size/S 32-99 changed lines labels Aug 11, 2026
@jbylund

jbylund commented Aug 20, 2026

Copy link
Copy Markdown
Owner

I think what I'd like to do here is have a separate prefer score which is scryfall's. We can obviously get the top 1 for that, but can we get the full ordering? that way we can reproduce their preference exactly. Then if we mount scryfall routes at /scryfall they can get closer to exact behavior and it would also be available as a prefer elsewhere. Would that work?

@daveycodez
daveycodez force-pushed the prefer-extended-art-sign branch from 3b343bc to 5d8b9e4 Compare August 21, 2026 23:33
daveycodez added a commit to daveycodez/sylvan_librarian that referenced this pull request Aug 21, 2026
Follows jbylund#920, which used Scryfall's `oracle_cards` dump as evidence for one
weight. This uses it as a LABEL: that dump is one card object per oracle_id, and
that object IS the printing Scryfall shows for the card. So it is ~38k external
preferences for exactly the question this score answers, against the 1,070 that
ten grading sessions produced for jbylund#720.

Measured on a 31,724-card corpus, agreement with Scryfall's representative:

    today                          66.2%
    with jbylund#920's extended_art fix   73.9%
    with this pin                  96.6%

The ceiling is the 3.4% of labels naming a printing a given corpus does not
carry, so 96.6% is the whole of what is available.

THE VETO IS THE PART WORTH REVIEWING, and it is why this is not "defer to
Scryfall". That dump optimises "most up-to-date recognizable version" -- measured
median ~4 months newer than this score's pick -- and newer increasingly means
licensed crossovers. On 213 cards it names an OFF-STYLE printing while an
on-style one exists: it would show Marvel Super Heroes Commander art for Birds of
Paradise, Harmonize, Shock and Skullclamp, which is precisely what `art_style`
was built to prevent on 177 of 177 labelled comparisons. So the pin yields to
`art_style` in that case and nowhere else, costing ~0.7 points of agreement to
keep jbylund#720's result intact.

Shape:

  - a side table, `magic.scryfall_representatives`, written wholesale each import
    rather than a column on magic.cards: one narrow row per card against 95k+
    card rows, and a LEFT JOIN means an EMPTY table scores every card exactly as
    before -- the extra dump is an optional input, never a hard dependency
  - populated before the backfill, for the same reason art tags are: scoring
    first would leave every card unpinned until the next import
  - `on_style_cards` CTE supplies the veto's precondition, so a card printed
    solely in licensed sets keeps its label rather than losing its representative

Tested against real Postgres, not mocks: a labelled printing gains the pin, an
unlabelled one is byte-identical across repeated backfills, and an off-style
label with an on-style sibling is NOT pinned. pytest api/tests: 578 passed,
ruff clean.
@daveycodez

Copy link
Copy Markdown
Contributor Author

Agreed — a separate prefer score that is Scryfall's own ordering is the better shape. Reworking this PR to do that: reproduce their full ordering (not just the top-1) so a /scryfall mount can match their choice exactly, and it's available as a prefer everywhere else.

Reworks this PR to the shape asked for in review: instead of re-weighting
extended_art inside the curated prefer_score, Scryfall's preference becomes a
SEPARATE score -- and a full per-card ordering, not just a top-1.

`scryfall_prefer_score` (new real column on magic.cards) is 0 for the printing
Scryfall itself shows for the card, then -1, -2, ... down the card's remaining
printings, so DESC reproduces their full ordering under the same higher-wins
machinery prefer_score already uses. The curated prefer_score is untouched;
neither score reads the other.

The ordering's two halves come from the two places Scryfall makes it
observable:

- The TOP is exact by construction: the `oracle_cards` bulk dump is one card
  object per oracle_id, and that object IS the representative printing they
  show. Those ids sync into magic.scryfall_representatives each import (same
  table and importer as jbylund#921, byte-identical so the two PRs merge cleanly in
  either order).
- The REST reproduce the prints listing every card object links as its
  `prints_search_uri` (`order=released&unique=prints`): release date, set
  code, collector number, ALL descending. The tie-breaks were verified
  against their live listing rather than assumed -- same set and day orders
  305 -> 184 -> 63 and "1638*" -> "1638" (numeric part first, text second);
  different sets, same day orders pfdn -> fdc, mb2 -> blc, and fbb -> 3ed,
  the last of which rules out set NAME ("Foreign Black Border" < "Revised")
  in favor of set CODE ('f' > '3').

One boundary, stated plainly: when a filter excludes their representative,
Scryfall's unique=cards fallback follows unpublished editorial rules, not
release order -- probed: Lightning Bolt minus its representative's set falls
back to clu 141 (2024) over five newer printings. That fallback is not
reproducible from any bulk source, so under such filters this ordering gives
their prints-listing order instead of their unpublished pick. Named lookups
and prints listings -- what a /scryfall mount would serve -- reproduce exactly.

Selectable everywhere prefers are:

- API: prefer=scryfall (PreferOrder.SCRYFALL -> scryfall_prefer_score DESC).
- Engine: Prefer::Scryfall reads the score off each printing; the store
  carries the new column, so ARCHIVE_FORMAT_VERSION bumps to 2026082701.
- UI: a Scryfall option in the prefer dropdown.

Import order: representatives sync after art tags, then
backfill_scryfall_prefer_scores ranks every printing (LEFT JOIN, so an empty
label table degrades to release order rather than failing the import).

Tests: full ordering against a live schema (pin beats newer printings; set-code
tie-break; unlabelled cards still fully ranked; prefer_score untouched), engine
prefer=scryfall picks the pinned-not-newest fixture printing, enum and SQL
compilation coverage. pytest: 3427 passed. cargo test: 160 passed.
@daveycodez
daveycodez force-pushed the prefer-extended-art-sign branch from 5d8b9e4 to eb7eaf5 Compare August 28, 2026 04:05
@github-actions github-actions Bot added rust Pull requests that update rust code frontend Changes to the static JS/HTML/CSS client UI database Schema, migrations, and PostgreSQL-related changes tests Test suite additions or changes card_engine Changes to the Rust query engine (card_engine) python size/L 317-999 changed lines and removed size/S 32-99 changed lines labels Aug 28, 2026
@daveycodez daveycodez changed the title prefer_score: Extended Art Is a Variant, Not the Canonical Printing prefer=scryfall: Scryfall's Own Printing Ordering, as Its Own Score Aug 28, 2026
@daveycodez

Copy link
Copy Markdown
Contributor Author

Reworked as discussed — the branch is rewritten, the extended-art reweighting is gone, and prefer_score is untouched.

The new scryfall_prefer_score stores Scryfall's full per-card ordering as a score: 0 for the printing they show for the card, counting down from there, so DESC reproduces the ordering. The top-1 comes from the oracle_cards dump (one object per oracle_id, and that object is their representative — same importer/table as #921, byte-identical so either merges first). The rest follow their prints listing (order=released): release date, set code, collector number, all descending — I verified the tie-breaks against their live listing rather than assuming them (the fbb-before-3ed pair is what pins set code over set name).

On "reproduce their preference exactly": named lookups and prints listings reproduce exactly. The one thing that can't be sourced is their fallback pick when a filter excludes the representative — I probed it, and it follows unpublished editorial rules (Bolt minus its rep's set falls back to a 2024 printing past five newer ones), so under such filters you get their prints-listing order instead. Everything needed for a /scryfall mount is here; the routes themselves I left for a follow-up.

It's selectable as prefer=scryfall in the API, the engine (new store column, archive format bumped), and the UI dropdown.

One conflict, in `ARCHIVE_FORMAT_VERSION`. Main bumped to 2026082501 for
`SortPermutations`' printing-span prefix sums; this branch holds 2026082701 for
`Printing`'s `scryfall_prefer_score`. Kept both paragraphs in date order and
kept this branch's newer value — the two changes are independent, and an equal
constant across two layouts is the one thing the header check cannot see
through.

`api/admin_resource.py` merged clean: jbylund#1011 rewrote the `is:hybrid` and
`is:phyrexian` expressions in `BOOLEAN_IS_TAGS`, well away from this branch's
`import_scryfall_representatives` call and `backfill_scryfall_prefer_scores`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Changes to the HTTP API / request handling in api_resource.py card_engine Changes to the Rust query engine (card_engine) database Schema, migrations, and PostgreSQL-related changes frontend Changes to the static JS/HTML/CSS client UI python rust Pull requests that update rust code size/L 317-999 changed lines tests Test suite additions or changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants