Pkg integration checklist: conventions, API mapping, rollout

[StefanKarpinski]

Umbrella for what Pkg-side integration needs from this package, collected from the #30/#57 review cycle (Pkg.jl PR 4738 targets the superseded #30 API and will need updating to #57's):

• API mapping: resolve(info, prob; with/without, diagnose) :: Union{Dict, Diagnosis, Nothing}; issatisfiable for preflight sweeps; holdbacks for the why-command; the three-tier cost ladder documented in the guide.
• Modeling conventions (these replace policy knobs, so they're load-bearing):
  • julia (and anything non-negotiable) is a dependency edge, never a requirement — that is what makes "drop requirement julia" unrepresentable (Diagnostics: goal-query resolve, verified fixes, and holdback analysis on the production SAT instance #57's dep-edge convention; bin/Registries.jl is the reference implementation).
  • Pkg.add(name, version)-style restrictions enter as Problem compat entries with labels = Dict(pkg => :requested) so reports say "you requested X at 1.7" rather than "your compat".
  • yanked pre-filtering + allow_yanked plumbing mirrors bin's.
• Presentation ownership: Pkg renders from the structured Diagnosis via the public toolkit (report knobs, or fully custom via render_fact/rank_upstream!/Bound.incidental/changes) — e.g. Pkg can drop incidental facts rather than demote them, and filter upstream suggestions by its own maintained-package knowledge. Guide section "Rendering your own reports" is the contract.
• pkg> why upgrade path: presence answers (why is X here / what would avoiding it cost) come from resolve(…; without = X); version answers from holdbacks/goal queries — one chain instead of the all-paths forest (the Suggest what to remove for satisfiable requirements #6 complaint).
• Rollout: KristofferC's suggestion from UNSAT diagnostics: explain unresolvable requirements and list verified fixes #30 — feature-flag the resolver with the new one as default so it gets tested, old resolver retained initially.
• Known gaps affecting parity: weak deps/extensions (Weak dependencies and extensions: model them #61); whole-registry T1 persistence for fast cold starts (T1 persistence: save/load/invalidate the universal registry cache (implementation plan) #55).

🤖 Opened with Claude Code at Stefan's request.
https://claude.ai/code/session_016p2sckwc5Gjjg5LG6gdEk7

[StefanKarpinski]

Status note, since this checklist was written against a design that has since been rebuilt.

Unchanged and still correct: the modelling conventions. They are the load-bearing part of this issue, because they replace policy knobs the resolver deliberately does not have.

• Platform packages — julia above all — are modelled as dependency edges, never as requirements. That is what makes drop requirement julia unrepresentable rather than suppressed by a flag. bin/Registries.jl is the reference implementation.
• Pkg.add(name, version)-style restrictions should enter as compat entries carrying a label, so a report says "you requested DataFrames at 1.7" rather than "your compat restricts". The labels mechanism ships with the presentation toolkit.
• Yanked versions are filtered at the provider, with allow_yanked to opt back in (Yanked versions are filtered out of the universe #68). The resolver never proposes a yanked version because it never sees one.
• Presentation belongs to the caller: Pkg renders from the structured Diagnosis rather than accepting our default show.

Stale: the API mapping. It describes the surface of the superseded #57. What is landing instead:

• resolve(data, prob_or_reqs; compat, pins, by, diagnose) :: Union{Dict, Diagnosis, Nothing} — a Diagnosis on failure, nothing when diagnose = false (Diagnose an unsatisfiable resolve #73, open).
• issatisfiable(...) :: Bool — the verdict alone, one solve, no descent (issatisfiable: the bare feasibility check #67, merged).
• Resolver.Diagnostics — the diagnosis machinery in its own submodule with an explicit export list; Resolver itself exports only Diagnosis, since that is resolve's return type.
• Goals, holdbacks and the toolkit follow; see Roadmap: what is done, what is next, and what is waiting on evidence #64 for the order.

Also stale: the Pkg.jl PR. JuliaLang/Pkg.jl#4738 targets the #57 API and will need updating against whatever ships. Worth doing once, after the diagnosis surface stops moving.

Not scheduling a test drive against a PR: it happens when the thing is genuinely ready to review.

[StefanKarpinski]

The API mapping in this checklist targets #57's surface, which was superseded. Updating it against what actually shipped in #73 and #79, on current main (2480f3f). The modelling conventions are unchanged and still correct.

API, as shipped:

• resolve(info, prob; by, order, diagnose) :: Union{Dict, Diagnosis, Nothing} — diagnose = true returns a Diagnosis in place of nothing; diagnose = false gives callers who only want the verdict the bare nothing. This part of the mapping survives as written.
• Resolver exports Diagnosis; the rest of the diagnosis surface (Conflict, Fix, Action, Fact, Requirement, Availability, action_phrase, diagnose) lives in Resolver.Diagnostics and is not re-exported.
• issatisfiable for preflight sweeps: still wanted, still not the shipped spelling — is_satisfiable(sat, reqs) is internal. Naming to settle before Pkg depends on it. (Rename the one-argument is_satisfiable to sat_solve #81 is renaming the one-argument form to sat_solve so the name cannot answer a staged question while looking total; it does not change the two-argument one.)
• holdbacks does not exist — Diagnostics should explain steering surprises: why a resolve was held back to an older julia (or package) version #52 is unstarted.

One convention needs rewriting, not just re-checking. This checklist specifies Pkg.add(name, version)-style restrictions entering as Problem compat entries with labels = Dict(pkg => :requested), so reports say "you requested X at 1.7" rather than "your compat". #79 removed the parallel labels mechanism: a problem's constraints are now one map keyed by kind, :compat and :pin are well-known kinds, and anything else is a custom kind whose name the report uses directly. So the :requested case is now expressed as its own constraint kind rather than a label attached to a compat entry — which is closer to what the checklist wanted, but it is a different API and the Pkg-side code has to be written against it.

Presentation ownership is the largest remaining gap. The report(io, d; knobs) / changes(sol₁, sol₂) / render_fact / rank_upstream! toolkit this section treats as the contract does not exist; #73 ships show(::IO, ::MIME"text/plain", ::Diagnosis) and nothing else. Pkg can render from the structured Diagnosis today, but not via the documented knobs, and the guide section "Rendering your own reports" is not written. This is item 4 on #64's list.

Rollout and the known parity gaps (#61 weak deps, #55 persistence) are unchanged.

Pkg.jl PR 4738 still targets the #30 API and will need rewriting against the above.

[StefanKarpinski]

Status against main at d59bcd2, after #84, #86, #90, #96, #98, #99, #104, #105, #106 and #107. The modelling conventions in the body are still correct and still load-bearing — julia as a dependency edge, restrictions as their own constraint kind, yanked filtered at the provider, presentation owned by the caller. Everything below is the API mapping, which has moved again.

Shipped, and stable enough to write Pkg code against

resolve(data, prob::Problem; by = identity, diagnose = true)
    -> Union{Dict{P,V}, Diagnosis{P,V}, Nothing}
resolve(data, reqs; by = identity, diagnose = true)
issatisfiable(data, prob) -> Bool

The issatisfiable naming question from the last update is settled. It is exported from Resolver and documented; sat_solve (#81) is the internal one-argument spelling and is no longer able to masquerade as it. Pkg can depend on this name.

diagnose = false matters for Pkg. The diagnosis costs several extra solves plus one resolve per offered fix. A preflight sweep should pass it.

Presentation ownership: much better than "show and nothing else"

The last update called this the largest remaining gap and said #73 ships show(::IO, ::MIME"text/plain", ::Diagnosis) and nothing else. That is no longer true. The whole structured surface is public and documented in docs/src/api.md:

• What a diagnosis is made of: Conflict, Alternative, selections, Fix, Action, action_phrase, Line, diagnose.
• The sentence layer, so a caller can build its own prose from the same pieces rather than re-deriving them: Resolver.Clauses with clause_phrase, range_phrase, version_order, letters, plus the clause algebra itself (literal, resolve_on, subsumes, isbottom).
• Self-check: report_problems(d) and print_conflict, so Pkg can assert on its own rendering in its own tests.

What still does not exist is the report(io, d; knobs) / changes(sol₁, sol₂) / render_fact / rank_upstream! toolkit this checklist names, and the guide section "Rendering your own reports" is unwritten. My read is that this is now a convenience gap rather than a capability gap: everything those knobs were meant to expose is reachable as plain data. Worth deciding whether Pkg wants the knobs at all before building them — writing them against a real Pkg renderer would produce a better API than guessing.

Still genuinely missing for the pkg> why upgrade path

Both of these are named in the body as the mechanism for why, and neither exists:

• Goals — resolve(…; with = X, without = X). Presence answers ("why is X here", "what does avoiding X cost") depend on this. Not started.
• holdbacks — version answers ("why is X at 1.2 and not 1.9"). Diagnostics should explain steering surprises: why a resolve was held back to an older julia (or package) version #52, still unstarted; it exists only on an unmerged branch.

So pkg> why cannot be built on this yet. Everything for the unsatisfiable path can be.

Corrections to the body

• Weak dependencies and extensions: model them #61 (weak deps/extensions) is done — closed by Test and document weak dependency semantics #82, with semantics tested and documented. It is no longer a parity gap; the body still lists it as one.
• T1 persistence: save/load/invalidate the universal registry cache (implementation plan) #55 (persistence) is the remaining parity gap and is the one that matters for interactive Pkg use: there is no cache at all today, so every resolve pays the full registry parse and pkg_info build. Still unstarted.
• labels = Dict(pkg => :requested) is dead, as flagged last time: One map, one constraint type #79 made constraints one map keyed by kind, so a Pkg.add(name, version) restriction is its own constraint kind whose name the report uses directly. Pkg-side code has to be written against that.

Rollout

KristofferC's feature-flag suggestion is unchanged and still right. JuliaLang/Pkg.jl#4738 targets the #30 API — three API generations stale now — and should be rewritten rather than rebased.

---

🤖 Triaged with Claude Code.