pkgdepends (and pak) fail when GitHub packages depend on each other and different versions are requested

[vandenman]

I noticed that pkgdepends fails to resolve multiple github packages that depend on each other:
A reprex:

on d59d4ee I get:

obj <- pkgdepends::new_pkg_deps(c(
  pkgdepends = "r-lib/pkgdepends@d59d4ee3b702d3ff2a38450ace2ae3050d17fc35",
  pkgcache   = "r-lib/pkgcache@93010c2f496705395f70c284c8925fe72bc092f3"
))

sol <- obj$solve()$get_solution()
print(sol)

gives output:

<pkg_solution>
+ result: FAILED
+ refs:
  - r-lib/pkgcache@93010c2f496705395f70c284c8925fe72bc092f3
  - r-lib/pkgdepends@d59d4ee3b702d3ff2a38450ace2ae3050d17fc35
+ constraints (62):
  - select pkgdepends exactly once
  - select pkgcache exactly once
  - select callr at most once
  - select cli at most once
  - select curl at most once
  - select desc at most once
  - select filelock at most once
  - select jsonlite at most once
  - select lpSolve at most once
  - select otel at most once
  ...
x failures:
* r-lib/pkgdepends@d59d4ee3b702d3ff2a38450ace2ae3050d17fc35: Can't install dependency r-lib/pkgcache@feature/ppm-sso (>= 2.2.0)
* r-lib/pkgcache@feature/ppm-sso: Conflicts with r-lib/pkgcache@93010c2f496705395f70c284c8925fe72bc092f3

there appears to be a conflict with the remote from the DESCRIPTION r-lib/pkgcache@feature/ppm-sso and the explicitly requested r-lib/pkgcache@93010c2f496705395f70c284c8925fe72bc092f3. This also happens when there are no branches listed, e.g., Remotes: r-lib/pkgcache would have also errored.

Originally observed when using renv to restore a lockfile using pak on CI, which failed to resolve the dependencies. This behavior appears to be in pkgdepends already since v0.8.0 (or earlier, I didn't test that far back). Perhaps this is intended, but it does complicate reproducibility. Note that the error did not surface with pak 0.9.5, only since 0.10.0. Still, this felt like the right place for the issue.

[gaborcsardi]

Yes, this is by design. You cannot load multiple versions of the same package.

[vandenman]

Perhaps I'm misunderstanding but I'm not "loading" multiple versions of a package ("loading" in the sense of library/ loadNamespace or so). I'm simply asking pkgdepends for a plan to:

• Install this GitHub package from commit A.
• install that GitHub package from commit B.

If I do this with development (or older versions) of rlang and roxygen2 this works. It works because even though roxygen2 depends on rlang, roxygen2 does not list Remotes: r-lib/rlang in the DESCRIPTION file. If you do it with pkgcache and pkgdepends as shown above, it fails. If that is truly intended behavior, I can understand of course but it does mean that for any form of reproducibility of R packages pkgdepends, and by extension pak is a no-go. I realize this is also not what these packages say they do, but the interaction with renv makes it appear as if they should.

[gaborcsardi]

r-lib/pkgdepends@d59d4ee3b702d3ff2a38450ace2ae3050d17fc35 needs r-lib/pkgcache@feature/ppm-sso:

> pak::pkg_deps_tree("r-lib/pkgdepends@d59d4ee3b702d3ff2a38450ace2ae3050d17fc35")
✔ Updated metadata database: 6.77 MB in 10 files.
✔ Updating metadata database ... done
r-lib/pkgdepends@d59d4ee3b702d3ff2a38450ace2ae3050d17fc35 0.9.1.9000 ✨👷🏿‍♀️🔧
[...]
├─r-lib/pkgcache@feature/ppm-sso 2.2.5.9000 ✨👷🏿‍♀️🔧
[...]

but you wanted r-lib/pkgcache@93010c2f496705395f70c284c8925fe72bc092f3 which is different (I assume).

[vandenman]

93010c2f496705395f70c284c8925fe72bc092f3 is just two commits prior on the same branch.

Perhaps I overcomplicated this by using pkgdepends as an example because the remotes in the DESCRIPTION lists a specific branch and not just a repository. If it makes things clearer, the minimal example in my own organization is:

obj <- pkgdepends::new_pkg_deps(c(
  jaspBase   = "jasp-stats/jaspBase@93f004588f6d56b46d23210e23bff003ae7b1ff4",
  jaspGraphs = "jasp-stats/jaspGraphs@1eb48e7e0fe9559bbd15da25f30703c7cca246bb"
))

sol <- obj$solve()$get_solution()
print(sol) # similar error as above

DESCRIPTION of jaspBase and DESCRIPTION of jaspGraphs (which has no GitHub dependencies).

So the task I'd like pkgdepends to successfully carry out is:

• install pkgA from user/pkgA@commitX
• install pkgB from user/pkgB@commitY
• pkg A has Remotes: user/pkgB

I'm wondering if that's intentionally not supported.

[gaborcsardi]

• pkg A has Remotes: user/pkgB

This means installing the latest commit of the default branch and that probably conflicts with user/pkgB@commitY

[vandenman]

and that probably conflicts with

Yeah that is exactly why the code currently throws an error. But should it? I explicitly requested user/pkgB@commitY and not user/pkgB. So in this case I would, perhaps naively, assume that what I explicitly requested has priority over whatever is in the DESCRIPTION (unless say, version bounds are incompatible or other situations).

[gaborcsardi]

user/pkgB means the latest commit of the default branch of that repo. If you don't want this commit, but any version of the package will do, then simply drop this from Remotes.

[vandenman]

Right, so suppose I remove Remotes from the DESCRIPTION.

Now if someone calls remotes::install_github("user/repo") or renv::install("user/repo"), it no longer works. The DESCRIPTION file also gives no hint as to where find that package.

For production purposes we use a lockfile with specific package versions, but for user friendliness not of course.

Since this also directly affects the interaction with renv and reproducibility, perhaps this should be discussed there further?

[gaborcsardi]

Now if someone calls remotes::install_github("user/repo") or renv::install("user/repo"), it no longer works.

So you do need that package from GitHub then? Then you need Remotes. I am sorry, it is a bit hard for me to follow the hypothetical examples.

But yes, currently all packages that specify a certain pkgB from GitHub need to specify the same commit, otherwise there is a conflict.

If I understand correctly, you are after some specification that says "any version from this package on GitHub"? That does not seem like a good idea to me, it is unlikely that this kind of dependency would be true in practice.

OTOH, there should be a way to resolve these kind of conflicts, by explicitly selecting the desired version of the conflicted package(s).