How to update inner packages that are github based ?

[liran-cohen-hs]

We are using github as our repository to install internal packages
i.e in our pyproject.toml file we have:
"<package_name> @ <git+ssh url>[email protected]"
when I want to add this package for the first time I use:
pdm add <git+ssh url>[email protected]

when I want to update this package I also use pdm add, the thing is, the resolver conflicts if the requirements of that package changed

I tried to use pdm update command but then I stumbled into another problem:
pdm update <git+ssh url>[email protected] results in <git+ssh url>[email protected] does not exist in inner dependencies.
and pdm update <package_name> does not identify the latest package

how to handle this ?

[frostming]

There are several unclear parts in your description, and I am unable to provide an answer.

First, does @X.Y.Z resolves to a single commit or does the commit it refers to changes?
Second,

when I want to update this package I also use pdm add, the thing is, the resolver conflicts if the requirements of that package changed

Did you use another ref than X.Y.Z, what kind of requirement change if the tag keeps the same?

[liran-cohen-hs]

I used another ref, actually just reproduced it with public library so it will be easier to explain
created new pdm project with clean pyproject.toml and ran:
pdm add git+https://github.com/boto/[email protected]
then I wanted to update so I ran:
pdm add git+https://github.com/boto/[email protected]
and the result is:

Adding packages to default dependencies: git+https://github.com/boto/[email protected]
🔒 Lock failed
Unable to find a resolution for botocore
because of the following conflicts:
  botocore<1.11.0,>=1.10.33 (from boto3@git+https://github.com/boto/[email protected])
  botocore<1.11.0,>=1.10.33 (from boto3@git+https://github.com/boto/boto3.git@a6d5dc3d2105918bf070ecc7982db0b90b16d873)
  botocore<1.33.0,>=1.32.3 (from boto3@git+https://github.com/boto/[email protected])
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/latest/usage/dependency/#solve-the-locking-failure for more details.
See /var/folders/nr/92733x4j3d54n5802mg7094h0000gp/T/pdm-lock-2gt3oe2p.log for detailed debug log.
[ResolutionImpossible]: Unable to find a resolution
Add '-v' to see the detailed traceback

the thing is that if I run: pdm add git+https://github.com/boto/[email protected], it does make the update (because it doesn't update the install_requires, yet it identifies that it is an update - pyproject.toml is update correctly)
so if I need to guess - it's a matter of the order of commands happening in the internals, i.e first identify that it's an update so you can ignore the resolving of the package requirements within itself in another version

another try:

pdm update git+https://github.com/boto/[email protected]
[ProjectError]: git+https://github.com/boto/[email protected] does not exist in default dependencies.

[frostming]

Okay, here's the deal.

Try pdm add "boto3 @ git+https://github.com/boto/[email protected]"

Yes, you need to give the package name explicitly to make PDM replace the entry correctly instead of adding a new one, which would result to a resolution conflict. PDM doesn't do any wild guess of the package name until it builds the package and retrieves it from the metadata.

And for pdm update, you need to give the package name, not the full URL. It acts as a key to identify the package.

[liran-cohen-hs]

thanks :)