Skip to content

Conversation

@nichmor
Copy link
Collaborator

@nichmor nichmor commented Nov 10, 2024

The PyPI - Conda name mapping that maps a PyPI package to it's known Conda counterpart.

This only works for packages that are available on the conda channels that it references.

This is available at https://conda-mapping.prefix.dev/pypi-to-conda-v0/{channel}/{pypi-normalized-name}.json where the channel is the name of the channel and the {pypi-normalized-name} is the normalized name of the package on PyPI.

Example:
for requests we can use https://conda-mapping.prefix.dev/pypi-to-conda-v0/conda-forge/requests.json, which will give you the corresponding json.

The schema will be saved at this location:
https://conda-mapping.prefix.dev/pypi-to-conda-v0/conda-forge/schema/schema.json

@maresb
Copy link

maresb commented Nov 13, 2024

Thanks a lot for this @nichmor!

It seems to me like this would be a lot simpler (in my mind at least) as a table instead of a mapping, as per #11 (comment).

So for PyPI→conda-forge requests you have a list of conda-forge archives that contain a PyPI package named requests. And that list can contain some other useful columns.

For the conda-forge→PyPI requests you have the same list but instead filtered to have PyPI packages that are defined by an archive from conda-forge's requests.

This way you only need one schema and it serves both mappings. Also, when you parse published conda-forge packages, you simply need to add rows to the master table. What do you think?

@maresb
Copy link

maresb commented Nov 13, 2025

Oh wow, thanks soooooo much everyone for taking into account my feedback, it really means a lot to me!!! 🙏 I believe that this will make it so much easier to use universally.

@tdejager
Copy link
Contributor

tdejager commented Nov 19, 2025

So this PR expanded by a lot, because of the functionality I needed to test things. Based on the previous work by @nichmor, @wolfv and the suggestions by @maresb .

I did a lot of work on this branch together with CLAUDE. And basically setup a new pipeline to do the conversion, also added a small explorer to have a rich cli to explore the mappings. We also modified the README to include all of this new information. The pipelines have also been merged into a templated one, I've triggered this in CI so that the PyPI mappings are now actually online behind the v1 url.

Next up, we made a full integration test with an in-memory s3, so that we can verify that the pipelines keep working.

Finally, we added a docker-compose.yml with a minio backend so that you can recreate the mappings locally.

When all this is merged I want to add a small marimo example for @maresb that he can use to explore the RelationTable as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants