Docs: How to single source dependencies?

[fohrloop]

I decided to ask about single sourcing dependencies in separate issue, since there might be slightly different approach than with version numbers.

Currently, I list my dependencies in setup.py:

import setuptools

setuptools.setup(
    name='mypackage',
    version=find_version(),
    author="Niko Pasanen",
    description="My package",
    packages=setuptools.find_packages(),
    install_requires=[
        'astropy>=3.2.1',  
        'click>=7.0',  
        'ipython>=7.4.0',  
        'geopy>=1.2.0', 
        'numba~=0.51.1', 
        'pandas~=1.1.2',
    ],
)

This is needed so that I can install the mypackage in editable state with

pip install -e <folder>

This is not, however the full list of packages, since all of the packages listed in install_requires might depend on other packages. Since I am using virtual environments, I can easily take a snapshot of everything I'm using with pip freeze. I could use this snapshotted list (with perhaps some manual editing, removing development time dependencies) then in the setup.py and installer.cfg. But if I modify anything, I have to remember modify two places. What would be the best way to single-source dependencies? I'm thinking that perhaps I could put all to installer.cfg and read them in setup.py. Would that be the way to go?

[takluyver]

As in the other issue (#216), one answer is to script the use of Pynsist. I tend to imagine simpler projects having a static config file and running pynsist installer.cfg by hand, and more complex projects running Pynsist under their own build scripts. E.g. you can have a prior step which collects all the wheels you need, and then use the local_wheels option with a glob pattern to build them into your installer. Certbot does this, for instance.

If you let Pynsist fetch wheels from PyPI, it requires that all dependencies are pinned to an exact version. This is a crude equivalent of what more sophisticated packaging tools call a lockfile. The idea is that if you check out your tag for version 3.5.2, you can quickly see what versions of dependencies the installer bundled - and ideally even create a similar installer a few years later, although I'm not sure how well that works in practice.

Perhaps there's a need for a tool that can go from a list of specified dependencies with version ranges to a list with pinned versions and all transitive dependencies. Tools like Poetry incorporate such a feature, but I'd hesitate to use something so application-y as a dependency.

I also experimented some years ago with a tool (kartoffel) to identify dependencies by running an application and checking what it imports. I still think it's an interesting approach, but I've never got round to making it actually useful.

I've got a bit off topic. 🙂 Pynsist definitely doesn't have a great story around bundling dependencies at the moment - but on the plus side, what it does have is pretty easy to understand and build on top of. I'm interested in thinking about what could improve it, but also keen to keep things as simple as possible.