Ensure __version__ is retrieved while all dependencies are available

[flying-sheep]

load_module is deprecated. And exec_module has a distinct advantage:

Since we try to get __version__ from a module that possibly doesn’t have its dependencies installed, we could partially execute the module by ignoring exceptions (ImportErrors would be sufficient). If the module manages to already set its __version__ before an ImportError is raised, flit install will still work.

This allows to only add those dependencies to build-requires which are needed to get the __version__, and let flit do the rest.

import importlib.util

def get_docstring_and_version_via_import(target):
    """
    Return a tuple like (docstring, version) for the given module,
    extracted by importing the module and pulling __doc__ & __version__
    from it.
    """
    log.debug("Loading module %s", target.file)
    spec = importlib.util.spec_from_file_location(target.name, str(target.file))
    m = importlib.util.module_from_spec(spec)
    with _module_load_ctx():
        spec.loader.exec_module(m)
    docstring = m.__dict__.get('__doc__', None)
    version = m.__dict__.get('__version__', None)
    return docstring, version

@contextmanager
def _module_load_ctx():
    """Preserve some global state that modules might change at import time.
    - Handlers on the root logger.
    - Ignores ImportErrors
    """
    logging_handlers = logging.root.handlers[:]
    try:
        yield
    except ImportError:
        pass
    finally:
        logging.root.handlers = logging_handlers

[takluyver]

Fine on moving away from the deprecated method. I'm less keen on the idea of ignoring import errors: I'm trying to think about how getting the version number could be made properly robust, not just a bit less likely to fail.

[flying-sheep]

I think that would simply mean that we don’t try to get the version until we’ve installed all dependencies.

[takluyver]

Either that, or limiting how packages can specify their version number so we can always get it without loading the whole package.

[flying-sheep]

As long as get_version keeps working 😄

[merwok]

I think I have seen AST browsing used to find an assignment to __version__.

[flying-sheep]

Sure, but that won’t work if there’s more complex code here. I want to retrieve the version from the latest git tag using setuptools_scm or get_version, and those need to call a function that gets passed __file__ or so.

[takluyver]

I've mentioned before that I'd consider breaking setuptools_scm type approaches. I haven't gone down that route yet, but I still might at some point. I know a lot of people like that kind of feature, but flit has always been an opinionated tool based around the way I like to do packaging, and that isn't part of it.

[flying-sheep]

Please give me a heads-up if you do that so I can move all of my packages to Poetry.

[merwok]

I want to retrieve the version from the latest git tag using setuptools_scm or get_version

Ah! that’s our difference, I’m in the camp of people who like having the canonical version in source code and have packaging metadata and VCS tags derived from it.

[flying-sheep]

I don’t want to start a discussion here, so let’s start a new issue to discuss this if you want to reply to this comment:

My main reasons to not do that is because of 1. friction and 2. human fallibility: Your approach means to always perfectly execute:

sed -Ei 's/"1\.3\.dev\d+"/"1\.4"' package/__init__.py
git add package/__init__.py
git commit -m 'v1.4'
sed -i 's/"1\.4"/"1\.4\.dev0"' package/__init__.py
git add package/__init__.py
git commit -m 'dev version'
git push
git push --tags

That’s a lot of work and a lot can go wrong compared to my preferred

git tag v1.4
git push --tags

The minor reasons: 3. It pollutes your commit history with pairs of meaningless commits that 4. contain information that’s duplicated in your git tags (no one true source of information). 5. Theoretically you’d also need to bump dev$x to dev${x+1} in every single other commit to correctly convey that the version changed, while I get this for free.

I know there’s tools who do the above dance for you, but 6. I can never remember how their CLI works, and if you need to specify the new version or how to bump the version and if it pushes for you or not or …