Pure Python pkgconfig implementation

[bruchar1]

The problem is that, especially on Windows, calling subprocesses for each dependency is slow. The second problem that installing pkgconfig on Windows usually requires a third-party package manager (e.g. chocolatey) 1

Different thigs where tried in the past:

• Having a Python package using libpkgconfig bindings [Use bindings to libpkgconf, when available #11972], and allow to use it from meson. However, there is no official package for those bindings, pkgconf is not interested maintaining them, and the way pkgconf is organized, there is a lot of logic still in the pkgconf client; therefore linking with libpkgconf alone is not sufficient. Furthermore, having a binary package requires generating it for every architectures, especially on Windows, which is tedious.
• Having a minimal pkgconfig implementation directly in meson (@xclaesse proposed something like this, but I cannot find it...) However, while the common case is simple, the pkgconfig implementation can quickly become a complex and error-prone task. Therefor, il is probably beyond the meson's scope.

I think the best option would be to implement a complete pure-python pkgconfig implementation in Python. I did a poc (not published yet) based on pkgconf code and algorithmsm abd frankly, I think this could be something viable. They, meson could use it directly as a Python lib, instead of relying on subprocesses, allowing to cache the .pc files parsing between invocations. There could be a syntax in machine files to opt in for that implementation, and a fallback mechanism to use it if the pkgconfig executable is not found, but that Python implementation is importable. That way, it would not break the current way pkgconfig is used.

What do you think about this idea? On the meson side, the changes would be minimal, since @xclaesse already refactored the pkgconfig dependency to allow different implementations #11973. I could publish the Python implementation of pkgconfig on my personal repo, but I think it would be better to have it on the mesonbuild organisation, to allow more maintainers (if any interested...)

[xclaesse]

My implementation is here: #7650.

pkg-config implementation is actually not that complex, especially since Meson only requires a subset of it and already implements parts of the needed logic like argument dedup.

[dcbaker]

I would be in favor of this. In the long term it makes sense because it allows us to mix and match dependency management solutions, which would be difficult to do with external tools.

[xclaesse]

Note that there is a downside in using any in-process pkg-config implementation (python or libpkgconf): CLI implementations are often wrappers that sets specific env. We can easily still fallback to CLI implementation when PKG_CONFIG env is pointing to one, but if user relies on a wrapper set in PATH we are going to break those setup. IMHO that's a downside we just have to accept and document.

[eli-schwartz]

In general, when the topic of "just implement pkgconfig yourself" comes up, I have a few major objections:

• it's not domain-appropriate for meson to include it as a bundled library, especially because nobody else would ever use it so we wouldn't be able to cross-compare results
• NIH vs "have an interoperability standard implemented by multiple alternatives" (e.g. the reason both GCC and clang exist) is a tough balance when talking about relatively small scoped projects, and the return on investment doesn't seem great to me. It feels like I've mainly seen people solve this by just not trying to guarantee good compatibility, which is unacceptable to me at least as a default (you should be forced to manually opt in)
• pkg-config bakes in knowledge of e.g. --libdir for the environment that your dependencies come from, which isn't necessarily the same as the environment you're installing into even when you're not using cross files. This may or may not be influenced by any of, $PKG_CONFIG, $PKG_CONFIG_LIBDIR, $PKG_CONFIG_SYSTEM_INCLUDE_PATH, etc. With pkgconf, you can extract this information by invoking the system pkg-config with the flag --dump-personality

Optionally using an externally installable module solves 1, and solves both 2 and 3 by making it opt in (and may also solve 2 by making it much more likely to have dedicated tests including possibly even importing the pkgconf testsuite verbatim?)

[xclaesse]

• pkg-config bakes in knowledge of e.g. --libdir for the environment that your dependencies come from, which isn't necessarily the same as the environment you're installing into even when you're not using cross files. This may or may not be influenced by any of, $PKG_CONFIG, $PKG_CONFIG_LIBDIR, $PKG_CONFIG_SYSTEM_INCLUDE_PATH, etc. With pkgconf, you can extract this information by invoking the system pkg-config with the flag --dump-personality

That's right, forgot about that issue, we can't have distro specific config that they set at pkg-config/pkgconf build time. I even had a FIXME in my PR for that: https://github.com/mesonbuild/meson/pull/7650/files#diff-2608eb7a6bbc9b311f10898835c193d407e7b07b528bdb29bc5d7c5b7946e7c8R662.

I did not know about --dump-personality, that's nice. What I would suggest is use python implementation if pkg-config CLI is not found (typically on Windows) or when the the CLI is pkgconf and we can use its config from --dump-personality. Otherwise keep using CLI.