add support for vendor profiles

[Centuriondan]

Currently the biggest part of devuans fork of jdg is for providing our own buildenv sub package. I'd like to see if we can find a way to genericise vendors and make it easy to provide a vendor provided package that will be automatically installed and used at build time.

Currently when setting up a builder we install:
jenkins-debian-glue
jenkins-debian-glue-buildenv
jenkins-debian-glue-buildenv-devuan (which contains our custom pbuilderrc)

I'd like to move to something similar to what lintian has for "vendors" - where a vendor can provide in their distribution a package with a `Provides: jenkins-debian-glue-vendor-profile'

With this we can then have jenkins-debian-glue add a "Recommends: jenkins-debian-glue-vendor-profile " which would install any or all (depending on the distrobutions availability and policy) of applicable vendor packages.

The vendor profile package will provide vendor profile(s) which can be sourced from the global pbuilderrc once the vendor and distribution are determined. We can rely on dpkg-vendor --query vendor to determine the default vendor to use if a "vendor' label isn't provided

[mika]

Sounds like a good idea to me, would appreciate if someone implements that!

[Centuriondan]

You happy with the proposed Provide and Recommends usage and virtual package name?

I think most of what's currently in /etc/jenkins-debian-glue would move to the vendor profiles as well as anything that would go in the pbuilderrc

This probably means providing a reasonable vendor profile for debian instead of jdg-buildenv

[mika]

Yeah, I currently don't see a problem with it.

[nuxwin]

@Centuriondan

It is not more a debootstrap issue? What the purpose of providing your own pbuilderrc file exactly? Overriding mirror and components for Devuan?

[Centuriondan]

@nuxwin that and other settings we find are neccesary for building devuan like setting additional hooks and PBUILDERSATISFYDEPENDS=classic (it's more reliable then aptitude) and other variables.

Potentially we could be be using our ci to provide some derivatives as well and being able to set the mirrors and other key parameters in a vendor profile would be really useful and allow selecting a vendor profile at build time or even build for multiple distrobutions in a single job.

[nuxwin]

@Centuriondan

that and other settings we find are neccesary for building devuan like setting additional hooks and PBUILDERSATISFYDEPENDS=classic (it's more reliable then aptitude) and other variables.

I see. I do that already through my /etc/jenkins/pbuilderrc file

# pbuilder configuration file automatically passed-in by jenkins-debian-glue
# See https://manpages.debian.org/stretch/pbuilder/pbuilderrc.5.en.html

# Only for cowbuilder (update process)
# cowbuilder doesn't care about the DIST variable...
DISTRIBUTION=$DIST

echo "Distribution set to ${DISTRIBUTION}"

# Set mirror and components to use, according distributor ID
if [ "$DEB_DIST_ID" = "debian" ]; then
  MIRRORSITE="http://deb.debian.org/debian"
  COMPONENTS="main contrib non-free"
else
  MIRRORSITE="http://fr.archive.ubuntu.com/ubuntu"
  COMPONENTS="main restricted universe multiverse"
fi

echo "Mirror set to ${MIRRORSITE}"
echo "Components set to ${COMPONENTS}"

# Setup dependencies resolver according distribution codename and architecture
# - aptitude resolver doesn't work with qemu-user-static
# - apt resolver is not provided by jessie, trusty and xenial
case "$DISTRIBUTION" in
 jessie|trusty|xenial) case "$ARCH" in
     i386|amd64) PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-aptitude" ;;
     *) PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-classic" ;;
 esac ;;
 *) PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-apt" ;;
esac

echo "Build dependencies resolver set to ${PBUILDERSATISFYDEPENDSCMD}"

I define the DEB_DIST_ID variable in my job which is responsible to build package for different distributor (Debian, Ubuntu). Then, in my pbuilderrc, I can set required settings for it.

# Distributor ID (debian/ubuntu)
# Used in the pbuilderrc conffile to determine both mirror and components to set
# Used to set release repository path
case "$distribution" in
	jessie|stretch|buster|bullseye|bookworm) export DEB_DIST_ID=debian ;;
	*) export DEB_DIST_ID=ubuntu ;;
esac

Indeed, a better way would be welcome ;) We should have an easy way for detection of the distributor ID in build job and select a specific profile according it.

Regarding the resolver, what you mean by "more reliable" ? As far as I known, the classic resolver is a bit slow compared to the aptitude resolver. My preference goes to the new apt resolver which isn't available in older distribution.

[Centuriondan]

@nuxwin
The idea is not so much to remove the need for a pbuilderrc as such, but to allow debian based distro's or derivatives to easily ship their own vendor profile in their distrobution and have it selected as the default vendor profile, along with it's own tunable pbuilderrc

With regard to the resolver, I've never much liked aptitudes resolver, and we had it break in some situations where classic worked fine. I'm keen to try the apt resolver, but again here it may need to be tied to a specific release which can be handled in the vendor profile.

With regards to architectures we use the matrix plugin to build multiple archs at the same time - for more on that see #207

[Centuriondan]

Hopefully I'll have time to get back to this and drop some code in the coming week ;-)