Use rpm repositories from bootstrap image at build time

[Romain-Geissler-1A]

This is issue a feature request for allowing to fully trust the rpm repositories already configured inside the bootstrap image. The use case for this would be people building package inside organization using their own rpm repositories (either because they don't have internet access for some reason, either because they have some extra repositories purely internal to that organization).

Let me describe what we do now inside Amadeus:

• I maintain for the company a set of official images, from RHEL 8 to 9, CentOS 10, and the latest fedora. These images are already configured "correctly" for someone working inside Amadeus: we remove all pre-set rpm repositories, to target instead our local mirror of the remote Red Hat, CentOS, Fedora rpm mirrors. We have some workloads using these images without internet access for security reason (but access to these internal mirrors).
• These images are provided to thousands of developers. So of them might need to package their own software for these plateforms, and use fedpkg/mock to do that. These developers have absolutely no idea which rpm repositories to use, this is an "implementation detail" of the images, which evolves over time (and are quite different between RHEL, CentOS and Fedora). These developers know how to use mock, but don't want to know the details of the rpm repositories configuration
• On my side, I don't think it would make much sense that I would provide to my users some extra mock config template, effectively copy/pasting what the upsteam mock project provides, but removing all rpm repositories from these config. Indeed, most of these config are implementation details of the mock project, or the distro maintainers, not really something I should know and follow.

Ideally, I would like to be able to provide my images, with rpm repositories correctly configured already, and my users would just need to call mock with an option to disable the mock logic rewritting the rpm repositories, while keeping all the other logic wrt dnf configuration override.

Right now, what we do is no great (according to me):

• We start our bootstrap images with podman, to extract the rpm repositories url
• We then run mock with as many --addrepo flags for each repository, since mock is totally removing the original dnf configuration
• We still inherit from the default rpm repositories configured by the mock core config template, which actually we would like not to use at all. It seems to work though (at least when internet access is available).

In term of code, it looks like this:

ALL_RPM_REPOSITORIES=($(podman run --rm "${BOOTSTRAP_IMAGE}" dnf repoinfo --quiet --enabled | grep -E -i '[^:]*base.?url[^:]*:' | sed -e 's/[^:]*:\s*//'))

MOCKBUILD_REPOSITORIES_ARGUMENTS=()

for RPM_REPOSITORY in "${ALL_RPM_REPOSITORIES[@]}"; do
    MOCKBUILD_REPOSITORIES_ARGUMENTS+=("--addrepo" "${RPM_REPOSITORY}")
done

# Actually build the packages.
fedpkg --release "${RELEASEVER}" mockbuild --mock-config "${MOCK_CONFIG_NAME}" -- --config-opts "bootstrap_image=${BOOTSTRAP_IMAGE}" "${MOCKBUILD_REPOSITORIES_ARGUMENTS[@]}"

Ideally I would like to have some new hypothetical boolean config option use_boostrap_image_rpm_repositories so that we would only do:

# Actually build the packages.
fedpkg --release "${RELEASEVER}" mockbuild --mock-config "${MOCK_CONFIG_NAME}" -- --config-opts "bootstrap_image=${BOOTSTRAP_IMAGE}" --config-opts use_boostrap_image_rpm_repositories=True

So this feature if accepted would mean more or less:

• splitting the the 'dnf.conf' config_opts everywhere in the templates into what is really purely dnf.conf from the actual repository configuration
• NOT removing the folder /etc/yum.repos.d from the bootstrap image, and using it inside 'dnf.conf'
• conditionally adding the mock template pre-configured repositories, or trusting the one from the bootstrap image

Would that make sense ?

Thanks,
Romain

[praiskup]

Thank you for the question/RFE. This is a hard one :) though.

Where do you / your users execute Mock? In Podman containers or on host? If you run mock-in-podman then you can easily ship mock configuration there (sets of configuration files, whatever you need). If "on host", it is a matter of shipping one config file per build target/chroot (whatever way you prefer) - it isn't that bad, you can use Copr to build your mock-cofiguration set, complementary to mock-core-configs.

The thing is that Mock is designed to have the configuration in /etc/mock + $HOME, there's no tooling (yet) to extract the config from the bootstrap image. Such a potential implementation would look like

a) parse configuration on host,
b) take a look how bootstrap image is named,
c) get the configuration options from that image,
d) combine the newly extracted configuration with the on-host configuration
e) start the build ... this seems pretty convoluted.

Can't you ship the mock config in container? Then you can do

podman run <your bootstrap image> cat /mock.cfg > /tmp/mock.cfg
mock - /tmp/mock.cfg <your.src.rpm>

What is inside "${MOCK_CONFIG_NAME}"?

[Romain-Geissler-1A]

Hi,

Where do you / your users execute Mock? In Podman containers or on host?

Right now we have one Fedora machine pre-configured with some /etc/mock config, and this is what I want to avoid using in the future. The team who created this machine described a bit what kind of /etc/mock config they have added, but didn't show the full config in their documentation, and I can't access their machine. It creates a blackbox for other external contributor like me. Plus it creates some long term maintenance burden as this team has to keep this specific fedora machine alive which we don't need for other usual development task. So I push so that we move it to container based mock, fully described in their git repositories so package builds are actually reproduceable locally.

What is inside "${MOCK_CONFIG_NAME}"?

fedora-41-x86_64 and centos-stream+epel-10-x86_64 (and similar ones for different OS version/arch). I know CentOS Stream isn't exactly RHEL, but it is for now close enough so that we can use it and not bother with subscription manager.

I know I could ship some /etc/mock config via a package, but then it would mean I would duplicate the informations of which repo to use for which OS both in the images I maintain already + in this /etc/mock config, which I would like to avoid. For now I prefer to extract it manually the list of repo from the bootstrap images and pass them as argument to mock via --addrepo, to avoid this duplication. In your high level description, I don't understand the first steps (especially about getting the host information). I would rather expect an option where we consider that the bootstrap image is "well configured" already when it comes to repository setup (and I don't know if we can assume this is setup in /etc/yum.repos.d rather than dnf.conf). mock would still override dnf.conf, but without disabling /etc/yuml.repos.d, and without providing in dnf.conf any repo, we would just trust what the bootstrap image has configured already. So naively it would look more like this:

a) Run mock with a specific bootstrap image and disable repo overriding, the mock config is provided (in my case it's one well known mock config like fedora or CentOS + epel)
b) Mock starts the container, overrides dnf.conf, keeps /etc/yum.repos.d enabled, doesn't delete the content of /etc/yum.repos.d
c) Build is started inside this container

For me this scenario would be opt-in, not a default one (at least for now, to keep the current behavior, not sure what to do for the long term). People using such an option would definitely know/accept that they become responsible for configuring repositories "the right" way. So it's more about not doing some of the existing cleanup/config steps than adding new non existing steps. And clearly separating repo config from dnf.conf config.

[praiskup]

The team who created this machine described a bit what kind of /etc/mock config they have added, but didn't show the full config in their documentation, and I can't access their machine.

They can execute mock -r <chroot> --debug-config for you, so you can make a diff.

In your high level description, I don't understand the first steps (especially about getting the host information).

Mock is installed with mock.rpm on host. Anytime you execute it it has to parse /etc/mock configuration.
Then, if we wanted to extract additional configuration from bootstrap image, we'd have to somehow merge the configuration with the /etc/mock configuration.

I would rather expect an option where we consider that the bootstrap image is "well configured" already when it comes to repository setup

This is tricky - the bootstrap chroot is typically the "base image" of target distribution, without having Mock installed at all. Set of repositories is typically related to container use-case, not "build system use-case". This wouldn't work in general for all chroots.

The thing is that this totally changes how mock works. Mock was always responsible for preparing a valid dnf.conf (or yum.conf), not to consume one (either from bootstrap or from build chroot). There's by far more logic than just taking care of the set of repositories.

This idea just feels a bit overcomplicated to me 🤷 considering you can just bake a specific Mock config into a well-prepared "mock container image" that you can distribute to your users. Anyway, if you want to experiment with this, feel free.
We probably can not give much support here, but if you provide a reasonable knob to opt this feature in (and well explaining documentation how to modify the bootstrap image), I think we could merge the feature upstream.

[Conan-Kudo]

The bootstrap image is not intended to be safe for building stuff. It's intended to provide an package manager stack to bootstrap the build environment. I do not think we should ever change this, because then it violates our ability to reliably reproduce things.