Adapt Fedora Change: Dropping of cert.pem file

[rezib]

I use latest version of mock from Fedora rawhide:

[root@rpm ~]# rpm -qa | grep mock
mock-filesystem-6.5-1.fc44.noarch
mock-core-configs-43.3-1.fc44.noarch
mock-6.5-1.fc44.noarch

Creating the build environment for opensuse-leap-16.0 fails with SSL error:

[root@rpm ~]# mock --init --root=opensuse-leap-16.0-x86_64
INFO: mock.py version 6.5 starting (python version = 3.14.0, NVR = mock-6.5-1.fc44), args: /usr/libexec/mock/mock --init --root=opensuse-leap-16.0-x86_64
Start(bootstrap): init plugins
INFO: selinux disabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux disabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start: clean chroot
Finish: clean chroot
Mock Version: 6.5
INFO: Mock Version: 6.5
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: /etc/mock/templates/opensuse-leap-16.0.tpl newer than root cache; cache will be rebuilt
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Package manager dnf4 detected and used (fallback)
Start(bootstrap): installing dnf tooling
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
[…]
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf4 detected and used (direct choice)
Start: installing minimal buildroot with dnf
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #2, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #3, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #4, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
ERROR: Command failed: 
 # /usr/bin/dnf-3 --installroot /var/lib/mock/opensuse-leap-16.0-x86_64/root/ --releasever 16.0 --setopt=deltarpm=False --setopt=allow_vendor_change=yes --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install patterns-devel-base-devel_rpm_build ca-certificates ca-certificates-mozilla
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

The workaround I found is to install ca-certificates-mozilla along with dnf:

[root@rpm ~]# echo "config_opts['dnf_install_command'] += ' ca-certificates-mozilla'" >> /etc/mock/opensuse-leap-16.0-x86_64.cfg

[xsuchy]

Cc @Conan-Kudo @lkocman

[scaronni]

I don't have this issue:

$ rpm -qa | grep mock
mock-core-configs-43.3-1.fc43.noarch
mock-filesystem-6.6-1.fc43.noarch
mock-6.6-1.fc43.noarch

@rezib do you still have the issue with the more recent mock version?

[rezib]

@rezib do you still have the issue with the more recent mock version?

Yes 😕

[root@rpm ~]# rpm -qa | grep mock
mock-filesystem-6.6-1.fc44.noarch
mock-core-configs-43.3-1.fc44.noarch
mock-6.6-1.fc44.noarch

[root@rpm ~]# mock --init --root=opensuse-leap-16.0-x86_64
INFO: mock.py version 6.6 starting (python version = 3.14.2, NVR = mock-6.6-1.fc44), args: /usr/libexec/mock/mock --init --root=opensuse-leap-16.0-x86_64
Start(bootstrap): init plugins
INFO: selinux disabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux disabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start: clean chroot
Finish: clean chroot
Mock Version: 6.6
INFO: Mock Version: 6.6
Start(bootstrap): chroot init
…
Start: installing minimal buildroot with dnf
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                       0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #2, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                       0.0  B/s |   0  B     00:01    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #3, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                       0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #4, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                       0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
ERROR: Command failed: 
 # /usr/bin/dnf-3 --installroot /var/lib/mock/opensuse-leap-16.0-x86_64/root/ --releasever 16.0 --setopt=deltarpm=False --setopt=allow_vendor_change=yes --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install patterns-devel-base-devel_rpm_build
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                       0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

[scaronni]

Ah, on Fedora 44. Can you try the following before running mock?

update-crypto-policies --show
update-crypto-policies --set LEGACY

And then trying again?

[rezib]

Same result 😕

[root@rpm ~]# update-crypto-policies --show
DEFAULT
[root@rpm ~]# update-crypto-policies --set LEGACY
Setting system policy to LEGACY
Note: System-wide crypto policies are applied on application start-up.
It is recommended to restart the system for the change of policies
to fully take place.
[root@rpm ~]# mock --init --root=opensuse-leap-16.0-x86_64
INFO: mock.py version 6.6 starting (python version = 3.14.2, NVR = mock-6.6-1.fc44), args: /usr/libexec/mock/mock --init --root=opensuse-leap-16.0-x86_64
Start(bootstrap): init plugins
INFO: selinux disabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux disabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start: clean chroot
Finish: clean chroot
Mock Version: 6.6
INFO: Mock Version: 6.6
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Package manager dnf4 detected and used (fallback)
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf4 detected and used (direct choice)
Start: installing minimal buildroot with dnf
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #2, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #3, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
WARNING: DNF4 command failed, retrying, attempt #4, sleeping 20s
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
ERROR: Command failed: 
 # /usr/bin/dnf-3 --installroot /var/lib/mock/opensuse-leap-16.0-x86_64/root/ --releasever 16.0 --setopt=deltarpm=False --setopt=allow_vendor_change=yes --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install patterns-devel-base-devel_rpm_build
Failed to set locale, defaulting to C
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Leap 16.0 - x86_64 - OSS                                                                                                                                                                              0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'opensuse-leap-oss':
  - Curl error (60): SSL peer certificate or SSH remote key was not OK for https://download.opensuse.org/distribution/leap/16.0/repo/oss/repodata/repomd.xml [SSL certificate problem: unable to get local issuer certificate]
Error: Failed to download metadata for repo 'opensuse-leap-oss': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

[scaronni]

"unable to get local issuer certificate" means it can't find the root or intermediate CAs, and unless it's a byproduct of the SSL crypto policies filtering a few hashes, it means you don't have it.

Can you check the second entry in the changelog here for Fedora 44? https://koji.fedoraproject.org/koji/buildinfo?buildID=2806358

• Tue Aug 12 2025 Frantisek Krenzelok fkrenzel@redhat.com - 2024.2.69_v8.0.401-8

• update-ca-trust: Added a temporary, compatibility option --rhbz2387674 to
  the extract command. This flag restores legacy certificate
  symlinks (e.g., /etc/ssl/cert.pem) to address issues with older software
  that has not yet adapted to their removal. This essentially provides a
  temporary way to revert the "Dropping of cert.pem file".

[scaronni]

If it works, then the issue is basically the compatibility for the new layout in mock.

[rezib]

Nice catch! 👏

[root@rpm ~]# update-ca-trust extract --rhbz2387674
----------------------------------------------------------------------------
** DEPRECATION WARNING **
----------------------------------------------------------------------------
The option --rhbz2387674 is a temporary workaround and will be removed in a
future release. Please do not use it in build scripts or automation.

----------------------------------------------------------------------------
** ACTION REQUIRED **
----------------------------------------------------------------------------
To ensure the affected package works correctly in the future, a bug report 
must
be filed.

1.  Check if a bug already exists for the affected package: 
https://bugzilla.redhat.com/buglist.cgi?component=ca-certificates&product=Fe
dora&short_desc=droppingOfCertPemFile%20package%3A&short_desc_type=allwordss
ubstr

2.  If no bug exists, please file a new one using this template: 
https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&component=ca-certif
icates&version=rawhide&short_desc=droppingOfCertPemFile+package:+<<package_n
ame>>+is+affected

Thank you for helping improve Fedora.
[root@rpm ~]# rm -rf /var/lib/mock/opensuse-leap-16.0-x86_64*
[root@rpm ~]# mock --init --root=opensuse-leap-16.0-x86_64
INFO: mock.py version 6.6 starting (python version = 3.14.2, NVR = mock-6.6-1.fc44), args: /usr/libexec/mock/mock --init --root=opensuse-leap-16.0-x86_64
Start(bootstrap): init plugins
INFO: selinux disabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux disabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start: clean chroot
Finish: clean chroot
Mock Version: 6.6
INFO: Mock Version: 6.6
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start(bootstrap): unpacking root cache
Finish(bootstrap): unpacking root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Package manager dnf4 detected and used (fallback)
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
…
timezone-2025b-160000.2.2.x86_64
update-alternatives-1.22.21-160000.2.2.x86_64
util-linux-2.41.1-160000.2.2.x86_64
which-2.23-160000.2.2.x86_64
wtmpdb-0.74.0+git20250509.272b109-160000.2.2.x86_64
xz-5.8.1-160000.2.2.x86_64
Finish: run
[root@rpm ~]# echo $?
0

[xsuchy]

IIRC - we should adapt Mock code to https://fedoraproject.org/wiki/Changes/droppingOfCertPemFile

As a temporary workaround for anyone hitting this issue is to run update-ca-trust extract --rhbz2387674