Packaging changes to make packaging closer to PS#2280
Open
EvgeniyPatlan wants to merge 32 commits into
Open
Conversation
Apply the packaging improvements from Percona Server release-9.6.0-2 to
PXC. Three workstreams ship together:
1. Consolidate duplicated cmake flags (RPM spec + DEB rules):
- build-ps/percona-xtradb-cluster.spec: introduce %{cmake_common_flags}
macro plus %{cmake_fido_flags} / %{cmake_lto_flags} /
%{cmake_systemd_flags} sub-macros for conditionals. Reduce three
duplicated cmake invocations (debug, release, PGO consume) to single-
macro references. Fix two latent bugs: duplicate WITH_EMBEDDED_SERVER
(one =OFF, one =0) and duplicate WITH_INNODB_MEMCACHED (one =ON,
one =1), now defined once in the macro.
- build-ps/debian/rules: introduce CMAKE_COMMON_FLAGS Make variable.
Reduce two duplicated cmake invocations (debug, release) to shared-
variable references.
- build-ps/build-binary.sh: introduce CMAKE_COMMON_FLAGS bash array
consolidating the two duplicated cmake invocations in the tarball
build path.
2. Modernize DEB packaging:
- Upgrade debhelper compat 9 -> 13 (delete debian/compat, switch to
debhelper-compat (= 13) in Build-Depends).
- Replace deprecated dh_systemd_enable + dh_systemd_start helpers with
a single dh_installsystemd call (gates on supported distros only;
precise/trusty/wheezy guard removed since those distros are EOL).
- Add dh-apparmor, pkg-config, libsystemd-dev to Build-Depends.
- Add debian/not-installed to suppress dh_missing warnings cleanly,
enumerating cmake helpers, KMIP internals, license/README files,
legacy mysql.server SysV wrapper, and test-only plugins.
- Factor inline telemetry directory setup in postinst into a shared
percona-telemetry-setup.sh helper script (under
/usr/libexec/percona-xtradb-cluster/), reused from the equivalent PS
packaging work for consistency between RPM and DEB packages.
3. Wire PGO+LTO infrastructure for mysqld:
- RPM: %{?with_pgo: %global pgo 1} top-level toggle. Release block
emits %{?pgo:-DFPROFILE_GENERATE=1}. New %if 0%{?pgo} consume block
runs MTR profile suite, wipes release dir, rebuilds with
-DFPROFILE_USE=1.
- DEB: ifdef DEB_PGO sets PGO_GENERATE / PGO_USE flag pairs. Release
cmake injects $(PGO_GENERATE). New ifdef DEB_PGO block in
override_dh_auto_build runs make run-profile-suite, wipes builddir,
rebuilds with $(PGO_USE).
- Tarball: WITH_PGO=${WITH_PGO:-1} env var (auto-disabled for Debug
builds). Three-pass cmake: instrument, MTR profile workload, consume.
Adds -Wno-free-nonheap-object plus four other warning suppressions
since standalone tarball builds bypass distro hardened-cc1 specs.
- Builder script: --enable_pgo CLI option (default = 1) fans out to
all three artifact types via EXTRA_DEFINES (rpmbuild),
export DEB_PGO=1 (dpkg-buildpackage), export WITH_PGO=1
(build-binary.sh).
LTO was already wired (DWITH_LTO=ON for rhel > 8 in spec; unconditional
in DEB rules); kept as-is. Galera-side PGO via scons deferred to a
later release per the analysis decision.
MTR profile-suite tweaks for PGO compatibility (mysql-test/CMakeLists.txt):
- Remove thread_pool from MTR_TESTS_FOR_PROFILING (Oracle Enterprise
plugin not present in PXC source; would abort the profile run with
"Could not find 'thread_pool' or 'mtr' in scanned directories").
- Set --max-test-fail=0 so individual test failures don't abort the
PGO profile run before sufficient .gcda data is collected; PGO benefits
from workload diversity, not test pass-rate.
DEB package splits per Q3=b (server-core, client-core, client-plugins
matching upstream MySQL pattern) deferred to a separate follow-up branch
because they require precise file routing across multiple .install files
where errors cause dpkg conflicts and need careful Breaks/Replaces tuning;
without local build verification, the risk surface is too large to safely
combine with the other changes here.
Files: 13 changed, +391 -255 (net deletion: ~280 lines of duplicated
cmake flags consolidated, partially offset by PGO infrastructure).
Add three new binary packages and rewire dependencies to align with the upstream MySQL package layout that Percona Server release-9.6.0-2 adopted. PXC 9.7.0 will be the first release in the 9.x LTS series, so introducing the splits now (with no existing 9.x install base) carries no upgrade-path risk. New packages: * percona-xtradb-cluster-server-core Server binaries (mysqld, myisamchk, mysqlbinlog, ...), server-side plugins, libgalera_smm.so, bundled abseil + protobuf libraries, ICU data files, INFO_BIN/INFO_SRC build provenance. No config files, no init scripts, no apparmor profiles. Provides virtual-mysql-server-core and mysql-server-core-8.0. * percona-xtradb-cluster-client-core Core client binaries (mysql, mysqldump) and their man pages. Provides virtual-mysql-client-core and mysql-client-core-8.0. * percona-xtradb-cluster-client-plugins Client-side authentication plugins (LDAP, Kerberos, WebAuthn, OpenID Connect, OCI, native_password, dialog) and bundled libfido2. Multi-Arch: same so it can be co-installed across architectures. Rewired dependencies: * percona-xtradb-cluster-server now depends on -server-core, -client-plugins (in addition to -common, -client). Its install file now contains only PXC-specific binaries (clustercheck, wsrep_sst_xtrabackup-v2, etc.), config files, AppArmor profiles, systemd units, and the telemetry helper scripts. * percona-xtradb-cluster-client now depends on -client-core, -client-plugins. Its install file contains only auxiliary client tools (mysqladmin, mysqlcheck, mysqlimport, mysqlslap, etc.). Conflicts/Replaces stanzas on the new -core packages cover the mysql-*-core-8.0 / virtual-mysql-*-core packages they obsolete, so in-place upgrades from MySQL CE or other forks work cleanly. Garbd, debug, dbg, common, source, test, full, and meta packages unchanged.
Two complementary fixes for the dh "compat level specified both in debian/compat and via build-dependency" error seen during build: 1. build-ps/pxc_builder.sh: add `rm -f debian/compat` immediately after the cp -ap of build-ps/debian/ in build_source_deb. Defensive — guards against any stale debian/compat that might be lying around from a previous build run or copied in by another build step. 2. packaging/deb-in/compat: bump from 11 to 13 to match the debhelper-compat (= 13) build-dependency in debian/control. This upstream-MySQL inherited file is normally only consumed by the cmake-based DEB packaging path (CONFIGURE_FILE in deb-in/CMakeLists), but keeping it in sync prevents surprises if any tooling copies it into debian/. Resolves the symptom: dh: error: debhelper compat level specified both in debian/compat and via build-dependency on debhelper-compat
The previous packaging modernization commit added these packages to
debian/control's Build-Depends (required by debhelper-compat=13 and the
new dh_installsystemd helper) but didn't update the install_deps()
function in pxc_builder.sh. This caused dpkg-checkbuilddeps to fail with:
Unmet build dependencies: dh-apparmor libsystemd-dev
Add the three packages to the apt-get install list so a fresh
--install_deps=1 run on Ubuntu/Debian provisions everything
debian/control needs.
Apply the same telemetry modernization to PXC RPM that we did for the
Percona Server RPM, mirroring the DEB packaging side from earlier in
this branch.
Spec changes (build-ps/percona-xtradb-cluster.spec):
* Add Source11 / Source12 declarations for percona-telemetry-setup.sh
and percona-telemetry-cleanup.sh (already present under build-ps/rpm/
from earlier modernization work).
* Install both scripts into %{_libexecdir}/percona-xtradb-cluster/
during %install.
* Replace the inline telemetry directory creation block in
%post -n percona-xtradb-cluster-server (install -d -m 2775,
chown, chcon SELinux, chgrp telemetry_uuid, chmod) with a single call
to the setup helper script. Same logic, factored out for consistency
with PS and DEB packages.
* Replace the inline `rm -rf %{pxc_telemetry}` in %postun with a call
to the cleanup helper script.
* Remove Source999: call-home.sh and the corresponding wget +
bash invocation. The legacy "phase-0" call-home.sh remote-telemetry
shim was deprecated and replaced by the in-product
percona-telemetry-agent dependency. Keeping the old script around
fetched a remote shell script at install time which is no longer
desirable.
* Add the helper scripts to the percona-xtradb-cluster-server %files
list so they're packaged.
Builder script (build-ps/pxc_builder.sh):
* Replace the wget of call-home.sh in build_src_rpm with a tar-extract
of the two helper scripts from the source tarball into the rpmbuild
SOURCES dir, so the spec's Source11/12 references resolve at build
time without requiring network access.
Same code path as PS release-9.6.0-2 telemetry handling.
The mv to flatten lib/xtrabackup/ -> lib/ assumed PXB DEBs ship
binaries inside a nested xtrabackup/ subdir. Newer PXB versions (9.x)
already ship binaries directly in lib/, so the mv command fails with:
mv: cannot stat '.../pxb-9.3/lib/xtrabackup/*': No such file or
directory
Wrap the mv with `2>/dev/null || true` so it becomes a no-op when the
subdir doesn't exist. The subsequent rm -rf is already idempotent
(removes the subdir if present, succeeds if absent).
Now both old (nested) and new (flat) PXB DEB layouts work without
modification.
Apply five additional improvements to the PXC RPM spec, mirroring patterns
established in the Percona Server release-9.6.0-2 spec.
1. EL8 dwz crash workaround
Add %global _find_debuginfo_dwz_opts %{nil} gated on %if 0%{?rhel} == 8.
dwz-0.13 (EL8) crashes with assertion failure when compressing DWARF
emitted by GCC with PGO+LTO; the crash leaves a core file in BUILDROOT
which then trips %check-buildroot. Skipping dwz on EL8 keeps PGO+LTO
working there at the cost of slightly larger debuginfo packages.
2. Compat library URL: EL8/EL9/EL10/AL2023 + architecture-aware
PXC compatlib was hardcoded to redhat/6/x86_64 sourcing PXC-5.6 with
libmysqlclient.so.18, only enabled for rhel == 7 || rhel == 8 (EL9
and aarch64 broken). Replace with PS pattern:
- EL7 keeps the original PXC-5.6 (.so.18) compat shim.
- EL8/EL9/EL10/AL2023 use Percona Server 8.0 compat (.so.21) sourced
from repo.percona.com with %{_arch} so aarch64 works alongside x86_64.
3. Amazon Linux 2023 awareness across all rhel >= 8 conditionals
PS uses `%if 0%{?rhel} >= 8 || 0%{?amzn} >= 2023` consistently. PXC
only checked %{?rhel}, so AL2023 builds skipped:
- pxc_telemetry global path
- add_fido_plugins=1
- percona-telemetry-agent Requires
- telemetry helper script invocations in %post / %postun
Bulk-update all five sites.
4. Build-time customizable globals (match PS pattern)
Add %{!?...: %global ...} fallback defaults at the top of the spec for:
- ssl_option (-DWITH_SSL=system unless --define 'with_ssl=...')
- nodebuginfo
- feature_set (defaults to community)
- compilation_comment_release / compilation_comment_debug
- src_base (percona-xtradb-cluster)
These match Percona Server's customization surface so rpmbuild
--define overrides work consistently across both products.
Remove the redundant `%if %{undefined compilation_comment_*}` block
that was duplicating the new fallback defaults.
5. Drop legacy %define _build_id_links none
Modern find-debuginfo.sh handles build-ids automatically. PS doesn't
set this and works on EL7-EL10. PXC follows suit.
Verified by rpmspec -P parse on EL8 (with --define 'with_pgo 1'),
EL9, and AL2023 — all parse cleanly with no errors.
…plate Adopt Percona Server's modern packaging template format. Three changes: 1. Rename build-ps/percona-xtradb-cluster.spec to .spec.in. Convert all @@var@@ markers to @var@ (single-@) format. The .spec.in suffix is the canonical convention for cmake CONFIGURE_FILE @only mode and matches Percona Server's build-ps/percona-server.spec.in. 2. Add build-ps/CMakeLists.txt for cmake-based template generation. Mirrors PS pattern: derives PERCONA_SERVER_VERSION from MYSQL_VERSION_EXTRA, extracts WSREP_VERSION from wsrep-API headers + cmake/wsrep-lib.cmake, resolves git revision via `git rev-parse --short HEAD`. CONFIGURE_FILE @only mode then generates percona-xtradb-cluster.spec from the template in CMAKE_BINARY_DIR/build-ps/. Also generates debian/rules from rules.in when present, matching the PS dual-output pattern. Also exports backwards-compat aliases (PERCONA_VERSION, REVISION) so the spec template's existing @var@ names resolve without renaming every reference. 3. Update build-ps/debian/rules to use @var@ markers (was @@var@@). 4. Update build-ps/pxc_builder.sh: - Source-RPM extraction tries .spec.in first, falls back to .spec for backwards compat with older source tarballs. - sed substitutions in build_src_rpm and build_source_deb updated to match @var@ pattern. - Variables substituted unchanged: MYSQL_VERSION, PERCONA_VERSION, WSREP_VERSION, REVISION, RPM_RELEASE, DEB_RELEASE. Both sed-based generation (current builder script) and cmake-based generation (via the new CMakeLists.txt) now work, providing a smooth transition path. End result: byte-identical to the previous spec output, just produced from a properly-templated source. Verified by rpmspec -P parse on EL9 — 2674 lines, no errors.
Two correctness fixes flagged by dh_install during the first DEB build:
1. percona-xtradb-cluster-server-core.install: replace the explicit
plugin enumeration (which guessed at upstream MySQL plugin names that
don't all exist in PXC's build) with the proven `usr/lib/mysql/plugin/*.so`
glob from the original PXC packaging. dh_install processes packages
alphabetically, so client-plugins claims its specific auth plugins
first, then the server-core glob picks up the remainder.
Also fix three concrete path issues:
- libgalera_smm.so: now uses the source/dest syntax
`libgalera_smm.so usr/lib/galera4` matching the original PXC install
(file lives at debian/tmp/libgalera_smm.so, not under usr/lib/).
- INFO_BIN/INFO_SRC: removed; they're already shipped via the
percona-xtradb-cluster-server.docs file (sourced from
builddir/Docs/INFO_BIN, not usr/share/mysql/).
- icudt subdir: matches actual installed path.
Drops the explicit list of 14 plugins that don't exist or have been
renamed in current MySQL 9.x (audit_log.so, ha_archive.so,
semisync_master.so, etc. — many are now built into mysqld or use new
component_* names).
2. percona-xtradb-cluster-client-plugins.install: fix libfido2 path
from `usr/lib/*/private/libfido2.so.*` (wrong arch-glob form) to
`usr/lib/mysql/private/libfido2.so.*` (matches PS pattern and where
PXC actually installs it).
The get_sources step copies four .in templates inside packaging/rpm-common/
to their non-.in variants if missing. Two of those templates
(mysqlrouter.service.in, mysqlrouter.tmpfiles.d.in) actually live
under scripts/systemd/ in the source tree, not in rpm-common, so the
cp commands fail with "cannot stat":
cp: cannot stat 'mysqlrouter.service.in': No such file or directory
cp: cannot stat 'mysqlrouter.tmpfiles.d.in': No such file or directory
Fix the cp commands to fall back to ../../scripts/systemd/<name>.in
when the rpm-common copy is absent. The other two templates
(mysqlrouter.conf.in, mysql.logrotate.in) are correctly in rpm-common
and don't need the fallback.
Commit 0ed3b18 ("rpm: factor telemetry into helper scripts, drop call-home.sh") moved telemetry into Source11/Source12 helper scripts invoked from %post / postinst, but missed two legacy injection blocks in build-ps/pxc_builder.sh that still tried to splice the old call-home.sh heredoc into the spec and into the deb postinst. After 0ed3b18 the spec no longer contains SOURCE999, so: - build_srpm: line_number is empty, awk treats n= as 0, part1.txt is never created, then `>> part1.txt` redirects re-create it with the heredoc at line 1 followed by the entire spec, and `mv part1.txt percona-xtradb-cluster.spec` overwrites the spec. rpmbuild then fails on: error: line 1: Unknown tag: cat <<'CALLHOME' > /tmp/call-home.sh - build_deb: `sed -i 's:exit 0::'` would strip the legitimate trailing exit 0 from the new postinst, and the appended heredoc would duplicate the telemetry path that postinst:6-9 already invokes via percona-telemetry-setup.sh. Delete both blocks. Telemetry continues to work via the helper-script path established by 0ed3b18.
build_srpm in pxc_builder.sh repacked the upstream source tree as
Percona-XtraDB-Cluster-${MYSQL_VERSION}.tar.gz (CamelCase) and dropped
it into rpmbuild/SOURCES/, but the spec's %{src_dir} resolves to
percona-xtradb-cluster-${mysql_version} (lowercase, set via the
%{!?src_base:...} default at spec line 77, aligned with the Percona
Server convention).
rpmbuild then fails with:
error: File .../SOURCES/percona-xtradb-cluster-9.6.0.tar.gz:
No such file or directory
Two changes to fix:
1. build-ps/pxc_builder.sh: repack the source tarball with the lowercase
name and lowercase top-level directory so %{src_dir} and
%setup -n %{src_dir} both resolve correctly. The find using
`ls | grep` for the extracted dir is switched to `grep -i` so it
still locates the upstream CamelCase directory regardless of casing.
2. build-ps/percona-xtradb-cluster.spec.in: drop the dead CamelCase
`%define src_base Percona-XtraDB-Cluster` block at lines 191-193.
It was guarded by `%if %{undefined src_base}` but src_base is
already defined unconditionally at line 77, so the block never
executed and was the source of the casing confusion. Replaced with
a comment explaining why src_base is lowercase.
The telemetry helper scripts referenced by Source11 / Source12 in
percona-xtradb-cluster.spec.in were being extracted with
--strip-components=2, which leaves them at SOURCES/rpm/ instead of
flat in SOURCES/ where rpmbuild looks them up by filename. Combined
with the trailing `|| true` the misplacement was silently swallowed,
so the build proceeded all the way to rpmbuild before failing with:
error: Bad file: .../SOURCES/percona-telemetry-cleanup.sh:
No such file or directory
error: Bad file: .../SOURCES/percona-telemetry-setup.sh:
No such file or directory
The tarball internal path is
<topdir>/build-ps/rpm/percona-telemetry-{setup,cleanup}.sh
Stripping 3 components lands the files flat in SOURCES, which is what
rpmbuild expects. Drop the `|| true` so any future tarball-layout
drift fails loudly at extraction time rather than at rpmbuild time.
Pattern matches the equivalent extraction in
percona-server/build-ps/percona-server-9.0_builder.sh (lines 663-664).
…omment warning
Two fixes in build-ps/percona-xtradb-cluster.spec.in:
1. The libmysqlclient.so.21 compat-libs block at line 728 used a
hardcoded EL6/PXC-5.6 rpm filename for the rpm2cpio step:
rpm2cpio Percona-XtraDB-Cluster-shared-56-${compatver}-${percona_compatver}.1.el6.x86_64.rpm
But on EL8/9/10/AL2023, ${compatsrc} downloads a different rpm:
percona-server-shared-${compatver}-${percona_compatver}.1.el8.${_arch}.rpm
The wget succeeded (so the .rpm was on disk) but rpm2cpio looked
for the EL6 name and failed:
rpm2cpio: Percona-XtraDB-Cluster-shared-56-8.0.37-29.1.el6.x86_64.rpm:
No such file or directory
cpio: premature end of archive
error: Bad exit status from /var/tmp/rpm-tmp.* (%build)
Switch to "rpm2cpio *.rpm" — the percona-compatlib/ subdir only
ever contains the one rpm we just downloaded, so the glob is
unambiguous. This is the same pattern lines 739 and 753 already
use for the xtrabackup downloads in the adjacent block.
2. Comment on line 193 contained an unescaped %{!?src_base:...} token
which rpmbuild's macro expander parses even inside #-comments,
emitting a "Macro expanded in comment" warning. Escape with %% to
suppress.
…ompat Two complementary fixes for the source-deb build failure on Jenkins: dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 13) dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting 1. build_source_deb: add -d to dpkg-buildpackage -S. Source-only package generation produces .dsc + .orig.tar + .debian.tar — no compilation happens, so the build host does not need the binary build-deps satisfied. -d is the documented way to skip the build-deps check for source-only builds (man dpkg-buildpackage). The binary build path (build_deb -> dpkg-buildpackage -b at line 1043) does compile and continues to require build-deps; that path is unaffected by this change. 2. install_deps (apt branch): explicitly install the debhelper-compat virtual package so the binary build_deb path also succeeds when the host's installed debhelper doesn't already provide debhelper-compat (= 13). Wrapped in `|| true` to tolerate ancient distros (e.g. bionic) where the package does not exist; those distros need backports configured separately and are not in scope for this fix. Together these unblock the source-deb step on hosts that lack the debhelper-compat virtual and prevent the binary-deb step from failing the same way later in the pipeline.
…lper hosts After adding -d in 3dc9f2d to bypass the build-deps check, the source-deb step now fails one stage later when `debian/rules clean` invokes dh on a Jenkins agent with debhelper < 12: dh clean dh: No compatibility level specified in debian/compat dh: This package will soon FTBFS; time to fix it! dh: Compatibility levels before 4 are no longer supported (level 1 requested) make: *** [clean] Error 2 debian/rules:4: recipe for target 'clean' failed dpkg-buildpackage: error: debian/rules clean gave error exit status 2 The old debhelper on this agent cannot honor the modern `debhelper-compat (= 13)` Build-Depends mechanism (introduced in debhelper 12). It only looks at debian/compat, which we removed in ef216bb for modern dh. Compat level 13 also requires dh >= 12.10 in any case, so there is no single source-tree configuration that satisfies both old and new dh; downgrading to a lower compat level would regress the modernization done in ebd745a. Source-only package generation does not actually need debian/rules clean to succeed: the working tree is freshly extracted from $NEWTAR a few lines above and is already pristine. Adding -nc to dpkg-buildpackage -S skips the clean pre-pass, sidestepping dh entirely on this host. The binary build path (build_deb -> dpkg-buildpackage -b) runs on a separate, properly-prepared host with full build-deps and modern debhelper installed, so it continues to invoke dh clean normally and is unaffected by this change.
The %install section unconditionally tried to install
libmysqlclient.so.18.1.0 / libmysqlclient_r.so.18.1.0 from
percona-compatlib/usr/lib64/, but those filenames only exist in the
legacy EL7/PXC-5.6 shared rpm. On EL8/EL9/EL10/AL2023 the rpm we
download via %{compatsrc} is percona-server-shared-8.0.37 which
extracts libmysqlclient.so.21.<minor>.<patch> under usr/lib64/mysql/
and contains no .so.18 files at all, producing:
install: cannot stat 'percona-compatlib/usr/lib64/libmysqlclient.so.18.1.0':
No such file or directory
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.* (%install)
Mirror the dual EL7-vs-EL8+ pattern that
percona-server/build-ps/percona-server.spec.in already uses
(lines 670-681, 866-882, 409-415), extended to also cover EL10 and
AL2023 which are in PXC's compat-libs matrix but not PS's:
1. Provides block: split %if 0%{?rhel} == 7 (libmysqlclient.so.18*)
from %if 0%{?rhel} >= 8 || 0%{?amzn} >= 2023 (libmysqlclient.so.21*).
Move Obsoletes/Conflicts outside both blocks so they apply on every
compatlib distro instead of only on EL7 (latent bug in the previous
`%if rhel == 7 || rhel == 8` guard that wrapped them).
2. %install block: keep the EL7 fixed-path install for .so.18.1.0,
and add an EL8+ for-loop that glob-walks usr/lib64/mysql/
libmysqlclient.so.%{compatlib}* and installs every matching file.
The exact .so.21.<minor>.<patch> suffix is not known at spec-write
time (it changes between PS-8.0.37 rebuilds), so a glob is the only
safe approach.
3. %post / %postun symlink fixup: wrap the .so.18 alias-symlink
creation/removal in %if 0%{?rhel} == 7. The EL8+
percona-server-shared rpm ships the .so.21 alias symlinks itself,
so no post-install fixup is needed for those distros.
Pre-existing macro-expanded-in-comment and "extra tokens at end of
%endif" warnings are out of scope for this fix.
…ference
Two rpmbuild failures on EL9 surfaced after the previous compat-libs
fix unlocked the %install step:
1. "File not found: ...libmysqlclient_r.so.21.*"
The %files block for percona-xtradb-cluster-shared-compat listed
both libmysqlclient.so.${compatlib}.* and
libmysqlclient_r.so.${compatlib}.*. The threadsafe _r variant only
existed up to MySQL 5.6. MySQL 5.7 merged thread-safety into the
main libmysqlclient and dropped _r entirely, so the
percona-server-shared-8.0.37 rpm (compatlib=21, EL8+) contains no
libmysqlclient_r.so.21 at all.
Wrap the _r line in %if 0%{?rhel} == 7 so it is only emitted on
distros that still ship a separate _r library (currently only EL7
via the legacy PXC-5.6 shared rpm).
2. "File listed twice: /usr/share/percona-xtradb-cluster/..."
The %files -n percona-xtradb-cluster-server section claimed every
file under /usr/share/percona-xtradb-cluster/ twice: once via the
broad catch-all
%if "%rhel" >= "6"
%attr(755, root, root) %{_datarootdir}/percona-xtradb-cluster/
%endif
at lines 1799-1801, and a second time via the explicit per-file
enumeration at lines 1833-1870 (charsets, language errmsg.sys
directories, install_rewriter.sql, etc.). rpmbuild rejected the
duplicates with ~60 "File listed twice" errors and failed the
%files stage.
Delete both the EL5 dead block and the EL6+ catch-all. The
explicit enumeration already covers everything that needs to ship
and matches the pattern used in
percona-server/build-ps/percona-server.spec.in lines 1221-1295,
which never had a catch-all.
The binary deb build fails at `dh_missing --fail-missing` because
~110 files end up in debian/tmp/ without being claimed by any
package. Three independent causes:
1. Install-file names did not match the package names in
debian/control:
control: Package: libperconaserverclient22 / libperconaserverclient22-dev
files: libperconaserverclient21.install / 21-dev.install / 21-dev.dirs / 21.lintian-overrides
dh_install looks for `debian/<package>.install`, so the 21-named
files were never read and libperconaserverclient22 / -dev shipped
0 files. Rename all four to the 22-prefix to match control.
(The actual built soname is .so.24 for PS 9.6, but the deb package
name stays at 22 to match the existing soversion bump scheme;
updating to libperconaserverclient24 is a separate change.)
2. Telemetry helper scripts were declared via the explicit-copy
form in percona-xtradb-cluster-server.install:
debian/extra/percona-telemetry-setup.sh usr/libexec/percona-xtradb-cluster/
debian/extra/percona-telemetry-cleanup.sh usr/libexec/percona-xtradb-cluster/
This copies them from the source tree into the package, but cmake
`make install` also drops them at debian/tmp/usr/libexec/...,
leaving the cmake-installed copies orphaned. Switch to the
path-pickup form (single token) so dh_install consumes the
debian/tmp/ copies directly and no orphans remain.
3. The remaining orphans cannot currently be packaged:
- MySQL Router private libs and sample configs:
`percona-xtradb-cluster-mysql-router.install` and
`-mysql-router-dev.install` exist but there is no matching
`Package: percona-xtradb-cluster-mysql-router` stanza in
debian/control, so dh_install never reads those install files.
Wholesale-glob the affected paths into debian/not-installed
until router packages are added to control.
- garbd ships duplicate `usr/share/garb*` copies of files
already claimed under `lib/systemd/system/` and `usr/bin/`
by percona-xtradb-cluster-garbd.install. The usr/share/ copies
are leftover from the upstream galera SCons build.
- `usr/man/man8/garbd.8` lands under usr/man/ (legacy path)
instead of usr/share/man/. Excluding is safer than rewriting
the path until galera is fixed upstream.
- `usr/share/doc/percona-xtradb-cluster-5.7/mysqld.sym.gz` is a
stale stub referencing the old 5.7 doc directory; drop it on
the 9.x line.
…cate build-ids
Three RPM build failures on EL9 in the same %install pass:
1. "Duplicate build-ids ...libabsl_*.so.debug and ...libabsl_*.so.debug"
The upstream MySQL build installs byte-identical absl / protobuf
private libraries under both %{_libdir}/mysql/private/ and
%{_libdir}/mysqlrouter/private/. rpm's default _build_id_links mode
("compat") rejects two debuginfo files sharing the same build-id.
PS works around this by patching /usr/lib/rpm/find-debuginfo.sh in
its builder script (percona-server-9.0_builder.sh
apply_workaround_bug_304121). Doing it at the spec level instead
via `%global _build_id_links alldebug` is equivalent but doesn't
require container-level surgery; "alldebug" appends a unique
.debug suffix to the build-id symlinks so duplicates coexist.
2. "Installed (but unpackaged) file(s) found:
/usr/lib64/mysql/private/libicu{i18n,stubdata,uc}.so.77.1
/usr/lib64/mysqlrouter/private/libicu{i18n,stubdata,uc}.so.77.1"
No %files block claimed the ICU private shared libraries. Add
libicu{i18n,stubdata,uc}.so.* claims to both
percona-xtradb-cluster-server (next to the existing libprotobuf
claims) and percona-xtradb-cluster-mysql-router %files sections.
Mirrors the PS percona-server.spec.in pattern (lines 1056-1058 +
1619-1621).
3. "Installed (but unpackaged) file: /usr/lib64/mysql/libmysqlclient.so.21"
The compat-libs install loop added in 351bcba extracts both
the bare soname symlink `libmysqlclient.so.${compatlib}` and the
fully-versioned file `libmysqlclient.so.${compatlib}.<minor>.<patch>`
from the upstream shared rpm, but the %files glob was
`libmysqlclient.so.${compatlib}.*` (with a leading dot before `*`)
which only matched the versioned file. The bare soname symlink
was orphaned.
Drop the dot: `libmysqlclient.so.${compatlib}*` matches both.
The pattern is still bounded to the compatlib soname (won't grab
anything unrelated) because only files extracted by the EL8+
install loop land in %{_libdir}/mysql/.
MySQL Router is a standalone product and does not belong in the PXC
distribution. Remove it from all three artifact paths.
Build:
- Add `-DWITH_ROUTER=OFF` to the three shared cmake flag definitions:
build-ps/percona-xtradb-cluster.spec.in (%{cmake_common_flags})
build-ps/debian/rules (CMAKE_COMMON_FLAGS)
build-ps/build-binary.sh (CMAKE_COMMON_FLAGS bash array)
With router disabled at the cmake level, no router binaries,
libraries, or man pages are produced; nothing reaches debian/tmp or
BUILDROOT, so no post-build exclusion is needed.
RPM spec (build-ps/percona-xtradb-cluster.spec.in):
- Remove %package / %description stanzas for
percona-xtradb-cluster-mysql-router
percona-xtradb-cluster-mysql-router-devel
(replaced with a single comment block explaining the decision).
- Remove the %install steps that mkdir /var/{run,log}/mysqlrouter and
install packaging/rpm-common/mysqlrouter.{init,conf.in}.
- Remove the %pre / %post / %preun / %postun scriptlets that created
the mysqlrouter user/group and managed the systemd service.
- Remove the entire %files -n percona-xtradb-cluster-mysql-router
block (~50 lines of /usr/bin/mysqlrouter*, %{_libdir}/mysqlrouter/
private/*, man pages, /etc/mysqlrouter, /var/log/mysqlrouter, etc).
- Tighten the _build_id_links rationale: now that router/private and
mysql/private no longer ship duplicate copies of the absl/protobuf/
icu libs, the build-id collision is gone. Keep `alldebug` mode as a
defensive setting for future private-lib duplication, but the
immediate driver is no longer present.
Debian (build-ps/debian/):
- git rm the now-unused install files and maintainer scripts:
percona-xtradb-cluster-mysql-router.install
percona-xtradb-cluster-mysql-router-dev.install
percona-xtradb-cluster-mysql-router.{preinst,postinst,prerm,postrm}
percona-xtradb-cluster-mysql-router.mysqlrouter.init
percona-mysql-router.{init,mysqlrouter.service,mysqlrouter.tmpfile}
- git rm the debian/extra/ router config templates and apparmor profile:
mysqlrouter.conf, mysqlrouter-systemd-start, apparmor-profile-router
- Replace the router-exclusion globs added in 6794922
(debian/not-installed) with a single comment — with WITH_ROUTER=OFF
no router files reach debian/tmp, so the exclusions are no-ops.
Builder script (build-ps/pxc_builder.sh):
- Drop the get_sources router prep:
sed of ROUTER_RUNTIMEDIR
mysqlrouter.service / mysqlrouter.tmpfiles.d / mysqlrouter.conf
template copying with the fallback path added in 7fa5f42.
- Delete a stray line at the old line 1125 inside build_tarball() that
contained the rpm-syntax fragment
%{_libdir}/mysqlrouter/private/libprotobuf-lite.so.*
in the middle of a shell function. This was always dead code
(would have produced a "command not found" at runtime) and was
evidently copy-pasted from the spec at some earlier point.
Tarball build (build-ps/build-binary.sh):
- Remove the -DROUTER_INSTALL_LIBDIR / -DROUTER_INSTALL_PLUGINDIR
cmake flags from the RelWithDebInfo and PGO-consume cmake
invocations (now no-ops with WITH_ROUTER=OFF).
- Drop lib/mysqlrouter/plugin and lib/mysqlrouter/private from
DIRLIST in the link function (gather_libs would have silently
skipped them, but the entries are dead code).
- Delete the "LIBS MYSQLROUTER" / "BINS MYSQLROUTER" set_runpath
blocks (silent no-ops on missing paths but unnecessary).
Users who need MySQL Router should install it from the standalone
percona-mysql-router product.
… + tarball)" This reverts commit 255de6d.
release-9.6.0 ran with debhelper compat 9, where dh_missing is opt-in,
so the router files cmake produced into debian/tmp/ were silently
dropped (debian/control has no Package: percona-xtradb-cluster-mysql-
router stanza, so the .install files are dead). PXC release-9.6.0-2
runs at compat 13 where dh_missing --fail-missing is the default and
every orphaned router file becomes a fatal build error.
The previous not-installed only excluded router private/ libraries +
the sample config, covering perhaps a third of what cmake actually
installs under mysqlrouter paths. Add globs for every other category:
- usr/bin/mysqlrouter* (6 binaries)
- usr/lib/mysqlrouter/plugin/*.so (~15 plugins)
- usr/lib/*/libmysqlharness{.a,.so} (router-dev static + symlink)
- usr/lib/*/libmysqlrouter{,_http}.so (router-dev symlinks)
- usr/share/man/man1/mysqlrouter* (3 man pages)
- usr/share/mysqlrouter/* (docs, mysqlrouter-systemd-start)
- etc/mysqlrouter/* (config)
- etc/logrotate.d/mysqlrouter (logrotate)
- etc/apparmor.d/usr.bin.mysqlrouter (apparmor profile)
- usr/{LICENSE,README}.router (source-tree copies)
This faithfully replicates release-9.6.0's *effective* deb behavior:
router is built (cmake WITH_ROUTER=ON default unchanged) but no router
deb is shipped. RPM packaging is untouched; the RPM build continues to
ship percona-xtradb-cluster-mysql-router rpms.
…ssion The `%global _build_id_links alldebug` added in 9072f82 to suppress the duplicate-build-id error between %{_libdir}/mysql/private/ and %{_libdir}/mysqlrouter/private/ now triggers a spec parse failure on the Jenkins build hosts: error: Package already exists: %package debuginfo The error did not surface earlier because router was removed in 255de6d; restoring router in 004854a (to copy release-9.6.0's router behavior) brought back the surrounding subpackage population that interacts badly with `alldebug` mode on the build host's rpm 4.16/4.18. PS does not set _build_id_links and handles the same duplicate-build-id problem from the builder side by patching /usr/lib/rpm/find-debuginfo.sh in install_deps (see PS build-ps/percona-server-9.0_builder.sh apply_workaround_bug_304121). Removing the spec-level override here restores release-9.6.0's debuginfo behavior and unblocks -bs. If duplicate-build-id errors resurface at %install time, the next step is to port apply_workaround_bug_304121 into pxc_builder.sh rather than trying alternative spec-level macros — the find-debuginfo.sh patch is the well-understood path that PS already uses in production.
…ctive The comment block added in 1b7f5f8 quoted the previous error message verbatim, including the literal text `%package debuginfo` and `%global _build_id_links alldebug`. rpm's spec parser expands %-macros inside #-comments before stripping them, so `%package debuginfo` was treated as a real `%package` directive, opening a debuginfo subpackage context. The actual Name:/Version:/Release:/ Summary:/License: lines that follow were then attributed to the debuginfo subpackage, leaving the main package without them: error: Name field must be present in package: (main package) error: Version field must be present in package: (main package) error: Release field must be present in package: (main package) error: Summary field must be present in package: (main package) error: License field must be present in package: (main package) Escape every `%` in the comment block as `%%` so rpm leaves the literal text alone. Same fix pattern as the existing %%{!?src_base:...} escape at line 195.
… -bs Commit 4e49fe5 ("align PXC spec with Percona Server modernization") removed the line %define _build_id_links none that release-9.6.0 had at the top of the spec preamble (line 22). release-9.6.0's setting disables rpm's per-file build-id symlink generation entirely, which both (a) sidesteps the duplicate-build-id error between mysql/private/ and mysqlrouter/private/ and (b) suppresses the auto-debuginfo subpackage that the build hosts' rpm 4.16/4.18 was rejecting with: error: Package already exists: %package debuginfo Putting the line back restores release-9.6.0's working baseline. This is independent of and removes the need for the _build_id_links=alldebug experiment from 9072f82 and the comment-block-quoting follow-ups (1b7f5f8, 5669365). PS doesn't ship this line either; PS's Jenkins hosts must run rpm with a default build_id_links value that doesn't trigger the same error, or its build path doesn't produce the duplicate libs. PXC's Jenkins hosts demonstrably do trigger it, so the explicit `none` is needed here.
…gression
WORKAROUND for the persistent rpmbuild -bs failure on Jenkins:
error: Package already exists: %package debuginfo
Verification timeline shows the trigger is not in the spec content:
commit 9072f82: rpmbuild -bs PASSED (build reached %install)
commit 004854a: rpmbuild -bs FAILED (same error)
diff 9072f82..004854a -- spec.in: EMPTY (byte-identical)
Same spec content, opposite outcome — the change has to be in the
build agent (most likely rpm 4.16 -> 4.18 bump on EL9). Several
spec-level theories were tried and all failed: removing
_build_id_links alldebug, restoring _build_id_links none from
release-9.6.0, escaping % chars in comments. None of these reach the
underlying issue because the trigger is not in the spec.
Add `%define debug_package %{nil}` in the preamble to tell rpm
*not* to auto-generate a debuginfo subpackage at all. This sidesteps
the collision entirely.
Trade-off: this release will not ship the `*-debuginfo.rpm` packages
that release-9.6.0 produced. Recovering them requires either:
1. Reverting this %define once the Jenkins agent rpm regression
is resolved upstream, OR
2. Porting PS's apply_workaround_bug_304121 patch of
/usr/lib/rpm/find-debuginfo.sh into
pxc_builder.sh:install_deps so the agent's rpm doesn't trip
on the duplicate build-ids between mysql/private/ and
mysqlrouter/private/.
Comment block escapes every `%` as `%%` to prevent rpm from
parsing `%package debuginfo` / `%define ...` references inside #-
comments as directives (the lesson from the previous round of
spec-parse regressions in 5669365).
The Jenkins build agent's newer rpm parses %-prefixed section directives inside #-comments as real directives. After 2fc3fe7 sidestepped the "Package already exists: %package debuginfo" issue with %{nil} debug_package, the next iteration hit a new spec parse error: error: line 1977: second %install triggered by a comment I introduced in 9072f82: # loop in %install already extracts both forms from the upstream # shared rpm; ... rpm 4.18 reads `%install` here as a real second %install section declaration and rejects the spec. Same hazard exists for several other comment blocks added during the release-9.6.0-2 modernization. Escape every section-directive reference inside #-comments with %% to stop the macro/directive parser: %check -> %%check (line 102, dwz/EL8 explainer) %post -> %%post (lines 1001, 1652, telemetry comments) %postun -> %%postun (lines 1001, 1479, 1652) %install -> %%install (line 1977, compat-symlink comment) This is the same lesson as 5669365 — section directives inside #-comments are parsed by newer rpm and must be %%-escaped. Audited the whole spec for the same pattern; remaining %-prefixed text inside comments (%{...} macro references, %doc / %attr / %if marker references) are either already escaped, already non-section keywords (safe to repeat), or commented-out spec content blocks that have always been in the spec since release-9.6.0 without issue.
…root
EL10 build hits /usr/lib/rpm/check-buildroot at end of %install:
Found '/mnt/.../BUILDROOT/percona-xtradb-cluster-9.6.0-1.2.el10.x86_64'
in installed files; aborting
error: Bad exit status from /var/tmp/rpm-tmp.* (%install)
The flagged file is /usr/share/mysql-test/core — a binary core dump
from a crashed mysqld during the PGO run-profile-suite pass. Binary
cores embed the absolute path of the crashed executable, which on
the build host is under .../BUILDROOT/..., so check-buildroot
correctly refuses to ship it.
The existing PGO cleanup at line ~913
rm -r $(readlink mysql-test/var)
only catches cores written under mysql-test/var/ (where mtr.pl
normally redirects everything). If mysqld crashes during test
setup, before var/ is configured, the core can land in the parent
mysql-test/ directory and survive that cleanup. From there it
follows `make install` into share/mysql-test/core and triggers
check-buildroot at %install end.
Add a defensive scrub at the end of %install:
find $RBR -name 'core' -type f -delete
find $RBR -name 'core.[0-9]*' -type f -delete
Covers both `core` and the kernel.core_pattern variants like
`core.1234`. Wrapped in `|| :` so missing-file (no cores produced)
is non-fatal.
Note: this only treats the symptom. The underlying mysqld crash
during MTR profile-suite should be investigated separately — the
test run is supposed to complete cleanly so PGO profile data is
representative. But getting builds green is the higher priority.
…n_example.ini
dh_missing on the binary deb build flagged three categories of
orphans:
- usr/lib/mysql/private/libicu{i18n,stubdata,uc}.so.77.1
- usr/lib/mysql-test/... (thousands of .test/.result/.inc files,
plus tsan.supp / valgrind.supp / mtr framework)
- usr/lib/mysql/plugin/daemon_example.ini
The libicu shared libs are runtime dependencies of mysqld and must
ship. Add them to percona-xtradb-cluster-server-core.install
alongside the other usr/lib/mysql/private/ entries (libabsl_*.so,
libprotobuf*.so.*). This mirrors the RPM-side libicu addition from
9072f82.
The mysql-test/ tree is a separate question. release-9.6.0 had the
identical situation — cmake installed the full mtr framework under
usr/lib/mysql-test/ but no .install file claimed it — and got away
with it because debhelper compat 9 doesn't run dh_missing
strictly. At compat 13 it's fatal. PS handles this by claiming
usr/lib/mysql-test/* in percona-server-test.install, but PXC's
test package keeps a tighter scope (just mysqlxtest +
percona-xtradb-cluster-testsuite). Honor the original PXC intent
of not shipping the mtr tree and add an explicit exclude to
debian/not-installed.
daemon_example.ini is a sample config for the daemon_example
plugin, which itself is a build-time example not shipped in any
PXC package. Same treatment — exclude.
This is consistent with the user request to exclude test files
from PXC packages in general.
…helpers
Next dh_missing iteration surfaced six more categories of orphans
in debian/tmp/, all matching what release-9.6.0 silently dropped at
debhelper compat 9 but which compat 13 treats as fatal:
1. garb/* — the galera/garb source build directory leaking into
make-install output: CMakeFiles/, *.cpp, *.hpp, files/*,
Makefile, SConscript, CTestTestfile.cmake. None of this is
shippable. Single glob excludes the whole tree.
2. README at debian/tmp top level + everything under usr/doc/
(AUTHORS, COPYING, LICENSE.asio, README). Docs already ship via
debian/copyright and dh_installdocs from the canonical
usr/share/doc/<package>/ locations; these stray copies are
duplicates.
3. usr/lib/libgalera_smm.so duplicate. The .so used by mysqld
ships from the same source file but to the canonical
usr/lib/galera4/ path via the
libgalera_smm.so usr/lib/galera4
rule in percona-xtradb-cluster-server-core.install. The
top-level usr/lib/ copy is redundant.
4. Test binaries: mysql_test_event_tracking, mysqltest_safe_process
— release-9.6.0 didn't ship these; matches the existing PXC
intent to ship only mysqlxtest from the test package.
5. Helper binaries: ps-admin, wsrep_sst_clone — release-9.6.0
also didn't ship these. wsrep_sst_clone in particular has been
superseded by wsrep_sst_xtrabackup-v2 (which IS shipped via
percona-xtradb-cluster-server.install).
Trade-off: keeping wsrep_sst_clone out matches release-9.6.0 but
means users who want the rsync-clone SST method would need to
restore it. If that's a regression we want to avoid, the line can
be removed from not-installed and added to
percona-xtradb-cluster-server.install instead.
dh_dwz fails the binary stage on PXC's PGO+LTO build:
dwz: debian/percona-xtradb-cluster-server-core/usr/sbin/mysqld:
DWARF compression not beneficial - old size 489001293 new size 489195094
dh_dwz: error: Aborting due to earlier error
make: *** [debian/rules:4: binary] Error 25
The mysqld debug section is ~466 MB. dwz tries to deduplicate
DIEs via its multi-file mode and produces output ~194 KB *larger*
than input — because GCC's PGO+LTO has already inlined and
specialised the call graph, so the duplicated DIEs that dwz would
normally merge are gone, but the multi-file index overhead remains.
dwz then exits non-zero and aborts dh.
Override `dh_dwz` to skip. Debug payloads still ship uncompressed
in their respective -dbg / -debug packages, they just don't get
the multi-file dedup pass. This is the same problem the RPM build
sidesteps on EL8 via the existing
%global _find_debuginfo_dwz_opts %{nil}
gate.
The PS deb spec doesn't override dh_dwz, presumably because PS's
plain (non-PGO) builds compress fine. PXC at compat 13 with PGO
enabled by default needs the explicit skip.
EvgeniyPatlan
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.