Skip to content

Commit fa4ed87

Browse files
brianjmurrellbrianjmurrell
committed
Build on Ubuntu
Skip-PR-comments: true PR-repos-ubuntu20: dtcmp@PR-6:lastBuild mpich@PR-64:lastBuild lwgrp@PR-6:lastBuild libcircle@PR-6:lastBuild hdf5-vol-daos-mpich@PR-38:lastBuild hdf5@PR-38:lastBuild daos@PR-11205:73 Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
1 parent 693bf54 commit fa4ed87

13 files changed

Lines changed: 262 additions & 88 deletions

‎debian/changelog‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
mpifileutils (0.11.1-4) focal; urgency=medium
2+
3+
* Initial packaging
4+
5+
-- Brian J. Murrell <brian.murrell@intel.com> Mon, 30 Jan 2023 08:02:22 -0500

‎debian/control‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
Source: mpifileutils
2+
Maintainer: DAOS <daos@daos.io>
3+
Uploaders: Brian J. Murrell <brian.murrell@intel.com>
4+
Section: devel
5+
Priority: optional
6+
Build-Depends: debhelper-compat (= 12),
7+
cmake,
8+
libbz2-dev,
9+
libssl-dev,
10+
uuid-dev,
11+
libattr1-dev,
12+
libdtcmp-mpich-dev,
13+
libcircle-mpich-dev,
14+
libhdf5-vol-daos-mpich-dev
15+
Standards-Version: 4.4.1.0
16+
Vcs-Browser: https://github.com/hpc/mpifileutils
17+
Vcs-Git: https://github.com/hpc/mpifileutils.git
18+
Homepage: https://github.com/hpc/mpifileutils.git
19+
20+
Package: mpifileutils
21+
Architecture: any
22+
Multi-Arch: foreign
23+
Depends: ${misc:Depends},
24+
${shlibs:Depends}
25+
Description: mpiFileUtils provides both a library called libmfu and a suite of MPI-based
26+
tools to manage large datasets, which may vary from large directory trees to
27+
large files. High-performance computing users often generate large datasets
28+
with parallel applications that run with many processes (millions in some
29+
cases). However those users are then stuck with single-process tools like cp
30+
and rm to manage their datasets. This suite provides MPI-based tools to handle
31+
typical jobs like copy, remove, and compare for such datasets, providing
32+
speedups of up to 20-30x. It also provides a library that simplifies the
33+
creation of new tools or can be used in applications.
34+
35+
Package: libmfu4-mpich
36+
Architecture: linux-any
37+
Depends: ${misc:Depends}, ${shlibs:Depends}
38+
Description: mpiFileUtils provides both a library called libmfu and a suite of MPI-based
39+
tools to manage large datasets, which may vary from large directory trees to
40+
large files. High-performance computing users often generate large datasets
41+
with parallel applications that run with many processes (millions in some
42+
cases). However those users are then stuck with single-process tools like cp
43+
and rm to manage their datasets. This suite provides MPI-based tools to handle
44+
typical jobs like copy, remove, and compare for such datasets, providing
45+
speedups of up to 20-30x. It also provides a library that simplifies the
46+
creation of new tools or can be used in applications.
47+
.
48+
This package contains the shared libraries for use with MPICH2.
49+
50+
Package: libmfu-mpich-dev
51+
Section: libdevel
52+
Architecture: linux-any
53+
Depends: libmfu4-mpich (= ${binary:Version}), ${misc:Depends}
54+
Description: mpiFileUtils provides both a library called libmfu and a suite of MPI-based
55+
tools to manage large datasets, which may vary from large directory trees to
56+
large files. High-performance computing users often generate large datasets
57+
with parallel applications that run with many processes (millions in some
58+
cases). However those users are then stuck with single-process tools like cp
59+
and rm to manage their datasets. This suite provides MPI-based tools to handle
60+
typical jobs like copy, remove, and compare for such datasets, providing
61+
speedups of up to 20-30x. It also provides a library that simplifies the
62+
creation of new tools or can be used in applications.
63+
.
64+
This package is needed to compile programs against libmfu4-mpich.
65+
It contains the header files and links needed for compiling.

‎debian/copyright‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
2+
Upstream-Name: mpifileutils
3+
Upstream-Contact: https://github.com/hpc/mpifileutils
4+
Source: https://github.com/hpc/mpifileutils
5+
Copyright: 2018, Lawrence Livermore National Security, LLC
6+
2006-2007,2011-2015, Los Alamos National Security, LLC
7+
2013-2015, UT-Battelle, LLC
8+
2015, DataDirect Networks, Inc.
9+
10+
Files: *
11+
Copyright: 2018, Lawrence Livermore National Security, LLC
12+
2006-2007,2011-2015, Los Alamos National Security, LLC
13+
2013-2015, UT-Battelle, LLC
14+
2015, DataDirect Networks, Inc.
15+
License: BSD 3-Clause "New" or "Revised" License
16+
BSD 3-Clause License
17+
18+
Copyright (c) 2018, Lawrence Livermore National Security, LLC
19+
Copyright (c) 2006-2007,2011-2015, Los Alamos National Security, LLC
20+
Copyright (2013-2015) UT-Battelle, LLC
21+
Copyright (c) 2015, DataDirect Networks, Inc.
22+
23+
All rights reserved.
24+
25+
Redistribution and use in source and binary forms, with or without
26+
modification, are permitted provided that the following conditions are met:
27+
28+
* Redistributions of source code must retain the above copyright notice, this
29+
list of conditions and the following disclaimer.
30+
31+
* Redistributions in binary form must reproduce the above copyright notice,
32+
this list of conditions and the following disclaimer in the documentation
33+
and/or other materials provided with the distribution.
34+
35+
* Neither the name of the copyright holder nor the names of its
36+
contributors may be used to endorse or promote products derived from
37+
this software without specific prior written permission.
38+
39+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
40+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
41+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
42+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
43+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
44+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
45+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
46+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
47+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
48+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎debian/libmfu-mpich-dev.install‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
usr/include/*
2+
usr/lib/x86_64-linux-gnu/lib*.so

‎debian/libmfu4-mpich.install‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
usr/lib/x86_64-linux-gnu/lib*.so.*

‎debian/mpifileutils.install‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
usr/bin/*

‎debian/rules‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#! /usr/bin/make -f
2+
3+
export DH_VERBOSE=1
4+
5+
DEB_HOST_MULTIARCH ?= $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
6+
7+
%:
8+
dh $@
9+
10+
override_dh_auto_configure:
11+
dh_auto_configure -- -DENABLE_DAOS=ON \
12+
-DENABLE_LIBARCHIVE=OFF \
13+
-DENABLE_HDF5=ON \
14+
-DHDF5_INCLUDE_DIRS=/usr/include/hdf5/mpich \
15+
-DHDF5_LIBRARIES=/usr/lib/x86_64-linux-gnu/hdf5/mpich/libhdf5.so
16+
17+
override_dh_auto_install-arch:
18+
dh_auto_install
19+
chrpath -d debian/tmp/usr/lib/*/*.so* debian/tmp/usr/bin/*
20+
rm -f debian/tmp/usr/lib/*/*.la

‎debian/source/format‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.0 (quilt)

‎packaging/Dockerfile.mockbuild‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -34,18 +34,24 @@ RUN echo "$USER:$PASSWD" | chpasswd
3434
# add the user to the mock group so it can run mock
3535
RUN usermod -a -G mock $USER
3636

37+
ARG CB0
38+
RUN dnf -y upgrade && \
39+
dnf clean all
40+
3741
# Monkey-patch rpmlint until a new release is made with
3842
# https://github.com/rpm-software-management/rpmlint/pull/795 in it
43+
# But make sure to patch after dnf upgrade so that an upgraded rpmlint
44+
# RPM doesn't wipe out our patch
3945
COPY packaging/rpmlint--ignore-unused-rpmlintrc.patch .
40-
RUN (cd $(python3 -c 'import site; print(site.getsitepackages()[-1])') && \
41-
patch -p1 && \
42-
rm -f rpmlint/__pycache__/{cli,lint}.*.pyc) < rpmlint--ignore-unused-rpmlintrc.patch && \
46+
RUN (cd $(python3 -c 'import site; print(site.getsitepackages()[-1])') && \
47+
if ! grep -e --ignore-unused-rpmlintrc rpmlint/cli.py; then \
48+
if ! patch -p1; then \
49+
exit 1; \
50+
fi; \
51+
rm -f rpmlint/__pycache__/{cli,lint}.*.pyc; \
52+
fi) < rpmlint--ignore-unused-rpmlintrc.patch; \
4353
rm -f rpmlint--ignore-unused-rpmlintrc.patch
4454

45-
ARG CB0
46-
RUN dnf -y upgrade && \
47-
dnf clean all
48-
4955
# show the release that was built
5056
ARG CACHEBUST
5157
RUN cat /etc/os-release

‎packaging/Dockerfile.ubuntu.20.04‎

Lines changed: 45 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -7,33 +7,60 @@
77
FROM ubuntu:20.04
88
LABEL org.opencontainers.image.authors="daos@daos.groups.io"
99

10-
# use same UID as host and default value of 1000 if not specified
11-
ARG UID=1000
12-
ARG REPO_URL=""
13-
ARG REPO_UBUNTU_20_04=""
10+
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
11+
curl gpg
12+
13+
ARG REPO_FILE_URL
14+
RUN if [ -n "$REPO_FILE_URL" ]; then \
15+
cd /etc/apt/sources.list.d && \
16+
curl -f -o daos_ci-ubuntu20.04-artifactory.list.tmp \
17+
"$REPO_FILE_URL"daos_ci-ubuntu20.04-artifactory.list && \
18+
true > ../sources.list && \
19+
mv daos_ci-ubuntu20.04-artifactory.list.tmp \
20+
daos_ci-ubuntu20.04-artifactory.list; \
21+
fi; \
22+
cd -; \
23+
curl -f -O "$REPO_FILE_URL"esad_repo.key; \
24+
gpg --no-default-keyring --keyring ./temp-keyring.gpg \
25+
--import esad_repo.key; \
26+
mkdir -p /usr/local/share/keyrings/; \
27+
gpg --no-default-keyring --keyring ./temp-keyring.gpg --export \
28+
--output /usr/local/share/keyrings/daos-stack-public.gpg; \
29+
rm ./temp-keyring.gpg; \
30+
url_prefix=https://downloads.linux.hpe.com/SDR/; \
31+
for url in hpPublicKey2048.pub \
32+
hpPublicKey2048_key1.pub \
33+
hpePublicKey2048_key1.pub; do \
34+
curl -f -O "$url_prefix$url"; \
35+
gpg --no-default-keyring --keyring ./temp-keyring.gpg \
36+
--import "$(basename $url)"; \
37+
done; \
38+
gpg --no-default-keyring --keyring ./temp-keyring.gpg --export \
39+
--output /usr/local/share/keyrings/hpe-sdr-public.gpg; \
40+
rm ./temp-keyring.gpg
1441

1542
# Install basic tools
1643
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
1744
autoconf bash ca-certificates curl debhelper dh-make \
18-
dpkg-dev dh-python doxygen gcc git git-buildpackage locales \
19-
make patch pbuilder pkg-config python3-dev python3-distro \
20-
python3-distutils rpm scons wget cmake valgrind
45+
dpkg-dev dh-python doxygen gcc git git-buildpackage \
46+
javahelper locales make patch pbuilder pkg-config \
47+
python3-dev python3-distro python3-distutils rpm scons wget \
48+
cmake valgrind rpmdevtools
2149

22-
# rpmdevtools
23-
RUN echo "deb [trusted=yes] ${REPO_URL}${REPO_UBUNTU_20_04} focal main" > /etc/apt/sources.list.d/daos-stack-ubuntu-stable-local.list
24-
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
25-
rpmdevtools
50+
# use same UID as host and default value of 1000 if not specified
51+
ARG UID=1000
2652

2753
# Add build user (to keep chrootbuild happy)
2854
ENV USER build
2955
RUN useradd -u $UID -ms /bin/bash $USER
3056

3157
# need to run the build command as root, as it needs to chroot
32-
RUN if ! grep "^#includedir /etc/sudoers.d" /etc/sudoers; then \
33-
echo "#includedir /etc/sudoers.d" >> /etc/sudoers; \
34-
fi; \
35-
echo "Defaults env_keep += \"DPKG_GENSYMBOLS_CHECK_LEVEL\"" > /etc/sudoers.d/build; \
36-
echo "build ALL=(ALL) NOPASSWD: /usr/sbin/pbuilder" >> /etc/sudoers.d/build; \
37-
chmod 0440 /etc/sudoers.d/build; \
38-
visudo -c; \
58+
RUN if ! grep "^#includedir /etc/sudoers.d" /etc/sudoers; then \
59+
echo "#includedir /etc/sudoers.d" >> /etc/sudoers; \
60+
fi; \
61+
echo "Defaults env_keep += \"DPKG_GENSYMBOLS_CHECK_LEVEL\"" > /etc/sudoers.d/build; \
62+
echo "build ALL=(ALL) NOPASSWD: /usr/bin/tee /root/.pbuilderrc" >> /etc/sudoers.d/build; \
63+
echo "build ALL=(ALL) NOPASSWD: /usr/sbin/pbuilder" >> /etc/sudoers.d/build; \
64+
chmod 0440 /etc/sudoers.d/build; \
65+
visudo -c; \
3966
sudo -l -U build

0 commit comments

Comments
 (0)