Skip to content

Commit 649e8e6

Browse files
gjbexgjbex
authored
Merge pull request #9 from gjbex/development
Changes for session on 2022-11-09
2 parents bb8ed35 + 32807ab commit 649e8e6

18 files changed

+303
-0
lines changed

‎examples/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -37,3 +37,6 @@ How to use Singularity, creating images/containers, and using them.
3737
to provide a server.
3838
1. `hpccm`: examples of using NVIDIA's hpccm to generate docker files and
3939
simgularity defintion files from a single description.
40+
1. `apps`: illustration of defining multiple applications in a single
41+
image.
42+
1. `multistage`: example of a multistage build file.

‎examples/apps/README.md

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Apps
2+
3+
Singularity allows to define multiple applications in an image using `%app` sections.
4+
5+
6+
## What is it?
7+
1. `apps.def`: simple Singularity definition file defining two applications,
8+
`hello` and `bye`.
9+
1. `hello.py`, `bye.py`: Python applications to run.
10+
11+
12+
## How to use?
13+
14+
The applications can be run by specifying the `--app` option, e.g.,
15+
```bash
16+
$ singularity run --app hello apps.sif
17+
$ singularity run --app bye apps.sif world
18+
```

‎examples/apps/apps.def

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
Bootstrap: docker
2+
From: docker.io/python
3+
4+
%labels
5+
Maintainer Geert Jan Bex
6+
Institute Vlaams Supercomputer Centrum
7+
8+
%files
9+
hello.py /bin/hello
10+
bye.py /bin/bye
11+
12+
%post
13+
chmod a+rx /bin/hello
14+
chmod a+rx /bin/bye
15+
16+
# hello app
17+
%apprun hello
18+
/bin/hello "$@"
19+
20+
%applabels hello
21+
BESTAPP HELLO
22+
23+
%apphelp hello
24+
say hello
25+
26+
# bye app
27+
%apprun bye
28+
/bin/bye "$@"
29+
30+
%applabels bye
31+
BESTAPP BYE
32+
33+
%apphelp bye
34+
say bye

‎examples/apps/bye.py

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env python
2+
3+
import sys
4+
5+
if len(sys.argv) > 1:
6+
name = sys.argv[1]
7+
else:
8+
name = 'anonymous'
9+
print(f'bye {name}')

‎examples/apps/hello.py

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env python
2+
3+
import sys
4+
5+
if len(sys.argv) > 1:
6+
name = sys.argv[1]
7+
else:
8+
name = 'anonymous'
9+
print(f'hello {name}')

‎examples/hpccm/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,9 @@ recipe.
77

88
## What is it?
99

10+
1. `simple.py`: very simple example.
1011
1. `development_node`: definition of a development node.
1112
1. `mssql_client`: defintion of a container with Microsoft
1213
SQL Server drivers, ODBC, FreeTDS and a conda environment
1314
for a client script.
15+
1. `r_container`: definition of an image to run R.

‎examples/hpccm/development_base.py

+54
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
'''Recipe to create either a docker container or Singularity image
2+
for a compute node on which users can log in using password authentication.
3+
The SQS queue system is available as well as a number of editors, git,
4+
make, CMake, GCC and Open-MPI
5+
6+
Usage:
7+
$ hpccm --recipe compute_node.py --format docker
8+
$ hpccm --recipe compute_node.py --format singularity
9+
'''
10+
11+
from pathlib import Path
12+
13+
14+
# Choose a base image
15+
Stage0.baseimage('ubuntu:21.10')
16+
17+
# Install editor and other tools
18+
Stage0 += apt_get(ospackages=['vim', 'less', 'ack', 'tmux', ])
19+
20+
# Install archive and compression software and utitlies
21+
Stage0 += apt_get(ospackages=['tar', 'gzip', 'bzip2', 'wget', 'ca-certificates', ])
22+
23+
# Install pip
24+
Stage0 += apt_get(ospackages=['python3-pip', ])
25+
26+
# Install version control
27+
Stage0 += apt_get(ospackages=['git', 'openssh-client', ])
28+
# Install build tools
29+
Stage0 += apt_get(ospackages=['build-essential', 'make', 'pkg-config', ])
30+
Stage0 += cmake(eula=True)
31+
32+
# Install GNU compilers (upstream)
33+
compilers = gnu()
34+
Stage0 += compilers
35+
36+
# install Open-MPI
37+
# This is the right thing to do, but it takes forever since it actually
38+
# builds Open MPI from source
39+
# Stage0 += openmpi(cuda=False, infiniband=False, version='4.0.2',
40+
# toolchain=compilers.toolchain, prefix='/usr/local/')
41+
# Fetching form repository is faster
42+
# Stage0 += apt_get(ospackages=['libopenmpi-dev', 'openmpi-common', 'openmpi-bin'])
43+
44+
# Install debugging tools
45+
Stage0 += apt_get(ospackages=['gdb', 'valgrind', 'strace', ])
46+
47+
48+
# Copy in some example code
49+
# source_dir = Path('source-code')
50+
# example_dir = '/sample_code'
51+
# for file in source_dir.glob('*'):
52+
# Stage0 += copy(src=f'{file}',
53+
# dest=f'{example_dir}/',
54+
# _mkdir=True)

‎examples/hpccm/development_intel_base.py

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
'''Recipe to create either a docker container or Singularity image
2+
for a compute node on which users can log in using password authentication.
3+
The SQS queue system is available as well as a number of editors, git,
4+
make, CMake, GCC and Open-MPI
5+
6+
Usage:
7+
$ hpccm --recipe development_intel.py --format docker
8+
$ hpccm --recipe development_intel.py --format singularity
9+
'''
10+
11+
from pathlib import Path
12+
13+
14+
# Choose a base image
15+
Stage0.baseimage('intel/oneapi-hpckit:latest')
16+
17+
# Install editor and other tools
18+
Stage0 += apt_get(ospackages=['vim', 'less', 'ack', 'tmux', ])
19+
20+
# Install archive and compression software and utitlies
21+
Stage0 += apt_get(ospackages=['tar', 'gzip', 'bzip2', 'wget', 'ca-certificates', ])
22+
23+
# Install version control
24+
Stage0 += apt_get(ospackages=['git', 'openssh-client', ])
25+
26+
# Install debugging tools
27+
Stage0 += apt_get(ospackages=['valgrind', 'strace', ])
28+
29+
# add run script, i.e., start bash
30+
Stage0 += runscript(commands=['/bin/bash'])

‎examples/hpccm/r_container/README.md

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# R container
2+
3+
Defintion of an image to run R.
4+
5+
## What is it?
6+
7+
1. `r_container.py`: hpccm definition of the image.
8+
1. `install_packages.R`: R script to install additional libraries.

‎examples/hpccm/r_container/install_packages.R

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
install.packages('raster')
2+
install.packages('sf')
3+
install.packages('sp')
4+
install.packages('rgdal')
5+
install.packages('landscapemetrics')
6+
install.packages('SpaDES')

‎examples/hpccm/r_container/r_container.py

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
'''Recipe to create either a docker container or Singularity image
2+
to perform data processing and rendering for the DSI covid-19
3+
dashboard.
4+
5+
Usage:
6+
$ hpccm --recipe dashboard.py --format docker
7+
$ hpccm --recipe dashboard.py --format singularity
8+
'''
9+
10+
from pathlib import Path
11+
12+
13+
# Choose a base image
14+
Stage0.baseimage('ubuntu:21.04')
15+
16+
# Install build tools
17+
Stage0 += apt_get(ospackages=['build-essential', 'make'])
18+
19+
# Install libraries
20+
Stage0 += apt_get(ospackages=['openssl', 'libxml2-dev', 'libcurl4-openssl-dev', 'libz-dev', 'libssl-dev',
21+
'libjpeg-dev', ])
22+
# Install GNU compilers (upstream)
23+
compilers = gnu()
24+
Stage0 += compilers
25+
26+
# install GDAL library
27+
Stage0 += apt_get(ospackages=['libgdal-dev', 'libudunits2-dev'])
28+
29+
# Install utilities
30+
Stage0 += apt_get(ospackages=['gist', 'wget', 'rclone', ])
31+
32+
# Install BLAS and Lapack
33+
Stage0 += apt_get(ospackages=['libopenblas-dev', 'liblapack-dev'])
34+
35+
# Install R
36+
Stage0 += apt_get(ospackages=['r-base', ])
37+
38+
# Install R packages
39+
Stage0 += copy(src='install_packages.R', dest='/setup/', _mkdir=True)
40+
Stage0 += shell(commands=['Rscript /setup/install_packages.R'])

‎examples/hpccm/simple.py

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
'''Recipe to create either a docker container or Singularity image
2+
for a very simple setup
3+
4+
Usage:
5+
$ hpccm --recipe simple.py --format docker
6+
$ hpccm --recipe simple.py --format singularity
7+
'''
8+
9+
# Choose a base image
10+
Stage0.baseimage('ubuntu:22.04')
11+
12+
# Install editor and other tools
13+
Stage0 += apt_get(ospackages=['vim', 'less', 'ack', 'tmux', ])
14+
15+
# Install archive and compression software and utitlies
16+
Stage0 += apt_get(ospackages=['tar', 'gzip', 'bzip2', 'wget', 'ca-certificates', ])

‎examples/multistage/README.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Multistage builds
2+
3+
Multi-stage builds have the advantage that libraries or tools
4+
required for software builds in the image but not required at
5+
runtime. This can potentially decrease the image size
6+
considerably.
7+
8+
## What is it?
9+
10+
1. `singlestage.def`: definition file for an statically
11+
build OpenMP application.
12+
1. `multistage.def`: multistage definition file for an statically
13+
build OpenMP application.
14+
1. `src`: source directory containing the OpenMP application's source
15+
file and the make file to build it.

‎examples/multistage/multistage.def

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Bootstrap: docker
2+
From: alpine:3.16.2
3+
Stage: build
4+
5+
%files
6+
src /src
7+
8+
%post
9+
apk update
10+
apk add make
11+
apk add gcc
12+
apk add musl-dev
13+
cd /src
14+
make
15+
16+
Bootstrap: docker
17+
From: alpine:3.16.2
18+
Stage: runtime
19+
20+
%files from build
21+
/src/hello_world /bin

‎examples/multistage/singlestage.def

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
Bootstrap: docker
2+
From: alpine:3.16.2
3+
4+
%files
5+
src /src
6+
7+
%post
8+
apk update
9+
apk add make
10+
apk add gcc
11+
apk add musl-dev
12+
cd /src
13+
make
14+
cp hello_world /bin

‎examples/multistage/src/Makefile

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
CC = gcc
2+
CFLAGS = -O3 -static -fopenmp -Wall -Wextra
3+
4+
hello_world: hello_world.o
5+
$(CC) $(CFLAGS) -o $@ $<
6+
7+
clean:
8+
$(RM) $(wildcard *.o) hello_world

‎examples/multistage/src/hello_world.c

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#include <omp.h>
2+
#include <stdio.h>
3+
4+
int main() {
5+
#pragma omp parallel
6+
{
7+
int nr_threads = 1;
8+
int thread_nr = 0;
9+
#ifdef _OPENMP
10+
nr_threads = omp_get_num_threads();
11+
thread_nr = omp_get_thread_num();
12+
#endif
13+
printf("hello from thread %d out of %d\n", thread_nr, nr_threads);
14+
}
15+
return 0;
16+
}

‎singularity.pptx

133 KB
Binary file not shown.

0 commit comments

Comments
 (0)