Skip to content

Commit 62da3bd

Browse files
EZoniEZoni
committed
Merge ECP-WarpX:development into EZoni:ci_docs_skip_checks 
2 parents 7a5e040 + 2cdcb77 commit 62da3bd

File tree

113 files changed

+2166
-674
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

113 files changed

+2166
-674
lines changed

‎.github/workflows/clang_sanitizers.yml

Lines changed: 4 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,7 @@ jobs:
2929
skip: ${{ env.SKIP_CHECKS }}
3030
build_UB_sanitizer:
3131
name: Clang UB sanitizer
32-
runs-on: ubuntu-22.04
33-
container: ubuntu:23.10
32+
runs-on: ubuntu-24.04
3433
needs: skip_checks
3534
if: ${{ github.event.pull_request.draft == false && needs.skip_checks.outputs.skip == 'false' }}
3635
env:
@@ -76,9 +75,6 @@ jobs:
7675
du -hs ~/.cache/ccache
7776
- name: Run with UB sanitizer
7877
run: |
79-
# We need these two lines because these tests run inside a docker container
80-
export OMPI_ALLOW_RUN_AS_ROOT=1
81-
export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1
8278
export OMP_NUM_THREADS=2
8379
#MPI implementations often leak memory
8480
export "ASAN_OPTIONS=detect_leaks=0"
@@ -88,10 +84,10 @@ jobs:
8884
mpirun -n 2 ./build/bin/warpx.3d Examples/Physics_applications/laser_acceleration/inputs_base_3d
8985
build_thread_sanitizer:
9086
name: Clang thread sanitizer
91-
runs-on: ubuntu-22.04
92-
container: ubuntu:23.10
87+
runs-on: ubuntu-24.04
88+
# TODO Fix data race conditions and re-enable job
9389
needs: skip_checks
94-
if: ${{ github.event.pull_request.draft == false && needs.skip_checks.outputs.skip == 'false' }}
90+
if: 0 #${{ github.event.pull_request.draft == false && needs.skip_checks.outputs.skip == 'false' }}
9591
env:
9692
CC: clang
9793
CXX: clang++
@@ -152,9 +148,6 @@ jobs:
152148
export PMIX_MCA_gds=hash
153149
export TSAN_OPTIONS='ignore_noninstrumented_modules=1'
154150
export ARCHER_OPTIONS="verbose=1"
155-
# We need these two lines because these tests run inside a docker container
156-
export OMPI_ALLOW_RUN_AS_ROOT=1
157-
export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1
158151
export OMP_NUM_THREADS=2
159152
mpirun -n 2 ./build/bin/warpx.rz Examples/Physics_applications/laser_acceleration/inputs_base_rz warpx.serialize_initial_conditions = 0
160153
mpirun -n 2 ./build/bin/warpx.1d Examples/Physics_applications/laser_acceleration/inputs_base_1d warpx.serialize_initial_conditions = 0

‎.github/workflows/cuda.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ jobs:
134134
export LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:${LD_LIBRARY_PATH}
135135
which nvcc || echo "nvcc not in PATH!"
136136
git clone https://github.com/AMReX-Codes/amrex.git ../amrex
137-
cd ../amrex && git checkout --detach 456c93c7d9512f1cdffac0574973d7df41417898 && cd -
137+
cd ../amrex && git checkout --detach b3f67385e62f387b548389222840486c0fffca57 && cd -
138138
make COMP=gcc QED=FALSE USE_MPI=TRUE USE_GPU=TRUE USE_OMP=FALSE USE_FFT=TRUE USE_CCACHE=TRUE -j 4
139139
ccache -s
140140
du -hs ~/.cache/ccache

‎.github/workflows/dependencies/clang17.sh

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,6 @@
77

88
set -eu -o pipefail
99

10-
# This dependency file is currently used within a docker container,
11-
# which does not come with sudo.
12-
apt-get -qqq update
13-
apt-get -y install sudo
14-
1510
# `man apt.conf`:
1611
# Number of retries to perform. If this is non-zero APT will retry
1712
# failed files the given number of times.

‎.github/workflows/dependencies/hip.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ sudo apt-get install -y --no-install-recommends \
5353
rocm-dev \
5454
rocfft-dev \
5555
rocprim-dev \
56+
rocsparse-dev \
5657
rocrand-dev \
5758
hiprand-dev
5859

‎.github/workflows/dependencies/nvcc11-3.sh

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@ sudo apt-get install -y \
4141
cuda-nvml-dev-11-3 \
4242
cuda-nvtx-11-3 \
4343
libcufft-dev-11-3 \
44-
libcurand-dev-11-3
44+
libcurand-dev-11-3 \
45+
libcusparse-dev-11-3
4546
sudo ln -s cuda-11.3 /usr/local/cuda
4647

4748
# if we run out of temporary storage in CI:

‎.github/workflows/dependencies/nvcc11-8.sh

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@ sudo apt-get install -y \
4141
cuda-nvml-dev-11-8 \
4242
cuda-nvtx-11-8 \
4343
libcufft-dev-11-8 \
44-
libcurand-dev-11-8
44+
libcurand-dev-11-8 \
45+
libcusparse-dev-11-8
4546
sudo ln -s cuda-11.8 /usr/local/cuda
4647

4748
# if we run out of temporary storage in CI:

‎.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ repos:
6969
# Python: Ruff linter & formatter
7070
# https://docs.astral.sh/ruff/
7171
- repo: https://github.com/astral-sh/ruff-pre-commit
72-
rev: v0.7.4
72+
rev: v0.8.3
7373
hooks:
7474
# Run the linter
7575
- id: ruff

‎CMakeLists.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Preamble ####################################################################
22
#
33
cmake_minimum_required(VERSION 3.24.0)
4-
project(WarpX VERSION 24.11)
4+
project(WarpX VERSION 24.12)
55

66
include(${WarpX_SOURCE_DIR}/cmake/WarpXFunctions.cmake)
77

‎Docs/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ openpmd-viewer # for checksumAPI
1313

1414
# PICMI API docs
1515
# note: keep in sync with version in ../requirements.txt
16-
picmistandard==0.31.0
16+
picmistandard==0.33.0
1717
# for development against an unreleased PICMI version, use:
1818
# picmistandard @ git+https://github.com/picmi-standard/picmi.git#subdirectory=PICMI_Python
1919

‎Docs/source/conf.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,9 +107,9 @@ def __init__(self, *args, **kwargs):
107107
# built documents.
108108
#
109109
# The short X.Y version.
110-
version = "24.11"
110+
version = "24.12"
111111
# The full version, including alpha/beta/rc tags.
112-
release = "24.11"
112+
release = "24.12"
113113

114114
# The language for content autogenerated by Sphinx. Refer to documentation
115115
# for a list of supported languages.

‎Docs/source/install/hpc.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ This section documents quick-start guides for a selection of supercomputers that
5050
hpc/perlmutter
5151
hpc/pitzer
5252
hpc/polaris
53-
hpc/quartz
53+
hpc/dane
5454
hpc/summit
5555
hpc/taurus
5656
hpc/tioga

‎Docs/source/install/hpc/dane.rst

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
.. _building-dane:
2+
3+
Dane (LLNL)
4+
=============
5+
6+
The `Dane Intel CPU cluster <https://hpc.llnl.gov/hardware/compute-platforms/dane>`_ is located at LLNL.
7+
8+
9+
Introduction
10+
------------
11+
12+
If you are new to this system, **please see the following resources**:
13+
14+
* `LLNL user account <https://lc.llnl.gov`__ (login required)
15+
* `Jupyter service <https://lc.llnl.gov/jupyter>`__ (`documentation <https://lc.llnl.gov/confluence/display/LC/JupyterHub+and+Jupyter+Notebook>`__, login required)
16+
* `Production directories <https://hpc.llnl.gov/hardware/file-systems>`_:
17+
18+
* ``/p/lustre1/$(whoami)`` and ``/p/lustre2/$(whoami)``: personal directory on the parallel filesystem
19+
* Note that the ``$HOME`` directory and the ``/usr/workspace/$(whoami)`` space are NFS mounted and *not* suitable for production quality data generation.
20+
21+
22+
.. _building-dane-preparation:
23+
24+
Preparation
25+
-----------
26+
27+
Use the following commands to download the WarpX source code.
28+
Note that these commands and the shell scripts all assume the bash shell.
29+
30+
.. code-block:: bash
31+
32+
git clone https://github.com/ECP-WarpX/WarpX.git /usr/workspace/${USER}/dane/src/warpx
33+
34+
We use system software modules, add environment hints and further dependencies via the file ``$HOME/dane_warpx.profile``.
35+
Create it now:
36+
37+
.. code-block:: bash
38+
39+
cp /usr/workspace/${USER}/dane/src/warpx/Tools/machines/dane-llnl/dane_warpx.profile.example $HOME/dane_warpx.profile
40+
41+
.. dropdown:: Script Details
42+
:color: light
43+
:icon: info
44+
:animate: fade-in-slide-down
45+
46+
.. literalinclude:: ../../../../Tools/machines/dane-llnl/dane_warpx.profile.example
47+
:language: bash
48+
49+
Edit the 2nd line of this script, which sets the ``export proj=""`` variable.
50+
For example, if you are member of the project ``tps``, then run ``vi $HOME/dane_warpx.profile``.
51+
Enter the edit mode by typing ``i`` and edit line 2 to read:
52+
53+
.. code-block:: bash
54+
55+
export proj="tps"
56+
57+
Exit the ``vi`` editor with ``Esc`` and then type ``:wq`` (write & quit).
58+
59+
.. important::
60+
61+
Now, and as the first step on future logins to Dane, activate these environment settings:
62+
63+
.. code-block:: bash
64+
65+
source $HOME/dane_warpx.profile
66+
67+
Finally, since Dane does not yet provide software modules for some of our dependencies, install them once:
68+
69+
.. code-block:: bash
70+
71+
bash /usr/workspace/${USER}/dane/src/warpx/Tools/machines/dane-llnl/install_dependencies.sh
72+
source /usr/workspace/${USER}/dane/venvs/warpx-dane/bin/activate
73+
74+
.. dropdown:: Script Details
75+
:color: light
76+
:icon: info
77+
:animate: fade-in-slide-down
78+
79+
.. literalinclude:: ../../../../Tools/machines/dane-llnl/install_dependencies.sh
80+
:language: bash
81+
82+
83+
.. _building-dane-compilation:
84+
85+
Compilation
86+
-----------
87+
88+
Use the following :ref:`cmake commands <building-cmake>` to compile the application executable:
89+
90+
.. code-block:: bash
91+
92+
cd /usr/workspace/${USER}/dane/src/warpx
93+
rm -rf build_dane
94+
95+
cmake -S . -B build_dane -DWarpX_FFT=ON -DWarpX_QED_TABLE_GEN=ON -DWarpX_DIMS="1;2;RZ;3"
96+
cmake --build build_dane -j 6
97+
98+
The WarpX application executables are now in ``/usr/workspace/${USER}/dane/src/warpx/build_dane/bin/``.
99+
Additionally, the following commands will install WarpX as a Python module:
100+
101+
.. code-block:: bash
102+
103+
rm -rf build_dane_py
104+
105+
cmake -S . -B build_dane_py -DWarpX_FFT=ON -DWarpX_QED_TABLE_GEN=ON -DWarpX_APP=OFF -DWarpX_PYTHON=ON -DWarpX_DIMS="1;2;RZ;3"
106+
cmake --build build_dane_py -j 6 --target pip_install
107+
108+
Now, you can :ref:`submit Dane compute jobs <running-cpp-dane>` for WarpX :ref:`Python (PICMI) scripts <usage-picmi>` (:ref:`example scripts <usage-examples>`).
109+
Or, you can use the WarpX executables to submit Dane jobs (:ref:`example inputs <usage-examples>`).
110+
For executables, you can reference their location in your :ref:`job script <running-cpp-dane>` or copy them to a location in ``$PROJWORK/$proj/``.
111+
112+
113+
.. _building-dane-update:
114+
115+
Update WarpX & Dependencies
116+
---------------------------
117+
118+
If you already installed WarpX in the past and want to update it, start by getting the latest source code:
119+
120+
.. code-block:: bash
121+
122+
cd /usr/workspace/${USER}/dane/src/warpx
123+
124+
# read the output of this command - does it look ok?
125+
git status
126+
127+
# get the latest WarpX source code
128+
git fetch
129+
git pull
130+
131+
# read the output of these commands - do they look ok?
132+
git status
133+
git log # press q to exit
134+
135+
And, if needed,
136+
137+
- :ref:`update the dane_warpx.profile file <building-dane-preparation>`,
138+
- log out and into the system, activate the now updated environment profile as usual,
139+
- :ref:`execute the dependency install scripts <building-dane-preparation>`.
140+
141+
As a last step, clean the build directory ``rm -rf /usr/workspace/${USER}/dane/src/warpx/build_dane`` and rebuild WarpX.
142+
143+
144+
.. _running-cpp-dane:
145+
146+
Running
147+
-------
148+
149+
.. _running-cpp-dane-CPUs:
150+
151+
Intel Sapphire Rapids CPUs
152+
^^^^^^^^^^^^^^^^^^^^^^^^^^
153+
154+
The batch script below can be used to run a WarpX simulation on 2 nodes on the supercomputer Dane at LLNL.
155+
Replace descriptions between chevrons ``<>`` by relevant values, for instance ``<input file>`` could be ``plasma_mirror_inputs``.
156+
157+
.. literalinclude:: ../../../../Tools/machines/dane-llnl/dane.sbatch
158+
:language: bash
159+
:caption: You can copy this file from ``Tools/machines/dane-llnl/dane.sbatch``.
160+
161+
To run a simulation, copy the lines above to a file ``dane.sbatch`` and run
162+
163+
.. code-block:: bash
164+
165+
sbatch dane.sbatch
166+
167+
to submit the job.

0 commit comments

Comments
 (0)