Merge branch 'main' into enable_semicoarsening

ajnonaka · ajnonaka · commit bcfeeea95c25 · 2025-10-03T09:43:13.000-07:00
diff --git a/Docs/source/GPU_Tutorial.rst b/Docs/source/GPU_Tutorial.rst
@@ -9,59 +9,10 @@
 GPU
 ==========================
 
-The tutorials in ``amrex-tutorials/ExampleCodes/GPU`` demonstrate the implementation
-of AMReX's GPU toolkit as well as provide GPU ported versions of CPU
-tutorials to help applications convert to GPUs.
-
-**Your first AMReX GPU application**
-------------------------------------
-
-This is a step-by-step guide to preparing, compiling and running your first
-AMReX GPU program.  This guide will use ``Tutorials/GPU/Launch``,
-and instructions will focus on ORNL's systems:
-
-1. Before compiling, the ``pgi`` and ``cuda`` software must be available.
-   On ORNL systems, the modules can be loaded directly by typing:
-
-.. highlight:: console
-
-::
-
-   module load pgi cuda
-
-2. Go to ``Tutorials/GPU/Launch`` to compile the executable.  Compile with
-   ``make USE_CUDA=TRUE COMP=pgi USE_MPI=TRUE USE_OMP=FALSE``, or edit the
-   ``GNUmakefile`` to match this configuration and run ``make``. This
-   should result in an executable: ``main3d.pgi.MPI.CUDA.ex``.
-
-3. On Summit systems, this executable can be submitted by using one of the run
-   scripts found in ``Tutorials/GPU``.  ``run.script`` can be used to run on
-   Summitdev, and ``run.summit`` can be used for Summit.  To change the number
-   of ranks and GPUs used in the simulation, change the number of resource sets,
-   ``n`` in the ``jsrun`` line.  For the first ``Launch`` tutorial, use ``n=1``
-   and set ``INPUTS=""`` because no input file is used in this example.
-
-When ready, submit the job script (on Summit: ``bsub run.script``).
-Congratulations! You've accelerated AMReX using GPUs!
-
-**Launch**
-----------
-
-Launch shows multiple examples of how GPU work can be offloaded using the tools
-available in AMReX. It includes examples of multiple AMReX macro launch methods,
-launching a Fortran function using CUDA and launching work using OpenACC and
-OpenMP offloading. This tutorial will be regularly updated with AMReX's
-preferred GPU launch methodologies.
+The tutorial in ``amrex-tutorials/ExampleCodes/GPU`` demonstrates how to implement a
+performance-portable application using AMReX's GPU offloading capabilities.
 
 **CNS**
 -------
 
-CNS is a direct GPU port of the ``ExampleCodes/EB/CNS`` example.
-
-**AmrCore**
------------
-
-AmrCore is a direct GPU port of the ``ExampleCodes/Amr/Advection_AmrCore`` tutorial
-that advects a single scalar field with a velocity field specified on faces, using
-strategies similar to HeatEquation and CNS.
-
+CNS solves the compressible Navier-Stokes equations on GPUs.
diff --git a/Docs/source/Python_Tutorial.rst b/Docs/source/Python_Tutorial.rst
@@ -7,12 +7,14 @@ Python
 These examples show how to use AMReX from Python.
 AMReX applications can also be interfaced to Python with the same logic.
 
-In order to build the Python tutorials, add ``-DTUTORIAL_PYTHON=ON`` to the CMake configuration options.
-Then install with ``cmake --build build --target pyamrex_pip_install``.
+In order to run the Python tutorials, you need to have pyAMReX installed.
+Please see `pyAMReX <https://github.com/AMReX-Codes/pyamrex/>`__ for more details.
+
+Alternatively, you can build the ExampleCodes in this repository with ``-DTUTORIAL_PYTHON=ON`` added to the CMake configuration options,
+then install with ``cmake --build build --target pyamrex_pip_install``, and pyamrex will be installed for you.
 
-Guided Tutorial Examples:
+Once pyAMReX is installed, you can run the following Guided Tutorial Examples:
 
 - :download:`MultiFab <../../GuidedTutorials/MultiFab/main.py>`
 - :download:`Heat Equation <../../GuidedTutorials/HeatEquation/Source/main.py>`
 
-Please see `pyAMReX <https://github.com/AMReX-Codes/pyamrex/>`__ for more details.
diff --git a/Docs/source/SUNDIALS_Tutorial.rst b/Docs/source/SUNDIALS_Tutorial.rst
@@ -4,20 +4,25 @@
 SUNDIALS and Time Integrators
 =============================
 
-This example code shows how to use the AMReX TimeIntegrator class
-with SUNDIALS backend for integration. It also include two example
-of AMReX native integrators, Forward Euler, and Explicit Runge Kutta.
-Each integration type can be chosen by selecting the corresponding
-inputs file:
+These example codes demonstrate how to use the AMReX TimeIntegrator class
+with SUNDIALS backend for integration.
 
-  - ``inputs_forward_euler`` -- Native AMReX Forward Euler integrator
+The first example code at ``amrex-tutorials/ExampleCodes/SUNDIALS/Single-Rate``
+solves the heat equation:
 
-  - ``inputs_rk3`` -- Native AMReX Explicit Runge Kutta
+.. math:: \frac{\partial\phi}{\partial t} = \nabla^2\phi.
 
-  - ``inputs_sundials_erk`` -- SUNDIALS backend
+The inputs file contains a template for single process time integration strategies.
 
-Both Runge Kutta and SUNDIALS have additional options which can
-be set by modifying the inputs file. Please see each respective inputs
-file or `AMReX User Guide:Time Integration`_ for more details.
+The second example code at ``amrex-tutorials/ExampleCodes/SUNDIALS/Reaction-Diffusion``
+solves the reaction-diffusion equation, where :math:`R` and :math:`D` are
+user-supplied reaction and diffusion coefficients:
+
+.. math:: \frac{\partial\phi}{\partial t} = D \nabla^2\phi - R \phi.
+
+The inputs file contains a template for MRI approaches, where the diffusion process
+can be treated as a "fast" partition relative to the reaction process.
+Please see each respective inputs file or
+`AMReX User Guide:Time Integration`_ for more details.
 
 .. _`AMReX User Guide:Time Integration`: https://amrex-codes.github.io/amrex/docs_html/TimeIntegration_Chapter.html#
diff --git a/ExampleCodes/Basic/HeatEquation_EX0_C/Exec/GNUmakefile b/ExampleCodes/Basic/HeatEquation_EX0_C/Exec/GNUmakefile
@@ -11,7 +11,7 @@ include $(AMREX_HOME)/Tools/GNUMake/Make.defs
 
 include ../Source/Make.package
 VPATH_LOCATIONS  += ../Source
-INCLUDE_LOCATIONS += ./Source
+INCLUDE_LOCATIONS += ../Source
 
 include $(AMREX_HOME)/Src/Base/Make.package
 
diff --git a/ExampleCodes/Particles/ElectromagneticPIC/Source/EMParticleContainerInit.cpp b/ExampleCodes/Particles/ElectromagneticPIC/Source/EMParticleContainerInit.cpp
@@ -140,7 +140,7 @@ InitParticles(const IntVect& a_num_particles_per_cell,
             unsigned int uiz = amrex::min(nz-1,amrex::max(0,iz));
             unsigned int cellid = (uix * ny + uiy) * nz + uiz;
 
-            int pidx = poffset[cellid] - poffset[0];
+            int pidx = poffset[cellid] + old_size;
 
             for (int i_part=0; i_part<num_ppc;i_part++)
             {
diff --git a/ExampleCodes/SUNDIALS/Reaction-Diffusion/Exec/README_sundials b/ExampleCodes/SUNDIALS/Reaction-Diffusion/Exec/README_sundials
@@ -1,64 +1,4 @@
-SUNDIALS installation guide:
-https://computing.llnl.gov/projects/sundials/faq#inst
+Refer to https://amrex-codes.github.io/amrex/docs_html/TimeIntegration_Chapter.html
+for SNUDIALS installation, build, and usage instructions.
 
-Installation
-
-# You need SUNDIALS v7.1.1 or later.
-# Check https://computing.llnl.gov/projects/sundials/sundials-software to see if it's available for download.
-# If so, download sundials-x.y.z.tar.gz and extract it at the same level as amrex using
->> tar -xzvf sundials-x.y.z.tar.gz # where x.y.z is the version of sundials
->> mv sundials-x.y.z sundials-src
-
-# If v7.1.1. is not available on the website, clone the git repo directly and use the latest version
-# At the same level that amrex is cloned, do:
-
->> git clone https://github.com/LLNL/sundials.git
->> mv sundials sundials-src
-
-# Next
-
->> mkdir sundials
->> cd sundials
-
-######################
-HOST BUILD 
-######################
-
->> mkdir instdir
->> mkdir builddir
->> cd builddir
-
->> cmake -DCMAKE_INSTALL_PREFIX=/pathto/sundials/instdir -DEXAMPLES_INSTALL_PATH=/pathto/sundials/instdir/examples -DENABLE_MPI=ON ../../sundials-src
-
-######################
-NVIDIA/CUDA BUILD
-######################
-
-# Navigate back to the 'sundials' directory and do:
-
->> mkdir instdir_cuda
->> mkdir builddir_cuda
->> cd builddir_cuda
-
->> cmake -DCMAKE_INSTALL_PREFIX=/pathto/sundials/instdir_cuda -DEXAMPLES_INSTALL_PATH=/pathto/sundials/instdir_cuda/examples -DENABLE_CUDA=ON -DENABLE_MPI=ON ../../sundials-src
-
-######################
-
->> make -j4
->> make install
-
-# in your .bashrc or preferred configuration file, add the following (and then "source ~/.bashrc")
-
-# If you have a CPU build:
-
->> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/pathto/sundials/instdir/lib/
-
-# If you have a NVIDIA/CUDA build:
-
->> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/pathto/sundials/instdir_cuda/lib/
-
-# now you are ready to compile amrex-tutorials/ExampleCodes/SUNDIALS/Exec with:
-
->> make -j4 # optional to have 'USE_CUDA=TRUE' as well
-
-# Refer to inputs to see how to enable different integrators
+Make sure that SUNDIALS_HOME in the GNUmakefile points to the installation directory.
diff --git a/ExampleCodes/SUNDIALS/Single-Rate/Exec/README_sundials b/ExampleCodes/SUNDIALS/Single-Rate/Exec/README_sundials
@@ -1,64 +1,4 @@
-SUNDIALS installation guide:
-https://computing.llnl.gov/projects/sundials/faq#inst
+Refer to https://amrex-codes.github.io/amrex/docs_html/TimeIntegration_Chapter.html
+for SNUDIALS installation, build, and usage instructions.
 
-Installation
-
-# You need SUNDIALS v7.1.1 or later.
-# Check https://computing.llnl.gov/projects/sundials/sundials-software to see if it's available for download.
-# If so, download sundials-x.y.z.tar.gz and extract it at the same level as amrex using
->> tar -xzvf sundials-x.y.z.tar.gz # where x.y.z is the version of sundials
->> mv sundials-x.y.z sundials-src
-
-# If v7.1.1. is not available on the website, clone the git repo directly and use the latest version
-# At the same level that amrex is cloned, do:
-
->> git clone https://github.com/LLNL/sundials.git
->> mv sundials sundials-src
-
-# Next
-
->> mkdir sundials
->> cd sundials
-
-######################
-HOST BUILD 
-######################
-
->> mkdir instdir
->> mkdir builddir
->> cd builddir
-
->> cmake -DCMAKE_INSTALL_PREFIX=/pathto/sundials/instdir -DEXAMPLES_INSTALL_PATH=/pathto/sundials/instdir/examples -DENABLE_MPI=ON ../../sundials-src
-
-######################
-NVIDIA/CUDA BUILD
-######################
-
-# Navigate back to the 'sundials' directory and do:
-
->> mkdir instdir_cuda
->> mkdir builddir_cuda
->> cd builddir_cuda
-
->> cmake -DCMAKE_INSTALL_PREFIX=/pathto/sundials/instdir_cuda -DEXAMPLES_INSTALL_PATH=/pathto/sundials/instdir_cuda/examples -DENABLE_CUDA=ON -DENABLE_MPI=ON ../../sundials-src
-
-######################
-
->> make -j4
->> make install
-
-# in your .bashrc or preferred configuration file, add the following (and then "source ~/.bashrc")
-
-# If you have a CPU build:
-
->> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/pathto/sundials/instdir/lib/
-
-# If you have a NVIDIA/CUDA build:
-
->> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/pathto/sundials/instdir_cuda/lib/
-
-# now you are ready to compile amrex-tutorials/ExampleCodes/SUNDIALS/Exec with:
-
->> make -j4 # optional to have 'USE_CUDA=TRUE' as well
-
-# Refer to inputs to see how to enable different integrators
+Make sure that SUNDIALS_HOME in the GNUmakefile points to the installation directory.

Original file line number	Diff line number	Diff line change
`@@ -140,7 +140,7 @@ InitParticles(const IntVect& a_num_particles_per_cell,`
`140`	`140`	`unsigned int uiz = amrex::min(nz-1,amrex::max(0,iz));`
`141`	`141`	`unsigned int cellid = (uix * ny + uiy) * nz + uiz;`
`142`	`142`
`143`		`- int pidx = poffset[cellid] - poffset[0];`
	`143`	`+ int pidx = poffset[cellid] + old_size;`
`144`	`144`
`145`	`145`	`for (int i_part=0; i_part<num_ppc;i_part++)`
`146`	`146`	`{`