Trace replay kernel

.github/workflows/build-cuda.yml

@@ -1,10 +1,8 @@
 name: Build and Test CUDA Backend
 
 on:
-  push:
-    branches: [ main ]
   pull_request:
-    branches: [ main ]
+    branches: [main, spatter-devel]
   schedule:
     - cron: '30 8 * * *'
 
@@ -15,4 +13,4 @@ jobs:
     - uses: actions/checkout@v4
     - name: Run batch file
       run: cd tests/misc && chmod +x run-crnch-cuda.sh && sbatch run-crnch-cuda.sh
-
+

.github/workflows/build-mpi.yml

@@ -1,13 +1,9 @@
 name: Build and Test MPI Backend
 
 on:
-  push:
-    branches: [ main ]
   pull_request:
-    branches: [ main ]
-  schedule:
-    - cron: '30 8 * * *'
-
+    branches: [main, spatter-devel]
+
 jobs:  
   build-and-run-mpi:
     runs-on: ubuntu-latest
@@ -21,5 +17,4 @@ jobs:
         run: 
           cmake -DUSE_MPI=1 -B build_mpi -S . && make -C build_mpi
       - name: Test-MPI
-        run: make test -C build_mpi
-
+        run: make test -C build_mpi

.github/workflows/build-serial-omp.yml

@@ -1,12 +1,8 @@
 name: Build and Test Serial and OpenMP backends
 
-on:
-  push:
-    branches: [ main ]
+on: 
   pull_request:
-    branches: [ main ]
-  schedule:
-    - cron: '30 8 * * *'
+    branches: [main, spatter-devel]
 
 jobs:
   build-and-run-serial:
@@ -27,4 +23,3 @@ jobs:
         cmake -DUSE_OPENMP=1 -B build_omp_gnu -S . && make -C build_omp_gnu
     - name: Test-OMP
       run: make test -C build_omp_gnu
-

.github/workflows/notebooks.yml

@@ -1,9 +1,8 @@
 name: Notebook Build
 
 on:
-  push:
-    branches:
-      - 'main'
+  pull_request:
+    branches: [main, spatter-devel]
 
 jobs:
   notebook-tests:

AUTHORS

@@ -4,17 +4,19 @@ Patrick Lavin
 	General code development, key gather/scatter kernels, maintainer
 
 Jeffrey Young
-	OpenCL backend and general updates
+	General code development, notebooks and CI/CD, maintainer
 
 Jered Dominguez-Trujillo
-	Support for combined gather/scatter kernel
+	Development of refactor for Spatter 2.0, MPI features, improved CUDA backend
+
+Connor Radelja
+	General bugfixes and analysis for Spatter 2.0 release; development of Spatter SST element
 
 Agustin Vaca Valverde
-	Unit testing and scripts
+	Unit testing and scripts, notebooks
 
 Vincent Huang
 	Scripting and OneAPI backend support
 
 James Wood
 	OneAPI backend support
-

CHANGELOG.md

@@ -1,10 +1,28 @@
-# Spatter 1.0 Release Notes
+# Spatter Release Notes
+
+## Spatter 2.0 Release Notes
+
+The 2.0 release of Spatter brings several important changes to the codebase, some of which are summarized in the MEMSYS 2024 paper, by Sheridan, et al., ["A Workflow for the Synthesis of Irregular Memory Access Microbenchmarks"](https://www.memsys.io/wp-content/uploads/ninja-forms/5/sheridan_et_al_workflow_irregular_patterns_paper_32_MEMSYS_2024-1.pdf). Specifically, this release includes these major changes:
+
+- Switch from C to C++ for the codebase, enabling more robus configuration options
+- Refactor of the CMake infrastructure
+- Different parsing mechanisms and deprecation of older parsing code requiring third-party submodules
+- Improvements to the CUDA backend including support for longer patterns and support for GPU throughput testing
+- MPI support for weak and strong scaling test scenarios
+- Removal of integrated support for PAPI
+- Addition of contributors guide, improved GH actions for testing, and templates for PRs and new issues.
+- Update to the Spatter wiki to describe [changes in Spatter 2.0 that may affect benchmark performance](https://github.com/hpcgarage/spatter/wiki/Spatter-2.0-Validation) and a [new guide for developing new backends](https://github.com/hpcgarage/spatter/wiki/Adding-New-Backends-to-Spatter).
+
+This release includes commits from Patrick Lavin, Jeff Young, Julio Agustin Vaca Valverde, Jered Dominguez-Trujillo, and Connor Radelja. We are greatly appreciative of the support of contributors from Los Alamos National Labs through [their work with Spatter](https://github.com/lanl/spatter) as well as new work on [generating input patterns with GS Patterns](https://github.com/lanl/gs_patterns/).
+
+
+## Spatter 1.0 Release Notes
 
 After 6 years of development, we are ready to release Spatter version 1.0! Over the years we have gone through a number of input and output formats, but they have not changed significantly in several years now, so we are confident that they are stable enough for our first major release. 
 
 Building Spatter has been a collaborative effort. This release includes commits from Patrick Lavin, Jeff Young, Julio Agustin Vaca Valverde, Jered Dominguez-Trujillo, James Wood, Vincent Huang, Sudhanshu Agarwal, Jeff Inman, and Jeff Hammond. Thank you to all of you for your effort and to those who submitted issues and advised us on the project as well.
 
-## New Features
+### New Features
 Since version 0.6, Spatter has added several major new features: 
 
  - multi-gather, multi-scatter: the multi- kernels perform two levels of indirection, e.g. multi-gather is dest[i] = src[idx1[idx2[i]] and multi-scatter is analogous. This greatly expands the class of patterns that Spatter can represent. 
@@ -14,6 +32,6 @@ MPI support allows Spatter to run the same pattern on many ranks and allows for
  - Testing and CI/CD has been greatly improved. The GPU backend is now included in the test suite, and we have expanded the set of automated tests used for the CPU backend as well.
  - Documentation now includes a Getting Started Jupyter notebook to demonstrate how to use the benchmark and how to plot outputs.
 
-## Ongoing Work
+### Ongoing Work
 We are still implementing a CI solution to automatically test commits to the GPU backend. 
 

CONTRIBUTING.md

@@ -0,0 +1,48 @@
+# Contributing to Spatter
+
+Thank you for your interest in contributing to the Spatter open source project. We appreciate your interest and contributions. Please review these guidelines for the project so that you can most effectively engage with the developers and maintainers of the project. 
+
+The license for the project is BSD-3 with some slight language additions from the main contributors at Los Alamos National Labs and Georgia Institute of Technology. 
+
+## Types of Contributions Accepted
+
+Spatter is a growing project and ecosystem, and there are several opportunities for contributions in the areas of new features, training and education, and general improvements to the codebase. 
+
+### Contribution Process
+In general, to contribute fixes or new code to Spatter, you would do the following:
+1) Fork the Spatter codebase
+2) Make your changes and run unit tests locally to see that changes do not break anything
+3) Rebase on spatter-devel branch in the main repo, as needed. This is our "test" branch whereas main is the "stable" branch.
+4) Open a PR and fill in the PR template with relevant information on your changes.
+5) Request reviewers and discuss/update the PR until it is ready to be accepted.
+
+### Reporting Bugs
+
+**NOTE**: If you find a security vulnerability in the codebase, please do not open an issue. Email the primary maintainers or post a Discussion message noting you have something you'd like to report. This provides the development team an opportunity to respond to your findings and prepare a patch that can be released to mitigate any vulnerabilities.
+
+Other general bugs can be reported using our [Bug Report Issue Template](https://github.com/hpcgarage/spatter/issues/new?assignees=&labels=bug&projects=&template=00-bug-report.yml&title=%F0%9F%90%9B+%5BBUG%5D+-+%3Ctitle%3E).
+
+### Suggesting Features or Enhancements
+
+If there is a feature that you would like to see in Spatter that doesn't currently exist, we also have a [feature request issue template](https://github.com/hpcgarage/spatter/issues/new?assignees=&labels=feature+request&projects=&template=01-feature-request.yml&title=%E2%9C%A8+%5BFEATURE+REQUEST%5D+-+%3Ctitle%3E) that you can use. This will create a new issue for further discussion with the development team. 
+
+### New Spatter Backends
+
+We have created a guide to developing new backends for Spatter [on our wiki here](https://github.com/hpcgarage/spatter/wiki/Adding-New-Backends-to-Spatter). If you would like to create a new backend for Spatter, you can create a new feature request, and the maintainers can link any 
+
+## Code review process
+
+We will aim to review your PR in 1-2 weeks with a goal of getting it merged within that time frame. If added changes are needed, we will follow up via the PR request and may make some suggestions to help your PR pass the CI unit tests and be able to be merged into the main line of the codebase. 
+
+### Code style conventions
+
+Spatter does not currently have strong checks for code formatting but please try to be consistent with the existing codebase with your changes. The `.clang-format` file in the top-level directory specifies the desired format for any commits. 
+
+## Community Channels for Discussion
+
+The primary discussion channels for this project are via [GitHub Issues](https://github.com/hpcgarage/spatter/issues) and the [GitHub Discussions space of this project](https://github.com/hpcgarage/spatter/discussions). You are welcome to post any general questions about Spatter to the Discussions channel or email the maintainers. 
+
+### Code of Conduct for Discussions
+The maintainers of the project aim to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. For more details on our standard of conduct, please see the [Contributor Covenant Guidelines](https://www.contributor-covenant.org/version/2/1/code_of_conduct/), which we follow to ensure constructive and welcoming discussions and engagement.s

README.md

@@ -2,7 +2,7 @@
 
 Spatter is a microbenchmark for timing Gather/Scatter kernels on CPUs and GPUs. View the [source](https://github.com/hpcgarage/spatter), and please submit an issue on Github if you run into any issues. 
 
-![Build Status](https://github.com/hpcgarage/spatter/actions/workflows/build.yml/badge.svg)
+![Build Status](https://github.com/hpcgarage/spatter/actions/workflows/build-serial-omp.yml/badge.svg)
 
 
 ## Purpose 

cmake/pkgs/MPISupport.cmake

@@ -3,6 +3,8 @@ option(USE_MPI "Enable support for MPI")
 if (USE_MPI)
     find_package(MPI)
     include_directories(${MPI_INCLUDE_PATH})
+    #Explicitly add directory for Ubuntu 22 to search
+    include_directories(/usr/lib/x86_64-linux-gnu/openmpi/include)
     set(COMMON_LINK_LIBRARIES ${COMMON_LINK_LIBRARIES} MPI::MPI_CXX)
     add_definitions(-DUSE_MPI)
 endif()

notebooks/requirements.txt

@@ -41,7 +41,7 @@ jupyter_client==8.6.2
 jupyter_core==5.7.2
 jupyter_server==2.14.1
 jupyter_server_terminals==0.5.3
-jupyterlab==4.2.2
+jupyterlab==4.2.5
 jupyterlab_pygments==0.3.0
 jupyterlab_server==2.27.2
 kiwisolver==1.4.5

src/CMakeLists.txt

@@ -5,6 +5,7 @@ add_subdirectory(Spatter)
 add_executable(spatter main.cc)
 #target_compile_options(spatter PUBLIC "-fnew-alignment 32")
 target_link_libraries(spatter ${COMMON_LINK_LIBRARIES} Spatter)
+target_link_libraries(spatter z)
 set_target_properties(spatter PROPERTIES
     COMPILE_DEFINITIONS "${COMMON_COMPILE_DEFINITIONS}"
     COMPILE_OPTIONS "${WARNING_FLAGS}"

src/Spatter/CMakeLists.txt

@@ -64,11 +64,15 @@ endif()
 target_link_libraries(Spatter
     PUBLIC
     ${COMMON_LINK_LIBRARIES}
+    z
+    stdc++fs
     )
 
 target_link_libraries(Spatter_shared
     PUBLIC
     ${COMMON_LINK_LIBRARIES}
+    z
+    stdc++fs
     )
 
 target_compile_options(Spatter