You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<!--Provide a general summary of your changes in the title above, for
2
+
example "Add AMR unit test for cell centered fields.". Please avoid
3
+
non-descriptive titles such as "Addresses issue #8576".-->
4
+
5
+
## PR Summary
6
+
7
+
<!--Please provide at least 1-2 sentences describing the pull request in
8
+
detail. Why is this change required? What problem does it solve?-->
9
+
10
+
<!--If it fixes an open issue, please link to the issue here.-->
11
+
12
+
## PR Checklist
13
+
14
+
<!-- Note that some of these check boxes may not apply to all pull requests -->
15
+
16
+
-[ ] Code passes cpplint
17
+
-[ ] New features are documented.
18
+
-[ ] Adds a test for any bugs fixed. Adds tests for new features.
19
+
-[ ] Code is formatted
20
+
-[ ] Changes are summarized in CHANGELOG.md
21
+
-[ ] Change is breaking (API, behavior, ...)
22
+
-[ ] Change is *additionally* added to CHANGELOG.md in the breaking section
23
+
-[ ] PR is marked as breaking
24
+
-[ ] Short summary API changes at the top of the PR (plus optionally with an automated update/fix script)
25
+
-[ ] Docs build
26
+
-[ ] Any contribution that was created or modified with the assistance of generative AI is disclosed here and in code following the [guidelines](https://github.com/parthenon-hpc-lab/athenapk/blob/main/CONTRIBUTING.md#use-of-agentic-coding)
Copy file name to clipboardExpand all lines: CHANGELOG.md
+84-1Lines changed: 84 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,88 @@
2
2
3
3
## Current develop (i.e., `main` branch)
4
4
5
+
### General notes
6
+
With the latest update of the Parthenon submodule several new features are now available, e.g.,
7
+
-[OpenPMD output](https://parthenon-hpc-lab.github.io/parthenon/pgrete/pmd-output/src/outputs.html#openpmd) including support for slices, data compression and coarse graining\
8
+
Note, the original naming convention (in the development branch) used for labeling components was not fully standard compliant.
9
+
New outputs are automatically written in the standard compliant version.
10
+
However, to keep the old scheme (e.g., when restarting from existing simulation data with the intent to keep the timeseries consistent add `openpmd_format_version=1` to the corresponding openpmd output blocks.
11
+
- A watchdog (to kill a simulation that hangs for whatever reason). Just run with `-w HH:MM:SS`.
12
+
- Support for (tracer) particles with AMR.
13
+
- Support for using more parallelism for the ghost zone exchange kernels, which speeds up simulation with few (<10) blocks per rank on device, see [here](https://github.com/parthenon-hpc-lab/parthenon/pull/1271).
14
+
A good starting point is `parthenon/mesh/minimum_number_of_teams_for_boundary_kernel=8`.
15
+
16
+
With the update of Kokkos to version 5.1.1 (or >5.0 in general) a performance regression was identified.
17
+
This is likely related to the Kokkos-internal use of int64 indices in the new `mdspan` based `View`s resulting more register usage (which. in turn, results in lower occupancy on devices).
18
+
The Kokkos team is aware of this and working on a fix.
19
+
If the current performance is (significantly) below expectation, one can try to use "legacy" views via `Kokkos_ENABLE_IMPL_VIEW_LEGACY=ON`.
20
+
21
+
**IMPORTANT** The latest Parthenon submodule includes a fix for a race condition in the flux correction communication routine when run with mesh refinement on GPUs, see [here](https://github.com/parthenon-hpc-lab/parthenon/pull/1405).
22
+
Please update immediately or rebuild AthenaPK with `PARTHENON_DISABLE_SPARSE=OFF` to mitigate the race condition.
-[[PR 172]](https://github.com/parthenon-hpc-lab/athenapk/pull/172) Fixed data race condition in few modes IFT (no practical implication)
30
+
31
+
### Infrastructure
32
+
-[[PR 167]](https://github.com/parthenon-hpc-lab/athenapk/pull/167) Bump Kokkos to 5.1.1 and `Parthenon` to upcoming 26.xx version (incl OpenPMD support)
-[[PR 167]](https://github.com/parthenon-hpc-lab/athenapk/pull/167) C++20 is now the minimum standard
39
+
-[[PR 167]](https://github.com/parthenon-hpc-lab/athenapk/pull/167) New Parthenon submodule changed input file parsing (removed `*pib = pin->pfirst_block;`), see [here](https://github.com/parthenon-hpc-lab/parthenon/pull/1385)
40
+
41
+
## Release 26.05
42
+
43
+
### General notes
44
+
45
+
Particle ids have been updated in Parthenon to be `uint64` and added by default.
46
+
Thus, the original "`id`" tracer variable has been removed in favor of the Parthenon default version.
47
+
Access to that `id` is done via `auto &id = swarm->Get<std::uint64_t>(swarm_position::id::name()).Get();`
48
+
and via `swarm.id` in the `phdf` python tools.
49
+
Other postprocessing tools, like VisIt will automatically identify the new field.
50
+
51
+
From updated Parthenon submodule:
52
+
53
+
-`packs_per_rank` can now be used instead of `pack_size` in the `<parthenon/mesh>` input block.
54
+
It is the new default (i.e., it's set automatically when it's not present in the input file)
55
+
because it result in better load balance.
56
+
- For simulation on (AMD) GPUs with AMR and many blocks per rank, the number of MPI messages in flight
57
+
(especially during mesh refinement) could sometimes cause "Memory access fault by GPU".
58
+
This is related to how the MPI library and hardware manage handles to communication buffer in device memory.
59
+
To circumvent this issue, Parthenon now supports [coalesced communication](https://parthenon-hpc-lab.github.io/parthenon/develop/src/boundary_communication.html#coalesced-mpi-communication)
60
+
where multiple messages between ranks are combined.
61
+
This comes at a small performance cost (due to the additional packing and unpacking of messages).
62
+
To enable, set `do_coalesced_comms=true` in the `<parthenon/mesh>` block of the input file.
63
+
- Input parameters can now be [automatically documented](https://github.com/parthenon-hpc-lab/parthenon/pull/1283)
64
+
by adding an optional string as last argument to any `ParameterInput``Get` or `GetOrAdd` call.
65
+
66
+
### Added (new features/APIs/variables/...)
67
+
-[[PR 162]](https://github.com/parthenon-hpc-lab/athenapk/pull/162) Add pgen for cloud shattering setup
68
+
-[[PR 158]](https://github.com/parthenon-hpc-lab/athenapk/pull/158) Update particle id handling (now automated `uint64`). Extend particle history lookback in turbulence pgen and include in turbulence test
69
+
-[[PR 157]](https://github.com/parthenon-hpc-lab/athenapk/pull/157) Support injection of blobs with density/temp contrast in turbulence simulations
70
+
71
+
### Changed (changing behavior/API/variables/...)
72
+
-[[PR 163]](https://github.com/parthenon-hpc-lab/athenapk/pull/163) Add normalization by volume for relative B field divergence in history file
0 commit comments