Skip to content

Conversation

AlexanderSinn
Copy link
Member

@AlexanderSinn AlexanderSinn commented Apr 7, 2025

Summary

This PR aims to explore adding a simpler version of ParticleTile #3570

It already assumes that a tile only has runtime components and always has a polymorphic arena allocator as described in #4380. To avoid the array of pointer style of the current particle tile, here a 2D array over particle ids and component ids is used. As a simplification both AoS and SoA members have been removed as well as all the push_back functions.

Performance test with HiPACE++:

dev, ParticleContainerPureSoA<11, 1>

TinyProfiler total time across processes [min...avg...max]: 21.44 ... 21.44 ... 21.44
TinyProfiler total time across processes [min...avg...max]: 21.42 ... 21.42 ... 21.42
TinyProfiler total time across processes [min...avg...max]: 21.4 ... 21.4 ... 21.4
TinyProfiler total time across processes [min...avg...max]: 21.43 ... 21.43 ... 21.43
TinyProfiler total time across processes [min...avg...max]: 21.39 ... 21.39 ... 21.39

ParticleContainerPureSoA<0, 0> using only runtime components

TinyProfiler total time across processes [min...avg...max]: 21.93 ... 21.93 ... 21.93
TinyProfiler total time across processes [min...avg...max]: 22.05 ... 22.05 ... 22.05
TinyProfiler total time across processes [min...avg...max]: 21.95 ... 21.95 ... 21.95
TinyProfiler total time across processes [min...avg...max]: 21.96 ... 21.96 ... 21.96
TinyProfiler total time across processes [min...avg...max]: 21.94 ... 21.94 ... 21.94

this PR, ParticleContainerPureSoA2<amrex::Real, int>

TinyProfiler total time across processes [min...avg...max]: 21.4 ... 21.4 ... 21.4
TinyProfiler total time across processes [min...avg...max]: 21.38 ... 21.38 ... 21.38
TinyProfiler total time across processes [min...avg...max]: 21.4 ... 21.4 ... 21.4
TinyProfiler total time across processes [min...avg...max]: 21.44 ... 21.44 ... 21.44
TinyProfiler total time across processes [min...avg...max]: 21.38 ... 21.38 ... 21.38

Additional background

ParticleReal and int are replaced with template types to make implementing the const versions easier and to allow for more flexibility in the future (use double and float in the same application, use double2 or float4, use 64 bit int etc).

Checklist

The proposed changes:

  • fix a bug or incorrect behavior in AMReX
  • add new capabilities to AMReX
  • changes answers in the test suite to more than roundoff level
  • are likely to significantly affect the results of downstream AMReX users
  • include documentation in the code and/or rst files, if appropriate

@atmyers
Copy link
Member

atmyers commented Jun 11, 2025

Hi @AlexanderSinn - is this still something you're working on? Do you think it would be worthwhile to keep pushing on this?

@AlexanderSinn
Copy link
Member Author

Yes. There is still a lot left to do. I am waiting for Axel to return and give feedback first

@ax3l ax3l requested review from atmyers and ax3l June 16, 2025 16:37
Copy link
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great start!

Let's finalize the naming to be more descriptive and implementation :)

@AlexanderSinn AlexanderSinn mentioned this pull request Jun 27, 2025
5 tasks
atmyers added a commit that referenced this pull request Jul 3, 2025
## Summary

In preparation of #4404, this PR reduces the reliance of SuperParticle.
The issue with getSuperParticle is that the compiler is instructed to
read in all the data of the particle into registers even if most of it
is not needed. Additionally, runtime data is not accessible through
SuperParticle.

## Additional background

The previous version of packIOData with RunOnGpu had a bug when some
particles were invalid.

## Checklist

The proposed changes:
- [x] fix a bug or incorrect behavior in AMReX
- [ ] add new capabilities to AMReX
- [ ] changes answers in the test suite to more than roundoff level
- [ ] are likely to significantly affect the results of downstream AMReX
users
- [ ] include documentation in the code and/or rst files, if appropriate

---------

Co-authored-by: Andrew Myers <[email protected]>
atmyers pushed a commit that referenced this pull request Aug 13, 2025
## Summary

This PR adds a `SetArena(Arena*)` function to ParticleContainerBase that
allows setting a memory arena that is used for all the particle vectors
if the allocator is PolymorphicArenaAllocator. The function has to be
called before particle tiles are defined.

This functionality is used to fix a bunch of places where a polymorphic
vector would previously not have its arena set properly.

Additionally the `RunOnGpu` logic in `AMReX_WriteBinaryParticleData.H`
is extended to work with a polymorphic allocator.

Uses changes extracted from
#4404.

## Additional background

Previously all components of all particle tiles needed their arena set
individually by the user if PolymorphicArenaAllocator was used. In case
this was done this PR is a braking change due to the
`AMREX_ALWAYS_ASSERT_WITH_MESSAGE(a_arena != nullptr` assert in
`ParticleTile::define()`.

## Checklist

The proposed changes:
- [ ] fix a bug or incorrect behavior in AMReX
- [x] add new capabilities to AMReX
- [ ] changes answers in the test suite to more than roundoff level
- [ ] are likely to significantly affect the results of downstream AMReX
users
- [ ] include documentation in the code and/or rst files, if appropriate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants