Skip to content

Commit 66162ee

Browse files
authored
Make sure we geometrically grow the capacity of the particle buffers (#6260)
This makes a significant performance difference on a problem setup @aeriforme gave me. In this simulation, many particles leave the domain and we need to store them all for analysis. On development: ``` TinyProfiler total time across processes [min...avg...max]: 4246 ... 4246 ... 4246 ------------------------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------------------------- ParticleBoundaryBuffer::gatherParticles::resize 8772 3843 3843 3843 90.49% WarpXOpenPMDPlot::SeriesFlush()() 96 89.46 89.46 89.46 2.11% ParticleBoundaryBuffer::gatherParticles 256 58.11 58.11 58.11 1.37% ParticleContainer::addParticles 48 53.54 53.54 53.54 1.26% ParticleBoundaryBuffer::gatherParticles::filterAndTransform 8772 27.26 27.26 27.26 0.64% ``` This PR: ``` TinyProfiler total time across processes [min...avg...max]: 494.6 ... 494.6 ... 494.6 ------------------------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------------------------- ParticleBoundaryBuffer::gatherParticles::resize 8754 126.3 126.3 126.3 25.53% WarpXOpenPMDPlot::SeriesFlush()() 96 90.3 90.3 90.3 18.26% ParticleContainer::addParticles 48 59.01 59.01 59.01 11.93% FFT::R2C::forward(in) 1934 27.05 27.05 27.05 5.47% ParticleBoundaryBuffer::gatherParticles::filterAndTransform 8754 26.8 26.8 26.8 5.42% ```
1 parent d9453b5 commit 66162ee

File tree

2 files changed

+16
-4
lines changed

2 files changed

+16
-4
lines changed

Source/Particles/ParticleBoundaryBuffer.cpp

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -459,7 +459,13 @@ void ParticleBoundaryBuffer::gatherParticlesFromDomainBoundaries (MultiParticleC
459459
auto dst_index = ptile_buffer.numParticles();
460460
{
461461
WARPX_PROFILE("ParticleBoundaryBuffer::gatherParticles::resize");
462-
ptile_buffer.resize(dst_index + amrex::get<0>(reduce_data.value()));
462+
auto np_to_add = amrex::get<0>(reduce_data.value());
463+
auto new_np = dst_index + np_to_add;
464+
amrex::Long capacity = ptile_buffer.capacity() / species_buffer.superParticleSize();
465+
// reserve space to avoid many small resize operations for performance reasons
466+
// the resize below will not shrink the capacity
467+
if (new_np > capacity) { ptile_buffer.reserve(2*new_np); }
468+
ptile_buffer.resize(new_np);
463469
}
464470
{
465471
WARPX_PROFILE("ParticleBoundaryBuffer::gatherParticles::filterAndTransform");
@@ -563,7 +569,13 @@ void ParticleBoundaryBuffer::gatherParticlesFromEmbeddedBoundaries (
563569
auto dst_index = ptile_buffer.numParticles();
564570
{
565571
WARPX_PROFILE("ParticleBoundaryBuffer::gatherParticles::resize_eb");
566-
ptile_buffer.resize(dst_index + amrex::get<0>(reduce_data.value()));
572+
auto np_to_add = amrex::get<0>(reduce_data.value());
573+
auto new_np = dst_index + np_to_add;
574+
amrex::Long capacity = ptile_buffer.capacity() / species_buffer.superParticleSize();
575+
// reserve space to avoid many small resize operations for performance reasons
576+
// the resize below will not shrink the capacity
577+
if (new_np > capacity) { ptile_buffer.reserve(2*new_np); }
578+
ptile_buffer.resize(new_np);
567579
}
568580
auto &warpx = WarpX::GetInstance();
569581
const auto dt = warpx.getdt(pti.GetLevel());

dependencies.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"version_amrex": "25.10",
44
"version_pyamrex": "25.10",
55
"version_picsar": "25.06",
6-
"commit_amrex": "26054b26bb653e594835a5a686893f6c20e7c5cd",
6+
"commit_amrex": "06ba8bd2e226823cbe11bc80a976c5df9e97d7bb",
77
"commit_pyamrex": "6f1b2dd4cd9374e196b0d47b1abf0051b51ce023",
88
"commit_picsar": "0c329e66010267662a82219f7de7abbd231463f4"
9-
}
9+
}

0 commit comments

Comments
 (0)