Dual ion impact ionization failure

[k-wolfinger]

Our case study: two separate ion species enter the simulation as beams and interact with a background gas through ion impact ionization.

A single species (+ collision) will result in a run without problem. Two species cause a seg-fault, specifically during the initialization of the second of the ion impact reactions.

[k-wolfinger]

Here is the output

(warpx) impact_ionization$ python PICMI_inputs_3d_n2.py 
Initializing AMReX (24.04-9-g2a3955a5f5aa)...
MPI initialized with 1 MPI processes
MPI initialized with thread support level 3
OMP initialized with 4 OMP threads
AMReX (24.04-9-g2a3955a5f5aa) initialized
PICSAR (23.09)
WarpX (19.02-6699-g9335803a01e3-dirty)

    __        __             __  __
    \ \      / /_ _ _ __ _ __\ \/ /
     \ \ /\ / / _` | '__| '_ \\  /
      \ V  V / (_| | |  | |_) /  \
       \_/\_/ \__,_|_|  | .__/_/\_\
                        |_|

Level 0: dt = 1e-10 ; dx = 0.03125 ; dy = 0.03125 ; dz = 0.03125

Grids Summary:
  Level 0   4 grids  131072 cells  100 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

-------------------------------------------------------------------------------
--------------------------- MAIN EM PIC PARAMETERS ----------------------------
-------------------------------------------------------------------------------
Precision:            | DOUBLE
Particle precision:   | DOUBLE
Geometry:             | 3D (XYZ)
Operation mode:       | Electrostatic
                      | - laboratory frame
                      | - vacuum
Poisson solver:       | multigrid
-------------------------------------------------------------------------------
Current Deposition:   | direct 
Particle Pusher:      | Boris 
Charge Deposition:    | standard 
Field Gathering:      | energy-conserving 
Particle Shape Factor:| 1
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
For full input parameters, see the file: warpx_used_inputs

--- INFO    : Writing openPMD file ./diags/june_impact_fusion_0v/particle000000
--- INFO    : Setting up collisions for hydrogen1 and n2_ions with:
             total non-ionization collision probability: 0.000000
             total ionization collision probability: 0.000021
--- INFO    : Setting up collisions for boron11 and n2_ions with:
             total non-ionization collision probability: 0.000000
             total ionization collision probability: 0.000021
Segfault
See Backtrace.0.0 file for details
application called MPI_Abort(MPI_COMM_WORLD, 11) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=11
:
system msg for write_line failure : Bad file descriptor
Assertion failed in file src/mpi/group/group_free.c at line 71: 0
/usr/lib64/mpich/lib/libmpi.so.12(MPL_backtrace_show+0x39) [0x7fa9db20cc79]
/usr/lib64/mpich/lib/libmpi.so.12(MPI_Group_free+0x228) [0x7fa9db078d38]
/home/vagrant/jupyter/StaffScratch/k-wolfinger/impact_ion_warpx/warpx/lib/libamrex_3d.so(_ZN5amrex15ParallelContext5FrameD2Ev+0x16) [0x7fa9dbb68946]
/home/vagrant/jupyter/StaffScratch/k-wolfinger/impact_ion_warpx/warpx/lib/libamrex_3d.so(_ZN5amrex6VectorINS_15ParallelContext5FrameESaIS2_EED2Ev+0x24) [0x7fa9dbb68e54]
/lib64/libc.so.6(+0x41015) [0x7fa9f5440015]
/lib64/libc.so.6(on_exit+0) [0x7fa9f5440190]
/usr/lib64/mpich/lib/libmpi.so.12(+0x3ac4bd) [0x7fa9db20e4bd]
/usr/lib64/mpich/lib/libmpi.so.12(MPI_Abort+0x230) [0x7fa9db085a60]
/home/vagrant/jupyter/StaffScratch/k-wolfinger/impact_ion_warpx/warpx/lib/libamrex_3d.so(_ZN5amrex11BLBackTrace7handlerEi+0xa41) [0x7fa9dbb8e881]
/lib64/libc.so.6(+0x3ea00) [0x7fa9f543da00]
/lib64/libc.so.6(+0xa734d) [0x7fa9f54a634d]
/lib64/libstdc++.so.6(_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_mutateEmmPKcm+0xa0) [0x7fa9df4f4c20]
/lib64/libstdc++.so.6(_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm+0x70) [0x7fa9df4f6690]
/home/vagrant/.pyenv/versions/warpx/lib/python3.9/site-packages/pywarpx/warpx_pybind_3d.cpython-39-x86_64-linux-gnu.so(_ZN22BackgroundMCCCollision12doCollisionsEddP22MultiParticleContainer+0x5cd) [0x7fa9da76b77d]
/home/vagrant/.pyenv/versions/warpx/lib/python3.9/site-packages/pywarpx/warpx_pybind_3d.cpython-39-x86_64-linux-gnu.so(_ZN16CollisionHandler12doCollisionsEddP22MultiParticleContainer+0x7a) [0x7fa9da74efda]
/home/vagrant/.pyenv/versions/warpx/lib/python3.9/site-packages/pywarpx/warpx_pybind_3d.cpython-39-x86_64-linux-gnu.so(_ZN22MultiParticleContainer12doCollisionsEdd+0x59) [0x7fa9da4607e9]
/home/vagrant/.pyenv/versions/warpx/lib/python3.9/site-packages/pywarpx/warpx_pybind_3d.cpython-39-x86_64-linux-gnu.so(_ZN5WarpX6EvolveEi+0x2ae) [0x7fa9da3101ce]
/home/vagrant/.pyenv/versions/warpx/lib/python3.9/site-packages/pywarpx/warpx_pybind_3d.cpython-39-x86_64-linux-gnu.so(+0x298104) [0x7fa9da2ba104]
/home/vagrant/.pyenv/versions/warpx/lib/python3.9/site-packages/pywarpx/warpx_pybind_3d.cpython-39-x86_64-linux-gnu.so(+0x276401) [0x7fa9da298401]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(+0xfc403) [0x7fa9f5824403]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyObject_MakeTpCall+0x8e) [0x7fa9f57e091e]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(+0xbb28a) [0x7fa9f57e328a]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x64b8) [0x7fa9f5795e88]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(+0x19fba1) [0x7fa9f58c7ba1]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyFunction_Vectorcall+0xad) [0x7fa9f57e074d]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(+0xbb248) [0x7fa9f57e3248]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x64b8) [0x7fa9f5795e88]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(+0x19fba1) [0x7fa9f58c7ba1]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyFunction_Vectorcall+0xad) [0x7fa9f57e074d]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x63ef) [0x7fa9f5795dbf]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(+0x19fba1) [0x7fa9f58c7ba1]
/home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so.1.0(_PyEval_EvalCodeWithName+0x52) [0x7fa9f58c7f42]
internal ABORT - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1
:
system msg for write_line failure : Bad file descriptor

[k-wolfinger]

Here are the lines for the ion impact collisions:

ionization_p_process={
    'ionization' : {'cross_section' : cross_sec_direc+'ion_p_ionization.dat',
                    'energy' : 15.578,
                    'species' : n2plus} #the produced ion species from the background
}
p_colls = picmi.MCCCollisions(
    name='p_coll',
    species=beams['hydrogen1']['species'], # species colliding with background
    background_density=n_bkgd,
    background_temperature=t_bkgd,
    ndt=1,
    electron_species=electrons, # the produced electrons
    scattering_processes=ionization_p_process
)
collisions.append(p_colls)

ionization_b11_process={
    'ionization' : {'cross_section' : cross_sec_direc+'ion_b11_ionization.dat',
                    'energy' : 15.578,
                    'species' : n2plus} #the produced ion species from the background
}
b11_colls = picmi.MCCCollisions(
    name='b11_coll',
    species=beams['boron11']['species'], # species colliding with background
    background_density=n_bkgd,
    background_temperature=t_bkgd,
    ndt=1,
    electron_species=electrons, # the produced electrons
    scattering_processes=ionization_b11_process
)
collisions.append(b11_colls)

[k-wolfinger]

And here is the backtrace file
Backtrace.txt

[RemiLehe]

Thanks for reporting this. It is unclear to me why this is SegFaulting.

Could you send us the files needed to reproduce this on our side?
In particular:

• could you send us the full input script (or a modified version thereof that still reproduces the bug), as well as the .dat files containing the cross-sections?
• could you let us know which version/branch of WarpX you are using?

[k-wolfinger]

The version/branch is found at: https://github.com/bruhwiler/WarpX/tree/ion-impact

The modified input script with the bug is: https://github.com/bruhwiler/WarpX/blob/ion-impact/Examples/Physics_applications/impact_ionization/PICMI_inputs_3d_n2_adjusted.py

The .dat files are attached (renamed as .txt so they could be uploaded)
ion_p_ionization.txt
ion_b11_ionization.txt