Does the neknek's sess file support precompilation?

[yslan]

Hi,

I am testing the eddyNekNek example on Summit using the default script with

nrsqsub_summit eddyNekNek.sess 2 00:30

cat eddyNekNek.sess
inside/inside:6;
outside/outside:6;

In the top level logfile from the submit system, it seems it fails during the per-compilation stage.

Wed May 31 09:51:01 EDT 2023
FATAL ERROR: size of sub-communicators does not match parent!
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI COMMUNICATOR 3 DUP FROM 0
with errorcode 1.

The sub-sessions still run fine with JIT compilation since it's only a 1 node job for each sessions. It's reasonable since the script submit a single mpi rank job but this won't work for large case and all system.

I can imagine the precompilation will probably work if I give each session a mpi rank.
But, that will need two session files?

Or, do we have command line support for the session file? Something like this will be nice

nekrs --session "inside/inside:6; outside/outside:6;"

Thanks,
Yu-Hsiang

[stgeke]

All nrsqsub scripts are not session file agnostic yet. You could modify your preferred script and use something like:

#/bin/bash

sessFile=$1
IFS=';' read -ra array <<< "`cat $sessFile | tr -d '\n'`"

nTasksGlobal=0
for i in "${array[@]}"
do
  sessionName=`echo $i | cut -d ":" -f 1`
  nTasks=`echo $i | cut -d ":" -f 2`
  mpirun -np 1 nekrs --setup $sessionName --buildOnly $nTasks
  let nTasksGlobal+=nTasks
done
mpirun -np $nTasksGlobal --session $sessFile

[imnusku]

Hi,

Can you please share with me a batch script for running neknek? Appreciate it, thanks!

[yslan]

For v26, all script should natively support neknek with resource distributed in the .sess file.

# NEKRS_HOME
export NEKRS_HOME<path-to-your-installation>

# local machine, 2 ranks
nrsbmpi eddyNekNek.sess 2

# hpc machines, 2 nodes, 30mins
PROJ_ID=<project id> nrsqsub_frontier eddyNekNek.sess 2 00:30

See also

• Submit script arguments
  https://nekrs.readthedocs.io/en/latest/problem_setup/running.html#running
• .sess format
  https://nekrs.readthedocs.io/en/latest/problem_setup/case.html#neknek-session-file-sess