Use Kokkos by default

[nksauter]

Problem statement: the way it is now GPU acceleration is only accessed if an environment variable is set:

DIFFBRAGG_USE_CUDA=1 simtbx.sim_view
DIFFBRAGG_USE_KOKKOS=1 simtbx.sim_view

As a user-friendly interface to diffBragg, the simtbx.sim_view command should be given default behavior that aligns with our current intended use case, namely always use Kokkos. Please implement this. Any number of options might work, but the simplest is just for the sim_view.py script to set the environment variable as soon as it starts. We need this done before returning the manuscript to the editor.

[dermen]

How about this :

$ git diff 
diff --git a/command_line/sim_view.py b/command_line/sim_view.py
index f528662..74cfa21 100644
--- a/command_line/sim_view.py
+++ b/command_line/sim_view.py
@@ -23,6 +23,9 @@ from matplotlib.backends.backend_tkagg import \
     FigureCanvasTkAgg, NavigationToolbar2Tk
 
 import libtbx.load_env
+if libtbx.env.build_options.enable_kokkos:
+    os.environ["DIFFBRAGG_USE_KOKKOS"]="1"
+
 from sim_erice.on_the_fly_simdata import run_simdata, get_SIM, randomize_orientation, set_
 from sim_erice.sim_phil import sim_view_phil_scope
 from simtbx.nanoBragg.tst_nanoBragg_multipanel import beam, whole_det

[irisdyoung]

Makes sense to me. This topic raises a few questions for me though:

• What happens if both variables are set?
• Should we first detect whether DIFFBRAGG_USE_KOKKOS is set to 0 so there is still a way to turn it off for a Kokkos-enabled build? (Is there any rational reason for doing this?)
• Does setting this environment variable here have downstream effects? Does it persist after closing the viewer?
• Are we supporting CUDA flags in the near future still or only Kokkos?

[dermen]

• If both are set, we should raise an exception (ideally this would happen in diffBragg/src/diffBrag.cpp).. I can add that fix in today.. I believe if both are set CUDA is actually done, based on the logic in diffBragg.cpp. We can also raise an exception in the viewer code...
• As far as I know, setting the var in a program is only temporary and it should all be as it was once the viewer is closed or placed in the background. And I believe the updated environment will only pertain to the process ID running the viewer..
• near future (next few months) id like to continue supporting the cuda flag

[dermen]

See here to address the double flag issue:

cctbx/cctbx_project#885

[irisdyoung]

Awesome, looks good to me. @nksauter will this address the issue?

[Trzs]

• Should we first detect whether DIFFBRAGG_USE_KOKKOS is set to 0 so there is still a way to turn it off for a Kokkos-enabled build? (Is there any rational reason for doing this?)
• Does setting this environment variable here have downstream effects? Does it persist after closing the viewer?

Iris, regarding your second question, I suggest to do it similar to the diffBragg tests, only now with a flag to not use kokkos. Using Derek's edits:

diff --git a/command_line/sim_view.py b/command_line/sim_view.py
index b9e2dfc..aeb7d05 100644
--- a/command_line/sim_view.py
+++ b/command_line/sim_view.py
@@ -22,7 +22,14 @@ from scitbx.matrix import col, sqr
 from matplotlib.backends.backend_tkagg import \
     FigureCanvasTkAgg, NavigationToolbar2Tk

+from argparse import ArgumentParser
+parser = ArgumentParser()
+parser.add_argument("--cpu-only", action="store_true")
+args = parser.parse_args()
+
 import libtbx.load_env
+if libtbx.env.build_options.enable_kokkos and not args.cpu-only:
+    os.environ["DIFFBRAGG_USE_KOKKOS"]="1"
 from sim_erice.on_the_fly_simdata import run_simdata, get_SIM, randomize_orientation, sweep
 from simtbx.nanoBragg.tst_nanoBragg_multipanel import beam, whole_det
 from simtbx.diffBragg import hopper_utils

[nksauter]

@irisdyoung since you ask I would follow the os.environ mechanism of @dermen, but implement the kokkos/cuda/cpu switch as a phil choice,

    context = *kokkos cuda cpu 
      .type = choice
      .optional = False
      .help = backend for parallel execution
      .expert_level=3

This way all three modalities are available for the determined power user.

Also I'd implement the switch within the if __name__ == '__main__': clause just after the phil params have been obtained.

[irisdyoung]

Closed with f0c88ed