Testing on GPU

[ambrad]

1a. to get gcc to do the same as -fp-model=strict, one uses
-ffp-contract=off.

1b. to get cuda to do the same, one adds --fmad=false to cuda_args in
nvcc_wrapper.

2. in addition, team reductions and scans will still diff. hommexx serializes
   these with a wrapper. however, these account for too much of the code in p3, so
   we can't use this route w/o losing tons of testing of important ||ism.

thus, for gpu tests, we should do non-bfb testing and set an appropriate tol. we
can augment the tolerance specification in run_and_cmp with something like

  if (OnGpu<Kokkos::DefaultExecutionSpace>::value)
    tol = 10*std::numeric_limits<Real>::epsilon();

we can still use 1a and 1b to get the diffs down to just the reduction/scan
ones.

[jgfouca]

min/max reductions should be bfb.

BFB should be attainable between fortran on P9 and GPU with the flag settings mentioned above with -O0 at least and maybe -O3.

[jgfouca]

See
https://github.com/E3SM-Project/HOMMEXX/blob/a3b98f9f810321c150818579b77fae6c8c45af8f/cmake/SetCompilerFlags.cmake