Make speculation safe for Intel compiler with floating point exceptions

[theurich]

Under #502 @climbfuji points us to

https://www.intel.com/content/www/us/en/docs/fortran-compiler/developer-guide-reference/2024-2/fp-speculation-qfp-speculation.html#GUID-5F6A2F05-0436-41B8-8ACE-49049135F06D

Setting compiler option -fp-speculation=safe on our Intel configurations for ESMF_BOPT=O seems the way to go to prevent issues with speculative floating point code that can lead to floating point exceptions. This would be the safe way to prevent the issues reported when user code builds with -fpe0.

I would think that for most of the ESMF code potential performance impact is minimal. Only maybe some portions of the weight generation part might have measurable performance impact with -fp-speculation=safe over the default fast option for optimized compilation?

[theurich]

There has been evidence under NEPTUNE development at NRL that -fp-speculation=safe is not actually safe. Specifically see https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename:'1',fontScale:14,fontUsePx:'0',j:1,lang:fortran,selection:(endColumn:1,endLineNumber:36,positionColumn:1,positionLineNumber:36,selectionStartColumn:1,selectionStartLineNumber:36,startColumn:1,startLineNumber:36),source:'%0Aprogram+Test%0A++++use,+intrinsic+::+ieee_exceptions,+only+:+ieee_set_halting_mode,+IEEE_USUAL%0A++++implicit+none%0A%0A++++real,+dimension(:),+allocatable+::+z,+dz,+rdz%0A++++integer+::+k%0A++++integer+::+nz%0A%0A++++nz+%3D+100%0A%0A++++call+ieee_set_halting_mode(IEEE_USUAL,+.TRUE.)%0A%0A++++allocate(z(nz))%0A++++allocate(dz(nz))%0A++++allocate(rdz(nz))%0A++++z+++%3D+0.0%0A++++dz++%3D+0.0%0A++++rdz+%3D+0.0%0A%0A++++do+k+%3D+1,+nz%0A++++++++z(k)+%3D+0.1+*+(k-1)%0A++++enddo%0A%0A++++do+k+%3D+2,+nz%0A++++++++dz(k)++%3D+z(k)+-+z(k-1)%0A%0A!!***+This+will+force+an+eval+before+usage+***%0A!!++++++++if(+dz(k)+/%3D+0.0+)then%0A++++++++++++rdz(k)+%3D+1.0+/+dz(k)%0A!!++++++++endif++++%0A++++enddo%0A%0A++++write(0,*)+%22+---+DONE+---+%22%0Aend+program+Test%0A'),l:'5',n:'0',o:'Fortran+source+%231',t:'0')),k:33.916616333720796,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:ifx202400,filters:(b:'0',binary:'1',binaryObject:'1',commentOnly:'0',debugCalls:'1',demangle:'0',directives:'0',execute:'0',intel:'0',libraryCode:'0',trim:'1',verboseDemangling:'0'),flagsViewOpen:'1',fontScale:14,fontUsePx:'0',j:1,lang:fortran,libs:!(),options:'-g+-traceback+-fp-model+consistent+-align+array64byte+-fimf-use-svml%3Dtrue+-fimf-precision%3Dhigh+-prec-div+-O2+-fp-speculation%3Dfast+',overrides:!(),selection:(endColumn:1,endLineNumber:1,positionColumn:1,positionLineNumber:1,selectionStartColumn:1,selectionStartLineNumber:1,startColumn:1,startLineNumber:1),source:1),l:'5',n:'0',o:'+x86-64+ifx+2024.0.0+(Editor+%231)',t:'0')),k:33.208051925634706,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:output,i:(compilerName:'x86-64+gfortran+15.2',editorid:1,fontScale:14,fontUsePx:'0',j:1,wrap:'1'),l:'5',n:'0',o:'Output+of+x86-64+ifx+2024.0.0+(Compiler+%231)',t:'0')),k:32.8753317406445,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4

[climbfuji]

There has been evidence under NEPTUNE development at NRL that -fp-speculation=safe is not actually safe. Specifically see https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename:'1',fontScale:14,fontUsePx:'0',j:1,lang:fortran,selection:(endColumn:1,endLineNumber:36,positionColumn:1,positionLineNumber:36,selectionStartColumn:1,selectionStartLineNumber:36,startColumn:1,startLineNumber:36),source:'%0Aprogram+Test%0A++++use,+intrinsic+::+ieee_exceptions,+only+:+ieee_set_halting_mode,+IEEE_USUAL%0A++++implicit+none%0A%0A++++real,+dimension(:),+allocatable+::+z,+dz,+rdz%0A++++integer+::+k%0A++++integer+::+nz%0A%0A++++nz+%3D+100%0A%0A++++call+ieee_set_halting_mode(IEEE_USUAL,+.TRUE.)%0A%0A++++allocate(z(nz))%0A++++allocate(dz(nz))%0A++++allocate(rdz(nz))%0A++++z+++%3D+0.0%0A++++dz++%3D+0.0%0A++++rdz+%3D+0.0%0A%0A++++do+k+%3D+1,+nz%0A++++++++z(k)+%3D+0.1+*+(k-1)%0A++++enddo%0A%0A++++do+k+%3D+2,+nz%0A++++++++dz(k)++%3D+z(k)+-+z(k-1)%0A%0A!!***+This+will+force+an+eval+before+usage+***%0A!!++++++++if(+dz(k)+/%3D+0.0+)then%0A++++++++++++rdz(k)+%3D+1.0+/+dz(k)%0A!!++++++++endif++++%0A++++enddo%0A%0A++++write(0,*)+%22+---+DONE+---+%22%0Aend+program+Test%0A'),l:'5',n:'0',o:'Fortran+source+%231',t:'0')),k:33.916616333720796,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:ifx202400,filters:(b:'0',binary:'1',binaryObject:'1',commentOnly:'0',debugCalls:'1',demangle:'0',directives:'0',execute:'0',intel:'0',libraryCode:'0',trim:'1',verboseDemangling:'0'),flagsViewOpen:'1',fontScale:14,fontUsePx:'0',j:1,lang:fortran,libs:!(),options:'-g+-traceback+-fp-model+consistent+-align+array64byte+-fimf-use-svml%3Dtrue+-fimf-precision%3Dhigh+-prec-div+-O2+-fp-speculation%3Dfast+',overrides:!(),selection:(endColumn:1,endLineNumber:1,positionColumn:1,positionLineNumber:1,selectionStartColumn:1,selectionStartLineNumber:1,startColumn:1,startLineNumber:1),source:1),l:'5',n:'0',o:'+x86-64+ifx+2024.0.0+(Editor+%231)',t:'0')),k:33.208051925634706,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:output,i:(compilerName:'x86-64+gfortran+15.2',editorid:1,fontScale:14,fontUsePx:'0',j:1,wrap:'1'),l:'5',n:'0',o:'Output+of+x86-64+ifx+2024.0.0+(Compiler+%231)',t:'0')),k:32.8753317406445,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4

I am confused. In that linked example, the program crashes with fp-speculation fast, but it passes with fp-speculation=safe?

[theurich]

@climbfuji interesting. I was going off Bryan's comment, see https://github.nrlmry.navy.mil/NEPTUNE/neptune-atmos/pull/1756#discussion_r14771 and link under https://github.nrlmry.navy.mil/NEPTUNE/neptune-atmos/pull/1756#discussion_r14772. But you are right, closer look, and this was with -fp-speculation fast. So then I wonder if we should have used fp-speculation safe under https://github.nrlmry.navy.mil/NEPTUNE/neptune-atmos/pull/1756 afterall!

[climbfuji]

@climbfuji interesting. I was going off Bryan's comment, see https://github.nrlmry.navy.mil/NEPTUNE/neptune-atmos/pull/1756#discussion_r14771 and link under https://github.nrlmry.navy.mil/NEPTUNE/neptune-atmos/pull/1756#discussion_r14772. But you are right, closer look, and this was with -fp-speculation fast. So then I wonder if we should have used fp-speculation safe under https://github.nrlmry.navy.mil/NEPTUNE/neptune-atmos/pull/1756 afterall!

I think so, yes.