Fortran compile fails with module dependencies

[james-thunes]

Describe the bug
scons does not appear to be correctly find dependencies on *.mod files in Fortran files.

My project includes a Fortran library that includes a number of source files and a module including some shared variables. When compiling on windows with ifort, I get the following error when doing a clean build:
error #7002: Error in opening the compiled module file. Check INCLUDE paths.
Subsequent builds complete successfully.

Investigation shows that the above compiler error is seen because the *.mod file required by the source files is not built before the source file compilation. Reordering the list of source files improves the situation, but it still fails periodically when building in parallel. Subsequent attempts to compile work as the *.mod file is created during the first scons attempt and are thus already available.

Per suggestion from discord, I ran scons with --tree=prune to check if the consumers of the module correctly listed the *.mod file as a build dependency. The output below (anonymized) shows the output for one of the source files:

  |   +-<path>\<objName>.staticrt.obj
  |   | +-<relativePath>\<srcFile>.f
  |   | +-C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2019.5.281\windows\bin\intel64\ifort.EXE

Note that the source file does not list the *.mod file as a dependency.

The project is cross-platform so I also checked the result of --tree=prune on linux (compiled with gfortran). Scons also did not recognize the dependency on the mod file (output was the same as above with the exception of the compiler path). However, the code compiles without issue on linux.

Since the above shows that the source files are not identifying the *.mod file as a build dependency, my assumption is that there is some issue in the way that scons is determining dependencies for Fortran files. I'm not sure why there's not an issue with gfortran on linux.

Required information

• Link to SCons Users thread discussing your issue.
  discussed on the scons discord (fortran-support channel)
• Version of SCons
  3.1.2
• Version of Python
  3.7.13
• Which python distribution if applicable (python.org, cygwin, anaconda, macports, brew,etc)
  cpython
• How you installed SCons
  pip install
• What Platform are you on? (Linux/Windows and which version)
  Windows 10 Enterprise version1909
• How to reproduce your issue? Please include a small self contained reproducer. Likely a SConstruct should do for most issues.
• How you invoke scons (The command line you're using "scons --flags some_arguments")
  scons -j8

[james-thunes]

I haven't been able to get a small test case set up yet. I will try to create one and post to this issue

[dnwillia-work]

I'll just add that when I was testing this locally it seems to only be reproducible when running a parallel build with -jN. A serial build would do things in the right order and I never saw that fail, but for parallel it's failing when a dependent source file is compiled concurrently before the module is complete.

[bdbaddog]

I'll just add that when I was testing this locally it seems to only be reproducible when running a parallel build with -jN. A serial build would do things in the right order and I never saw that fail, but for parallel it's failing when a dependent source file is compiled concurrently before the module is complete.

That's just luck... If parallel builds fail due to missing dependencies (for any build system), it means the build system doesn't have all the appropriate dependencies and so isn't building everything needed for the failing build step. I run into this often (though mainly with non-scons based build systems where it's harder to get this right)..

[dnwillia-work]

Yeah, fair enough. Even for the parallel builds I'll add that it would sometimes work, sometimes not.

I wonder since it seems to be a windows specific thing then is something else going on. eg: the .mod file is still locked by the OS so the dependent module cannot link against it. We have run into this kind of issue with .lib (import library) files on windows parallel builds before as well if an Install() was used to move the .lib file to some other place before it was linked into dependent module despite the fact that we would add Depends() on the install task to the dependent module. We removed the use of Install() targets for this as a result and it got better.

[bdbaddog]

Ugh.. the joys of NTFS..

[bdbaddog]

How are you creating your Environment()? Specifically are you setting FORTRANSUFFIXES (or any of the F*SUFFIXES env vars)?

[bdbaddog]

Try adding this to your Environment()

FORTRANFILESUFFIXES=['.f90'],

[james-thunes]

this particular library has all *.f files, but I can try setting FORTRANFILESUFFIXES=['.f'],

[dnwillia-work]

How are you creating your Environment()? Specifically are you setting FORTRANSUFFIXES (or any of the F*SUFFIXES env vars)?

We definitely do not configure that particular variable currently. We do configure these variables related to this issue

    env["FORTRANMODDIR"] = "${TARGET.dir}"
    env["F90PATH"] = "${TARGET.dir}"

It's the same for ifort and gfortran.

[mwichmann]

Okay, I've located a simple example and don't see the problem with it. Here's the tree dump:

+-.
  +-SConstruct
  +-list-module.f90
  +-list-module.o
  | +-list-module.f90
  | +-/bin/gfortran
  +-listmodule.mod
  | +-list-module.f90
  | +-/bin/gfortran
  +-main-list
  | +-main-list.o
  | | +-main-list.f90
  | | +-[listmodule.mod]
  | | +-/bin/gfortran
  | +-[list-module.o]
  | +-/bin/gfortran
  +-main-list.f90
  +-[main-list.o]

main-list.o is shown as depending on its source file, on listmodule.mod, and on the compiler. That looks right to me? What am I missing?

[bdbaddog]

@mwichmann - I think this is only an intel fortran issue. And I think it's due to loading default tools, then ifort and having files named .f when ifort looks like it only wants files named .i or .i90?

[bdbaddog]

this particular library has all *.f files, but I can try setting FORTRANFILESUFFIXES=['.f'],

Did that work?

[mwichmann]

I still have to look at the .i/.i90 thing, I think it should work just find on ordinary suffixes and I don't know what those actually mean.

[james-thunes]

this particular library has all *.f files, but I can try setting FORTRANFILESUFFIXES=['.f'],

Did that work?

Sorry, been busy with other tasks. Will try to get to this as soon as I can.

[james-thunes]

this particular library has all *.f files, but I can try setting FORTRANFILESUFFIXES=['.f'],

Did that work?

Ok, I was able to spend a bit of time with this today. I'm a bit confused to be honest. Setting FORTRANFILESUFFIXES=['.f'], did indeed appear to resolve the issue. I was able to see the module dependencies when looking at the output with --tree=prune. However, removing the flag and doing a clean rebuild of the code I see that the module dependencies is still seen. It definitely wasn't last week...