Delayed creation of default Environment overwrites scanner assignments

[bdbaddog]

This issue was originally created at: 2009-10-08 13:49:35.
This issue was reported by: jgershon.

jgershon said at 2009-10-08 13:49:36

The default environment is created lazily. If SourceFileScanner.add_scanner() is called in a SConstruct/SConscript file before the default environment has been created, the later creation of the default environment will re-initialize tools which may overwrite the scanner assignments made by the user.

An example is attached. In some cases, C-style preprocessor #include directives can be used in Fortran files. This requires that .F files be scanned with ClassicCPP rather than F90Scanner. However, if the user calls

SourceFileScanner.add_scanner('.F', SCons.Scanner.C.CScanner())

before the default environment has been created, when the default environment does get created it will call

SourceFileScanner.add_scanner('.F', SCons.Scanner.Fortran.FortranScan())

when setting up its tools, which overrides the scanner assignment the user made.

Build the attached example using scons --tree=all and note that the source file fscan.F has an implicit dependency on fscan.inc. Then comment out the first line, which will defer creation of the default environment until after the scanner assignment has been made. Call scons --tree=all again and note that fscan.inc is no longer picked up as a dependency.

Some suggested options for fixing this problem:

• (Best) Make SourceCodeScanner an instance variable in Environment rather than a global. Not only does this fix the problem, it also allows different environments to be customized with their own scanners.
• (Okay) Check for existing scanner mappings when the default environment is created and don't override them if they've already been assigned. Note that this has to be restricted to ONLY calls made from the default environment setup; in other cases, overriding existing mappings must be allowed.
• (Not So Hot) Do away with lazy initialization of the default environment. Initialize it before the scripts are read.

jgershon said at 2009-10-08 13:52:05

Created an attachment (id=636)
Example project illustrating the problem

gregnoel said at 2009-12-16 15:24:39

Bug party triage. Steven to research and report back (unless he can fix it right away).

As part of fixing this issue, the third reference to SourceFileScanner in the man page should be rewritten. It is in a sentence ending in a colon, but there is no clause after the colon. The next paragraph may well have been intended to be the clause, but probably all of that should be revised to be similar to the way that the first two references are worded.

jgershon said at 2009-12-17 05:31:10

Created an attachment (id=663)
Re-trying previous attachment. (Original seems to be corrupt.)

jgershon said at 2009-12-17 05:36:40

My attachments are getting corrupted somehow. If you experience the same problem, I can mail fscan.tgz to you.

stevenknight said at 2010-04-30 15:18:52

Fixed the documentation per Greg's observations in r4820:

http://scons.tigris.org/source/browse/scons?view=rev&revision=4820

stevenknight said at 2010-04-30 15:24:56

Investigated this (during some airplane time).

Jay's exactly correct that the right solution here is to make SourceFileScanner a per-environment object. To do that, we need lazy evaluation of the source_scanner argument, probably by allowing it to be a string that refers to the name of the desired scanner object. Right now source_scanner must be a scanner object itself.

In the code, a first step would be to change the Node.scan() method to use the get_source_scanner() method, instead of reaching in directly to the self.builder.source_scanner attribute. (get_source_scanner() is actually unused except for rendering the include tree for debugging...)

However, we already have a slightly different mechanism for lazy evaluation of scanner objects, namely the SCANNERS dictionary. I'm not sure at this point how that would interact with other potential lazy evaluation of source_scanner.

Based on that analysis, I'm going to suggest that we prioritize this as 2.x p2. I'm changing the Target Milestone back to -unspecified- so we can discuss at the next bug party.

gregnoel said at 2010-05-19 06:32:08

Bug party triage.

jgershon attached fscan.tgz at 2009-10-08 13:52:05.

Example project illustrating the problem

jgershon attached fscan.tgz at 2009-12-17 05:31:10.

Re-trying previous attachment. (Original seems to be corrupt.)