Workaround for java scanner bugs (1594/2046/2547/2733)

[bdbaddog]

This issue was originally created at: 2011-03-16 03:35:39.
This issue was reported by: nijtmans.

nijtmans said at 2011-03-16 03:35:39

Here is a proposed workaround for 4 java-scanner bugs.

It works by extending the jar tool recognizing this situation, and adding the missing .class files to the jar command line, so the end result is correct.

I know that it would be much better to fix the scanner, but until that is done, at least this fix results in a correct build, which was not the case before.

A test-case (bug.zip) is attached. Currently, running scons in the directory prints:

scons output

scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
javac -d . -sourcepath . test.java
jar tf test.jar
META-INF/
META-INF/MANIFEST.MF
test$inner.class
test.class
scons: done building targets.

After applying jar.py.patch, the scons output is:

scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Java scanner missed a class file, added now: test$1.class
Java scanner missed a class file, added now: test$1.class
Java scanner missed a class file, added now: test$1.class
Java scanner missed a class file, added now: test$1.class
jar cf test.jar -C . test$inner.class -C . test.class -C . test$1.class
Java scanner missed a class file, added now: test$1.class
Java scanner missed a class file, added now: test$1.class
Java scanner missed a class file, added now: test$1.class
jar tf test.jar
META-INF/
META-INF/MANIFEST.MF
test$inner.class
test.class
test$1.class
scons: done building targets.

Ideas for improvements are welcome, but my main concern is providing a correct build.

Regards,
Jan Nijtmans

nijtmans said at 2011-03-16 03:36:51

Created an attachment (id=843)
proposed fix

nijtmans said at 2011-03-16 03:37:47

Created an attachment (id=844)
test case

nijtmans said at 2011-03-17 08:01:34

Created an attachment (id=845)
improved patch, eliminating import of fnmatch and string

nijtmans said at 2011-03-17 08:02:49

Created an attachment (id=846)
improved fix, but full file

nijtmans said at 2011-03-17 08:12:32

Added an improved version of jar.py.patch, where the use of fnmatch and string import are eliminated. The same could be done with simple string operations.

Here is some explanation about the changes:

• The variable "present" keeps track of the '$'-containing class-files that have been found, preventing that they are put twice at the command line.
• The section between the SPECIAL HANDLING marks does the following:
  • if _chdir and str(src)[-6:] == '.class':
    this line checks if the src.attributes.java_classdir attribute was set, and if the extension is .class this means that the file is produced by the Java scanner. If not, the file is untouched.
  • if str(src).find('$') != -1:
    If the file-name contains '$', it was correctly generated by the scanner. Record this file in the "present" variable
  • else:
    If the file contains no '$', then glob for its companion .class files that might have been missed by the scanner. Any file found, which was not recorded in "present" will be added to the command line.

This way, if no files are found which were missed by the Java scanner, the command line will be exactly as before.

nijtmans said at 2011-03-21 07:47:54

Created an attachment (id=847)
improved patch (please, ignore earlier patches!)

nijtmans said at 2011-03-21 07:53:11

Here is one more attempt, with further improvements (see 2742.patch):

• Changed line 76 in jar.py from:
  srcs.append(env.File(x))
  to
  srcs.append(env.File(SCons.Subst.Literal(x)))
  This has the effect that when filenames are present containing $[a-zA-Z], no attempt is done to substitute variables in it!
• Modified FS.Unlink such that together with foo>.class other files named foo$1.class>are removed before starting javac.

Please review!

nijtmans attached jar.py.patch at 2011-03-16 03:36:51.

proposed fix

nijtmans attached bug.zip at 2011-03-16 03:37:47.

test case

nijtmans attached jar.py.patch at 2011-03-17 08:01:34.

improved patch, eliminating import of fnmatch and string

nijtmans attached jar.py at 2011-03-17 08:02:49.

improved fix, but full file

nijtmans attached 2742.patch at 2011-03-21 07:47:54.

improved patch (please, ignore earlier patches!)

[mwichmann]

Adding this comment to make "live" the links to the issues in the subject line: referenced issues #1594 #2046 #2547 #2733