Compatibility with annotation processors (eg. Immutables)

[ephemerist]

In our codebase we have a few classes like

import org.immutables.builder.Builder

public record Foo(
  String bar,
  String baz
) {
  @Builder.Constructor
  public Foo {}
}

At compile-time, an additional class FooBuilder is generated, but zinc doesn't know how to map that back to the original Foo.java source file, so gives up and logs a warning Could not determine source for class FooBuilder. As a result, any time anything in the module changes, all classes that depend on FooBuilder get invalidated.

Is mapping from classes generated by annotation processors back to the original source supported by zinc? If not, how about adding def sourceFileName(classFile: ClassFile): Option[String] to Lookup to allow build tools the chance to add this feature for the annotation processors they support?

[eed3si9n]

Is mapping from classes generated by annotation processors back to the original source supported by zinc?

In theory, https://github.com/sbt/zinc/blob/1.10.x/zinc/src/main/scala/sbt/internal/inc/javac/AnalyzingJavaCompiler.scala 's work I think is to associate the class files to source files. If there's a process that Java processor performs that's somehow opaque to Zinc, we might need to tweak something?

[ephemerist]

Can I have your thoughts on ephemerist@d965cb8? This captures more information from the classfile and optionally delegates to ExternalLookup to map the classfile back to the original source. Needs some tests obviously, but didn't want to go too far down the rabbit hole before soliciting feedback.

[eed3si9n]

I think the overall approach is ok, but my concern is the interface part:

+      sourceFileLookup: ClassFile => Option[String]

Assuming that changing the function implementation would change the build semantics, we should invalidate the build when it happens, but with a function we won't have a mechanism to hash it easily. If we could somehow express the same information using a map, then that can be hashed.

[ephemerist]

Unfortunately, because the input to sourceFileLookup is the ClassFile then we can only create a map with the results of the compilation. In theory, we could create a map by parsing all the classfiles from the previous output (eg. the previous class jar), but that feels like an overhead most users wouldn't want to pay.

[eed3si9n]

Looking at

zinc/internal/zinc-classfile/src/main/scala/sbt/internal/inc/classfile/JavaAnalyze.scala

Lines 83 to 93 in f8cf093

	for {
	newClass <- newClasses
	classFile = Parser(newClass, log)
	_ <- classFile.sourceFile orElse guessSourceName(newClass.getFileName.toString)
	source <- guessSourcePath(sourceMap, classFile, log)
	binaryClassName = classFile.className
	loadedClass <- load(
	binaryClassName,
	Some("Error reading API from class file: " + binaryClassName)
	)
	} {

the first parameter into guessSourcePath is called sourceNameMap: Map[String, Set[VirtualFile]]. Currently the sourceNameMap is calculated like this

zinc/internal/zinc-classfile/src/main/scala/sbt/internal/inc/classfile/JavaAnalyze.scala

Lines 43 to 45 in f8cf093

	val sourceMap = sources
	.toSet[VirtualFile]
	.groupBy(_.name)

so if we expose a way for OBT to pass in a custom map, right before compilation starts, you should able to augment the map Map("Foo.java" -> vf(a/b/Foo.java)) to make Map("Foo.java" -> vf(a/b/Foo.java), "FooBuilder.java" -> vf(a/b/Foo.java))?

[ephemerist]

The problem is that I need the classfile information to be able to augment the map (or at least some kind of AST representation). In the case of Immutables, I'm looking for an annotation on the FooBuilder class which tells me the type from which this class was generated (ie. Foo in the case of FooBuilder).