`MemberNotFoundException` when invoking `tpe` on the tree of a java annotation

[alexishorner]

Hi, I am currently rewriting the ExtractAPI phase of dotty to use TASTy Query and I encountered the following exception when implementing apiAnnotation (dotty/tools/dotc/sbt/ExtractAPI.scala:891):

def apiAnnotation(annot: Annotation): api.Annotation = {
    // Like with inline defs, the whole body of the annotation and not just its
    // type is part of its API so we need to store its hash, but Zinc wants us
    // to extract the annotation type and its arguments, so we use a dummy
    // annotation argument to store the hash of the tree. We still need to
    // extract the annotation type in the way Zinc expects because sbt uses this
    // information to find tests to run (for example junit tests are
    // annotated @org.junit.Test).
    api.Annotation.of(
      apiType(annot.tree.tpe), // Used by sbt to find tests to run
      Array(api.AnnotationArgument.of("TREE_HASH", treeHash(annot.tree, inlineOrigin = NoSymbol).toString)))
  }

Here is the stack trace when running through sbt-test/scripted:

tastyquery.Exceptions$MemberNotFoundException: Member <init> not found in TypeRef(PackageRef(java.lang), Deprecated)
tastyquery.Types$TermRef.doResolve(Types.scala:1076)
tastyquery.Types$TermRef.computeAndStore$2(Types.scala:1050)
tastyquery.Types$TermRef.resolved(Types.scala:1051)
tastyquery.Types$TermRef.underlyingOrMethodic(Types.scala:1085)
tastyquery.Types$TermType.widenTermRef(Types.scala:456)
tastyquery.Trees$Apply.computeAndStore$2(Trees.scala:354)
tastyquery.Trees$Apply.methodType(Trees.scala:357)
tastyquery.Trees$Apply.instantiateMethodType(Trees.scala:363)
tastyquery.Trees$Apply.calculateType(Trees.scala:367)
tastyquery.Trees$TermTree.computeAndStore$1(Trees.scala:98)
tastyquery.Trees$TermTree.tpe(Trees.scala:99)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiAnnotation(ExtractAPITasty.scala:881)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiAnnotations$$anonfun$1(ExtractAPITasty.scala:736)
scala.collection.immutable.List.foreach(List.scala:333)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiAnnotations(ExtractAPITasty.scala:732)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiDef(ExtractAPITasty.scala:406)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiDefinition(ExtractAPITasty.scala:325)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiDefinitions$$anonfun$1(ExtractAPITasty.scala:304)
scala.collection.immutable.List.map(List.scala:250)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiDefinitions(ExtractAPITasty.scala:304)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.$anonfun$11(ExtractAPITasty.scala:260)
dotty.tools.dotc.sbt.ThunkHolder.$anonfun$1(ThunkHolder.scala:28)
xsbti.api.SafeLazy$Impl.get(SafeLazy.java:64)
dotty.tools.dotc.sbt.ThunkHolder.forceThunks$$anonfun$1(ThunkHolder.scala:19)
scala.collection.immutable.List.foreach(List.scala:333)
dotty.tools.dotc.sbt.ThunkHolder.forceThunks(ThunkHolder.scala:19)
dotty.tools.dotc.sbt.ThunkHolder.forceThunks$(ThunkHolder.scala:12)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.forceThunks(ExtractAPITasty.scala:82)
dotty.tools.dotc.sbt.ExtractAPITastyCollector.apiSource(ExtractAPITasty.scala:164)
dotty.tools.dotc.sbt.ExtractAPITasty.run$$anonfun$2(ExtractAPITasty.scala:62)
scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
scala.collection.IterableOnceOps.foreach(IterableOnce.scala:576)
scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:574)
scala.collection.AbstractIterable.foreach(Iterable.scala:933)
scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:903)
dotty.tools.dotc.sbt.ExtractAPITasty.run(ExtractAPITasty.scala:57)
dotty.tools.dotc.transform.Pickler.runOn(Pickler.scala:488)
dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:343)
scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
dotty.tools.dotc.Run.runPhases$1(Run.scala:336)
dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:384)
dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:396)
dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:69)
dotty.tools.dotc.Run.compileUnits(Run.scala:396)
dotty.tools.dotc.Run.compileSources(Run.scala:282)
dotty.tools.dotc.Run.compile(Run.scala:267)
dotty.tools.dotc.Driver.doCompile(Driver.scala:37)
dotty.tools.xsbt.CompilerBridgeDriver.run(CompilerBridgeDriver.java:141)
dotty.tools.xsbt.CompilerBridge.run(CompilerBridge.java:22)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:193)
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:183)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:163)
sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:163)
sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:180)
sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:178)
sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:464)
sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:419)
sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:406)
sbt.internal.inc.Incremental$.apply(Incremental.scala:172)
sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2371)
sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2321)
sbt.internal.server.BspCompileTask$.$anonfun$compute$1(BspCompileTask.scala:31)
sbt.internal.io.Retry$.apply(Retry.scala:47)
sbt.internal.io.Retry$.apply(Retry.scala:29)
sbt.internal.io.Retry$.apply(Retry.scala:24)
sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:31)
sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2319)
scala.Function1.$anonfun$compose$1(Function1.scala:49)
sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
sbt.std.Transform$$anon$4.work(Transform.scala:69)
sbt.Execute.$anonfun$submit$2(Execute.scala:283)
sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
sbt.Execute.work(Execute.scala:292)
sbt.Execute.$anonfun$submit$1(Execute.scala:283)
sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.base/java.lang.Thread.run(Thread.java:840)

The issue occurs when invoking tpe on the tree of a java annotation.

Minimal reproducible example

build.sbt

scalaVersion := "3.4.2-RC1"

libraryDependencies ++= Seq("ch.epfl.scala" %% "tasty-query" % "1.3.0")

src/main/scala/poc.scala

import java.nio.file.*
import java.net.URI

import tastyquery.Contexts.*
import tastyquery.Classpaths.*
import tastyquery.jdk.ClasspathLoaders
import tastyquery.Names.*
import tastyquery.Trees.*
import tastyquery.Symbols.*
import tastyquery.Types.*

@main def main: Unit =
    val cpPaths = List(
        Paths.get("target/scala-3.4.2-RC1/classes"),
        Paths.get("/path/to/scala3-library_3-3.4.2-RC1.jar"),
        Paths.get("/path/to/1.3.0/tasty-query_3-1.3.0.jar"),
        Paths.get("/path/to/semanticdb-javac-0.9.9.jar"),
        Paths.get("/path/to/scala-library-2.13.12.jar"),
        FileSystems.getFileSystem(java.net.URI.create("jrt:/")).getPath("modules", "java.base"),
    )

    val cp = ClasspathLoaders.read(cpPaths)
    given Context = Context.initialize(cp)

    val pkg = ctx.findPackage("java.lang")
    val obj = pkg.findDecl(typeName("Object")).asClass
    val decls = obj.declarations
    
    val finalize = decls.find(_.name == termName("finalize")).get
    val deprecatedAnnot = finalize.annotations.head
    val tree = deprecatedAnnot.tree.asInstanceOf[Apply]
    val tpe = tree.tpe
end main

Observed behavior

tastyquery.Exceptions$MemberNotFoundException: Member <init> not found in TypeRef(PackageRef(java.lang), Deprecated)
        at tastyquery.Types$TermRef.doResolve(Types.scala:1076)
        at tastyquery.Types$TermRef.computeAndStore$2(Types.scala:1050)
        at tastyquery.Types$TermRef.resolved(Types.scala:1051)
        at tastyquery.Types$TermRef.underlyingOrMethodic(Types.scala:1085)
        at tastyquery.Types$TermType.widenTermRef(Types.scala:456)
        at tastyquery.Trees$Apply.computeAndStore$2(Trees.scala:354)
        at tastyquery.Trees$Apply.methodType(Trees.scala:357)
        at tastyquery.Trees$Apply.instantiateMethodType(Trees.scala:363)
        at tastyquery.Trees$Apply.calculateType(Trees.scala:367)
        at tastyquery.Trees$TermTree.computeAndStore$1(Trees.scala:98)
        at tastyquery.Trees$TermTree.tpe(Trees.scala:99)
        at poc$package$.main(poc.scala:32)
        at main.main(poc.scala:12)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)

Expected behavior

No exception should be thrown when calling tree.tpe. In dotty, annot.tree.tpe returns something similar to TypeRef(ThisType(TypeRef(NoPrefix,module class lang)),trait Deprecated).

Findings

Custom java annotations

The issue also occurs with a custom class annotated with custom java annotations, like

package customannotations;

public @interface CustomAnnotation {
    String description() default "";
}

package example

import customannotations.*

class A:
    @CustomAnnotation("custom annotation")
    def baz = 42

The issue does not occur with scala annotations.

Origin

After some digging, I found that tree.tpe ends up calling fun.tpe.widenTermRef (tastyquery/Trees.scala:354), which ends up calling findMember (tastyquery/Symbols:1476), which returns None if name == nme.Constructor.

Thanks in advance for your help !