Skip to content

Anvil causes Gradle tasks to be non-cacheable #1039

Open
@1gravity

Description

@1gravity

When analyzing our Gradle builds, we see that the Kotlin compile task cannot be cached by Gradle. The error message always shows for the app module and sometimes also for library modules:

Overlapping outputs: Gradle does not know how file 'build/anvil/devDebug/caches/generated-file-cache.bin' was created (output property '$2'). Task output caching requires exclusive access to output paths to guarantee correctness (i.e. multiple tasks are not allowed to produce output in the same location).

Kind TASK  
Path :mobile:compileDevDebugKotlin
Type org.jetbrains.kotlin.gradle.tasks.KotlinCompile

Dagger 2.51.1
Anvil 2.5.0-beta09
Kotlin 1.9.23

com.squareup.anvil.trackSourceFiles=true

This is an issue for us since non-cacheable tasks increase the local builds times considerably

Activity

changed the title Anvil causes Anvil causes Gradle tasks to be non-cacheable on Jul 22, 2024
RBusarow

RBusarow commented on Jul 22, 2024

@RBusarow
Collaborator

Thanks for the report.

That warning message means that some other task is declaring that directory, that file, or some other parent directory as its own output. The warning will only happen when both of the tasks are actually configured and in the task graph, which might explain why it only sometimes happens with library modules.

The Anvil plugin creates cache directories that are unique to each build variant (e.g., devDebug), and they're all direct children of $buildDir/anvil, so Anvil won't create overlaps on its own.

To find the other task, you can add an assertion against the output files of other tasks. For example, to find out the culprit for :mobile:compileDevDebugKotlin, you could add this to that project's build.gradle.kts:

val overlapped = generateSequence(
  file("build/anvil/devDebug/caches/generated-file-cache.bin"),
) { it.parentFile }
  .takeWhile { it != rootDir }
  .toList()

tasks.named { it != "compileDevDebugKotlin" }.configureEach {
  doLast {
    val overlaps = overlapped.filter { outputs.files.contains(it) }
    check(overlaps.isEmpty()) {
      "Overlapping output files:\n${overlaps.joinToString("\n")}"
    }
  }
}

After that, if you just run the same task(s) that prompted the warning before, it should tell you where the overlap is coming from.

1gravity

1gravity commented on Jul 22, 2024

@1gravity
Author

Thanks @RBusarow for getting back to me so quickly. I tried this but none of the tasks seem to have an overlap while I still get the overlapping output message in the Gradle build scan. I had to convert the script to Groovy because we haven't converted all of our build files to Kotlin yet but I think this should work:

def overlapped = []
def currentFile = file("build/anvil/devDebug/caches/generated-file-cache.bin")
while (currentFile != rootDir) {
    overlapped.add(currentFile)
    currentFile = currentFile.parentFile
}

tasks.configureEach {
    if (name != "compileDevDebugKotlin") {
        doLast {
            def overlaps = overlapped.findAll { outputs.files.contains(it) }
            if (!overlaps.isEmpty()) {
                throw new GradleException("Overlapping output files:\n${overlaps.join('\n')}")
            }
        }
    }
}

I put in print statements to verify that there's indeed no overlap (there isn't). I am working through past releases of our app to see when this starts happening. I don't see the issue with Anvil 2.4.9 but will keep testing.

1gravity

1gravity commented on Jul 22, 2024

@1gravity
Author

I can say with a high level of certainty that the issue was introduced with a 2.5.0-beta0X version. I don't see it with 2.4.9 and it shows the moment we updated to 2.5.0-beta03 which we did because of #693.
I compared build times for a one line of code change in the app module and times went from around 1m 30s to 2m 30s with beta03 and are now at 5m 10s with beta09.
I took our latest stable version and reverted back to Anvil 2.4.9 and the build times dropped from >5m to 1m 45s for that simple test case I used.
I don't know if that's all due to the Overlapping outputs issue but that seems to be a good starting point.

I just tested with 2.5.0-beta10 and still the same issue.

1gravity

1gravity commented on Jul 23, 2024

@1gravity
Author

Duplicate of #821 mentioned in #693 (comment)?

1gravity

1gravity commented on Jul 23, 2024

@1gravity
Author

After running more tests, I found that three of our Gradle projects show that error when we run all tasks (--rerun-tasks):

image

What they have in common is that kapt is running the Dagger compiler. I removed it from the two library modules using generateDaggerFactories.set(true) and that eliminates the error from those two library modules. However I cannot disable kapt/Dagger in the app module since we need Dagger to create the components.

My assumption is that there is some interaction between kapt and the Anvil plugin.

1gravity

1gravity commented on Jul 24, 2024

@1gravity
Author

I was able to reproduce the issue in a sample app that I pushed to Github: https://github.com/1gravity/anvil-tests.

When I make a change in the app module and run ./gradlew :app:assembleDebug --scan I can see the error in the Gradle scan: https://scans.gradle.com/s/waavwoxnjlvj4/timeline?cacheability=any_non_cacheable&hide-timeline&kind=task&outcome=success,failed&sort=longest

I would very much appreciate if someone could try this and give me some guidance on what needs to change

RBusarow

RBusarow commented on Jul 24, 2024

@RBusarow
Collaborator

Thanks for the repro. It seems to be an issue with KAPT. If you run this, the problem goes away:

./gradlew :app:compileDebugKotlin -x :app:kaptDebugKotlin

It's not immediately apparent to me why that's happening. I should have time to dig into it more this evening.

chrisbanes

chrisbanes commented on Aug 1, 2024

@chrisbanes

Just +1'ing this. We're seeing the same.

nwagu

nwagu commented on Sep 3, 2024

@nwagu

This is an issue only when trackSourceFiles is enabled, right?

bmuschko

bmuschko commented on Sep 12, 2024

@bmuschko

I am seeing the same issue. +1

WendyYanto

WendyYanto commented on Nov 28, 2024

@WendyYanto

+1, facing the same issue with trackSourceFiles enabled.

Kind | TASK |  
Path | :app:compileInternalDebugKotlin |  
Type | org.jetbrains.kotlin.gradle.tasks.KotlinCompile |

Image

Anvil Version: 2.5.0-beta11

rbrauwerstala

rbrauwerstala commented on Jan 20, 2025

@rbrauwerstala

+1 facing this issue.
Anvil: 2.5.0

Image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @chrisbanes@bmuschko@1gravity@RBusarow@nwagu

      Issue actions

        Anvil causes Gradle tasks to be non-cacheable · Issue #1039 · square/anvil