No executable tests were found during the mutation test with android.app.* import

[icelandld815]

Dear Author,

When I tried to execute the mutation test on my Android application, below warning displayed:

7:57:20 PM PIT >> INFO : MINION : 7:57:20 PM PIT >> FINE : Expecting 259 tests classes from parent
7:57:20 PM PIT >> INFO : MINION : 7:57:20 PM PIT >> FINE : Tests classes received
7:57:20 PM PIT >> INFO : MINION : 7:57:20 PM PIT >> INFO : Checking environment
7:57:21 PM PIT >> INFO : MINION : 7:57:21 PM PIT >> INFO : Found  0 tests
**7:57:21 PM PIT >> INFO : MINION : 7:57:21 PM PIT >> WARNING : No executable tests were found after examining the 259 test classes supplied. This may indicate an issue with the classpath or a missing test plugin (e.g for JUnit 5).**
7:57:21 PM PIT >> INFO : MINION : 7:57:21 PM PIT >> INFO : 0 tests discovered
7:57:21 PM PIT >> INFO : MINION : 7:57:21 PM PIT >> INFO : 0All tests were executed as part of discovery.
7:57:21 PM PIT >> FINE : Coverage generator Minion exited ok

And i use "pl.droidsonroids.gradle:gradle-pitest-plugin:0.2.18" and config the pitest as below:

pitest{
     targetClasses=['weather.models.helper.*']
    pitestVersion = '1.9.4'
    mutationThreshold = 80
    outputFormats = ['HTML']
    excludeMockableAndroidJar = true
    avoidCallsTo = ['kotlin.jvm.internal']
    verbose = true
}

and here is my test config:

   testImplementation "io.mockk:mockk:$mockk_version"
   testImplementation "junit:junit:4.13.2"
   testImplementation "androidx.arch.core:core-testing:2.2.0"
   testImplementation "org.jetbrains.kotlinx:kotlinx-coroutines-test:$kotlinx_coroutines_test_version"**

Could you please kindly help to give the insights?

[NamigGadir]

Hello. Could you pls share one example project to take a look?

[icelandld815]

Hi @NamigGadir It's a little time consuming for me to prepare the sample project. However, there's some other findings for me.
For my current set up, it works for the junit test which only relay on org.junit and io.mockk like below:

import io.mockk.MockKAnnotations
import io.mockk.mockk
import io.mockk.unmockkAll
import org.junit.After
import org.junit.Assert
import org.junit.Before
import org.junit.Test

However, the test can't be load if it contains other android.app.* relevant import like

import android.app.AlarmManager
import android.app.PendingIntent
import android.content.Context
import androidx.test.core.app.ApplicationProvider
import io.mockk.*
import io.mockk.impl.annotations.MockK
import org.junit.After
import org.junit.Before
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.RuntimeEnvironment
import org.robolectric.annotation.Config
import java.time.LocalTime

and it reports the error

3:23:55 PM PIT >> FINE : MINION : Installing PIT agent
3:23:55 PM PIT >> INFO : Sending 1 test classes to minion
3:23:55 PM PIT >> INFO : Sent tests to minion
3:23:55 PM PIT >> FINE : MINION : java.lang.TypeNotPresentException: Type [unknown] not present
3:23:55 PM PIT >> FINE : MINION :       at java.base/java.lang.reflect.Method.getDefaultValue(Method.java:725)
3:23:55 PM PIT >> FINE : MINION :       at java.base/sun.reflect.annotation.AnnotationType.<init>(AnnotationType.java:133)
3:23:55 PM PIT >> FINE : MINION :       at java.base/sun.reflect.annotation.AnnotationType.getInstance(AnnotationType.java:85)
3:23:55 PM PIT >> FINE : MINION :       at java.base/sun.reflect.annotation.AnnotationParser.parseAnnotation2(AnnotationParser.java:262)
3:23:55 PM PIT >> FINE : MINION :       at java.base/sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:121)
3:23:55 PM PIT >> FINE : MINION :       at java.base/sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:73)
3:23:55 PM PIT >> FINE : MINION :       at java.base/java.lang.Class.createAnnotationData(Class.java:4069)
3:23:55 PM PIT >> FINE : MINION :       at java.base/java.lang.Class.annotationData(Class.java:4058)
3:23:55 PM PIT >> FINE : MINION :       at java.base/java.lang.Class.getAnnotation(Class.java:3941)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.junit.JUnit4SuiteFinder.apply(JUnit4SuiteFinder.java:30)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.junit.JUnit4SuiteFinder.apply(JUnit4SuiteFinder.java:26)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.extension.common.CompoundTestSuiteFinder.apply(CompoundTestSuiteFinder.java:20)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.extension.common.CompoundTestSuiteFinder.apply(CompoundTestSuiteFinder.java:9)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.mutationtest.config.PrioritisingTestSuiteFinder.apply(PrioritisingTestSuiteFinder.java:18)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.mutationtest.config.PrioritisingTestSuiteFinder.apply(PrioritisingTestSuiteFinder.java:8)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.testapi.execute.FindTestUnits.findTestUnits(FindTestUnits.java:54)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.testapi.execute.FindTestUnits.getTestUnits(FindTestUnits.java:47)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.testapi.execute.FindTestUnits.findTestUnitsForAllSuppliedClasses(FindTestUnits.java:36)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.coverage.execute.CoverageMinion.discoverTests(CoverageMinion.java:167)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(CoverageMinion.java:153)
3:23:55 PM PIT >> FINE : MINION :       at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:84)
3:23:55 PM PIT >> FINE : MINION : Caused by: java.lang.NoClassDefFoundError: android/app/Application

From above analysis, i am just wondering does this plugin support the pitest if the test has an android.app.* import on the UT case? To avoid the confusion, the test is under the src/test/java but has the android.app.* import

Looking forward for your insights.

[NamigGadir]

@icelandld815 could you pls check this project? -> https://github.com/NamigGadir/pitest_issue

[dingn-otaku]

Hi, @NamigGadir ,thank you for providing the sample project for us. However, upon reviewing the project, I noticed that there are no "android.." imports in the unit test cases under "android/src/test/java/com/example/android" and "app/src/test/java/com/example/pitestandroidtestproject".

We would like to clarify the following:
For unit tests, if we import Android-related packages (as shown in the sample below), Is Robolectric framework the mandatory to execute mutation tests with your plugin?

package com.example.models.extension

import android.content.Context
import io.mockk.MockKAnnotations
import io.mockk.every
import io.mockk.impl.annotations.MockK
import io.mockk.mockkObject
import io.mockk.verify
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Before
import org.junit.Test

class AlarmExtensionTest {

    @MockK
    private lateinit var mockContext: Context


    @Before
    fun setUp() {
        MockKAnnotations.init(this, relaxed = true)
        mockkObject(DateUtil)
        every { mockContext.getString(R.string.update_time) } returns "update at"
        every { mockContext.getString(R.string.warning_source) } returns warningSource
    }

    @Test
    fun `test get alarm content`() {
        val desc =
            "warning。"
        val date = "2023-06-11T16:33:01+08:00"
        val timeText = "2023-06-11 16:33"
        val alarm = generateAlarm(desc = desc, date = date)

        every { DateUtil.getTimeText(date) } returns timeText

        val content = alarm.getContent(mockContext)
        val expected = "$desc$warningSource\n\n update at $timeText"
        assertEquals(expected, content)
    }

[NamigGadir]

Hi @dingn-otaku . I will check your question and return to you soon.

[NamigGadir]

Hi @dingn-otaku I have checked this code snippet and it is working

package pitest.test

import android.content.Context
import io.mockk.MockKAnnotations
import io.mockk.every
import io.mockk.impl.annotations.MockK
import org.junit.Assert.assertEquals
import org.junit.Before
import org.junit.Test
import pl.drodsonroids.pitest.R

class AlarmExtensionTest {

@MockK
private lateinit var mockContext: Context


@Before
fun setUp() {
    MockKAnnotations.init(this, relaxed = true)
    every { mockContext.getString(any()) } returns "1"
}

@Test
fun `test get alarm content`() {
    val desc =
        "warning。"
    val date = "2023-06-11T16:33:01+08:00"
    val timeText = "2023-06-11 16:33"

    every { mockContext.getString(R.string.app_name) } returns timeText
    val result = mockContext.getString(R.string.app_name)

    assertEquals(timeText, result)
}

}