Allow running the same test class multiple times with multiple annotations

[daniel-frak]

I have a use case where I need to test my repositories against several databases. For that, I use a SomeRepositoryAbstractTest class, where I write all my tests. Then, I extend the class with children like SomeRepositoryPostgresTest and SomeRepositoryH2Test, with each child using a different annotation like @PostgresRepositoryTest or @H2RepositoryTest. This is a bit tedious and, if I ever want to add or remove a database, I will have to go through every repository test in my project.

Each annotation does things like setting up a Spring Boot context, starting Testcontainers, configuring mocks and choosing the correct Spring profiles.

An example annotation could look like this:

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Testcontainers
@DataJpaTest
@EnableJpaAuditing
@TestPropertySource(properties = {"spring.jpa.hibernate.ddl-auto=validate"})
@Import({
        // Common
        SharedJpaRepositoryBeanConfig.class,
        FakeTimeBeanConfig.class,

        // Specific
        SomeJpaRepositoryBeanConfig.class
})
@MockitoBean(types = DomainEventPublisher.class)
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@ActiveProfiles("postgres")
@ContextConfiguration(initializers = {PostgresContainerInitializer.class}) // Initializes Testcontainer for Spring Boot
public @interface PostgresRepositoryTest {
}

I would like to be able to do something like:

@MultiAnnotationTest({PostgresRepositoryTest.class, H2RepositoryTest.class})
class SomeRepositoryAbstractTest {

  // My tests here

}

And have Junit run the test class several times, each time applying another annotation. It would be even better if I could wrap that annotation in another annotation (e.g. a custom @MultiDbRepositoryTest), for reusability.

I have tried achieving this with the current Junit API, but it seems there is no way to add the annotation (e.g. using ByteBuddy) before annotation processing kicks in.

Deliverables

• A mechanism to run the same test class multiple times, each time with different annotations applied
• Making the mechanism work with complex tests, such as those using @ParameterizedTest

[marcphilipp]

The upcoming 5.13 release will introduce @ClassTemplate for running a test class multiple times with different contexts.

@sbrannen Is there a way to programmatically interact with the Spring extension to "inject" a bunch of annotations that influence how the application/test context is built?

[scordio]

Each annotation does things like setting up a Spring Boot context, starting Testcontainers, configuring mocks and choosing the correct Spring profiles.

As a workaround for similar use cases, I usually use/misuse Spring Boot's ApplicationContextRunner to programmatically define the context I want to test, and this usually plays nicely with @ParameterizedTest.

[sbrannen]

Is there a way to programmatically interact with the Spring extension to "inject" a bunch of annotations that influence how the application/test context is built?

No, unfortunately not.

Many components (including third-party components) look up Spring-related configuration annotations based solely on the current "test class".

I have pondered many times what could possibly be done to improve the situation, to allow Spring integration tests to participate in parameterization (like with the new @ParameterizedClass support in Jupiter), but I have failed to come up with something reasonable.

And... funny that you should ask me, because I've been meaning to ask you to brainstorm on this topic, keeping in mind that the SpringExtension is likely not the only extension in the wild that will encounter issues trying to integrate with @ParameterizedClass support.

Perhaps we can do that in the coming week(s)?