Skip to content

Latest commit

 

History

History
1255 lines (1009 loc) · 57.3 KB

extensions.adoc

File metadata and controls

1255 lines (1009 loc) · 57.3 KB

Extension Model

Overview

In contrast to the competing Runner, TestRule, and MethodRule extension points in JUnit 4, the JUnit Jupiter extension model consists of a single, coherent concept: the Extension API. Note, however, that Extension itself is just a marker interface.

Registering Extensions

Extensions can be registered declaratively via @ExtendWith, programmatically via @RegisterExtension, or automatically via Java’s ServiceLoader mechanism.

Declarative Extension Registration

Developers can register one or more extensions declaratively by annotating a test interface, test class, test method, or custom composed annotation with @ExtendWith(…​) and supplying class references for the extensions to register. As of JUnit Jupiter 5.8, @ExtendWith may also be declared on fields or on parameters in test class constructors, in test methods, and in @BeforeAll, @AfterAll, @BeforeEach, and @AfterEach lifecycle methods.

For example, to register a WebServerExtension for a particular test method, you would annotate the test method as follows. We assume the WebServerExtension starts a local web server and injects the server’s URL into parameters annotated with @WebServerUrl.

@Test
@ExtendWith(WebServerExtension.class)
void getProductList(@WebServerUrl String serverUrl) {
	WebClient webClient = new WebClient();
	// Use WebClient to connect to web server using serverUrl and verify response
	assertEquals(200, webClient.get(serverUrl + "/products").getResponseStatus());
}

To register the WebServerExtension for all tests in a particular class and its subclasses, you would annotate the test class as follows.

@ExtendWith(WebServerExtension.class)
class MyTests {
	// ...
}

Multiple extensions can be registered together like this:

@ExtendWith({ DatabaseExtension.class, WebServerExtension.class })
class MyFirstTests {
	// ...
}

As an alternative, multiple extensions can be registered separately like this:

@ExtendWith(DatabaseExtension.class)
@ExtendWith(WebServerExtension.class)
class MySecondTests {
	// ...
}
Tip
Extension Registration Order

Extensions registered declaratively via @ExtendWith at the class level, method level, or parameter level will be executed in the order in which they are declared in the source code. For example, the execution of tests in both MyFirstTests and MySecondTests will be extended by the DatabaseExtension and WebServerExtension, in exactly that order.

If you wish to combine multiple extensions in a reusable way, you can define a custom composed annotation and use @ExtendWith as a meta-annotation as in the following code listing. Then @DatabaseAndWebServerExtension can be used in place of @ExtendWith({ DatabaseExtension.class, WebServerExtension.class }).

@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@ExtendWith({ DatabaseExtension.class, WebServerExtension.class })
public @interface DatabaseAndWebServerExtension {
}

The above examples demonstrate how @ExtendWith can be applied at the class level or at the method level; however, for certain use cases it makes sense for an extension to be registered declaratively at the field or parameter level. Consider a RandomNumberExtension which generates random numbers that can be injected into a field or via a parameter in a constructor, test method, or lifecycle method. If the extension provides a @Random annotation that is meta-annotated with @ExtendWith(RandomNumberExtension.class) (see listing below), the extension can be used transparently as in the following RandomNumberDemo example.

link:../../../../src/test/java/example/extensions/Random.java[role=include]
link:../../../../src/test/java/example/extensions/RandomNumberDemo.java[role=include]

The following code listing provides an example of how one might choose to implement such a RandomNumberExtension. This implementation works for the use cases in RandomNumberDemo; however, it may not prove robust enough to cover all use cases — for example, the random number generation support is limited to integers; it uses java.util.Random instead of java.security.SecureRandom; etc. In any case, it is important to note which extension APIs are implemented and for what reasons.

Specifically, RandomNumberExtension implements the following extension APIs:

  • BeforeAllCallback: to support static field injection

  • TestInstancePostProcessor: to support non-static field injection

  • ParameterResolver: to support constructor and method injection

link:../../../../src/test/java/example/extensions/RandomNumberExtension.java[role=include]
Tip
Extension Registration Order for @ExtendWith on Fields

Extensions registered declaratively via @ExtendWith on fields will be ordered relative to @RegisterExtension fields and other @ExtendWith fields using an algorithm that is deterministic but intentionally nonobvious. However, @ExtendWith fields can be ordered using the @Order annotation. See the Extension Registration Order tip for @RegisterExtension fields for details.

Tip
Extension Inheritance

Extensions registered declaratively via @ExtendWith on fields in superclasses will be inherited.

See Extension Inheritance for details.

Note
@ExtendWith fields may be either static or non-static. The documentation on Static Fields and Instance Fields for @RegisterExtension fields also applies to @ExtendWith fields.

Programmatic Extension Registration

Developers can register extensions programmatically by annotating fields in test classes with {RegisterExtension}.

When an extension is registered declaratively via @ExtendWith, it can typically only be configured via annotations. In contrast, when an extension is registered via @RegisterExtension, it can be configured programmatically — for example, in order to pass arguments to the extension’s constructor, a static factory method, or a builder API.

Tip
Extension Registration Order

By default, extensions registered programmatically via @RegisterExtension or declaratively via @ExtendWith on fields will be ordered using an algorithm that is deterministic but intentionally nonobvious. This ensures that subsequent runs of a test suite execute extensions in the same order, thereby allowing for repeatable builds. However, there are times when extensions need to be registered in an explicit order. To achieve that, annotate @RegisterExtension fields or @ExtendWith fields with {Order}.

Any @RegisterExtension field or @ExtendWith field not annotated with @Order will be ordered using the default order which has a value of Integer.MAX_VALUE / 2. This allows @Order annotated extension fields to be explicitly ordered before or after non-annotated extension fields. Extensions with an explicit order value less than the default order value will be registered before non-annotated extensions. Similarly, extensions with an explicit order value greater than the default order value will be registered after non-annotated extensions. For example, assigning an extension an explicit order value that is greater than the default order value allows before callback extensions to be registered last and after callback extensions to be registered first, relative to other programmatically registered extensions.

Tip
Extension Inheritance

Extensions registered via @RegisterExtension or @ExtendWith on fields in superclasses will be inherited.

See Extension Inheritance for details.

Note
@RegisterExtension fields must not be null (at evaluation time) but may be either static or non-static.
Static Fields

If a @RegisterExtension field is static, the extension will be registered after extensions that are registered at the class level via @ExtendWith. Such static extensions are not limited in which extension APIs they can implement. Extensions registered via static fields may therefore implement class-level and instance-level extension APIs such as BeforeAllCallback, AfterAllCallback, TestInstancePostProcessor, and TestInstancePreDestroyCallback as well as method-level extension APIs such as BeforeEachCallback, etc.

In the following example, the server field in the test class is initialized programmatically by using a builder pattern supported by the WebServerExtension. The configured WebServerExtension will be automatically registered as an extension at the class level — for example, in order to start the server before all tests in the class and then stop the server after all tests in the class have completed. In addition, static lifecycle methods annotated with @BeforeAll or @AfterAll as well as @BeforeEach, @AfterEach, and @Test methods can access the instance of the extension via the server field if necessary.

Registering an extension via a static field in Java
link:../../../../src/test/java/example/registration/WebServerDemo.java[role=include]
Static Fields in Kotlin

The Kotlin programming language does not have the concept of a static field. However, the compiler can be instructed to generate a private static field using the @JvmStatic annotation in Kotlin. If you want the Kotlin compiler to generate a public static field, you can use the @JvmField annotation instead.

The following example is a version of the WebServerDemo from the previous section that has been ported to Kotlin.

Registering an extension via a static field in Kotlin
link:../../../../src/test/kotlin/example/registration/KotlinWebServerDemo.kt[role=include]
Instance Fields

If a @RegisterExtension field is non-static (i.e., an instance field), the extension will be registered after the test class has been instantiated and after each registered TestInstancePostProcessor has been given a chance to post-process the test instance (potentially injecting the instance of the extension to be used into the annotated field). Thus, if such an instance extension implements class-level or instance-level extension APIs such as BeforeAllCallback, AfterAllCallback, or TestInstancePostProcessor, those APIs will not be honored. Instance extensions will be registered before extensions that are registered at the method level via @ExtendWith.

In the following example, the docs field in the test class is initialized programmatically by invoking a custom lookUpDocsDir() method and supplying the result to the static forPath() factory method in the DocumentationExtension. The configured DocumentationExtension will be automatically registered as an extension at the method level. In addition, @BeforeEach, @AfterEach, and @Test methods can access the instance of the extension via the docs field if necessary.

An extension registered via an instance field
link:../../../../src/test/java/example/registration/DocumentationDemo.java[role=include]

Automatic Extension Registration

In addition to declarative extension registration and programmatic extension registration support using annotations, JUnit Jupiter also supports global extension registration via Java’s {ServiceLoader} mechanism, allowing third-party extensions to be auto-detected and automatically registered based on what is available in the classpath.

Specifically, a custom extension can be registered by supplying its fully qualified class name in a file named org.junit.jupiter.api.extension.Extension within the /META-INF/services folder in its enclosing JAR file.

Enabling Automatic Extension Detection

Auto-detection is an advanced feature and is therefore not enabled by default. To enable it, set the junit.jupiter.extensions.autodetection.enabled configuration parameter to true. This can be supplied as a JVM system property, as a configuration parameter in the LauncherDiscoveryRequest that is passed to the Launcher, or via the JUnit Platform configuration file (see [running-tests-config-params] for details).

For example, to enable auto-detection of extensions, you can start your JVM with the following system property.

-Djunit.jupiter.extensions.autodetection.enabled=true

When auto-detection is enabled, extensions discovered via the {ServiceLoader} mechanism will be added to the extension registry after JUnit Jupiter’s global extensions (e.g., support for TestInfo, TestReporter, etc.).

Filtering Auto-detected Extensions

The list of auto-detected extensions can be filtered using include and exclude patterns via the following configuration parameters:

junit.jupiter.extensions.autodetection.include=<patterns>

Comma-separated list of include patterns for auto-detected extensions.

junit.jupiter.extensions.autodetection.exclude=<patterns>

Comma-separated list of exclude patterns for auto-detected extensions.

Include patterns are applied before exclude patterns. If both include and exclude patterns are provided, only extensions that match at least one include pattern and do not match any exclude pattern will be auto-detected.

See [running-tests-config-params-deactivation-pattern] for details on the pattern syntax.

Extension Inheritance

Registered extensions are inherited within test class hierarchies with top-down semantics. Similarly, extensions registered at the class-level are inherited at the method-level. This applies to all extensions, independent of how they are registered (declaratively or programmatically).

This means that extensions registered declaratively via @ExtendWith on a superclass will be registered before extensions registered declaratively via @ExtendWith on a subclass.

Similarly, extensions registered programmatically via @RegisterExtension or @ExtendWith on fields in a superclass will be registered before extensions registered programmatically via @RegisterExtension or @ExtendWith on fields in a subclass, unless @Order is used to alter that behavior (see Extension Registration Order for details).

Note
A specific extension implementation can only be registered once for a given extension context and its parent contexts. Consequently, any attempt to register a duplicate extension implementation will be ignored.

Conditional Test Execution

{ExecutionCondition} defines the Extension API for programmatic, conditional test execution.

An ExecutionCondition is evaluated for each container (e.g., a test class) to determine if all the tests it contains should be executed based on the supplied ExtensionContext. Similarly, an ExecutionCondition is evaluated for each test to determine if a given test method should be executed based on the supplied ExtensionContext.

When multiple ExecutionCondition extensions are registered, a container or test is disabled as soon as one of the conditions returns disabled. Thus, there is no guarantee that a condition is evaluated because another extension might have already caused a container or test to be disabled. In other words, the evaluation works like the short-circuiting boolean OR operator.

See the source code of {DisabledCondition} and {Disabled} for concrete examples.

Deactivating Conditions

Sometimes it can be useful to run a test suite without certain conditions being active. For example, you may wish to run tests even if they are annotated with @Disabled in order to see if they are still broken. To do this, provide a pattern for the junit.jupiter.conditions.deactivate configuration parameter to specify which conditions should be deactivated (i.e., not evaluated) for the current test run. The pattern can be supplied as a JVM system property, as a configuration parameter in the LauncherDiscoveryRequest that is passed to the Launcher, or via the JUnit Platform configuration file (see [running-tests-config-params] for details).

For example, to deactivate JUnit’s @Disabled condition, you can start your JVM with the following system property.

-Djunit.jupiter.conditions.deactivate=org.junit.*DisabledCondition

Pattern Matching Syntax

Test Instance Pre-construct Callback

{TestInstancePreConstructCallback} defines the API for Extensions that wish to be invoked prior to test instances being constructed (by a constructor call or via {TestInstanceFactory}).

This extension provides a symmetric call to {TestInstancePreDestroyCallback} and is useful in combination with other extensions to prepare constructor parameters or keeping track of test instances and their lifecycle.

Note
Accessing the test-scoped ExtensionContext

You may override the getTestInstantiationExtensionContextScope(…​) method to return TEST_METHOD to make test-specific data available to your extension implementation or if you want to keep state on the test method level.

Test Instance Factories

{TestInstanceFactory} defines the API for Extensions that wish to create test class instances.

Common use cases include acquiring the test instance from a dependency injection framework or invoking a static factory method to create the test class instance.

If no TestInstanceFactory is registered, the framework will invoke the sole constructor for the test class to instantiate it, potentially resolving constructor arguments via registered ParameterResolver extensions.

Extensions that implement TestInstanceFactory can be registered on test interfaces, top-level test classes, or @Nested test classes.

Warning

Registering multiple extensions that implement TestInstanceFactory for any single class will result in an exception being thrown for all tests in that class, in any subclass, and in any nested class. Note that any TestInstanceFactory registered in a superclass or enclosing class (i.e., in the case of a @Nested test class) is inherited. It is the user’s responsibility to ensure that only a single TestInstanceFactory is registered for any specific test class.

Note
Accessing the test-scoped ExtensionContext

You may override the getTestInstantiationExtensionContextScope(…​) method to return TEST_METHOD to make test-specific data available to your extension implementation or if you want to keep state on the test method level.

Test Instance Post-processing

{TestInstancePostProcessor} defines the API for Extensions that wish to post process test instances.

Common use cases include injecting dependencies into the test instance, invoking custom initialization methods on the test instance, etc.

For a concrete example, consult the source code for the {MockitoExtension} and the {SpringExtension}.

Note
Accessing the test-scoped ExtensionContext

You may override the getTestInstantiationExtensionContextScope(…​) method to return TEST_METHOD to make test-specific data available to your extension implementation or if you want to keep state on the test method level.

Test Instance Pre-destroy Callback

{TestInstancePreDestroyCallback} defines the API for Extensions that wish to process test instances after they have been used in tests and before they are destroyed.

Common use cases include cleaning dependencies that have been injected into the test instance, invoking custom de-initialization methods on the test instance, etc.

Parameter Resolution

{ParameterResolver} defines the Extension API for dynamically resolving parameters at runtime.

If a test class constructor, test method, or lifecycle method (see [writing-tests-definitions]) declares a parameter, the parameter must be resolved at runtime by a ParameterResolver. A ParameterResolver can either be built-in (see {TestInfoParameterResolver}) or registered by the user. Generally speaking, parameters may be resolved by name, type, annotation, or any combination thereof.

If you wish to implement a custom {ParameterResolver} that resolves parameters based solely on the type of the parameter, you may find it convenient to extend the {TypeBasedParameterResolver} which serves as a generic adapter for such use cases.

For concrete examples, consult the source code for {CustomTypeParameterResolver}, {CustomAnnotationParameterResolver}, and {MapOfListsTypeBasedParameterResolver}.

Warning

Due to a bug in the byte code generated by javac on JDK versions prior to JDK 9, looking up annotations on parameters directly via the core java.lang.reflect.Parameter API will always fail for inner class constructors (e.g., a constructor in a @Nested test class).

The {ParameterContext} API supplied to ParameterResolver implementations therefore includes the following convenience methods for correctly looking up annotations on parameters. Extension authors are strongly encouraged to use these methods instead of those provided in java.lang.reflect.Parameter in order to avoid this bug in the JDK.

  • boolean isAnnotated(Class<? extends Annotation> annotationType)

  • Optional<A> findAnnotation(Class<A> annotationType)

  • List<A> findRepeatableAnnotations(Class<A> annotationType)

Note
Accessing the test-scoped ExtensionContext

You may override the getTestInstantiationExtensionContextScope(…​) method to return TEST_METHOD to support injecting test specific data into constructor parameters of the test class instance. Doing so causes a test-specific {ExtensionContext} to be used while resolving constructor parameters, unless the test instance lifecycle is set to PER_CLASS.

Tip
Parameter resolution for methods called from extensions

Other extensions can also leverage registered ParameterResolvers for method and constructor invocations, using the {ExecutableInvoker} available via the getExecutableInvoker() method in the ExtensionContext.

Parameter Conflicts

If multiple implementations of ParameterResolver that support the same type are registered for a test, a ParameterResolutionException will be thrown, with a message to indicate that competing resolvers have been discovered. See the following example:

Conflicting parameter resolution due to multiple resolvers claiming support for integers
link:../../../../src/test/java/example/extensions/ParameterResolverConflictDemo.java[role=include]

If the conflicting ParameterResolver implementations are applied to different test methods as shown in the following example, no conflict occurs.

Fine-grained registration to avoid conflict
link:../../../../src/test/java/example/extensions/ParameterResolverNoConflictDemo.java[role=include]

If the conflicting ParameterResolver implementations need to be applied to the same test method, you can implement a custom type or custom annotation as illustrated by {CustomTypeParameterResolver} and {CustomAnnotationParameterResolver}, respectively.

Custom type to resolve duplicate types
link:../../../../src/test/java/example/extensions/ParameterResolverCustomTypeDemo.java[role=include]

A custom annotation makes the duplicate type distinguishable from its counterpart:

Custom annotation to resolve duplicate types
link:../../../../src/test/java/example/extensions/ParameterResolverCustomAnnotationDemo.java[role=include]

JUnit includes some built-in parameter resolvers that can cause conflicts if a resolver attempts to claim their supported types. For example, {TestInfo} provides metadata about tests. See [writing-tests-dependency-injection] for details. Third-party frameworks such as Spring may also define parameter resolvers. Apply one of the techniques in this section to resolve any conflicts.

Parameterized tests are another potential source of conflict. Ensure that tests annotated with @ParameterizedTest are not also annotated with @Test and see [writing-tests-parameterized-tests-consuming-arguments] for more details.

Test Result Processing

{TestWatcher} defines the API for extensions that wish to process the results of test method executions. Specifically, a TestWatcher will be invoked with contextual information for the following events.

  • testDisabled: invoked after a disabled test method has been skipped

  • testSuccessful: invoked after a test method has completed successfully

  • testAborted: invoked after a test method has been aborted

  • testFailed: invoked after a test method has failed

Note
In contrast to the definition of "test method" presented in [writing-tests-definitions], in this context test method refers to any @Test method or @TestTemplate method (for example, a @RepeatedTest or @ParameterizedTest).

Extensions implementing this interface can be registered at the class level, instance level, or method level. When registered at the class level, a TestWatcher will be invoked for any contained test method including those in @Nested classes. When registered at the method level, a TestWatcher will only be invoked for the test method for which it was registered.

Warning

If a TestWatcher is registered via a non-static (instance) field – for example, using @RegisterExtension – and the test class is configured with @TestInstance(Lifecycle.PER_METHOD) semantics (which is the default lifecycle mode), the TestWatcher will not be invoked with events for @TestTemplate methods (for example, @RepeatedTest or @ParameterizedTest).

To ensure that a TestWatcher is invoked for all test methods in a given class, it is therefore recommended that the TestWatcher be registered at the class level with @ExtendWith or via a static field with @RegisterExtension or @ExtendWith.

If there is a failure at the class level — for example, an exception thrown by a @BeforeAll method — no test results will be reported. Similarly, if the test class is disabled via an ExecutionCondition — for example, @Disabled — no test results will be reported.

In contrast to other Extension APIs, a TestWatcher is not permitted to adversely influence the execution of tests. Consequently, any exception thrown by a method in the TestWatcher API will be logged at WARNING level and will not be allowed to propagate or fail test execution.

Warning

Any instances of ExtensionContext.Store.CloseableResource stored in the Store of the provided {ExtensionContext} will be closed before methods in the TestWatcher API are invoked (see Keeping State in Extensions). You can use the parent context’s Store to work with such resources.

Test Lifecycle Callbacks

The following interfaces define the APIs for extending tests at various points in the test execution lifecycle. Consult the following sections for examples and the Javadoc for each of these interfaces in the {extension-api-package} package for further details.

  • {BeforeAllCallback}

    • {BeforeClassTemplateInvocationCallback} (only applicable for class templates)

      • {BeforeEachCallback}

        • {BeforeTestExecutionCallback}

        • {AfterTestExecutionCallback}

      • {AfterEachCallback}

    • {AfterClassTemplateInvocationCallback} (only applicable for class templates)

  • {AfterAllCallback}

Note
Implementing Multiple Extension APIs
Extension developers may choose to implement any number of these interfaces within a single extension. Consult the source code of the {SpringExtension} for a concrete example.

Before and After Test Execution Callbacks

{BeforeTestExecutionCallback} and {AfterTestExecutionCallback} define the APIs for Extensions that wish to add behavior that will be executed immediately before and immediately after a test method is executed, respectively. As such, these callbacks are well suited for timing, tracing, and similar use cases. If you need to implement callbacks that are invoked around @BeforeEach and @AfterEach methods, implement BeforeEachCallback and AfterEachCallback instead.

The following example shows how to use these callbacks to calculate and log the execution time of a test method. TimingExtension implements both BeforeTestExecutionCallback and AfterTestExecutionCallback in order to time and log the test execution.

An extension that times and logs the execution of test methods
link:../../../../src/test/java/example/timing/TimingExtension.java[role=include]

Since the TimingExtensionTests class registers the TimingExtension via @ExtendWith, its tests will have this timing applied when they execute.

A test class that uses the example TimingExtension
link:../../../../src/test/java/example/timing/TimingExtensionTests.java[role=include]

The following is an example of the logging produced when TimingExtensionTests is run.

INFO: Method [sleep20ms] took 24 ms.
INFO: Method [sleep50ms] took 53 ms.

Exception Handling

Exceptions thrown during the test execution may be intercepted and handled accordingly before propagating further, so that certain actions like error logging or resource releasing may be defined in specialized Extensions. JUnit Jupiter offers API for Extensions that wish to handle exceptions thrown during @Test methods via {TestExecutionExceptionHandler} and for those thrown during one of test lifecycle methods (@BeforeAll, @BeforeEach, @AfterEach and @AfterAll) via {LifecycleMethodExecutionExceptionHandler}.

The following example shows an extension which will swallow all instances of IOException but rethrow any other type of exception.

An exception handling extension that filters IOExceptions in test execution
link:../../../../src/test/java/example/exception/IgnoreIOExceptionExtension.java[role=include]

Another example shows how to record the state of an application under test exactly at the point of unexpected exception being thrown during setup and cleanup. Note that unlike relying on lifecycle callbacks, which may or may not be executed depending on the test status, this solution guarantees execution immediately after failing @BeforeAll, @BeforeEach, @AfterEach or @AfterAll.

An exception handling extension that records application state on error
link:../../../../src/test/java/example/exception/RecordStateOnErrorExtension.java[role=include]

Multiple execution exception handlers may be invoked for the same lifecycle method in order of declaration. If one of the handlers swallows the handled exception, subsequent ones will not be executed, and no failure will be propagated to JUnit engine, as if the exception was never thrown. Handlers may also choose to rethrow the exception or throw a different one, potentially wrapping the original.

Extensions implementing {LifecycleMethodExecutionExceptionHandler} that wish to handle exceptions thrown during @BeforeAll or @AfterAll need to be registered on a class level, while handlers for BeforeEach and AfterEach may be also registered for individual test methods.

Registering multiple exception handling extensions
link:../../../../src/test/java/example/exception/MultipleHandlersTestCase.java[role=include]

Pre-Interrupt Callback

{PreInterruptCallback} defines the API for Extensions that wish to react on timeouts before the Thread.interrupt() is called.

Please refer to [writing-tests-declarative-timeouts-debugging] for additional information.

Intercepting Invocations

{InvocationInterceptor} defines the API for Extensions that wish to intercept calls to test code.

The following example shows an extension that executes all test methods in Swing’s Event Dispatch Thread.

An extension that executes tests in a user-defined thread
link:../../../../src/test/java/example/interceptor/SwingEdtInterceptor.java[role=include]
Note
Accessing the test-scoped ExtensionContext

You may override the getTestInstantiationExtensionContextScope(…​) method to return TEST_METHOD to make test-specific data available to your extension implementation of interceptTestClassConstructor or if you want to keep state on the test method level.

Providing Invocation Contexts for Class Templates

A {ClassTemplate} class can only be executed when at least one {ClassTemplateInvocationContextProvider} is registered. Each such provider is responsible for providing a Stream of {ClassTemplateInvocationContext} instances. Each context may specify a custom display name and a list of additional extensions that will only be used for the next invocation of the {ClassTemplate}.

The following example shows how to write a class template as well as how to register and implement a {ClassTemplateInvocationContextProvider}.

A class template with accompanying extension
link:../../../../src/test/java/example/ClassTemplateDemo.java[role=include]

In this example, the class template will be invoked twice, meaning all test methods in the class template will be executed twice. The display names of the invocations will be apple and banana as specified by the invocation context. Each invocation registers a custom {TestInstancePostProcessor} which is used to inject a value into a field. The output when using the ConsoleLauncher is as follows.

└─ ClassTemplateDemo ✔
   ├─ apple ✔
   │  ├─ notNull() ✔
   │  └─ wellKnown() ✔
   └─ banana ✔
      ├─ notNull() ✔
      └─ wellKnown() ✔

The {ClassTemplateInvocationContextProvider} extension API is primarily intended for implementing different kinds of tests that rely on repetitive invocation of all test methods in a test class albeit in different contexts — for example, with different parameters, by preparing the test class instance differently, or multiple times without modifying the context. Please refer to the implementations of Parameterized Classes which uses this extension point to provide its functionality.

Providing Invocation Contexts for Test Templates

A {TestTemplate} method can only be executed when at least one {TestTemplateInvocationContextProvider} is registered. Each such provider is responsible for providing a Stream of {TestTemplateInvocationContext} instances. Each context may specify a custom display name and a list of additional extensions that will only be used for the next invocation of the {TestTemplate} method.

The following example shows how to write a test template as well as how to register and implement a {TestTemplateInvocationContextProvider}.

A test template with accompanying extension
link:../../../../src/test/java/example/TestTemplateDemo.java[role=include]

In this example, the test template will be invoked twice. The display names of the invocations will be apple and banana as specified by the invocation context. Each invocation registers a custom {ParameterResolver} which is used to resolve the method parameter. The output when using the ConsoleLauncher is as follows.

└─ testTemplate(String) ✔
   ├─ apple ✔
   └─ banana ✔

The {TestTemplateInvocationContextProvider} extension API is primarily intended for implementing different kinds of tests that rely on repetitive invocation of a test-like method albeit in different contexts — for example, with different parameters, by preparing the test class instance differently, or multiple times without modifying the context. Please refer to the implementations of [writing-tests-repeated-tests] or Parameterized Tests which use this extension point to provide their functionality.

Keeping State in Extensions

Usually, an extension is instantiated only once. So the question becomes relevant: How do you keep the state from one invocation of an extension to the next? The ExtensionContext API provides a Store exactly for this purpose. Extensions may put values into a store for later retrieval. See the TimingExtension for an example of using the Store with a method-level scope. It is important to remember that values stored in an ExtensionContext during test execution will not be available in the surrounding ExtensionContext. Since ExtensionContexts may be nested, the scope of inner contexts may also be limited. Consult the corresponding Javadoc for details on the methods available for storing and retrieving values via the {ExtensionContext_Store}.

Note
AutoCloseable
An extension context store is bound to its extension context lifecycle. When an extension context lifecycle ends it closes its associated store. As of JUnit 5.13, all stored values that are instances of AutoCloseable are notified by an invocation of their close() method in the inverse order they were added in. (unless the junit.jupiter.extensions.store.close.autocloseable.enabled configuration parameter is set to false). Older versions supported CloseableResource, which is still available for backward compatibility but is no longer recommended.

An example implementation of AutoCloseable is shown below, using an HttpServer resource.

HttpServer resource implementing AutoCloseable
link:../../../../src/test/java/example/extensions/HttpServerResource.java[role=include]

This resource can then be stored in the desired ExtensionContext. It may be stored at class or method level, if desired, but this may add unnecessary overhead for this type of resource. For this example it might be prudent to store it at root level and instantiate it lazily to ensure it’s only created once per test run and reused across different test classes and methods.

Lazily storing in root context with Store.getOrComputeIfAbsent
link:../../../../src/test/java/example/extensions/HttpServerExtension.java[role=include]
A test case using the HttpServerExtension
link:../../../../src/test/java/example/HttpServerDemo.java[role=include]

Migration Note for Resource Cleanup

Starting with JUnit Jupiter 5.13, the framework automatically closes resources stored in the ExtensionContext.Store that implement AutoCloseable when auto-close is enabled (which is the default behavior). Prior to 5.13, only resources implementing Store.CloseableResource were automatically closed.

If you’re developing an extension that needs to support both JUnit Jupiter 5.13+ and earlier versions, and your extension stores resources that need to be cleaned up, you should implement both interfaces:

public class MyResource implements Store.CloseableResource, AutoCloseable {
    @Override
    public void close() throws Exception {
        // Resource cleanup code
    }
}

This ensures that your resource will be properly closed regardless of which JUnit Jupiter version is being used.

Supported Utilities in Extensions

The junit-platform-commons artifact provides maintained utilities for working with annotations, classes, reflection, classpath scanning, and conversion tasks. These utilities can be found in the {junit-platform-support-package} and its subpackages. TestEngine and Extension authors are encouraged to use these supported utilities in order to align with the behavior of the JUnit Platform and JUnit Jupiter.

Annotation Support

AnnotationSupport provides static utility methods that operate on annotated elements (e.g., packages, annotations, classes, interfaces, constructors, methods, and fields). These include methods to check whether an element is annotated or meta-annotated with a particular annotation, to search for specific annotations, and to find annotated methods and fields in a class or interface. Some of these methods search on implemented interfaces and within class hierarchies to find annotations. Consult the Javadoc for {AnnotationSupport} for further details.

Note
The isAnnotated() methods do not find repeatable annotations. To check for repeatable annotations, use one of the findRepeatableAnnotations() methods and verify that the returned list is not empty.

Class Support

ClassSupport provides static utility methods for working with classes (i.e., instances of java.lang.Class). Consult the Javadoc for {ClassSupport} for further details.

Reflection Support

ReflectionSupport provides static utility methods that augment the standard JDK reflection and class-loading mechanisms. These include methods to scan the classpath in search of classes matching specified predicates, to load and create new instances of a class, and to find and invoke methods. Some of these methods traverse class hierarchies to locate matching methods. Consult the Javadoc for {ReflectionSupport} for further details.

Modifier Support

ModifierSupport provides static utility methods for working with member and class modifiers — for example, to determine if a member is declared as public, private, abstract, static, etc. Consult the Javadoc for {ModifierSupport} for further details.

Conversion Support

ConversionSupport (in the org.junit.platform.commons.support.conversion package) provides support for converting from strings to primitive types and their corresponding wrapper types, date and time types from the java.time package, and some additional common Java types such as File, BigDecimal, BigInteger, Currency, Locale, URI, URL, UUID, etc. Consult the Javadoc for {ConversionSupport} for further details.

Field and Method Search Semantics

Various methods in AnnotationSupport and ReflectionSupport use search algorithms that traverse type hierarchies to locate matching fields and methods – for example, AnnotationSupport.findAnnotatedFields(…​), ReflectionSupport.findMethods(…​), etc.

As of JUnit 5.11 (JUnit Platform 1.11), field and method search algorithms adhere to standard Java semantics regarding whether a given field or method is visible or overridden according to the rules of the Java language.

Prior to JUnit 5.11, the field and method search algorithms applied what we now refer to as "legacy semantics". Legacy semantics consider fields and methods to be hidden, shadowed, or superseded by fields and methods in super types (superclasses or interfaces) based solely on the field’s name or the method’s signature, disregarding the actual Java language semantics for visibility and the rules that determine if one method overrides another method.

Although the JUnit team recommends the use of the standard search semantics, developers may optionally revert to the legacy semantics via the junit.platform.reflection.search.useLegacySemantics JVM system property.

For example, to enable legacy search semantics for fields and methods, you can start your JVM with the following system property.

-Djunit.platform.reflection.search.useLegacySemantics=true

Note
Due to the low-level nature of the feature, the junit.platform.reflection.search.useLegacySemantics flag can only be set via a JVM system property. It cannot be set via a configuration parameter.

Relative Execution Order of User Code and Extensions

When executing a test class that contains one or more test methods, a number of extension callbacks are called in addition to the user-supplied test and lifecycle methods.

User and Extension Code

The following diagram illustrates the relative order of user-supplied code and extension code. User-supplied test and lifecycle methods are shown in orange, with callback code implemented by extensions shown in blue. The grey box denotes the execution of a single test method and will be repeated for every test method in the test class.

extensions lifecycle
User code and extension code

The following table further explains the sixteen steps in the User code and extension code diagram.

  1. interface org.junit.jupiter.api.extension.BeforeAllCallback
    extension code executed before all tests of the container are executed

  2. annotation org.junit.jupiter.api.BeforeAll
    user code executed before all tests of the container are executed

  3. interface org.junit.jupiter.api.extension.LifecycleMethodExecutionExceptionHandler #handleBeforeAllMethodExecutionException
    extension code for handling exceptions thrown from @BeforeAll methods

  4. interface org.junit.jupiter.api.extension.BeforeClassTemplateInvocationCallback
    extension code executed before each class template invocation is executed (only applicable if the test class is a class template)

  5. interface org.junit.jupiter.api.extension.BeforeEachCallback
    extension code executed before each test is executed

  6. annotation org.junit.jupiter.api.BeforeEach
    user code executed before each test is executed

  7. interface org.junit.jupiter.api.extension.LifecycleMethodExecutionExceptionHandler #handleBeforeEachMethodExecutionException
    extension code for handling exceptions thrown from @BeforeEach methods

  8. interface org.junit.jupiter.api.extension.BeforeTestExecutionCallback
    extension code executed immediately before a test is executed

  9. annotation org.junit.jupiter.api.Test
    user code of the actual test method

  10. interface org.junit.jupiter.api.extension.TestExecutionExceptionHandler
    extension code for handling exceptions thrown during a test

  11. interface org.junit.jupiter.api.extension.AfterTestExecutionCallback
    extension code executed immediately after test execution and its corresponding exception handlers

  12. annotation org.junit.jupiter.api.AfterEach
    user code executed after each test is executed

  13. interface org.junit.jupiter.api.extension.LifecycleMethodExecutionExceptionHandler #handleAfterEachMethodExecutionException
    extension code for handling exceptions thrown from @AfterEach methods

  14. interface org.junit.jupiter.api.extension.AfterEachCallback
    extension code executed after each test is executed

  15. interface org.junit.jupiter.api.extension.AfterClassTemplateInvocationCallback
    extension code executed after each class template invocation is executed (only applicable if the test class is a class template)

  16. annotation org.junit.jupiter.api.AfterAll
    user code executed after all tests of the container are executed

  17. interface org.junit.jupiter.api.extension.LifecycleMethodExecutionExceptionHandler #handleAfterAllMethodExecutionException
    extension code for handling exceptions thrown from @AfterAll methods

  18. interface org.junit.jupiter.api.extension.AfterAllCallback
    extension code executed after all tests of the container are executed

In the simplest case only the actual test method will be executed (step 9); all other steps are optional depending on the presence of user code or extension support for the corresponding lifecycle callback. For further details on the various lifecycle callbacks please consult the respective Javadoc for each annotation and extension.

All invocations of user code methods in the above table can additionally be intercepted by implementing InvocationInterceptor.

Wrapping Behavior of Callbacks

JUnit Jupiter always guarantees wrapping behavior for multiple registered extensions that implement lifecycle callbacks such as BeforeAllCallback, AfterAllCallback, BeforeClassTemplateInvocationCallback, AfterClassTemplateInvocationCallback, BeforeEachCallback, AfterEachCallback, BeforeTestExecutionCallback, and AfterTestExecutionCallback.

That means that, given two extensions Extension1 and Extension2 with Extension1 registered before Extension2, any "before" callbacks implemented by Extension1 are guaranteed to execute before any "before" callbacks implemented by Extension2. Similarly, given the two same two extensions registered in the same order, any "after" callbacks implemented by Extension1 are guaranteed to execute after any "after" callbacks implemented by Extension2. Extension1 is therefore said to wrap Extension2.

JUnit Jupiter also guarantees wrapping behavior within class and interface hierarchies for user-supplied lifecycle methods (see [writing-tests-definitions]).

  • @BeforeAll methods are inherited from superclasses as long as they are not overridden. Furthermore, @BeforeAll methods from superclasses will be executed before @BeforeAll methods in subclasses.

    • Similarly, @BeforeAll methods declared in an interface are inherited as long as they are not overridden, and @BeforeAll methods from an interface will be executed before @BeforeAll methods in the class that implements the interface.

  • @AfterAll methods are inherited from superclasses as long as they are not overridden. Furthermore, @AfterAll methods from superclasses will be executed after @AfterAll methods in subclasses.

    • Similarly, @AfterAll methods declared in an interface are inherited as long as they are not overridden, and @AfterAll methods from an interface will be executed after @AfterAll methods in the class that implements the interface.

  • @BeforeEach methods are inherited from superclasses as long as they are not overridden. Furthermore, @BeforeEach methods from superclasses will be executed before @BeforeEach methods in subclasses.

    • Similarly, @BeforeEach methods declared as interface default methods are inherited as long as they are not overridden, and @BeforeEach default methods will be executed before @BeforeEach methods in the class that implements the interface.

  • @AfterEach methods are inherited from superclasses as long as they are not overridden. Furthermore, @AfterEach methods from superclasses will be executed after @AfterEach methods in subclasses.

    • Similarly, @AfterEach methods declared as interface default methods are inherited as long as they are not overridden, and @AfterEach default methods will be executed after @AfterEach methods in the class that implements the interface.

The following examples demonstrate this behavior. Please note that the examples do not actually do anything realistic. Instead, they mimic common scenarios for testing interactions with the database. All methods imported statically from the Logger class log contextual information in order to help us better understand the execution order of user-supplied callback methods and callback methods in extensions.

Extension1
link:../../../../src/test/java/example/callbacks/Extension1.java[role=include]
Extension2
link:../../../../src/test/java/example/callbacks/Extension2.java[role=include]
AbstractDatabaseTests
link:../../../../src/test/java/example/callbacks/AbstractDatabaseTests.java[role=include]
DatabaseTestsDemo
link:../../../../src/test/java/example/callbacks/DatabaseTestsDemo.java[role=include]

When the DatabaseTestsDemo test class is executed, the following is logged.

@BeforeAll AbstractDatabaseTests.createDatabase()
@BeforeAll DatabaseTestsDemo.beforeAll()
  Extension1.beforeEach()
  Extension2.beforeEach()
    @BeforeEach AbstractDatabaseTests.connectToDatabase()
    @BeforeEach DatabaseTestsDemo.insertTestDataIntoDatabase()
      @Test DatabaseTestsDemo.testDatabaseFunctionality()
    @AfterEach DatabaseTestsDemo.deleteTestDataFromDatabase()
    @AfterEach AbstractDatabaseTests.disconnectFromDatabase()
  Extension2.afterEach()
  Extension1.afterEach()
@BeforeAll DatabaseTestsDemo.afterAll()
@AfterAll AbstractDatabaseTests.destroyDatabase()

The following sequence diagram helps to shed further light on what actually goes on within the JupiterTestEngine when the DatabaseTestsDemo test class is executed.

extensions DatabaseTestsDemo
DatabaseTestsDemo

JUnit Jupiter does not guarantee the execution order of multiple lifecycle methods that are declared within a single test class or test interface. It may at times appear that JUnit Jupiter invokes such methods in alphabetical order. However, that is not precisely true. The ordering is analogous to the ordering for @Test methods within a single test class.

Note

Lifecycle methods that are declared within a single test class or test interface will be ordered using an algorithm that is deterministic but intentionally non-obvious. This ensures that subsequent runs of a test suite execute lifecycle methods in the same order, thereby allowing for repeatable builds.

In addition, JUnit Jupiter does not support wrapping behavior for multiple lifecycle methods declared within a single test class or test interface.

The following example demonstrates this behavior. Specifically, the lifecycle method configuration is broken due to the order in which the locally declared lifecycle methods are executed.

  • Test data is inserted before the database connection has been opened, which results in a failure to connect to the database.

  • The database connection is closed before deleting the test data, which results in a failure to connect to the database.

BrokenLifecycleMethodConfigDemo
link:../../../../src/test/java/example/callbacks/BrokenLifecycleMethodConfigDemo.java[role=include]

When the BrokenLifecycleMethodConfigDemo test class is executed, the following is logged.

Extension1.beforeEach()
Extension2.beforeEach()
  @BeforeEach BrokenLifecycleMethodConfigDemo.insertTestDataIntoDatabase()
  @BeforeEach BrokenLifecycleMethodConfigDemo.connectToDatabase()
    @Test BrokenLifecycleMethodConfigDemo.testDatabaseFunctionality()
  @AfterEach BrokenLifecycleMethodConfigDemo.disconnectFromDatabase()
  @AfterEach BrokenLifecycleMethodConfigDemo.deleteTestDataFromDatabase()
Extension2.afterEach()
Extension1.afterEach()

The following sequence diagram helps to shed further light on what actually goes on within the JupiterTestEngine when the BrokenLifecycleMethodConfigDemo test class is executed.

extensions BrokenLifecycleMethodConfigDemo
BrokenLifecycleMethodConfigDemo
Tip

Due to the aforementioned behavior, the JUnit Team recommends that developers declare at most one of each type of lifecycle method (see [writing-tests-definitions]) per test class or test interface unless there are no dependencies between such lifecycle methods.