Parameterized tests: misleading error if exception was thrown in static initializer

[sebastianhaberey]

When a parameterized test is executed, but an exception is thrown in the test class' static initializer, the error message is:

"Configuration error: You must configure at least one set of arguments for this @ParameterizedTest"

This distracts from the actual source of the problem. I know static initializers are generally frowned upon, but they are recommended for some edge cases, e.g. by the Testcontainers project.

Steps to reproduce

class FooTest {

    static {
        if (true) throw new RuntimeException("some exception");
    }

    private static Stream<Arguments> getArguments() {
        return Stream.of("foo", "bar").map(Arguments::of);
    }

    @ParameterizedTest
    @MethodSource("getArguments")
    void shouldProcessSmartFile(String testString) {
        assertThat(testString).hasSize(3);
    }

}

Context

• Used versions (Jupiter/Vintage/Platform): junit-jupiter-params: 5.8.1
• Build Tool/IDE: IntelliJ IDEA 2022.2.1

Full error message

click here

java.lang.ExceptionInInitializerError
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
  at org.junit.jupiter.params.provider.MethodArgumentsProvider.lambda$provideArguments$1(MethodArgumentsProvider.java:46)
  at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
  at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
  at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
  at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
  at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
  at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
  at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
  at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
  at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
  at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
  at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
  at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
  at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
  at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
  at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
  at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
  at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
  at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
  at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
  at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
  at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
  at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
  at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:107)
  at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:42)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
  at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
  at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
  at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
  at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
  at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
  at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
  at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
  at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
  at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
  at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
  at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
  at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
  at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
  at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
  at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
  at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
  at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
  at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
  at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
  at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
  at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
  at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
  at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
  at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
  at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  Suppressed: org.junit.platform.commons.PreconditionViolationException: Configuration error: You must configure at least one set of arguments for this @ParameterizedTest
  	at org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:281)
  	at org.junit.jupiter.params.ParameterizedTestExtension.lambda$provideTestTemplateInvocationContexts$5(ParameterizedTestExtension.java:98)
  	at java.base/java.util.stream.AbstractPipeline.close(AbstractPipeline.java:323)
  	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
  	... 69 more
Caused by: java.lang.RuntimeException: some exception
  at de.xxx.mds.e2etest.tests.FooTest.<clinit>(FooTest.java:14)
  ... 96 more

[sbrannen]

When a parameterized test is executed, but an exception is thrown in the test class' static initializer, the error message is:

"Configuration error: You must configure at least one set of arguments for this @ParameterizedTest"

That's the "suppressed" exception.

The actual exception that is thrown is the java.lang.ExceptionInInitializerError with the following cause:

Caused by: java.lang.RuntimeException: some exception

In Eclipse IDE, one sees the ExceptionInInitializerError by default and has to scroll to find the suppressed exception.

Does IntelliJ display the suppressed exception instead of the top-level exception?

[sebastianhaberey]

I am not familiar with the concept of "suppressed" exceptions. Like all Java developers I am used to scrolling down lengthy stacktraces hoping for a clue about what's going on. In my experience it's the plain text phrases that provide the most help. Such as "Configuration error: You must configure at least one set of arguments for this @ParameterizedTest".

Only in this case, that's not what's going on at all. My suggestion would be to simply not have that in the stack trace.