JUnit Jupiter Extension fails on constructor injection with multiple test methods

[mkobit]

Jenkins and plugins versions report

Environment

mvn --version
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: /usr/local/Cellar/maven/3.9.6/libexec
Java version: 17.0.9, vendor: Homebrew, runtime: /usr/local/Cellar/openjdk@17/17.0.9/libexec/openjdk.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "14.2.1", arch: "x86_64", family: "mac"

What Operating System are you using (both controller, and any agents involved in the problem)?

N/A

Reproduction steps

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.hasItem;
import static org.hamcrest.collection.IsCollectionWithSize.hasSize;
import static org.hamcrest.collection.IsEmptyCollection.empty;

import java.io.IOException;
import java.util.Objects;

import org.junit.jupiter.api.Test;
import org.jvnet.hudson.test.JenkinsRule;

@WithJenkins
class JenkinsRuleConstructorResolverTest {

    private final JenkinsRule rule;

    JenkinsRuleConstructorResolverTest(JenkinsRule rule) {
        this.rule = Objects.requireNonNull(rule);
    }

    @Test
    void firstTest() throws IOException {
        assertThat(rule.jenkins.getJobNames(), empty());
        rule.createFreeStyleProject("job-0");
        assertThat(rule.jenkins.getJobNames(), hasSize(1));
        assertThat(rule.jenkins.getJobNames(), hasItem("job-0"));
    }

    @Test
    void secondTest() throws IOException {
        assertThat(rule.jenkins.getJobNames(), empty());
        rule.createFreeStyleProject("job-1");
        assertThat(rule.jenkins.getJobNames(), hasSize(1));
        assertThat(rule.jenkins.getJobNames(), hasItem("job-1"));
    }
}

Expected Results

Tests should execute successfully, preusumably with a "reset Jenkins" instance since the default operating mode for Jenkins test instance lifecycle is per-method (see docs and @TestInstance)

Actual Results

[ERROR] org.jvnet.hudson.test.junit.jupiter.JenkinsRuleConstructorResolverTest.secondTest -- Time elapsed: 0.002 s <<< ERROR!
org.junit.jupiter.api.extension.ParameterResolutionException: Failed to bind to localhost/127.0.0.1:58115
	at org.jvnet.hudson.test.junit.jupiter.JenkinsExtension.resolveParameter(JenkinsExtension.java:107)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.io.IOException: Failed to bind to localhost/127.0.0.1:58115
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
	at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:304)
	at org.eclipse.jetty.server.Server.lambda$doStart$0(Server.java:402)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.eclipse.jetty.server.Server.doStart(Server.java:398)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
	at org.jvnet.hudson.test.JenkinsRule._createWebServer(JenkinsRule.java:885)
	at org.jvnet.hudson.test.JenkinsRule.createWebServer(JenkinsRule.java:803)
	at org.jvnet.hudson.test.JenkinsRule.createWebServer(JenkinsRule.java:791)
	at org.jvnet.hudson.test.JenkinsRule.newHudson(JenkinsRule.java:741)
	at org.jvnet.hudson.test.JenkinsRule.before(JenkinsRule.java:409)
	at org.jvnet.hudson.test.junit.jupiter.JenkinsExtension.resolveParameter(JenkinsExtension.java:104)
	... 3 more
Caused by: java.net.BindException: Address already in use
	at java.base/sun.nio.ch.Net.bind0(Native Method)
	at java.base/sun.nio.ch.Net.bind(Net.java:555)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.netBind(ServerSocketChannelImpl.java:337)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:294)
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:339)
	... 23 more

Test fails to execute due to a binding exception

Anything else?

No response

Are you interested in contributing a fix?

Yes, would like to improve the JUnit 5 support in a few other ways with other issues I've found (like with @JenkinsRecipe as well)

[strangelookingnerd]

This could be fixed by adding

    @Override
    public ExtensionContextScope getTestInstantiationExtensionContextScope(@NonNull ExtensionContext rootContext) {
        return ExtensionContextScope.TEST_METHOD;
    }

to the JenkinsExtension. Otherwise the #afterEach will not be called after a test, leaving the JenkinsRule instantiated, failing following tests.

This apparently will become default behavior in a future version anyways (https://docs.junit.org/6.0.1/api/org.junit.jupiter.api/org/junit/jupiter/api/extension/TestInstantiationAwareExtension.ExtensionContextScope.html#DEFAULT) so we might want to try it out. I can't really tell if this has any other implications..