JUnit 5 tests don't apply a timeout

[timja]

Jenkins and plugins versions report

https://github.com/jenkinsci/pipeline-graph-view-plugin/blob/8099553260272937bec41f266799e5bf8551d7d7/src/test/java/io/jenkins/plugins/pipelinegraphview/consoleview/PipelineConsoleViewInputTest.java#L55

What Operating System are you using (both controller, and any agents involved in the problem)?

ci.jenkins.io

Reproduction steps

run a test and set -Djenkins.test.timeout=1, expect it to fail like it does with any JUnit 4 test, it succeeds

Expected Results

A timeout based on jenkins.test.timeout

Actual Results

No timeout as far as I can tell, the ci.jenkins.io test ran for approx 50 mins before being killed

Anything else?

No response

Are you interested in contributing a fix?

No response

[timja]

cc @strangelookingnerd

[strangelookingnerd]

The ideal and easiest solution would be to configure junit.jupiter.execution.timeout.default (https://junit.org/junit5/docs/current/user-guide/#writing-tests-declarative-timeouts) and let JUnit handle the timeouts. This however would not only affect the JenkinsRule tests, but all tests (non-issue I'd say).

This approach would make JenkinsRule#timeout useless, which is unfortunate and could cause some confusion when migrating tests. Yet I do not think this would have a huge impact, as there are only few instances where this is overriden anyways.

From what I can tell it is not possible to programmatically set JUnit5 configuration parameters during runtime in an Extension, as they are resolved before that. Implementing our own TimeoutExtension as counter-part to JenkinsRule#apply(org.junit.runners.model.Statement, org.junit.runner.Description would be an option. Yet I do not like adding more code for something that is already kind of solved by the underlying framework.

I may need some more time to investigate it further.