The JUnit Platform does not receive complete support from IDEs and build tools

[mpkorstanje]

Currently the user-guide states:

First-class support for the JUnit Platform also exists in popular IDEs (see IntelliJ IDEA, Eclipse, NetBeans, and Visual Studio Code) and build tools (see Gradle, Maven, and Ant).

However of these I know that:

• IDEA

  • https://youtrack.jetbrains.com/issue/IDEA-276463
  • https://youtrack.jetbrains.com/issue/IDEA-276477
  • https://youtrack.jetbrains.com/issue/IDEA-227508
  • https://youtrack.jetbrains.com/issue/IDEA-292853
  • https://youtrack.jetbrains.com/issue/IDEA-292662
  • https://youtrack.jetbrains.com/issue/IDEA-153780

• Eclipse

  • No issue exists.

• Maven Surefire

  • https://issues.apache.org/jira/browse/SUREFIRE-1724

• Gradle

  • JUnit 5: Allow resource-based testing gradle/gradle#4773

do not support the use of all discovery selector primitives (either directly or indirectly) and don't fully use the hierarchal test concept (often limited to the extent needed to support JUnit 4). As such I don't believe we can claim these IDEs and build tools have first-class support for the JUnit Platform. At best I believe we may claim some support for JUnit Jupiter.

[sormuras]

You're right. Most support in 3rd-party tools is Jupiter-centric.

I know at least of one additional Maven plugin that tries to treat each test engine alike. ;-)

• Maven
  • https://github.com/sormuras/junit-platform-maven-plugin supports most selectors of the JUnit Platform

[mpkorstanje]

Some additional context on why I believe support for all discovery selectors is important.

A typical use case is to run a single test. For example:

mvn test -Dtest=MyExampleTest

Which is effectively the same as using a DiscoverySelectors.selectClass(MyExampleTest.class). However for the IDEs and tools listed above this is also the only supported selector.

Ideally this would be:

mvn test -Dselect-class=MyExampleTest
mvn test -Dselect-file=path/to/example.file

Currently users of Cucumber who use the junit platform are running into this problem. And as such I'm considering adding functionality in cucumber/cucumber-jvm#2498 that makes it possible to do:

mvn test -Dcucumber.features=path/to/example.feature

but this pretty much nullifies the advantage of having the JUnit Platform create the discovery request.

[mpkorstanje]

You're right. Most support in 3rd-party tools is Jupiter-centric.

It's almost as if everyone ignored the "What is JUnit 5?" section of the user guide and reused the support code from JUnit 4. 🤣

I know at least of one additional Maven plugin that tries to treat each test engine alike. ;-)

Neat!