Run only a test of a class with Parameterized runner

[diogoeag]

When using the Parameterized runner there is no possibility to run only a test method of a test class.

For example when running in maven: mvn test -Dtest=TesClass#testName

No tests are run when using the Parameterized runner.

That would be great to have this feature, has for development is comon to the developers trying to run only the test they are working on.

[migueljbento]

Fixing this would be great, we end up having to comment UTs in large test classes which is undesirable.

[kcooney]

The Maven Surefire plugin allows to to specify globs (see http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html). Have you tried that?

[diogoeag]

Yes, that's how we would like to use, but it doesn't work. Are you saygin that this is a problem of the surefire plugin and not from junit?

[kcooney]

@diogoeag I believe it's a problem with the Surefire plugin. You can verify that by writing a class with a main method that uses JUnitCore to run the tests of a class that used @Parameterized and have your main method apply a filter to select one of the tests

[diogoeag]

Hi Kevin,

Take a look at these two samples:

First test without parameterized:

package org;

import org.junit.Test;
import org.junit.internal.requests.FilterRequest;
import org.junit.runner.Description;
import org.junit.runner.JUnitCore;
import org.junit.runner.manipulation.Filter;

public class NoParameterTest {

    public static void main(String[] args) {
        new JUnitCore().run(FilterRequest.aClass(NoParameterTest.class).filterWith(new Filter() {
            @Override
            public boolean shouldRun(Description description) {
                System.out.println("Should run test: classname[" + description.getClassName() + "] method name[" + description.getMethodName() + "]");
                return description != null && description.getMethodName() != null && description.getMethodName().equals("test");
            }

            @Override
            public String describe() {
                return null;
            }
        }).getRunner());

    }

    @Test
    public void test() {
        System.out.println("Running Test: test");
    }

    @Test
    public void test2() {
        System.out.println("Running Test: test2");
    }
}

output

Should run test: classname[org.NoParameterTest] method name[test]
Should run test: classname[org.NoParameterTest] method name[test2]
Running Test: test

Second test with parameterized:

package org;

import org.junit.Test;
import org.junit.internal.requests.FilterRequest;
import org.junit.runner.Description;
import org.junit.runner.JUnitCore;
import org.junit.runner.RunWith;
import org.junit.runner.manipulation.Filter;
import org.junit.runners.Parameterized;

import java.util.Arrays;
import java.util.Collection;

@RunWith(Parameterized.class)
public class ParameterTest {

    public static void main(String[] args) {
        new JUnitCore().run(FilterRequest.aClass(ParameterTest.class).filterWith(new Filter() {
            @Override
            public boolean shouldRun(Description description) {
                System.out.println("Should run test: classname[" + description.getClassName() + "] method name[" + description.getMethodName() + "]");
                return description != null && description.getMethodName() != null && description.getMethodName().equals("test");
            }

            @Override
            public String describe() {
                return null;
            }
        }).getRunner());

    }


    @Parameterized.Parameters
    public static Collection<Object[]> data() {
        return Arrays.asList(new Object[]{"Paramter 1"}, new Object[]{"Parameter 2"});
    }

    private String localParameter;

    public ParameterTest(String parameter) {

        this.localParameter = parameter;
    }

    @Test
    public void test() {
        System.out.println("Running Test: test");
        System.out.println("Parameter: " + localParameter);
    }

    @Test
    public void test2() {
        System.out.println("Running Test: test2");
        System.out.println("Parameter: " + localParameter);
    }
}

output

Should run test: classname[[0]] method name[null]
Should run test: classname[[1]] method name[null]

The Description passed to the Filter by the Parameterized is not correct right?

[kcooney]

@diogoeag Thanks for the code example. What version of JUnit did you try this on? I think we fixed a related issue in 4.12 but I am not sure.

[diogoeag]

@kcooney, I've tested with the current master 4.12-SNAPSHOT and also with 4.11. Both have the same problem.

[kcooney]

Thanks. Fixing this would like require changes that are not backwards compatible. Setting milestone to 5.0

[diogoeag]

Just to clarify, fixing this would only need to go to BlockJUnit4ClassRunnerWithParameters and remove the override of the #testName method. That is replacing the default

    protected String testName(FrameworkMethod method) {
        return method.getName();
    }

by

    @Override
    protected String testName(FrameworkMethod method) {
        return method.getName() + getName();
    }

And the consequencies would be that the reporting of the tests would not have the parameter name in the testHeader (at least the tests that have failed were only these)?

[kcooney]

@diogoeag but wouldn't we want the parameter name in the description?

I think the fixes would best be made on top of the Description builder we are working on for JUnit 5.0.

[diogoeag]

@kcooney Yes to be correct we should have the parameter name in the description, but the description should not affect the test name. I agree with you that the Description should have a test name that should be the correct name and the display name that should be used in user facing descriptions and include the parameter.

Do you have any estimates on the target date to 5.0 ?

I was looking to the usages of getDisplayName to submit a pull request to 5.0 with the fix, however it is used 23 times in the project (sources and tests) and I don't have the enough knowledge of JUnit source to make such change. Meanwhile I will make a fork for my personal usage with that intermediate fix for me. Having the parameter names is not critical, however allowing us to run single tests saves us many hours in many developers running tests in our source.

Thanks

[diogoeag]

If anyone needs to have access to this temporary fix: feedzai@be41d29

[Tibor17]

@diogoeag
See this. Would it help in surefire 2.18 ?
apache/maven-surefire#46