Is it possible to set a different Test key for each dynamic test iteration?

[paulmwhite]

Given a dynamic test like the below I can't see how it's possible to use an annotation and have a unique key for each iteration:

class UnitTest {

    @Nested
    inner class Something {
        @TestFactory
        fun `something() successfully`(): List<DynamicTest> {
            fun test(
                something: String,
            ): DynamicTest {
                return DynamicTest.dynamicTest(
                    "something: $something",
                ) {
                    //    Test implementation
                }
            }

            return listOf(
                test(
                    "one"
                ),
                test(
                    "two"
                ),
            )
        }
    }
}

Is there any workaround or solution, perhaps a way to set the key in the test body?

Thanks

[bitcoder]

Can you have a look at #54 and the PR that came out of it; I think it might be possible (?).
Right now I dont have the mental bandwith to look at this in more detail, but if it's possible it would be through that implementation as addressed in #55

[mk868]

It's tricky, but it works:

package xyz;

import static org.junit.jupiter.api.DynamicTest.dynamicTest;

import java.util.List;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

class DynTest {

  @TestFactory
  List<DynamicTest> something() {
    return List.of(
        dynamicTest("token$1", () -> {
          // ...
        }),
        dynamicTest("token$2", () -> {
          // ...
        }),
        dynamicTest("token$3", () -> {
          // ...
        })
    );
  }

}

Custom XrayTestMetadataReader:

package xyz;

import app.getxray.xray.junit.customjunitxml.DefaultXrayTestMetadataReader;
import java.util.Optional;
import org.junit.platform.launcher.TestIdentifier;

public class CustomTestMetadataReader extends DefaultXrayTestMetadataReader {

  @Override
  public Optional<String> getKey(TestIdentifier testIdentifier) {
    var token = testIdentifier.getDisplayName();
    if (Consts.TOKEN_TO_KEY.containsKey(token)) {
      return Optional.of(Consts.TOKEN_TO_KEY.get(token));
    }
    return super.getKey(testIdentifier);
  }

  @Override
  public Optional<String> getSummary(TestIdentifier testIdentifier) {
    var token = testIdentifier.getDisplayName();
    if (Consts.TOKEN_TO_SUMMARY.containsKey(token)) {
      return Optional.of(Consts.TOKEN_TO_SUMMARY.get(token));
    }
    return super.getSummary(testIdentifier);
  }

  @Override
  public Optional<String> getDescription(TestIdentifier testIdentifier) {
    var token = testIdentifier.getDisplayName();
    if (Consts.TOKEN_TO_DESCRIPTION.containsKey(token)) {
      return Optional.of(Consts.TOKEN_TO_DESCRIPTION.get(token));
    }
    return super.getDescription(testIdentifier);
  }
}

Key/summary/description definitions for dynamic tests:

package xyz;

import java.util.Map;

public class Consts {

  public static final Map<String, String> TOKEN_TO_KEY = Map.ofEntries(
      Map.entry("token$1", "ABC-100"),
      Map.entry("token$2", "ABC-200"),
      Map.entry("token$3", "ABC-300")
  );

  public static final Map<String, String> TOKEN_TO_SUMMARY = Map.ofEntries(
      Map.entry("token$1", "Summary 1"),
      Map.entry("token$2", "Summary 2"),
      Map.entry("token$3", "Summary 3")
  );

  public static final Map<String, String> TOKEN_TO_DESCRIPTION = Map.ofEntries(
      Map.entry("token$1", "description 1"),
      Map.entry("token$2", "description 2"),
      Map.entry("token$3", "description 3")
  );
}

xray-junit-extensions.properties:

test_metadata_reader=xyz.CustomTestMetadataReader

[mk868]

If you really need to set the key/summary/description from the test method, you can do something like this:

package xyz;

import java.util.HashMap;
import java.util.Map;

public class Consts {

  public static final Map<String, String> TOKEN_TO_KEY = new HashMap<>();
  public static final Map<String, String> TOKEN_TO_SUMMARY = new HashMap<>();
  public static final Map<String, String> TOKEN_TO_DESCRIPTION = new HashMap<>();
}

package xyz;

import static org.junit.jupiter.api.DynamicTest.dynamicTest;

import java.util.List;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

class DynTest {

  @TestFactory
  List<DynamicTest> something() {
    return List.of(
        dynamicTest("token$1", () -> {
          Consts.TOKEN_TO_KEY.put("token$1", "ABC-100");
          Consts.TOKEN_TO_SUMMARY.put("token$1", "Summary 1");
          Consts.TOKEN_TO_DESCRIPTION.put("token$1", "description 1");

          // ...
        }),
        dynamicTest("token$2", () -> {
          Consts.TOKEN_TO_KEY.put("token$2", "ABC-200");
          Consts.TOKEN_TO_SUMMARY.put("token$2", "Summary 2");
          Consts.TOKEN_TO_DESCRIPTION.put("token$2", "description 2");

          // ...
        }),
        dynamicTest("token$3", () -> {
          Consts.TOKEN_TO_KEY.put("token$3", "ABC-300");
          Consts.TOKEN_TO_SUMMARY.put("token$3", "Summary 3");
          Consts.TOKEN_TO_DESCRIPTION.put("token$3", "description 3");

          // ...
        })
    );
  }

}

[mk868]

Oh wait...
I started to reconsider this case and it looks like we have a simpler solution:

CustomTestMetadataReader.java

package xyz;

import app.getxray.xray.junit.customjunitxml.DefaultXrayTestMetadataReader;
import java.net.URI;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Optional;
import org.junit.platform.engine.support.descriptor.UriSource;
import org.junit.platform.launcher.TestIdentifier;

public class CustomTestMetadataReader extends DefaultXrayTestMetadataReader {

  @Override
  public Optional<String> getKey(TestIdentifier testIdentifier) {
    var keyFromUriOpt = readQueryValue(testIdentifier, "key");
    if (keyFromUriOpt.isPresent()) {
      return keyFromUriOpt;
    }
    return super.getKey(testIdentifier);
  }

  @Override
  public Optional<String> getSummary(TestIdentifier testIdentifier) {
    var summaryFromUriOpt = readQueryValue(testIdentifier, "summary");
    if (summaryFromUriOpt.isPresent()) {
      return summaryFromUriOpt;
    }
    return super.getSummary(testIdentifier);
  }

  @Override
  public Optional<String> getDescription(TestIdentifier testIdentifier) {
    var descriptionFromUriOpt = readQueryValue(testIdentifier, "description");
    if (descriptionFromUriOpt.isPresent()) {
      return descriptionFromUriOpt;
    }
    return super.getDescription(testIdentifier);
  }

  private static Optional<String> readQueryValue(TestIdentifier testIdentifier, String key) {
    return testIdentifier.getSource()
        .filter(source -> source instanceof UriSource)
        .map(source -> (UriSource) source)
        .map(UriSource::getUri)
        .map(URI::getQuery)
        .stream()
        .map(query -> query.split("&"))
        .flatMap(Arrays::stream)
        .filter(query -> query.startsWith(key + "="))
        .map(query -> query.substring(key.length() + 1))
        .map(value -> URLDecoder.decode(value, StandardCharsets.UTF_8))
        .findFirst();
  }
}

DynTest.java

package xyz;

import static org.junit.jupiter.api.DynamicTest.dynamicTest;

import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.List;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

class DynTest {

  @TestFactory
  List<DynamicTest> something() {
    return List.of(
        dynamicTest("Test 1",
            testToURI(DynTest.class, "something/test1", "ABC-100", "Summary 1", "Description 1"), () -> {
              // ...
            }),
        dynamicTest("Test 2",
            testToURI(DynTest.class, "something/test2", "ABC-200", "Summary 2", "Description 2"), () -> {
              // ...
            }),
        dynamicTest("Test 3",
            testToURI(DynTest.class, "something/test3", "ABC-300", "Summary 3", "Description 3"), () -> {
              // ...
            })
    );
  }

  public static URI testToURI(Class<?> testClass, String testMethod, String key, String summary,
      String description) {
    return URI.create("test://" + testClass.getName() + "/" + testMethod
        + "?key=" + URLEncoder.encode(key, StandardCharsets.UTF_8)
        + "&summary=" + URLEncoder.encode(summary, StandardCharsets.UTF_8)
        + "&description=" + URLEncoder.encode(description, StandardCharsets.UTF_8));
  }
}

Output report:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="JUnit Jupiter" tests="3" skipped="0" failures="0" errors="0" time="0" hostname="XXXXXX" timestamp="XXXXXX">
<properties>
  <!-- ... -->
</properties>
<testcase name="something" classname="xyz.DynTest" time="0" started-at="XXXXXX" finished-at="XXXXXX">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:xyz.DynTest]/[test-factory:something()]/[dynamic-test:#2]
display-name: Test 2
]]></system-out>
<properties>
<property name="test_key" value="ABC-200"/>
<property name="test_description"><![CDATA[Description 2]]></property>
<property name="test_summary" value="Summary 2"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="something" classname="xyz.DynTest" time="0" started-at="XXXXXX" finished-at="XXXXXX">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:xyz.DynTest]/[test-factory:something()]/[dynamic-test:#3]
display-name: Test 3
]]></system-out>
<properties>
<property name="test_key" value="ABC-300"/>
<property name="test_description"><![CDATA[Description 3]]></property>
<property name="test_summary" value="Summary 3"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="something" classname="xyz.DynTest" time="0" started-at="XXXXXX" finished-at="XXXXXX">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:xyz.DynTest]/[test-factory:something()]/[dynamic-test:#1]
display-name: Test 1
]]></system-out>
<properties>
<property name="test_key" value="ABC-100"/>
<property name="test_description"><![CDATA[Description 1]]></property>
<property name="test_summary" value="Summary 1"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]
display-name: JUnit Jupiter
]]></system-out>
</testsuite>

In this example test metadata like key/summary/description is passed in the TestSource (see UriSource).
The test source URI is in the form: test://{{class}}/{{method}}/{{testId}}?key={{key}}&summary={{summary}}&description={{description}}, for example test://xyz.DynTest/something/test2?key=ABC-200&summary=Summary+2&description=Description+2.
It requires more tests and verification if working as expected, but looks promising.

[mk868]

BTW.
If a solution with a test source URI works for all cases and doesn't conflict with JUnit practices, then I think we can enrich xray-junit-extensions project with the definition of the URI schema for the dynamic tests.
Then we can enhance the DefaultXrayTestMetadataReader to read information from the URI according to a predefined schema + add builder/factory methods for users to generate URIs for their code

[bitcoder]

Thanks for the input @mk868 . I don't have a strong opinion on this but I think the last approach you suggest could be feasible, as long as we keep it simple for anyone that doesn't wish all this level of customization. Do you want to prepare a PR?