Expose capability to configure TestNG via a service loading implementation

[krmahadevan]

As and when TestNG starts adding new capabilities, the ability to configure it also starts growing. So build tools also need to absorb these changes into it so that end users can configure/tweak TestNG behavior to suite their needs.

As an example here's one such issue that talks about adding these newer capabilities into Gradle gradle/gradle#18828

What if TestNG exposes the ability to configure itself via an implementation that can be loaded up via a service loader. It could look something like this

import org.testng.CommandLineArgs;

public interface IConfigureTestNG {
    CommandLineArgs tweakArgs();
}

TestNG could then look up this implementation via a service loader and then merge all tweaks (since this is a service loader, we may end up finding one or more implementations and so may need to merge) and then apply the tweaks to TestNG to configure it.

This way end users would no longer have to depend on build tools to expose these capabilities and they could tweak the behavior of TestNG in yet another way.

[krmahadevan]

@juherr - What do you think?

[krmahadevan]

Yes that will help with changing configurations at suite level. I think testng has configs that are not relevant to suite files.

For example setting the suite thread pool size is beyond a suite configuration. Same goes for parallel suite execution.

[juherr]

Got it.

Here's what I suggest:

• Define an appropriate name for the configuration model.
• Design an interceptor interface to allow modifications to the configuration.
• Integrate the configuration model into the suite model.
• Deprecate global configurations from the suite and replace them with the new configuration model

Does this roadmap align with your vision? Let me know your thoughts.

[krmahadevan]

@juherr - I didnt understand the suggestion. Can you please help explain a bit more ?

Currently we have the below configurations (I have summarised them into a table)

Config name	Only available via config parameters	Can be altered via suite/listener
-log	No	Yes
-verbose	No	Yes
-groups	No	Yes
-excludegroups	No	Yes
-d	No	Yes
-mixed	No	Yes
-listener	No	Yes
-listenercomparator	No	Yes
-methodselectors	No	Yes
-objectfactory	No	Yes
-parallel	No	Yes
-configfailurepolicy	No	Yes
-threadcount	No	Yes
-dataproviderthreadcount	No	Yes
-suitename	No	Yes
-testname	No	Yes
-reporter	Yes	No
-usedefaultlisteners	Yes	No
-skipfailedinvocationcounts	Yes	No
-testclass	No	Yes
-testnames	No	Yes
-ignoreMissedTestNames	Yes	No
-testjar	Yes	No
-xmlpathinjar	Yes	No
-testrunfactory	Yes	No
-listenerfactory	Yes	No
-methods	No	Yes
-suitethreadpoolsize	Yes	No
-randomizesuites	Yes	No
-alwaysrunlisteners	Yes	No
-threadpoolfactoryclass	Yes	No
-dependencyinjectorfactory	Yes	No
-failwheneverythingskipped	Yes	No
-spilistenerstoskip	Yes	No
-overrideincludedmethods	No	Yes
-includeAllDataDrivenTestsWhenSkipping	Yes	No
-propagateDataProviderFailureAsTestFailure	Yes	No
-generateResultsPerSuite	Yes	No
-shareThreadPoolForDataProviders	No	Yes
-useGlobalThreadPool	No	Yes

The intent is to be able to set the config parameters that have a Yes in the column Only available via config parameters via this new configuration setting mechanism (TestNG currently lets me set them only via command parameters).

[juherr]

@krmahadevan Thanks for the configuration summary! Could you also add the target level of the configuration? method/suite/run/...

[krmahadevan]

@juherr - Eventually all configurations are meant to be affecting methods only. So it would be hard for me to add the target level. Can you let me know what do you have in mind :) Maybe I am not understanding the bigger picture here.