Test framework support for loading plugin instances #6138
Conversation
Also, added integration tests for drop_events and parse_json processors. Signed-off-by: David Venable <[email protected]>
…or loading plugin instances. Not all features are supported, so some tests are still using the classic approach. Signed-off-by: David Venable <[email protected]>
dlvenable
requested review from
KarstenSchnitter,
chenqi0805,
dinujoh,
engechas,
graytaylor0,
kkondaka,
oeyh,
san81,
sb2k16 and
srikanthjg
as code owners
Signed-off-by: David Venable <[email protected]>
Signed-off-by: David Venable <[email protected]>
| final Map<String, Object> messageMap = Map.of( | ||
| "stringKey", UUID.randomUUID().toString(), | ||
| "integerKey", random.nextInt(10_000) + 10, | ||
| "objectKey", Map.of( | ||
| "nestedKey", UUID.randomUUID().toString() | ||
| ), | ||
| "arrayKey", List.of( | ||
| UUID.randomUUID().toString(), | ||
| UUID.randomUUID().toString() | ||
| ) | ||
| ); |
There was a problem hiding this comment.
Like we were discussing, we can avoid these lines repeating between the test methods, if we can generalize this part as well.
There was a problem hiding this comment.
Yes, I added #5907 to track this idea. I think it would be a nice addition.
|
|
||
| final PluginSetting pluginSetting = generatePluginSetting(testClass, configurationFile); | ||
|
|
||
| final PluginFactory pluginFactory = getOrCreatePluginFactory(extensionContext); | ||
|
|
||
| final Class<?> pluginType = annotation.pluginType(); | ||
|
|
||
| return pluginFactory.loadPlugin(pluginType, pluginSetting); | ||
| } | ||
|
|
||
| private PluginSetting generatePluginSetting(final Class<?> testClass, final String configurationFile) { |
There was a problem hiding this comment.
Hope we can't avoid this method returning PluginSetting as some of the processors still depend on it! Or we shouldn't add this and convert those processors before we support test framework on them?
There was a problem hiding this comment.
The plugin framework still takes PluginSetting. So until that is removed we need it here. But, this is kept private so that it doesn't go into the plugins themselves.
|
|
||
| final Class<?> pluginType = annotation.pluginType(); | ||
|
|
||
| return pluginFactory.loadPlugin(pluginType, pluginSetting); |
There was a problem hiding this comment.
May be not required for this PR but we should test this with one of the plugin that has its PluginConfig injected. We are not explicitly registering PluginConfig object as a bean here but I think it is not required since we are scanning the entire DefaultPluginFactory.class package where are already initializing PluginConfig bean.
|
|
||
| private static PipelinesDataFlowModel readPipelinesDataFlowModel(final Class<?> testClass, final String configurationFile) { | ||
| final PipelinesDataFlowModel pipelinesDataFlowModel; | ||
| try(final InputStream resourceStream = testClass.getResourceAsStream(configurationFile)) { |
There was a problem hiding this comment.
We are expecting these files to be in the same package as the Plugin class. It may not be a problem but if the number of scenarios goes high, there may be a need to organize these into folders.
There was a problem hiding this comment.
Right now, it loads relative to the test class actually. So there is some additional flexibility. It should also support sub-directories, though I've not tested that.
3 participants
Description
This adds a new feature to the Data Prepper plugin test framework. It provides the ability for plugin authors to write tests that will load plugin instances from a YAML file. The core feature is the addition of the
@PluginConfigurationFileannotation that can be applied to method parameters. It uses JUnit's support for parameter injection to load the actual plugin and provide it to the plugin author's unit test.The biggest new code change is the addition of
PluginInstanceParameterResolverwhich perform the actual loading.This also provides support for plugin tests to get an injected
EventFactoryorEventKeyFactoryto help with authoring tests.This PR also adds plugin test framework support to the
drop_eventsandparse_jsonplugins.Additionally, I updated the
GrokProcessorITto make as many tests as possible use this new approach. These tests are now using the framework to load plugin configurations rather than mutatingPluginSettingdirectly. However, there were some tests that I couldn't yet update. These were:I do plan to look into this in a future PR.
Issues Resolved
Resolves #5917
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.