Fixed Order Dependent Flakiness in testValidFlowNamespaceWithAllValidCharsTypes

[LucasEby]

Fixes #1463

Motivation

SlangBuilderTest.testValidFlowNamespaceWithAllValidCharsTypes fails when executed on it's own because it is an Order-Dependent test that requires SlangBuilderTest.testValidFlowNamespaceCaseInsensitive to mock and set the behavior of metadataExtractor.extractMetadataModellingResult() to return EMPTY_METADATA beforehand. As a result, the test can fail due to external factors not setting up it's state correctly, despite the test and the code that it is testing remaining unchanged. Order dependent flaky tests can lead to unreliable results from CI and can erode developer trust in the test suite.

Modifications

A few lines were introduced at the beginning of SlangBuilderTest.testValidFlowNamespaceWithAllValidCharsTypes to mock and set the required behavior of metadataExtractor.extractMetadataModellingResult() to return EMPTY_METADATA. Previously, the test would only pass if another test had configured this mock beforehand. The mock configuration persists between tests because the tests use Autowired Spring beans which are shared across tests, and the Before method does not reset the metadataExtractor mock.

As a result, testValidFlowNamespaceWithAllValidCharsTypes is now self-contained and no longer depends on execution order, as it properly initializes its own required mock state.

Affected Tests

This change affects the existing tests:

• io.cloudslang.lang.tools.build.SlangBuilderTest#testValidFlowNamespaceWithAllValidCharsTypes

Tools Used

iDFlakies was utilized to identify the order dependent polluter(s), victim(s), and cleaner(s) in the test suite(s). The tool functions by repeatedly running a project's JUnit tests under many different deterministic test orders to detect flaky tests which pass and fail inconsistently, and then re-executes the failing orders to classify each flaky test as either order-dependent or non-order-dependent. iFixFlakies was executed on the output to further diagnose the source of the problems and additionally generate test helper patches from existing cleaners that were found. With the combination of these tools, I was able to more easily diagnose the source(s) of the problem(s) and create effective solution(s) for them.