-
Notifications
You must be signed in to change notification settings - Fork 83
Description
ExecutableStepsTest.testBoundInputEvent fails when executed on it's own because it is an Order-Dependent test that requires RunEnvironmentSensitiveTest.testRunEnvironmentAllSensitive to set up it's state beforehand. As a result, the test can fail due to external factors not setting up it's state correctly, despite the test and the code that it is testing remaining unchanged. Order dependent flaky tests can lead to unreliable results from CI and can erode developer trust in the test suite.
You can replicate the errors with the following command:
mvn -pl cloudslang-runtime -Dtest=io.cloudslang.lang.runtime.steps.ExecutableStepsTest#testBoundInputEvent test -e
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.cloudslang.lang.runtime.steps.ExecutableStepsTest
[ERROR] OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.123 s <<< FAILURE! - in io.cloudslang.lang.runtime.steps.ExecutableStepsTest
[ERROR] io.cloudslang.lang.runtime.steps.ExecutableStepsTest.testBoundInputEvent Time elapsed: 0.024 s <<< ERROR!
java.lang.RuntimeException: Application context bean is missing.
at io.cloudslang.lang.runtime.steps.ExecutableStepsTest.testBoundInputEvent(ExecutableStepsTest.java:161)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ExecutableStepsTest.testBoundInputEvent:161 » Runtime Application context bean...
[INFO]
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0