You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tests: Allow post_action to determine exit status (#2713)
Add new attr `post_action_determines_exit_code` to let the test runner
post action determine the exit status. Our use case is allow soft
failing of certain test targets.
```starlark
ios_xctestrun_runner(
name = "ios_sim_runner_with_soft_fail",
device_type = "iPhone Xs",
post_action = ":post_action_soft_fail",
post_action_determines_exit_code = True,
)
```
@@ -143,6 +148,12 @@ A binary to run prior to test execution. Runs after simulator creation. Sets any
143
148
cfg="exec",
144
149
doc="""
145
150
A binary to run following test execution. Runs after testing but before test result handling and coverage processing. Sets the `$TEST_EXIT_CODE` environment variable, in addition to any other variables available to the test runner.
151
+
""",
152
+
),
153
+
"post_action_determines_exit_code": attr.bool(
154
+
default=False,
155
+
doc="""
156
+
When true, the exit code of the test run will be set to the exit code of the post action. This is useful for tests that need to fail the test run based on their own criteria.
@@ -191,6 +196,12 @@ A binary to run prior to test execution. Runs after simulator creation. Sets the
191
196
cfg="exec",
192
197
doc="""
193
198
A binary to run following test execution. Runs after testing but before test result handling and coverage processing. Sets the `$TEST_EXIT_CODE`, `$TEST_LOG_FILE`, and `$SIMULATOR_UDID` environment variables, the `$TEST_XCRESULT_BUNDLE_PATH` environment variable if the test run produces an XCResult bundle, and any other variables available to the test runner.
199
+
""",
200
+
),
201
+
"post_action_determines_exit_code": attr.bool(
202
+
default=False,
203
+
doc="""
204
+
When true, the exit code of the test run will be set to the exit code of the post action. This is useful for tests that need to fail the test run based on their own criteria.
@@ -139,6 +144,12 @@ A binary to run prior to test execution. Sets any environment variables availabl
139
144
cfg="exec",
140
145
doc="""
141
146
A binary to run following test execution. Runs after testing but before test result handling and coverage processing. Sets the `$TEST_EXIT_CODE`, `$TEST_LOG_FILE`, and `$TEST_XCRESULT_BUNDLE_PATH` environment variables, in addition to any other variables available to the test runner.
147
+
""",
148
+
),
149
+
"post_action_determines_exit_code": attr.bool(
150
+
default=False,
151
+
doc="""
152
+
When true, the exit code of the test run will be set to the exit code of the post action. This is useful for tests that need to fail the test run based on their own criteria.
Rule to identify an iOS runner that runs tests for iOS.
@@ -557,6 +557,7 @@ Outputs:
557
557
| <aid="ios_test_runner-execution_requirements"></a>execution_requirements | Dictionary of strings to strings which specifies the execution requirements for the runner. In most common cases, this should not be used. | <ahref="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional |`{"requires-darwin": ""}`|
558
558
| <aid="ios_test_runner-os_version"></a>os_version | The os version of the iOS simulator to run test. The supported os versions correspond to the output of `xcrun simctl list runtimes`. ' 'E.g., 11.2, 9.3. By default, it is the latest supported version of the device type.' | String | optional |`""`|
559
559
| <aid="ios_test_runner-post_action"></a>post_action | A binary to run following test execution. Runs after testing but before test result handling and coverage processing. Sets the `$TEST_EXIT_CODE` environment variable, in addition to any other variables available to the test runner. | <ahref="https://bazel.build/concepts/labels">Label</a> | optional |`None`|
560
+
| <aid="ios_test_runner-post_action_determines_exit_code"></a>post_action_determines_exit_code | When true, the exit code of the test run will be set to the exit code of the post action. This is useful for tests that need to fail the test run based on their own criteria. | Boolean | optional |`False`|
560
561
| <aid="ios_test_runner-pre_action"></a>pre_action | A binary to run prior to test execution. Runs after simulator creation. Sets any environment variables available to the test runner. | <ahref="https://bazel.build/concepts/labels">Label</a> | optional |`None`|
561
562
| <aid="ios_test_runner-test_environment"></a>test_environment | Optional dictionary with the environment variables that are to be propagated into the XCTest invocation. | <ahref="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional |`{}`|
562
563
@@ -665,8 +666,9 @@ of the attributes inherited by all test rules, please check the
This rule creates a test runner for iOS tests that uses xctestrun files to run
@@ -719,6 +721,7 @@ in Xcode.
719
721
| <aid="ios_xctestrun_runner-device_type"></a>device_type | The device type of the iOS simulator to run test. The supported types correspond to the output of `xcrun simctl list devicetypes`. E.g., iPhone X, iPad Air. By default, it reads from --ios_simulator_device or falls back to some device. | String | optional |`""`|
720
722
| <aid="ios_xctestrun_runner-os_version"></a>os_version | The os version of the iOS simulator to run test. The supported os versions correspond to the output of `xcrun simctl list runtimes`. E.g., 15.5. By default, it reads --ios_simulator_version and then falls back to the latest supported version. | String | optional |`""`|
721
723
| <aid="ios_xctestrun_runner-post_action"></a>post_action | A binary to run following test execution. Runs after testing but before test result handling and coverage processing. Sets the `$TEST_EXIT_CODE`, `$TEST_LOG_FILE`, and `$SIMULATOR_UDID` environment variables, the `$TEST_XCRESULT_BUNDLE_PATH` environment variable if the test run produces an XCResult bundle, and any other variables available to the test runner. | <ahref="https://bazel.build/concepts/labels">Label</a> | optional |`None`|
724
+
| <aid="ios_xctestrun_runner-post_action_determines_exit_code"></a>post_action_determines_exit_code | When true, the exit code of the test run will be set to the exit code of the post action. This is useful for tests that need to fail the test run based on their own criteria. | Boolean | optional |`False`|
722
725
| <aid="ios_xctestrun_runner-pre_action"></a>pre_action | A binary to run prior to test execution. Runs after simulator creation. Sets the `$SIMULATOR_UDID` environment variable, in addition to any other variables available to the test runner. | <ahref="https://bazel.build/concepts/labels">Label</a> | optional |`None`|
723
726
| <aid="ios_xctestrun_runner-random"></a>random | Whether to run the tests in random order to identify unintended state dependencies. | Boolean | optional |`False`|
724
727
| <aid="ios_xctestrun_runner-reuse_simulator"></a>reuse_simulator | Toggle simulator reuse. The default behavior is to reuse an existing device of the same type and OS version. When disabled, a new simulator is created before testing starts and shutdown when the runner completes. | Boolean | optional |`True`|
0 commit comments