Skip to content

Commit 8be4648

Browse files
committed
docs: Restructure test-run troubleshooting guide
1 parent 8268ac0 commit 8be4648

10 files changed

+142
-169
lines changed

Diff for: docs/guide/investigating-test-failure.mdx

-41
This file was deleted.

Diff for: docs/img/app-loader.jpeg

-9.09 KB
Loading

Diff for: docs/img/transient-ui-element.png

74.9 KB
Loading

Diff for: docs/introduction/debugging.mdx

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ In the Detox world, you can debug either Detox itself (i.e. run it step by step)
2020

2121
Detox tests can be run step-by-step either using an IDE or by using Chrome debugger.
2222

23-
Start by running Detox using the Detox CLI alongside the inspection argument (`--inspec-brk`) and the file in which the test resides. For example:
23+
Start by running Detox using the Detox CLI alongside the inspection argument (`--inspect-brk`) and the file in which the test resides. For example:
2424

2525
```bash
2626
detox test --inspect-brk -c android.emu.debug e2e/starter.test.js

Diff for: docs/introduction/your-first-test.mdx

+2-2
Original file line numberDiff line numberDiff line change
@@ -204,8 +204,8 @@ If you haven't changed the generated `e2e/starter.test.js`, you are likely to se
204204
205205
```
206206

207-
If you have created your own test, and it is failing, examine the error message, check out our [Investigating Failures](../guide/investigating-test-failure.mdx)
208-
and [Debugging](debugging.mdx) guides, and run your tests again after you fix the issue.
207+
If you have created your own test, and it is failing, examine the error message, check out our [Troubleshooting](../troubleshooting/running-tests.md)
208+
and [Debugging](debugging.mdx) guides..
209209

210210
[matchers]: ../api/matchers.md
211211
[`by.id()`]: ../api/matchers.md#byidid

Diff for: docs/troubleshooting/element-matching.md

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Dealing with Element Matching Issues
2+
3+
The preferred matching technique is always matching based on test ID's.
4+
5+
React Native only supports the `testID` prop on the native built-in components. If you’ve created a custom composite component, you will have to support this prop yourself. You should probably propagate the `testID` prop to one of your rendered children (a built-in component):
6+
7+
```jsx
8+
export class MyCompositeComponent extends Component {
9+
render() {
10+
return (
11+
<TouchableOpacity testID={this.props.testID}>
12+
<View>
13+
<Text>Something something</Text>
14+
</View>
15+
</TouchableOpacity>
16+
);
17+
}
18+
}
19+
```
20+
21+
Now, adding `testID` to your composite component should work:
22+
23+
```jsx
24+
render() {
25+
return <MyCompositeComponent testID='MyUniqueId123' />;
26+
}
27+
```
28+
29+
## Debug View Hierarchy
30+
31+
You can also investigate the app’s native view hierarchy, this might shed some light on how the app’s view hierarchy is laid out and therefore why an element matching doesn't work (perhaps a test ID should be used, or the matcher should be improved).
32+
33+
On iOS, one way this can be done is by using `xcode`. Do the following:
34+
35+
1. Start a debuggable app (not a release build) in your simulator
36+
1. Open Xcode
37+
1. Attach Xcode to your app’s process
38+
![attach to process](../img/attach-to-process.jpg)
39+
1. Press the `Debug View Hierarchy` button
40+
![debug view hierarchy](../img/debug-view-hierarchy.jpg)
41+
1. This will open the hierarchy viewer, and will show a breakdown of your app’s native view hierarchy. Here you can browse through the views
42+
1. React Native testIDs are manifested as _accessibility identifiers_ in the native view hierarchy
43+
44+
Let’s see an example. We will find the following view in the native hierarchy:
45+
46+
```jsx
47+
<TouchableOpacity onPress={this.onButtonPress.bind(this, 'ID Working')}>
48+
<Text testID='UniqueId345' style={{color: 'blue', marginBottom: 20}}>ID</Text>
49+
</TouchableOpacity>
50+
```
51+
52+
This is the hierarchy viewer, pointing to the native view just mentioned:
53+
54+
![hierarchy viewer](../img/hierarchy-viewer.jpg)
55+
56+
There are other techniques for doing this with xcode, and also on Android -- coming soon!

Diff for: docs/troubleshooting/running-tests.md

+62-116
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,8 @@
11
# Dealing With Problems With Running Tests
22

3-
This page is about issues related to executing your Detox tests, typically triggered when running `detox test` (and not `detox build`, for example).
4-
5-
## Trace Mode
3+
<!-- markdownlint-disable MD024 -->
64

7-
It’s a good idea to get as much information as possible about what’s going on. We can enable trace mode during tests by running our tests with:
8-
9-
```bash
10-
detox test --loglevel trace
11-
```
5+
This page is about issues related to executing your Detox tests, typically triggered when running `detox test` (and not `detox build`, for example).
126

137
## No simulators found (iOS)
148

@@ -18,125 +12,104 @@ If you’re missing a simulator, make sure Xcode is installed and use it to down
1812

1913
Once the desired simulator is installed and returned by `xcrun simctl list`, double check its name in the list and make sure this name is found in the `detox` configuration entry in `package.json`. The reference for the configuration options is available [here](../config/devices.mdx).
2014

21-
## Tests execution hangs
15+
## Detox starts but my test doesn't actually run
16+
17+
### Issue
2218

23-
**Issue:** A while after running Detox, you get a message about failure to connect to the running app, in the logs:
19+
- A while after running Detox, things seem to hang and you get a message about a failure to connect to the running app, in Detox's logs:
2420

2521
```plain text
2622
Detox can’t seem to connect to the test app(s)!
2723
```
2824

25+
### Course of action
26+
2927
This can be a result of various reasons. It is generally up to you to debug and find the root cause. In any case, below are the common ones.
3028

31-
### If you do not see your app running on the device
29+
#### If you don't see your app running on the device
3230

3331
- You might have forgotten to run `device.launchApp()` in the beginning of your test.
3432
- The app might have crashed before Detox has had a chance to connect to it. To get the crash details, you can run Detox tests with `--record-logs all` CLI option and then inspect the device logs in the artifacts' folder.
3533
- **On Android**, there might be a problem with the native test code in the `DetoxTest.java` file. Revisit the [associated section](../introduction/project-setup.mdx#step-4-additional-android-configuration) in the setup guide.
3634
- **On Android**, your `Network Security Config` may not be recognized. Revisit the [associated section](../introduction/project-setup.mdx#43-enabling-unencrypted-traffic-for-detox) in the setup guide.
3735

38-
### If you _do_ see your app running on the device
36+
**If you _do_ see your app running on the device**
3937

4038
- **On Android with SDK≥28**, the app’s connection to the Detox test server is blocked due to clear-traffic blockage (as reported in issue [#1450](https://github.com/wix/Detox/issues/1450)).
4139
The main step for getting this fixed is to revisit the [associated section](../introduction/project-setup.mdx#step-4-additional-android-configuration) in the setup guide, which discusses network-security. Alternatively, the `android:usesCleartextTraffic="true"` attribute can be configured in the `<application>` tag of the app’s `AndroidManifest.xml`, but **that is highly discouraged**.
4240
- If you’ve applied the above suggestion but the app fails to connect to the Detox test server, nonetheless: Refer to the device’s logs, which should contain messages about failed connection attempts (get them using the `--record-logs all` argument)
4341
- The app could be running without Detox native code injected. In this case, first, make sure you’re not trying to run in manual launch mode (where this behavior is valid). If so, examine the logs from the device (get them using the `--record-logs all` argument). If you see a crash related to Detox’s native code, you are welcome to report it on our GitHub tracker.
44-
- If you are in fact debugging your native code integration with Detox, our [Debugging tutorial](../introduction/debugging.mdx) may prove helpful.
42+
- Try reading our [Debugging tutorial](../introduction/debugging.mdx).
4543

46-
## Syntax Error: Unexpected Token
44+
## The test runs but the app looks stuck / test times-out
4745

48-
**Issue:** Running tests immediately throws the following error:
46+
### Issue
4947

50-
```js
51-
beforeEach(async () => {
52-
^
53-
SyntaxError: Unexpected token (
54-
at Object.exports.runInThisContext (vm.js:76:16)
55-
at Module._compile (module.js:545:28)
56-
at loader (/Users/builduser/buildAgent/work/34eee2d16ef6c34b/node_modules/babel-register/lib/node.js:144:5)
57-
at Object.require.extensions.(anonymous function) [as .js] (/Users/builduser/buildAgent/work/34eee2d16ef6c34b/node_modules/babel-register/lib/node.js:154:7)
58-
...
59-
child_process.js:531
60-
throw err;
61-
```
48+
- **The test runs and at some point the app gets stuck on the same screen / state until the end of the test** - for example, displaying a loader animation. This can be observed directly if Detox is run locally, or otherwise post-factum by checking Detox's [video recording artifact](https://wix.github.io/Detox/docs/config/artifacts/#enabling-artifacts):
49+
50+
![App loader example](../img/app-loader.jpeg)
51+
52+
- Detox logs might also show synchronization warnings, repeatedly. For example:
6253

63-
**Solution:** This error means that your version of Node does not support `async-await` syntax. You should do the following:
54+
```plain text
55+
19:07:20.140 detox[1907] i The app is busy with the following tasks:
56+
• UI elements are busy:
57+
- View animations pending: 2.
58+
- Layers pending animations: 7.
59+
- Layers needs layout: 147.
60+
- View needs layout: 98.
61+
- View needs display: 67.
62+
- Layers needs display: 82.
63+
• 1 enqueued native timers:
64+
- Timer #1:
65+
+ Fire date: none.
66+
+ Time until fire: 0.000.
67+
+ Repeat interval: 0.
68+
+ Is recurring: YES.
69+
• 1 network requests with URLs:
70+
- URL #1: https://example.org/something?id=1337
71+
```
6472

65-
1. Update Node to a version **8.3.0 or higher**.
73+
- The test failure reason might be associated with time-outs, or you see time-outs in the _device_ (not Detox) log (available as a Detox [test artifact](https://wix.github.io/Detox/docs/config/artifacts/#enabling-artifacts)).
6674

67-
## Can’t Find My Component Even Though I Added a `testID` to Its Props
75+
### Course of action
6876

69-
**Issue:** Detox fails to match a component even though it has a `testID`. Detox will throw the following error:
77+
This might be related somehow to your test code, but **could definitely stem from a an app bug.** Take an in-depth look at the [synchronization troubleshooting guide](./synchronization.md).
7078

71-
```plain text
72-
Error: Cannot find UI Element.
73-
Exception with Assertion: {
74-
"Assertion Criteria" : "assertWithMatcher: matcherForSufficientlyVisible(>=0.750000)",
75-
"Element Matcher" : "(((respondsToSelector(accessibilityIdentifier) && accessibilityID('Welcome')) && !kindOfClass('RCTScrollView')) || (kindOfClass('UIScrollView') && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && ancestorThatMatches(((respondsToSelector(accessibilityIdentifier) && accessibilityID('Welcome')) && kindOfClass('RCTScrollView'))))))",
76-
"Recovery Suggestion" : "Check if element exists in the UI, modify assert criteria, or adjust the matcher"
77-
}
78-
79-
Error Trace: [
80-
{
81-
"Description" : "Interaction cannot continue because the desired element was not found.",
82-
"Domain" : "com.google.earlgrey.ElementInteractionErrorDomain",
83-
"Code" : "0",
84-
"File Name" : "GREYElementInteraction.m",
85-
"Function Name" : "-[GREYElementInteraction matchedElementsWithTimeout:error:]",
86-
"Line" : "119"
87-
}
88-
]
89-
```
79+
## Detox can't find or interact with an element even though it's on the screen
9080

91-
**Solution:** React Native only supports the `testID` prop on the native built-in components. If you’ve created a custom composite component, you will have to support this prop yourself. You should probably propagate the `testID` prop to one of your rendered children (a built-in component):
92-
93-
```jsx
94-
export class MyCompositeComponent extends Component {
95-
render() {
96-
return (
97-
<TouchableOpacity testID={this.props.testID}>
98-
<View>
99-
<Text>Something something</Text>
100-
</View>
101-
</TouchableOpacity>
102-
);
103-
}
104-
}
105-
```
81+
### Issue
10682

107-
Now, adding `testID` to your composite component should work:
83+
Either one of the following 2 scenarios:
10884

109-
```jsx
110-
render() {
111-
return <MyCompositeComponent testID='MyUniqueId123' />;
112-
}
113-
```
85+
1. The test runs, the app starts on the device and is maybe even being successfully navigated-through by the test, onto an inner screen; The app/screen opens properly and you know for sure that your element matcher is spot-on - **yet Detox says the element cannot be found (this can even happen intermittently, in a flaky test).**
86+
2. The test runs, the app starts and displays a transient UI element (e.g. a Toast) for a limited amount of time and is then removed automatically. For example: A temporary "Success" cheering-effect that fades out to the screen below:
11487

115-
## Test Tries to Find My Component Before It’s Created
88+
![Transient UI element](../img/transient-ui-element.png)
11689

117-
**Issue:** Due to a synchronization issue, the test tries to perform an expectation and fails because it runs the expectation too soon. Consider this example:
90+
### Course of action
11891

119-
```js
120-
await element(by.text('Login')).tap();
121-
await expect(element(by.text('Welcome'))).toBeVisible();
122-
```
92+
For the 1st option, you'd have to go deeper by exploring the [synchronization troubleshooting guide](./synchronization.md#last-resort-Switching-to-manual-synchronization).
12393

124-
In the test above, after tapping the Login button, the app performs several complex asynchronous operations until the Welcome message is displayed post-login. These can include querying a server, waiting for a response and then running an animated transition to the Welcome screen. Detox attempts to simplify your test code by synchronizing _automatically_ with these asynchronous operations. What happens if for some reason the automatic synchronization doesn’t work? As a result, Detox will not wait correctly until the Welcome screen appears and instead will continue immediately to the next line and try to run the expectation. Since the screen is not there yet, the test will fail.
94+
For the 2nd option, if possible, start by revisiting your matching technique, as explained in the dedicated [matching troubleshooting guide](./element-matching.md). If that doesn't solve it, consider switching to using `waitFor()` --
12595

126-
**Solution:** When you suspect that automatic synchronization didn’t work, you have a fail-safe by synchronizing manually with `waitFor`. Using `waitFor` will poll until the expectation is met. This isn’t a recommended approach so please use it as a workaround and open and issue to resolve the synchronization issue.
96+
#### Switching to the polling-based `waitFor()` API's
12797

128-
This is what the fixed test would look like:
98+
you'd have to resort to Detox's fail-safe `waitFor()` family of API's, which poll your matching criteria within a specified time frame defined by you as a timeout. To take the screen above as an example, you'd have to change your test from:
12999

130100
```js
131-
await element(by.text('Login')).tap();
132-
await waitFor(element(by.text('Welcome'))).toBeVisible().withTimeout(2000);
101+
await element(by.id('join-button')).tap();
102+
await expect(element(by.text('Success!'))).toBeVisible();
133103
```
134104

135-
## Can’t synchronize the test with my app
105+
to something like:
136106

137-
If you suspect that the test is failing because Detox fails to synchronize the test steps with your app, take a look at this in-depth [synchronization troubleshooting tutorial](synchronization.md).
107+
```js
108+
await element(by.id('join-button')).tap();
109+
await waitFor(element(by.text('Success!'))).toBeVisible().withTimeout(2_000);
110+
```
138111

139-
## An Element is Not Visible
112+
## Detox says an element is not visible enough
140113

141114
**On iOS**, you may run in a situation, when one of the interactions (tap, scroll, etc.) on an element fails with an error like:
142115

@@ -160,48 +133,21 @@ If you are developing a React Native app, then the following applies. If, for in
160133

161134
If you see that your issue cannot be solved via testID replacement or a simple hierarchy rearrangement, then there’s a chance this is a bug in Detox. Make sure to provide your `ui.viewhierarchy` artifact, the generated `DETOX_VISIBILITY_*` pictures and a comprehensive description of the issue backed up with sound arguments.
162135

163-
## Debug View Hierarchy
164-
165-
**Issue:** I added the `testID` prop, but I still can’t find the view by id in my tests.
166-
167-
**Solution:** You can investigate the app’s native view hierarchy, this might shed some light on how the app’s view hierarchy is laid out.
168-
169-
Do the following:
170-
171-
1. Start a debuggable app (not a release build) in your simulator
172-
1. Open Xcode
173-
1. Attach Xcode to your app’s process
174-
![attach to process](../img/attach-to-process.jpg)
175-
1. Press the `Debug View Hierarchy` button
176-
![debug view hierarchy](../img/debug-view-hierarchy.jpg)
177-
1. This will open the hierarchy viewer, and will show a breakdown of your app’s native view hierarchy. Here you can browse through the views
178-
1. React Native testIDs are manifested as _accessibility identifiers_ in the native view hierarchy
179-
180-
Let’s see an example. We will find the following view in the native hierarchy:
181-
182-
```jsx
183-
<TouchableOpacity onPress={this.onButtonPress.bind(this, 'ID Working')}>
184-
<Text testID='UniqueId345' style={{color: 'blue', marginBottom: 20}}>ID</Text>
185-
</TouchableOpacity>
186-
```
187-
188-
This is the hierarchy viewer, pointing to the native view just mentioned:
189-
190-
![hierarchy viewer](../img/hierarchy-viewer.jpg)
136+
## Explore More Options
191137

192-
## Compare to a Working Setup
138+
### Compare to a Working Setup
193139

194140
If you feel lost, try starting from a working example for sanity.
195141

196142
There are multiple working examples included in this repo, such as [demo-react-native](https://github.com/wix/Detox/tree/master/examples/demo-react-native).
197143

198144
First, install, build and make sure the tests are indeed passing. If they are, try comparing this setup with what you have.
199145

200-
## Take a Look at Past Issues
146+
### Take a Look at Past Issues
201147

202148
Before opening a new issue, search the [list of issues](https://github.com/wix/detox/issues?utf8=%E2%9C%93\&q=is%3Aissue) on GitHub. There’s a good chance somebody faced the same problem you are having.
203149

204-
## How to Open a New Issue
150+
### How to Open a New Issue
205151

206152
Before opening a new issue, please follow the entire troubleshooting guide and go over past issues.
207153

0 commit comments

Comments
 (0)