Skip to content

Commit

Permalink
Update integration testing docs for Android and iOS. (#11680)
Browse files Browse the repository at this point in the history 
_Description of what this PR is changing or adding, and why:_

Separate the iOS and Android integration testing steps so that it's
easier for our Android and iOS devs to parse this content.

Notes:

I noticed some issues with the guide while making the original updates
and made a few updates to fix the issues. This includes:
- Strengthened the introduction.
- Removed the 'recipe' wording (this isn't structured like our recipes
nor in the recipe directory).
- There were quite a few coding errors and grammatical flow problems in
the original Firebase Test Lab section for Android. I've cleaned these
up but I'm also not 100% certain that those commands are accurate today.
We might want to follow up on that section and make sure the steps work.

_Issues fixed by this PR (if any):_

_PRs or commits this PR depends on (if any):_

## Presubmit checklist

- [x] This PR is marked as draft with an explanation if not meant to
land until a future stable release.
- [x] This PR doesn’t contain automatically generated corrections
(Grammarly or similar).
- [x] This PR follows the [Google Developer Documentation Style
Guidelines](https://developers.google.com/style) — for example, it
doesn’t use _i.e._ or _e.g._, and it avoids _I_ and _we_ (first person).
- [x] This PR uses [semantic line
breaks](https://github.com/dart-lang/site-shared/blob/main/doc/writing-for-dart-and-flutter-websites.md#semantic-line-breaks)
of 80 characters or fewer.
  • Loading branch information
@antfitch
antfitch authored Feb 6, 2025
1 parent a8b3d5e commit b3f9e58
Showing 1 changed file with 124 additions and 79 deletions.
203 changes: 124 additions & 79 deletions src/content/testing/integration-tests/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,24 @@ description: Learn how to write integration tests

<?code-excerpt path-base="testing/integration_tests/how_to"?>

This recipe describes how to use the
[`integration_test`][] package to run integration tests.
The Flutter SDK includes the `integration_test` package.
Integration tests using this package have the following
properties.
## Introduction

* Use the `flutter drive` command to run tests
on a physical device or emulator.
* Run on [Firebase Test Lab][],
to automate testing on a variety of devices.
* Use [flutter_test][] APIs to enable tests to be written in a style similar to [widget tests][]
This guide describes how to run integration tests with your Flutter app. With
it, you'll learn how to do the following:

In this recipe, learn how to test a counter app.
* Set up integration tests.
* Verify if an app displays specific text.
* Tap specific widgets.
* Run integration tests.

* how to set up integration tests
* how to verify if an app displays specific text
* how to tap specific widgets
* how to run integration tests
The guide references the `counter_app` project that comes with
Flutter and the Flutter [`integration_test`][] package. The
`integration_test` package lets you:

This recipe uses the following steps:

1. Create an app to test.
2. Add the `integration_test` dependency.
3. Create the test files.
4. Write the integration test.
5. Run the integration test.
* Use the `flutter drive` command to run tests on a physical device or emulator.
* Run on [Firebase Test Lab][], to automate testing on a variety of devices.
* Use [flutter_test][] APIs to write tests in a style similar to
[widget tests][].

## Create a new app to test

Expand Down Expand Up @@ -337,20 +329,49 @@ Based on platform, the command result should resemble the following output.

---

### Test on a mobile device
### Test on an Android device

To test on a real iOS or Android device,
To test on a real Android device,
complete the following tasks.

1. Connect the device.
1. Connect the Android device.

1. Run the following command from the root of the project.

```console
$ flutter test integration_test/app_test.dart
```

The result should resemble the following output.

```console
$ flutter test integration_test/app_test.dart
00:04 +0: loading /path/to/counter_app/integration_test/app_test.dart
00:15 +0: loading /path/to/counter_app/integration_test/app_test.dart
00:18 +0: loading /path/to/counter_app/integration_test/app_test.dart 2,387ms
Installing build/app/outputs/flutter-apk/app.apk... 612ms
00:21 +1: All tests passed!
```

1. Verify that the test removed the Counter App when it finished.
If not, subsequent tests fail. If needed, press on the app and choose
**Remove App** from the context menu.

---

### Test on an iOS device

To test on a real iOS device, complete the following tasks.

1. Connect the iOS device.

1. Run the following command from the root of the project.

```console
$ flutter test integration_test/app_test.dart
```

The result should resemble the following output. This example uses iOS.
The result should resemble the following output.

```console
$ flutter test integration_test/app_test.dart
Expand Down Expand Up @@ -479,21 +500,15 @@ To learn more, see the

---

### Test using the Firebase Test Lab
### Test in Firebase Test Lab (Android)

To test both Android and iOS targets,
you can use the Firebase Test Lab.
You can use Firebase Test Lab to test Android targets.

#### Android setup

Follow the instructions in the [Android Device Testing][]
section of the README.

#### iOS setup

Follow the instructions in the [iOS Device Testing][]
section of the README.

#### Test Lab project setup

1. Launch your [Firebase Console][].
Expand All @@ -506,71 +521,102 @@ section of the README.

#### Upload an Android APK

1. Create an APK using Gradle.
Complete the following steps to upload an Android APK.

```console
$ pushd android
# flutter build generates files in android/ for building the app
flutter build apk
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=integration_test/<name>_test.dart
$ popd
```
1. Create an APK using Gradle.

Where `<name>_test.dart` is the file created in the
**Project Setup** section.
```console
// Go to the Android directory which contains the gradlew script
$ pushd android

:::note
To use `--dart-define` with `gradlew:
// Build the APK for Flutter with gradlew
$ flutter build apk

1. Encode all parameters with `base64`.
1. Pass the parameters to gradle in a comma-separated list.
// Build an Android test APK
$ ./gradlew app:assembleAndroidTest

```console
./gradlew project:task -Pdart-defines="{base64 (key=value)},[...]"
```
// Build a debug APK by passing in an integration test
$ ./gradlew app:assembleDebug -Ptarget=integration_test/<name>_test.dart
```

* `<name>_test.dart`: The file created in the **Project Setup** section.

:::
1. If needed, pass parameters into the integration test as a comma-separated
list. Encode all parameters as `base64`.

To start a Robo test and run other tests, drag the "debug" APK from
`<flutter_project_directory>/build/app/outputs/apk/debug`
into the **Android Robo Test** target on the web page.
```console
$ ./gradlew project:task --defines="{base64 (key=value)}[, ...]"
```

<img src='/assets/images/docs/integration-test/test-lab-2.png' alt="Firebase Test Lab upload">
* `(key=value)}[, ...]`: Replace this with a comma-separated list of
key value pairs.

1. Click **Run a test**.
1. Return to your previous directory.

1. Select the **Instrumentation** test type.
```console
$ popd
```

1. Add the App APK to the **App APK or AAB** box.
For additional instructions, see the
[Firebase Test Lab section of the README][].

`<flutter_project_directory>/build/app/outputs/apk/debug/<file>.apk`
#### Start Robo test

1. Add the Test APK to the **Test APK** box.
To use Robo test to run integration tests, complete the following steps.

`<flutter_project_directory>/build/app/outputs/apk/androidTest/debug/<file>.apk`
1. Drag the debug APK from
`<flutter_project_directory>/build/app/outputs/apk/debug`
into the **Android Robo Test** target on the web page. For example:

<img src='/assets/images/docs/integration-test/test-lab-3.png' alt="Firebase Test Lab upload two APKs">
<img src='/assets/images/docs/integration-test/test-lab-2.png' alt="Firebase Test Lab upload">

If a failure occurs, click the red icon to view the output:
1. Click **Run a test**.

<img src='/assets/images/docs/integration-test/test-lab-4.png' alt="Firebase Test Lab test results">
1. Select the **Instrumentation** test type.

#### Upload an Android APK from the command line
1. Add the App APK to the **App APK or AAB** box.

See the [Firebase Test Lab section of the README][]
for instructions on uploading the APKs from the command line.
`<flutter_project_directory>/build/app/outputs/apk/debug/<file>.apk`

#### Upload Xcode tests
1. Add the Test APK to the **Test APK** box.

To learn how to upload the .zip file,
consult the [Firebase TestLab iOS instructions][]
on the Firebase TestLab section of the Firebase Console.
`<flutter_project_directory>/build/app/outputs/apk/androidTest/debug/<file>.apk`

#### Upload Xcode tests from the command line
<img src='/assets/images/docs/integration-test/test-lab-3.png' alt="Firebase Test Lab upload two APKs">

To learn how to upload the .zip file from the command line,
consult the [iOS Device Testing][] section in the README.
1. If a failure occurs, click the red icon to view the output:

<img src='/assets/images/docs/integration-test/test-lab-4.png' alt="Firebase Test Lab test results">

---

### Test in Firebase Test Lab (iOS)

You can use Firebase Test Lab to test iOS targets.

#### iOS setup

Follow the [iOS Device Testing instructions][].

#### Test Lab project setup

1. Launch your [Firebase Console][].

1. Create a new Firebase project if necessary.

1. Navigate to **Quality > Test Lab**.

<img src='/assets/images/docs/integration-test/test-lab-1.png' alt="Firebase Test Lab Console">

#### Upload Xcode tests through the Firebase Console

To learn how to upload tests from a ZIP file, using the
Firebase Test Lab Console, consult the [Firebase Test Lab iOS instructions][].

#### Upload Xcode tests to Firebase Console with the command line

To learn how to upload tests from a ZIP file from the command line to the
Firebase Test Lab Console, consult the [iOS Device Testing instructions][].

[`integration_test`]: {{site.repo.flutter}}/tree/main/packages/integration_test#integration_test
[Android Device Testing]: {{site.repo.flutter}}/tree/main/packages/integration_test#android-device-testing
Expand All @@ -580,13 +626,12 @@ consult the [iOS Device Testing][] section in the README.
[Firebase Console]: http://console.firebase.google.com/
[Firebase Test Lab section of the README]: {{site.repo.flutter}}/tree/main/packages/integration_test#firebase-test-lab
[Firebase Test Lab]: {{site.firebase}}/docs/test-lab
[Firebase TestLab iOS instructions]: {{site.firebase}}/docs/test-lab/ios/firebase-console
[Firebase Test Lab iOS instructions]: {{site.firebase}}/docs/test-lab/ios/firebase-console
[flutter_test]: {{site.api}}/flutter/flutter_test/flutter_test-library.html
[Integration testing]: /testing/integration-tests
[iOS Device Testing]: {{site.repo.flutter}}/tree/main/packages/integration_test#ios-device-testing
[iOS Device Testing instructions]: {{site.repo.flutter}}/tree/main/packages/integration_test#ios-device-testing
[Running Flutter driver tests with web]: {{site.repo.flutter}}/blob/main/docs/contributing/testing/Running-Flutter-Driver-tests-with-Web.md
[widget tests]: /testing/overview#widget-tests

[flutter_driver]: {{site.api}}/flutter/flutter_driver/flutter_driver-library.html
[integration_test usage]: {{site.repo.flutter}}/tree/main/packages/integration_test#usage
[samples]: {{site.repo.samples}}
Expand Down

0 comments on commit b3f9e58

Please sign in to comment.