Skip to content

Commit 0f9ce0f

Browse files
LoopedBard3LoopedBard3
authored
Add doc for android startup scenario local testing. (#4664)
Added doc for android startup scenario local testing and added a new scenario folder named genericandroidstartup.
1 parent 304c424 commit 0f9ce0f

File tree

5 files changed

+120
-1
lines changed

5 files changed

+120
-1
lines changed

‎docs/android-startup-scenarios.md

+64
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
2+
# Android Startup Test
3+
4+
## Prereqs
5+
6+
- Ensure `python` is installed and available. Any currently supported `python` 3.* version should work. Downloads are available at https://www.python.org/downloads/.
7+
- Ensure `dotnet` is installed and available with the `dotnet` command for easy xharness installation. Any supported .NET Core version should work. [Dotnet Download](https://dotnet.microsoft.com/en-us/download) or [Daily Dotnet Download](https://github.com/dotnet/sdk/blob/main/documentation/package-table.md).
8+
- Ensure `xharness` is installed and available with the `xharness` command. The current version in use can be found in the `eng/performance/maui_scenarios_android.proj` file at line 7 (under the tag `MicrosoftDotNetXHarnessCLIVersion`), although any recent version should work. [XHarness Install Instructions](https://github.com/dotnet/xharness?tab=readme-ov-file#installation-and-usage).
9+
- Have an Android app APK available for testing.
10+
- Have an Android Device (with developer mode enabled) or emulator connected to computer, and viewable with `xharness android device` or `xharness android adb -- devices -l`.
11+
12+
## Steps
13+
14+
1. Initialize the environment (note the . for bash):
15+
16+
```sh
17+
cd src/scenarios
18+
. ./init.sh # or `.\init.ps1` on Windows. Can specify custom dotnet install with -dotnetdir <dir>, but dotnet install should not impact Android Startup testing itself.
19+
```
20+
21+
2. Navigate to the `genericandroidstartup` scenario directory:
22+
23+
```sh
24+
cd genericandroidstartup
25+
```
26+
27+
3. Copy the APK into the `genericandroidstartup` directory.
28+
4. Run the test:
29+
30+
```sh
31+
python test.py devicestartup --device-type android --package-path <path-to-apk> --package-name <apk-package-name> [--disable-animations] [--use-fully-drawn-time --fully-drawn-extra-delay <delay-in-sec>]
32+
```
33+
34+
* Refer to the [Notes](./android-startup-scenarios.md#notes) below about specifying --use-fully-drawn-time --fully-drawn-extra-delay parameters.
35+
36+
5. Read the output:
37+
38+
During the running of the test you will see the loop of the activity being started to get the startup times.
39+
Once the testing is completed, you will see output similar to the following:
40+
41+
```txt
42+
[2025/01/29 11:15:44][INFO] Found Value (ms): 713
43+
[2025/01/29 11:15:44][INFO] Found Value (ms): 715
44+
[2025/01/29 11:15:44][INFO] Found Value (ms): 728
45+
[2025/01/29 11:15:44][INFO] Found Value (ms): 716
46+
[2025/01/29 11:15:44][INFO] Found Value (ms): 715
47+
[2025/01/29 11:15:44][INFO] Found Value (ms): 734
48+
[2025/01/29 11:15:44][INFO] Found Value (ms): 716
49+
[2025/01/29 11:15:44][INFO] Found Value (ms): 718
50+
[2025/01/29 11:15:44][INFO] Found Value (ms): 713
51+
[2025/01/29 11:15:44][INFO] Found Value (ms): 706
52+
[2025/01/29 11:15:44][INFO] Device Startup - Maui Android Default NoAnimation
53+
[2025/01/29 11:15:44][INFO] Metric |Average |Min |Max
54+
[2025/01/29 11:15:44][INFO] ----------------|---------------|---------------|---------------
55+
[2025/01/29 11:15:44][INFO] Generic Startup |717.400 ms |706.000 ms |734.000 ms
56+
```
57+
58+
The Found Value's are the individual test run startup times with the overall stats at the bottom. The stats provided include the following startup stats: average, minimum, and maximum times.
59+
60+
## Notes
61+
62+
- Specific example command such as when using the runtime android example app: `python test.py devicestartup --device-type android --package-path HelloAndroid.apk --package-name net.dot.HelloAndroid`.
63+
- Other example commands and additional logic can be found in the `maui_scenarios_android.proj` and `runner.py` files in the `performance` repository.
64+
- If using `[--use-fully-drawn-time --fully-drawn-extra-delay <delay in sec>]` arguments, the Android app must have reportFullyDrawn() called on a ComponentActivity. Reference: https://developer.android.com/topic/performance/vitals/launch-time#retrieve-TTFD.

‎docs/scenarios-workflow.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
## Overview
55

6-
Our existing scenario tests are under `src\scenarios` in this repo, where each subdirectory contains a test asset that can be combined with a specific set of commands to do measurements. Currently we have scenario tests for [SDK](./sdk-scenarios.md), [Crossgen](./crossgen-scenarios.md), [Blazor](./blazor-scenarios.md) and [other scenarios](./basic-scenarios.md).
6+
Our existing scenario tests are under `src\scenarios` in this repo, where each subdirectory contains a test asset that can be combined with a specific set of commands to do measurements. Currently we have scenario tests for [SDK](./sdk-scenarios.md), [Crossgen](./crossgen-scenarios.md), [Blazor](./blazor-scenarios.md), [Android Startup](./android-startup-scenarios.md), and [other scenarios](./basic-scenarios.md).
77

88
## Running scenario tests
99

@@ -12,6 +12,7 @@ This is a general guideline on how the scenario tests are arranged in this repo.
1212
- [How to run SDK scenario tests](./sdk-scenarios.md)
1313
- [How to run Crossgen scenario tests](./crossgen-scenarios.md)
1414
- [How to run Blazor tests](./blazor-scenarios.md)
15+
- [How to run Android Startup tests](./android-startup-scenarios.md)
1516
- [How to run other Scenario tests](./basic-scenarios.md)
1617

1718
### Prerequisites
@@ -138,4 +139,5 @@ Some command options are only applicable for certain test assets. Refer to the c
138139
- [SDK Command Matrix](./sdk-scenarios.md#command-matrix)
139140
- [Crossgen Command Matrix](./crossgen-scenarios.md#command-matrix)
140141
- [Blazor Command Matrix](./blazor-scenarios.md#command-matrix)
142+
- [How to run Android Startup tests](./android-startup-scenarios.md)
141143
- [Other Scenarios Command Matrix](./basic-scenarios.md#command-matrix)

‎src/scenarios/genericandroidstartup/post.py

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
'''
2+
post cleanup script
3+
'''
4+
5+
from shared.postcommands import clean_directories
6+
7+
clean_directories()

‎src/scenarios/genericandroidstartup/pre.py

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
'''
2+
pre-command
3+
'''
4+
import sys
5+
import os
6+
from zipfile import ZipFile
7+
from performance.logger import setup_loggers, getLogger
8+
from shutil import copyfile
9+
from shared.precommands import PreCommands
10+
from shared.const import PUBDIR
11+
from argparse import ArgumentParser
12+
13+
setup_loggers(True)
14+
15+
parser = ArgumentParser()
16+
parser.add_argument(
17+
'--apk-name',
18+
dest='apk',
19+
required=True,
20+
type=str,
21+
help='Name of the APK to setup (with .apk)')
22+
args = parser.parse_args()
23+
24+
if not os.path.exists(PUBDIR):
25+
os.mkdir(PUBDIR)
26+
apkname = args.apk
27+
if not os.path.exists(apkname):
28+
getLogger().error('Cannot find %s' % (apkname))
29+
exit(-1)
30+
else:
31+
copyfile(apkname, os.path.join(PUBDIR, apkname))
32+
33+
34+

‎src/scenarios/genericandroidstartup/test.py

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
'''
2+
C# Console app
3+
'''
4+
from shared.runner import TestTraits, Runner
5+
6+
EXENAME = 'GenericAndroidStartup'
7+
8+
if __name__ == "__main__":
9+
traits = TestTraits(exename=EXENAME,
10+
guiapp='false',
11+
)
12+
Runner(traits).run()

0 commit comments

Comments
 (0)