Skip to content

ci(e2e): run the Android signup e2e test on a CI emulator with video evidence - #16

Open
arnoldcastro5000 wants to merge 1 commit into
Greenstand:mainfrom
arnoldcastro5000:e2e/android-emulator-ci
Open

ci(e2e): run the Android signup e2e test on a CI emulator with video evidence#16
arnoldcastro5000 wants to merge 1 commit into
Greenstand:mainfrom
arnoldcastro5000:e2e/android-emulator-ci

Conversation

@arnoldcastro5000

Copy link
Copy Markdown

Summary

This adds an automated test that runs our Android app by itself on GitHub
Actions (the free CI system built into GitHub). Nobody needs a phone, and
nobody needs to set up a laptop. It is the first GitHub Action in this repo.

When it runs, it does this on its own:

  • builds the Android app
  • starts a phone emulator (a virtual Android phone) on GitHub's servers
  • installs the app on that emulator
  • runs the 02_signup_flow test from start to finish
  • saves a video and step-by-step screenshots as a downloadable file

The 02_signup_flow test does the full offline signup, the same way a real
new user would: pick a language, accept the privacy policy, enter a phone
number, enter a name, read the selfie tutorial, take a selfie (the emulator
gives a fake camera image), review and approve, and arrive at the main
dashboard (UPLOAD / MESSAGES / TRACK). The test passes from start to finish.

This PR changes:

  • .github/workflows/android-e2e.yml - the CI job. It builds the app, boots
    the emulator, runs the test, and uploads the video and screenshots.
  • apps/e2e/utils/artifacts.ts - records the video and the per-step
    screenshots.
  • apps/e2e/utils/helpers.ts - the position-based tap helpers that drive the
    app's custom buttons (see the note below).
  • apps/e2e/wdio.conf.ts - connects the video and screenshot recording to the
    test run.
  • apps/e2e/features/step-definitions/steps.ts - small step wiring.
  • apps/e2e/.env.example - example settings, so others can run it locally.
  • .gitignore - keep the apps/e2e/.env.example template tracked.

How to run it

  • It runs by itself on any pull request that changes apps/e2e/, the Android
    app, or this workflow. You will see the result as a check on the PR.
  • It also runs when you push to a branch whose name contains android-e2e.
  • You can also start it by hand: open the Actions tab, pick "Android E2E", and
    click "Run workflow".
  • When it finishes, open the run and download the e2e-evidence file. Inside
    are the video and the screenshots.

One thing reviewers should know

Our app's buttons are built in a way that hides them from the test tool. The
test tool cannot find a button by name. It can only tap a position on the
screen. So this test taps buttons by position (for example, "tap 82% across
and 88% down"). This works, but it depends on the screen layout. If the layout
changes, these positions may need small adjustments.

The clean long-term fix is a small change inside the app: give the shared
button a test "tag" (an id the test tool can see). One change to the shared
button would make every screen testable by name, and we could then remove the
position-based taps. That is a follow-up, not part of this PR.

Note: 02_signup_flow is fully offline and needs no secrets, so it works even
on pull requests from a fork. A later test, 03_capture_setup, will need real
cloud (AWS) secrets. Pull requests from forks do not get repo secrets, so that
test will need to run after merge, or use a trusted trigger.

Testing

Green in CI on a personal fork (arnoldcastro5000/treetracker). The last run
passed all 16 checks with result "success", and the dashboard screenshot showed
UPLOAD / MESSAGES / TRACK after signup finished.

Ran by hand:

# start the workflow on the fork, watch it, and confirm it passes
gh workflow run "Android E2E" --ref ci/android-e2e-emulator
gh run watch

The uploaded e2e-evidence artifact holds the full-scenario video and the
per-step screenshots for that run.

Add a GitHub Actions workflow that builds the debug APK, boots an
accelerated Android emulator, and runs the apps/e2e WebdriverIO suite
(02_signup_flow) end to end. It uploads a full-scenario video and
per-step screenshots as a build artifact.

The app's custom Compose buttons expose no automation ids, so the suite
drives them by screen coordinate. Coordinate-tap helpers, screen
recording, and per-scenario hooks are added under apps/e2e, along with
an .env.example template and a Windows local-run guide.
@arnoldcastro5000

Copy link
Copy Markdown
Author

Note for reviewers about the CI check.

This is the first GitHub Actions workflow in the repo, and it comes from a
fork. GitHub does not run a workflow from a fork by itself. A maintainer needs
to click "Approve and run" on the Checks tab of this PR to start the
emulator run. (If Actions is not enabled on the repo yet, that has to be turned
on first, in Settings, Actions, General.)

Until it is approved here, the proof is the run on my fork, which passed all 16
checks with result "success". The workflow uploads a full run video plus
per-step screenshots as the e2e-evidence artifact, so you can watch the whole
signup happen.

02_signup_flow is fully offline and uses only placeholder keys, so it needs
no repo secrets and is safe to approve on a fork PR.

@arnoldcastro5000

Copy link
Copy Markdown
Author

@dadiorchen for your review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant