Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(e2e): Adds Feedback Widget Maestro E2E tests #4604

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

antonis
Copy link
Collaborator

@antonis antonis commented Feb 26, 2025

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

Note: I'm targeting main with this PR since the changes involve only additional tests. For the feedback related feature changes/additions (e.g.) I'll use feedback-ui-2

📜 Description

Adds Feedback Widget Maestro E2E tests

💡 Motivation and Context

Part of #4302

💚 How did you test it?

CI, Manual

📝 Checklist

  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

Copy link
Contributor

github-actions bot commented Feb 26, 2025

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 399.69 ms 407.27 ms 7.57 ms
Size 17.75 MiB 20.12 MiB 2.37 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
3261206 476.43 ms 465.96 ms -10.48 ms
5571a20 410.55 ms 441.06 ms 30.51 ms
255d28b 424.09 ms 441.96 ms 17.87 ms
1faf8e3 449.18 ms 432.08 ms -17.10 ms
690220d 468.28 ms 480.06 ms 11.78 ms
57448c5 443.47 ms 440.20 ms -3.26 ms
6e8584e 447.10 ms 474.71 ms 27.61 ms
5446992 403.40 ms 426.70 ms 23.30 ms
b7eb05d 435.98 ms 466.19 ms 30.21 ms
e5bc97b 438.96 ms 437.39 ms -1.57 ms

App size

Revision Plain With Sentry Diff
3261206 17.75 MiB 20.11 MiB 2.37 MiB
5571a20 17.73 MiB 19.93 MiB 2.19 MiB
255d28b 17.75 MiB 20.12 MiB 2.37 MiB
1faf8e3 17.74 MiB 20.08 MiB 2.34 MiB
690220d 17.74 MiB 20.08 MiB 2.35 MiB
57448c5 17.74 MiB 20.08 MiB 2.34 MiB
6e8584e 17.73 MiB 19.86 MiB 2.12 MiB
5446992 17.73 MiB 19.85 MiB 2.12 MiB
b7eb05d 17.75 MiB 20.12 MiB 2.37 MiB
e5bc97b 17.74 MiB 20.08 MiB 2.34 MiB

Copy link
Contributor

github-actions bot commented Feb 26, 2025

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1217.85 ms 1223.90 ms 6.04 ms
Size 2.63 MiB 3.75 MiB 1.12 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
ac41368+dirty 1226.65 ms 1237.90 ms 11.24 ms
457e29f+dirty 1253.94 ms 1269.18 ms 15.24 ms
ae2f6bc+dirty 1226.13 ms 1227.71 ms 1.58 ms
4a6664f+dirty 1209.49 ms 1208.63 ms -0.86 ms
4ed9c54+dirty 1215.39 ms 1224.66 ms 9.27 ms
94fc925+dirty 1231.12 ms 1223.85 ms -7.27 ms
d197b5c+dirty 1217.61 ms 1242.66 ms 25.05 ms
86818c7+dirty 1228.06 ms 1226.42 ms -1.64 ms
60ad48c+dirty 1214.22 ms 1227.41 ms 13.19 ms
b1e8712+dirty 1256.02 ms 1265.14 ms 9.12 ms

App size

Revision Plain With Sentry Diff
ac41368+dirty 2.36 MiB 3.14 MiB 793.46 KiB
457e29f+dirty 2.36 MiB 2.87 MiB 520.67 KiB
ae2f6bc+dirty 2.63 MiB 3.70 MiB 1.06 MiB
4a6664f+dirty 2.36 MiB 3.04 MiB 696.39 KiB
4ed9c54+dirty 2.63 MiB 3.75 MiB 1.12 MiB
94fc925+dirty 2.63 MiB 3.69 MiB 1.05 MiB
d197b5c+dirty 2.36 MiB 2.82 MiB 462.86 KiB
86818c7+dirty 2.63 MiB 3.75 MiB 1.12 MiB
60ad48c+dirty 2.63 MiB 3.75 MiB 1.12 MiB
b1e8712+dirty 2.36 MiB 2.84 MiB 488.84 KiB

Copy link
Contributor

github-actions bot commented Feb 26, 2025

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 422.13 ms 417.56 ms -4.56 ms
Size 7.15 MiB 8.39 MiB 1.23 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
c398f67+dirty 315.08 ms 345.60 ms 30.52 ms
5a22220+dirty 384.61 ms 419.06 ms 34.45 ms
a38594f+dirty 393.83 ms 422.12 ms 28.29 ms
4297324+dirty 385.33 ms 435.68 ms 50.35 ms
5852d77+dirty 378.83 ms 381.89 ms 3.06 ms
94fc925+dirty 365.44 ms 422.74 ms 57.30 ms
9cd0e9f+dirty 383.65 ms 418.65 ms 35.00 ms
acadc0f+dirty 259.04 ms 304.67 ms 45.63 ms
d43a46b+dirty 417.65 ms 472.98 ms 55.33 ms
397a69c+dirty 365.67 ms 347.08 ms -18.59 ms

App size

Revision Plain With Sentry Diff
c398f67+dirty 7.15 MiB 8.21 MiB 1.07 MiB
5a22220+dirty 7.15 MiB 8.21 MiB 1.06 MiB
a38594f+dirty 7.15 MiB 8.38 MiB 1.23 MiB
4297324+dirty 7.15 MiB 8.35 MiB 1.20 MiB
5852d77+dirty 7.15 MiB 8.38 MiB 1.22 MiB
94fc925+dirty 7.15 MiB 8.38 MiB 1.23 MiB
9cd0e9f+dirty 7.15 MiB 8.35 MiB 1.20 MiB
acadc0f+dirty 7.15 MiB 8.03 MiB 903.20 KiB
d43a46b+dirty 7.15 MiB 8.34 MiB 1.19 MiB
397a69c+dirty 7.15 MiB 8.39 MiB 1.23 MiB

Copy link
Contributor

github-actions bot commented Feb 26, 2025

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1228.27 ms 1233.98 ms 5.71 ms
Size 3.19 MiB 4.32 MiB 1.13 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
ac41368+dirty 1226.69 ms 1229.96 ms 3.27 ms
457e29f+dirty 1256.71 ms 1258.50 ms 1.79 ms
ae2f6bc+dirty 1220.82 ms 1217.93 ms -2.88 ms
4a6664f+dirty 1218.77 ms 1221.07 ms 2.30 ms
4ed9c54+dirty 1219.96 ms 1230.49 ms 10.53 ms
94fc925+dirty 1231.04 ms 1231.60 ms 0.55 ms
d197b5c+dirty 1234.80 ms 1249.20 ms 14.40 ms
86818c7+dirty 1239.80 ms 1239.82 ms 0.01 ms
60ad48c+dirty 1225.79 ms 1233.51 ms 7.72 ms
b1e8712+dirty 1284.11 ms 1297.82 ms 13.71 ms

App size

Revision Plain With Sentry Diff
ac41368+dirty 2.92 MiB 3.69 MiB 794.29 KiB
457e29f+dirty 2.92 MiB 3.43 MiB 524.75 KiB
ae2f6bc+dirty 3.19 MiB 4.26 MiB 1.08 MiB
4a6664f+dirty 2.92 MiB 3.60 MiB 702.09 KiB
4ed9c54+dirty 3.19 MiB 4.32 MiB 1.13 MiB
94fc925+dirty 3.19 MiB 4.25 MiB 1.07 MiB
d197b5c+dirty 2.92 MiB 3.37 MiB 464.41 KiB
86818c7+dirty 3.19 MiB 4.32 MiB 1.13 MiB
60ad48c+dirty 3.19 MiB 4.32 MiB 1.13 MiB
b1e8712+dirty 2.92 MiB 3.40 MiB 494.15 KiB

@antonis antonis changed the title test(e2e): Feedback Widget Maestro tests test(e2e): Adds Feedback Widget Maestro E2E tests Mar 5, 2025
@antonis
Copy link
Collaborator Author

antonis commented Mar 12, 2025

⚠️ Keeping this in draft till I figure out why the Android tests fail on CI (work locally) while iOS tests fail to detect input and button fields. This might be related with Maestro (e.g. issue 1, issue 2) and might require changes in the form.

iOS Android
Screenshot 2025-03-12 at 2 02 58 PM Screenshot 2025-03-12 at 1 59 49 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant