Add FXIOS-12039 [Homepage] section viewed telemetry

[cyndichin]

📜 Tickets

Jira ticket
Github issue

💡 Description

Replace item_viewed telemetry event (previous PR) with section_viewed labeled_counter.
Added a throttler to delay because when we scrollToTop it was still tracking impressions for the sections on the bottom when from the user perspectives they don't see those sections.
Separate logic for top sites items and sections.

Cases should be tested
For both new homepage + old homepage, impressions are triggered:
Load webpage -> tap on home button -> view homepage
Load webpage -> Add new tab from tab tray -> view homepage
Load webpage -> long pressing on tab tray to open new tab -> view homepage
Load webpage -> Add new tab from menu -> view homepage
Select homepage -> View webpage -> navigate back to homepage
Load webpage -> select on URL -> view homepage
On Homepage -> Add new tab from tab tray -> view homepage
On Homepage -> Add new tab from menu-> view homepage
Oh Homepage -> long pressing on tab tray to open new tab -> view homepage

Note: If on homepage, show modal (i.e. settings, menu), no impression is triggered

📝 Checklist

You have to check all boxes before merging

• Filled in the above information (tickets numbers and description of your work)
• Updated the PR name to follow our PR naming guidelines
• Wrote unit tests and/or ensured the tests suite is passing
• When working on UI, I checked and implemented accessibility (minimum Dynamic Text and VoiceOver)
• If needed, I updated documentation / comments for complex code and public methods
• If needed, added a backport comment (example @Mergifyio backport release/v120)

Video Capture

Telemetry_demo.mov

[mobiletest-ci-bot]

	Messages
📖	Project coverage: 36.04%
📖	Edited 12 files
📖	Created 1 files

Client.app: Coverage: 34.92

File	Coverage
Throttler.swift	91.67%	✅
TopSitesAction.swift	100.0%	✅
HomepageTelemetry.swift	98.28%	✅
TopSitesMiddleware.swift	94.07%	✅
HomepageViewController.swift	46.11%	⚠️
HomepageAction.swift	100.0%	✅
HomepageMiddleware.swift	100.0%	✅

Generated by 🚫 Danger Swift against 03e2145

[cyndichin]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?

Clicks and impression are required to understand Click through rate. Which helps us understand if the content we are showing to users is meeting a need or helping them complete a task. The contents usefulness or lack thereof will be measured by increases or decreases in click thru rate.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

Do users want us to show their bookmarks on the homescreen? Do they want us to recommend articles online they haven’t seen? Do we want to recommend items from history or previous searching sessions as quick links? Understanding Clicks/Impressions will be a core way we answer those questions

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

Previously, we added item_viewed, but this new metric simplifies the call and is sufficient for calculating CTR.

4. Can current instrumentation answer these questions?

Yes for the old homepage, we have existing telemetry events, but will be replacing those old telemetry events with these new ones. Also, we are replacing the metric item_viewed.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description	Data Collection Category	Tracking Bug #
homepage.section_viewed	2	FXIOS-12039

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

This collection is documented in the Glean Dictionary at https://dictionary.telemetry.mozilla.org/

7. How long will this data be collected? Choose one of the following:

I want to permanently monitor this data: @[email protected]

8. What populations will you measure?

All channels, countries, and locales. No filters.

9. If this data collection is default on, what is the opt-out mechanism for users?

User has the general option of disable telemetry or/and studies.

10. Please provide a general description of how you will analyze this data.

We will analyze aggregated information: items tapped from the homepage and the most common items that were tapped.

11. Where do you intend to share the results of your analysis?

Mozilla Internal (Product and Dev Team).

12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection? If so:

No. We will collect the info using Glean. We will analyze the data with the traditional Looker/SQL database that Mozilla uses for all other telemetry.

[cyndichin]

@mattreaganmozilla Throttler was added to the ClientTests target membership and so WindowSimpleTabsCoordinator was as well. I didn't see why they should be added so I removed it. Let me know if any issues!

[cyndichin]

@mergify backport release/v138

[mergify]

backport release/v138

✅ Backports have been created

• #26281 Add FXIOS-12039 [Homepage] section viewed telemetry (backport #26234) has been created for branch release/v138

[cyndichin]

unrelated testOpenTabsInSearchSuggestions also failing on other PRs

[ih-codes]

Just wanted to double check, you rolled back some of your past changes, so we still have the bug where if you open the tab tray, then close homepage tabs, we are logging section_viewed events despite not actually looking at the homepage? (I saw it in testing)

I just had a meeting today where I learned more about the baseline/metrics/events pings. Didn't realize labelled counters are on the metrics ping. We may want to add this to the baseline ping as well... and since it's a bit more work to build table joins between metrics and events we may want to do something extra to get the other homepage data into the same DB table. I have another meeting tomorrow to follow up on some of this. We already missed the v138 cutoff so if it's ok maybe we can keep this open til my meeting tomorrow afternoon just to see what the glean team suggests we do here. 🤔

[cyndichin]

Just wanted to double check, you rolled back some of your past changes, so we still have the bug where if you open the tab tray, then close homepage tabs, we are logging section_viewed events despite not actually looking at the homepage? (I saw it in testing)

I just had a meeting today where I learned more about the baseline/metrics/events pings. Didn't realize labelled counters are on the metrics ping. We may want to add this to the baseline ping as well... and since it's a bit more work to build table joins between metrics and events we may want to do something extra to get the other homepage data into the same DB table. I have another meeting tomorrow to follow up on some of this. We already missed the v138 cutoff so if it's ok maybe we can keep this open til my meeting tomorrow afternoon just to see what the glean team suggests we do here. 🤔

If you are referring to this this bug, then discussed with John that we don't need to address for v138. Waiting for results of tab tray experiment to decide which direction to address the issue.

Sounds good, this is targeted for weekly release, so as long as I can get it in for then, that's okay!

[ih-codes]

If you are referring to this this bug, then discussed with John that we don't need to address for v138. Waiting for results of tab tray experiment to decide which direction to address the issue.

Sounds good, this is targeted for weekly release, so as long as I can get it in for then, that's okay!

Ok thanks for the reminder about the bug, that was the one I believe! 👍

Update from meetings: It sounds like data science can do a join on the metrics and events tables so this shouldn't be a concern. Worst case is a PM will have to request a query from them. One possible suggestion was to also record both the counter AND the event... but I think just the counter is sufficient for his needs right now. So I will approve!

[cyndichin]

If you are referring to this this bug, then discussed with John that we don't need to address for v138. Waiting for results of tab tray experiment to decide which direction to address the issue.
Sounds good, this is targeted for weekly release, so as long as I can get it in for then, that's okay!

Ok thanks for the reminder about the bug, that was the one I believe! 👍

Update from meetings: It sounds like data science can do a join on the metrics and events tables so this shouldn't be a concern. Worst case is a PM will have to request a query from them. One possible suggestion was to also record both the counter AND the event... but I think just the counter is sufficient for his needs right now. So I will approve!

great! thanks for the follow up!

[cyndichin]

@adudenamedruby data request is ready for review - #26234 (comment) thanks!

[adudenamedruby]

r+

Data Review Form

1) Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, telemetry is documented at https://dictionary.telemetry.mozilla.org/apps/firefox_ios

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, Firefox iOS provides data controls in the app settings

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

This will be reviewed by a data-steward on the Firefox iOS team (@adudenamedruby) every 6 months

4) Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Cat 2

5) Is the data collection request for default-on or default-off?

Default on

6) Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7) Is the data collection covered by the existing Firefox privacy notice?

Yes

8) Does the data collection use a third-party collection tool?

No