Premium Analytics: Backfill WooCommerce Analytics lookup data once by louwie17 · Pull Request #53728 · Automattic/jetpack

louwie17 · 2026-10-16T14:27:11Z

Fixes #

Proposed changes

Stores that ran the standalone WooCommerce Analytics plugin before Premium Analytics hold order stats on WordPress.com but no product, tax or coupon lookup rows: the plugin strips that data from the shared woocommerce_analytics sync module. Nothing resends it after Premium Analytics takes over. Incremental sync only sends newly imported orders, the dashboard doesn't start a sync because the milestone is already set from the plugin's sync, and a checksum didn't repair it on a test store.

Add Sync\Lookup_Data_Backfill, registered from Configuration::configure_sync() so it runs only with WooCommerce active. It checks for a backfill:
- when the dashboard polls GET /jetpack/v4/sync/status (rest_post_dispatch, priority 11, after Sync_Status_Tracker sets the milestone from the stored full sync status at 10);
- when the Premium Analytics dashboard loads (current_screen + Analytics::is_dashboard_request()). This covers stores whose milestone was already set and whose dashboard no longer polls.
It schedules one Action Scheduler action when nothing has recorded the lookup data as synced, the milestone is set (an earlier analytics full sync finished), the standalone plugin's woocommerce_analytics_initial_full_sync_finished is set, wc_order_stats has rows and at least one lookup table has rows. The plugin's option is only set by that plugin and survives deactivation; without it Premium Analytics sent every order's lookup data itself. Without that option, with no orders or with empty lookup tables, it marks the data as synced and stops without asking WordPress.com.
Before starting, the action asks WordPress.com whether the history arrived. It takes the oldest completed order with product lookup rows and requests wpcom/v2/sites/<id>/analytics/reports/products (signed as the blog) for a day either side of its date_created, with date_type=created. The products report is built from the product lookup table joined with order stats, so the product is missing exactly when the lookup rows are. A completed order is never in the report's default excluded statuses. Only an old order can tell: orders synced after the takeover carry lookup rows.
- Product present with its type: mark the data as synced, no full sync.
- Product missing, present without a type (the catalog never reached WordPress.com), or no completed order to check: start a full sync with context premium_analytics_lookup_backfill: woocommerce_analytics, plus terms, term_relationships and posts limited to product and variation IDs when the store has products.
- No answer from WordPress.com, Jetpack disconnected, sync not allowed or another full sync running: retry every 10 minutes for up to a day. Without an answer by then, start the full sync anyway.
The full sync also resends the product catalog. WordPress.com builds its wc_product_lookup table (name, SKU, type, stock status) from synced product posts and their product_type terms, but only once its synced jetpack_connection_active_plugins lists an analytics plugin. A catalog synced before that has no rows there, so reports show products without names or types and type-filtered widgets such as Top performing products stay empty. Modules are sent in config order, so terms go before posts. posts gets only product and variation IDs, up to 5,000; above that it resends all posts, because full sync stores its config in the status option, rewrites it after every batch and sends it in jetpack_full_sync_start. term_relationships can't be limited, so it resends all relationships. The normal full syncs (site registration, the dashboard's initial sync, manual syncs) are unchanged.
Depends on WordPress.com seeing jetpack-premium-analytics in that option, which enabling Premium Analytics alone doesn't store; that fix is in a separate PR.
jetpack_premium_analytics_lookup_backfill records the run (scheduled_at, started_at), so it runs once per store and a second action for the same run does nothing.
A run that can't go on doesn't block the store: when the retry window ends, its action is canceled or fails before it starts, or another full sync replaces the backfill's before it finishes, the state option is cleared and the next dashboard check schedules a new run. A finished backfill sync whose end the tracker missed is marked synced from the stored status.
Sync_Status_Tracker sets jetpack_premium_analytics_lookup_data_synced when a full sync covering the whole woocommerce_analytics module ends, even when the milestone already exists (on these stores it came from the plugin's sync). A store whose first analytics full sync runs under Premium Analytics sets it there and never checks.
Load the woocommerce-packages Phan stubs for the Action Scheduler functions, and drop the suppressions they made unnecessary.

Follows the structure of RefundFixFullSync in woocommerce/woocommerce-analytics#1823.

Related product discussion/links

Does this pull request change what data or activity we track or use?

No. It resends lookup data Premium Analytics already syncs for new orders, and reads one products report from WordPress.com.

Testing instructions

Create a Jurassic Ninja store with WooCommerce and about 320 orders with tax and shipping, some of them completed.
Install and connect the WooCommerce Analytics plugin. Wait for its full sync to finish, so WordPress.com has order stats but no lookup data.
Install Jetpack from this branch, enable Premium Analytics, and deactivate the WooCommerce Analytics plugin.
Open the Premium Analytics dashboard. Under Tools › Scheduled Actions, a pending jetpack_premium_analytics_lookup_backfill action appears. Other admin pages don't schedule it.
Let it run (or run it from that screen). Check that a full sync starts with woocommerce_analytics, terms, term_relationships and posts (a list of product IDs) in its config and context premium_analytics_lookup_backfill:
wp eval 'print_r( \Automattic\Jetpack\Sync\Modules::get_module( "full-sync" )->get_status() );'
After it finishes, check that wp option get jetpack_premium_analytics_lookup_data_synced holds a timestamp and the product, tax and coupon reports on the WooCommerce tab include past orders, and Top performing products lists product names.
Reload the dashboard. No new action is scheduled and no further full sync starts.
To check the WordPress.com probe on a store whose history already arrived: wp option delete jetpack_premium_analytics_lookup_data_synced jetpack_premium_analytics_lookup_backfill, open the dashboard and run the action. It sets jetpack_premium_analytics_lookup_data_synced without starting a full sync.
On a store whose first analytics full sync ran under Premium Analytics, no backfill is scheduled.

🤖 Generated with Claude Code

github-actions · 2026-10-16T14:28:33Z

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

To test on WoA, go to the Plugins menu on a WoA dev site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin (Jetpack), and enable the add/premium-analytics-lookup-data-backfill branch.
To test on Simple, run the following command on your sandbox:

bin/jetpack-downloader test jetpack add/premium-analytics-lookup-data-backfill

Interested in more tips and information?

In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
Read more about our development workflow here: PCYsg-eg0-p2
Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

github-actions · 2026-10-16T14:28:44Z

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

✅ Include a description of your PR changes.
✅ Add a "[Status]" label (In Progress, Needs Review, ...).
✅ Add testing instructions.
✅ Specify whether this PR includes any changes to data or privacy.
✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

Follow this PR Review Process:

Ensure all required checks appearing at the bottom of this PR are passing.
Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
You can use GitHub's Reviewers functionality to request a review.
When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

Jetpack plugin:

No scheduled milestone found for this plugin.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

Premium Analytics plugin:

No scheduled milestone found for this plugin.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

jp-launch-control · 2026-10-16T14:42:31Z

Code Coverage Summary

Coverage changed in 2 files.

File	Coverage	Δ%	Δ Uncovered
projects/packages/premium-analytics/src/Sync/class-sync-status-tracker.php	68/75 (90.67%)	-0.51%	1 ❤️‍🩹
projects/packages/premium-analytics/src/Sync/class-configuration.php	38/43 (88.37%)	0.28%	0 💚

1 file is newly checked for coverage.

File	Coverage
projects/packages/premium-analytics/src/Sync/class-lookup-data-backfill.php	94/140 (67.14%) 💚

Full summary · PHP report · JS report

chihsuan

Nice catch on the missing lookup history! @louwie17 The happy path worked on a fresh store, and I left inline questions about the resend scope and stuck runs.

chihsuan · 2026-10-20T06:47:49Z

+				'woocommerce_analytics' => true,
+				'terms'                 => true,
+				'term_relationships'    => true,
+				'posts'                 => true,


Could we send only product and variation IDs here instead of posts => true? The posts module accepts an ID list, and WordPress.com only builds its product table from product and product_variation posts. On a test store with 25 products this resent 98 posts, so stores with large blogs would resend everything.

Good call, updated it to only send the product and variation IDs. I did add a cap of 5,000 IDs though, since full sync stores its config in the status option and rewrites it after every batch. Above that it falls back to resending all posts, which for a store that size are mostly products anyway.

Fixed in 16562de, and the 5,000 cap in 3d30b66.

chihsuan · 2026-10-20T06:47:49Z

+			}
+		}
+
+		if ( $can_retry ) {


Should we clear the state option when a run can't go on, so a later dashboard load schedules it again? I noticed two paths leave it set for good: the retry window ends while sync isn't allowed, or another full sync with an explicit module list replaces this one after started_at is set.

Nice catch on both! When the retry window runs out it now clears the state, so the next dashboard load schedules a new run. A started run also gets released on the next check if another full sync replaced it, and if our own sync finished but the tracker missed the end, it marks the data synced from the stored status instead.

Fixed in 16562de.

chihsuan · 2026-10-20T06:47:49Z

+		}
+
+		// Wait for the first analytics full sync; if it runs under Premium Analytics, the tracker marks the data synced.
+		if ( (int) get_option( Sync_Status_Tracker::INITIAL_ANALYTICS_SYNC_OPTION, 0 ) <= 0 ) {


Would it be worth also checking the standalone plugin's woocommerce_analytics_initial_full_sync_finished option here? It's only set by that plugin and survives deactivation, so stores that never ran it could skip the WordPress.com probe, and a wrong probe answer couldn't start a full sync on them.

Agreed, added this. Without that option Premium Analytics sent the lookup data itself, so we now mark it as synced without asking WordPress.com. The only edge case is a store that used the plugin's "reset initial full sync" tool before switching, which I think is fine to accept.

Fixed in 47863da.

chihsuan · 2026-10-20T06:47:50Z

 			}
 		);
-		if ( ! $active ) {
+		// An array of IDs is a partial sync, which leaves the other orders without lookup data.


nit: The woocommerce_analytics module reads an array config as a start_date / end_date range, not IDs (same for the partial analytics sync test row), so maybe:

Suggested change

// An array of IDs is a partial sync, which leaves the other orders without lookup data.

// An array config is a date range, a partial sync that leaves the other orders without lookup data.

Thanks, applied this and updated the partial analytics sync test row to use a date range as well.

Fixed in 16562de.

louwie17 · 2026-10-20T10:34:27Z

Thanks for the thorough review @chihsuan! Addressed all four inline, would you mind giving it another look? :)

chihsuan

Thanks for the updates! @louwie17 This looks ready to merge once the inline note is addressed. 👍

Just a thought, I wonder if it makes sense to move WooCommerce-specific sync code like this backfill to woocommerce-stats in a follow-up, keeping PA's sync generic for all consumers? #53282 already starts that, but it hasn't merged yet.

chihsuan · 2026-10-21T03:21:49Z

+		}
+
+		$state = get_option( self::STATE_OPTION );
+		if ( false !== $state && ! $this->forget_lost_run( $state ) ) {


Could we also release a scheduled run whose action is gone? It's an edge case: if the pending action is canceled in Tools › Scheduled Actions, or fails because Jetpack was off, the state stays at started_at: 0. Every later check then returns here, so the backfill never runs again.

Good catch, fixed in 977726d. A scheduled run is now released when its action is no longer pending or running, so the next dashboard load schedules a new one.

Stores that synced orders through the standalone WooCommerce Analytics plugin hold order stats on WordPress.com without product, tax and coupon lookup rows. Start one woocommerce_analytics full sync through Action Scheduler when an earlier analytics full sync finished, orders and lookup rows exist, and no full sync under Premium Analytics has sent the data yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…kfilling Check for a backfill when the dashboard polls /jetpack/v4/sync/status or loads, instead of on every admin_init. Before starting the full sync, ask WordPress.com's products report for the oldest completed order's product; when it is there, the history already arrived and no full sync starts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ress.com accepts WordPress.com rejected the probe with HTTP 400: date_type takes created, paid or completed, and add_query_arg() left the + of the date offsets unencoded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ckfill WordPress.com builds its product table from synced product posts and their product_type terms, and skips them while it doesn't know an analytics plugin is active. On a store that synced its catalog before then, reports show products without name, SKU or type, and type-filtered widgets stay empty. Add terms, term_relationships and posts to the backfill's full sync, and treat a reported product without a type as missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Send only product and variation IDs in the backfill's posts full sync, and skip the catalog modules when the store has no products. Forget a run whose retries ran out, or whose full sync another one replaced, so the next dashboard check schedules a new one; settle a finished run the tracker missed. Correct the tracker comment: an array config for woocommerce_analytics is a date range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… ran the standalone plugin Without the plugin's woocommerce_analytics_initial_full_sync_finished, Premium Analytics sent every order's lookup data itself, so mark the data synced without asking WordPress.com. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…is large Full sync stores its config in the status option, rewrites it after every batch and sends it in jetpack_full_sync_start, so above 5,000 product and variation IDs resend all posts instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A canceled or failed action left the state at started_at 0, so every later check stopped there and the backfill never ran again. Forget the run when no action is pending or running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

louwie17 · 2026-10-21T12:16:51Z

Thanks @chihsuan! Agreed on moving the WooCommerce-specific sync code to woocommerce-stats, that makes sense, I'll pick that up as a follow up once #53282 lands, given both touch the sync tracker.

chihsuan

Nice work @louwie17 Looks good 🚀

chihsuan · 2026-10-22T02:31:00Z

This PR needs a trunk merge or rebase.

Just a heads-up, I merged trunk into this branch to bring it up to date.

louwie17 added [Status] In Progress [Type] Bug labels Oct 16, 2026

louwie17 self-assigned this Oct 16, 2026

github-actions Bot added [Package] Premium Analytics [Plugin] Jetpack Issues about the Jetpack plugin. https://wordpress.org/plugins/jetpack/ [Plugin] Premium Analytics [Tests] Includes Tests labels Oct 16, 2026

louwie17 mentioned this pull request Oct 19, 2026

Premium Analytics: Refresh the stored connection slug when the dashboard turns on or off #53779

Closed

louwie17 marked this pull request as ready for review October 19, 2026 14:52

louwie17 requested a review from a team as a code owner October 19, 2026 14:52

louwie17 added [Status] Needs Review This PR is ready for review. and removed [Status] In Progress labels Oct 19, 2026

chihsuan reviewed Oct 20, 2026

View reviewed changes

louwie17 requested a review from chihsuan October 20, 2026 10:34

chihsuan reviewed Oct 21, 2026

View reviewed changes

louwie17 and others added 8 commits October 21, 2026 14:16

louwie17 force-pushed the add/premium-analytics-lookup-data-backfill branch from 7e8df5a to 977726d Compare October 21, 2026 12:16

louwie17 requested a review from chihsuan October 21, 2026 15:04

chihsuan approved these changes Oct 22, 2026

View reviewed changes

Merge branch 'trunk' into add/premium-analytics-lookup-data-backfill

0fa2554

	// An array of IDs is a partial sync, which leaves the other orders without lookup data.
	// An array config is a date range, a partial sync that leaves the other orders without lookup data.

Conversation

louwie17 commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Related product discussion/links

Does this pull request change what data or activity we track or use?

Testing instructions

Uh oh!

github-actions Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jp-launch-control Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Summary

Uh oh!

chihsuan left a comment

Choose a reason for hiding this comment

Uh oh!

chihsuan Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

louwie17 Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chihsuan Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

louwie17 Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chihsuan Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

louwie17 Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chihsuan Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

louwie17 Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

louwie17 commented Oct 20, 2026

Uh oh!

chihsuan left a comment

Choose a reason for hiding this comment

Uh oh!

chihsuan Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

louwie17 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

louwie17 commented Oct 21, 2026

Uh oh!

chihsuan left a comment

Choose a reason for hiding this comment

Uh oh!

chihsuan commented Oct 22, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

louwie17 commented Oct 16, 2026 •

edited

Loading

github-actions Bot commented Oct 16, 2026 •

edited

Loading

github-actions Bot commented Oct 16, 2026 •

edited

Loading

jp-launch-control Bot commented Oct 16, 2026 •

edited

Loading

louwie17 Oct 20, 2026 •

edited

Loading

louwie17 Oct 20, 2026 •

edited

Loading

louwie17 Oct 20, 2026 •

edited

Loading

louwie17 Oct 20, 2026 •

edited

Loading