Prior ROI Mismatch When Specifying Calibration Period in MediaSummary

[violasiii]

I'm encountering discrepancies in prior ROI values when specifying a calibration period for channels with lift tests. Here's the detailed scenario:

​Expected Behavior​
When setting a prior ROI for a channel and calibrating it to a specific period (e.g., lift test dates), the MediaSummary table should reflect the prior ROI close to the manually defined value.

​Actual Behavior​
​Without calibration period: Prior ROI matches the explicitly set value.
​With calibration period: Prior ROI deviates significantly from the set value.

​Reproduction Steps​

1. Define a prior ROI for a channel with lift test data.
   roi_m = build_media_channel_args( googleAds_SEARCH=estimate_lognormal_dist(2.0,0.3) )
2. Verify ROI via MediaSummary without time filtering - works as expected.
   media_summary = visualizer.MediaSummary(mmm)
media_summary.summary_table()
3. Specify selected_times corresponding to the lift test period.
   roi_period = { 'googleAds_SEARCH': [ '2025-03-17', '2025-03-24' ] }
roi_calibration_period = np.zeros((len(data.time), len(data.media_channel)))
for i in roi_period.items(): roi_calibration_period[ np.isin(data.time.values, i[1]), data.media_channel.values == i[0] ] = 1
roi_calibration_period[ :, ~np.isin(data.media_channel.values, list(roi_period.keys())) ] = 1
4. Observe distorted prior ROI values in the summary table.
   media_summary = visualizer.MediaSummary( mmm, selected_times=[ '2025-03-17', '2025-03-24' ] )
media_summary.summary_table()
   ​Key Questions​
   Is there a recommended way to lock priors to user-defined values while calibrating posteriors to specific periods?

[dattatreyam23]

Hi @violasiii ,

Thank you for contacting us.

The reason the results do not align is because ModelSpec.roi_calibration_period and MediaSummary.selected_times have different functions.

• roi_calibration_period sets the time window of the media whose effect is being captured in the ROI numerator.
  • In your case, the ROI numerator is the difference in expected revenue between the counterfactual where media is set to historical values against the counterfactual where Search is set to zero during weeks 2025-03-17 and 2025-03-24. The incremental revenue is summed over the entire modeling time window, which includes up to max_lag time periods after 2025-03-24.
• selected_times in MediaSummary sets the time window over which the incremental outcome is captured in the ROI numerator.
  • In your case, the ROI numerator is the incremental revenue generated during 2025-03-17 to 2025-03-24 by all Search execution during and up to max_lag time periods before this window.
• In both cases, the ROI denominator is the spend during 2025-03-17 to 2025-03-24.
• By default roi_calibration_period is set to the full modeling time window, so the ROI numerator of the prior aligns with MediaSumary.

There is not a way to obtain the calibration window ROI prior or posterior distribution using MediaSummary, but there are other options:

• Extract Meridian.inference_data.prior.roi_m and Meridian.inference_data.posterior.roi_m.
• Call Analyzer.incremental_outcome() with media_selected_times set to align with roi_calibration_period. You can set the use_posterior argument to obtain either the prior or posterior distribution. Incremental outcome is the ROI numerator, so to get the ROI you need to divide by the Search spend during 2025-03-17 to 2025-03-24.

Feel free to reach out if you have any questions or suggestions regarding Meridian.

Thank you,

Google Meridian Support Team