Why doesn't Meridian have any specific forecasting functionality?

[FaneBastinBellroy]

Everything in the documentation talks about causal inference, ie. finding out what caused what in a segment of existing data. But to actually get any value of that information, we have to assume that the future is going to look like the past - otherwise none of that can inform our decisions about what to do next.

So then, why doesn't Meridian have any explicit forecasting ability, given that this is basically what anyone using Meridian wants to get out of it?

I would appreciate anything that can help my understanding here.

[eddieligo]

Meridian's causal inference framework helps understand cause-and-effect relationships between marketing activities and business results. By uncovering the causal impact of your past media spending, you can make more informed and strategic decisions for the future, leading to optimized budgets and improved ROI. Budget optimization for future media spend is not currently supported. However, to forecast future performance with different media spend scenarios, you can approximate this by selecting a previous time period in your data that is representative of the future time period and run BudgetOptimizer. For detailed instructions, please see GitHub - Budget Optimization Forecast #267.

The Meridian team is aware that budget optimization for future media spend is a request from many users, and it's an area we will consider exploring.

[rkgoyal26]

Budget Optimization for future media spend - Does Meridian has this functionality now ?? Or can we expect it in near future ?

[dattatreyam23]

Hi @rkgoyal26,

Thank you for reaching out to the Google Meridian support team.

Meridian doesn't currently support budget optimization for future media spend but this feature is coming soon! Please look out for the feature in our CHANGELOG.

If you would like to run the budget optimization for future data, you'll need future media, media_spend, and revenue_per_kpi data. For now, here are the steps to use this data for optimization.

1. Concatenate the new future data with the historical data used to train the model and create the InputData including both the historical and future data.
2. Create a ModelSpec object with the holdout_id selecting the future times of the data. This will allow you to train the model using the historical data but the model will have access to the future data as well.
3. Create a Meridian object with the above InputData (historical + future data) and ModelSpec (with holdout_id).
4. After training, create a BudgetOptimization object and run optimize(selected_times=(future_start_date, future_end_date)). This will only run the optimization on the future data.

Feel free to reach out if you have any questions or suggestions regarding Meridian.

Thank you,

Google Meridian Support Team

[sonriks6]

Hello @dattatreyam23

With release v1.0.9 the support for future optimizations was included, by passing a new_data argument. So I've created this DataTensor using future media and times in near future (but not included in the trained set)

new_data = analyzer.DataTensors( media=tf.expand_dims(media_tensor, axis=0), media_spend=tf.expand_dims(media_tensor, axis=0), revenue_per_kpi=tf.expand_dims(revenue_per_kpi_tensor, axis=0), time=time_tensor )

When running optimize module, it drops a warning saying 'media_spend' not expected so not used:

optimization_results = budget_optimizer.optimize( new_data=new_data, use_kpi=True, )

After optimize ends, I can't run .output_optimization_summary() because time_tensor strings are not present in TimeCoordinates!

What I'm doing wrong? Should I train model with forecast data included but using holdout_id ?

Thanks for your support!

[dattatreyam23]

Hi @sonriks6,

Thank you for contacting us.

The warnings regarding the spend data have been fixed with pr #691 and will be in the next patch.

As for the output_optimization_summary(), the visualizations do not yet support the use of new_data. You can access the non-optimized and optimized data in the OptimizationResults.nonoptimized_data and OptimizationResults.optimized_data. However, you will not be able to use the visualizations yet. We are working on this support and will update the CHANGELOGs when it becomes available.

In order to see the visualizations, please use the holdout_id workaround explained above.

Feel free to reach out if you have any questions or suggestions regarding the same.

Thank you,

Google Meridian Support Team

[Fin02070]

Hi @dattatreyam23 ,

I am having an issue with the holdout_id workaround explained above.

When running training a model with concatenated future data and holdout_id set for that future data, the media spend is still used to train the model, as noted in the Meridian documentation here. This is problematic because the future, falsified data changes the response curve which affects overall mode performance and budget optimization on future data.

Is this is known issue with this workaround, or do you have any suggestions on how more completely hold out the future data?
Thank you!

[sonriks6]

Hello,

We had more or less the same issue, we've considered the holdout_id approach so we feed the future media plans available from business side, and forecasted the non-media variables and also the target KPI using any time-series forecasting model (i.e Prophet). This way you can optimize this future media spends. You may refine the process to better predict those future values.

The second option you have is by using new_data argument in optimize() so you prepare a DataTensor with new media, new media_spend, new time_coordinates, and new revenue_per_kpi, this does not require any forecasted value but is not fully supported yet, you only have access to the non_optimized_data and optimized_data xarray datasets (which are more than enough in our use case).

[dattatreyam23]

Hi @Fin02070,

I’d like to inform you that the media data for the holdout observations are still used in the model training. This is necessary because media spend in one period affects adstock in subsequent periods, which might have some indirect effect on parameter estimation. For the falsified data, we recommend picking a time period that: 1) has a similar flighting pattern to what you anticipate in the future quarter; and 2) had normal spend volume on all channels preceding the time period, as mentioned in Issue #267.

The goal of Meridian is causal inference of the treatment variables, not forecasting into the future. We recommend using a holdout sample that is fairly balanced across geos and time periods. In other words, use a holdout sample that has approximately the same number of holdout observations for each geo and approximately the same number of holdout observations for each time period. Avoid holding out large, contiguous-in-time, chunks of data, such as at the end of the MMM time window, to assess forecast error in the KPI. Meridian isn't designed for forecasting the KPI, especially if there is strong trend and seasonality in the KPI. For more information, you may check our documentation here.

I would recommend using the new_data argument in the Optimizer class. The new_data argument allows one to use Meridian’s causal inference to estimate quantities for any hypothetical media execution or flighting pattern.

Feel free to reach out if you have any query or suggestion regarding the same.

Thank you

Google Meridian Support Team

[alexwakim]

The Meridian team just added a new page that's relevant to this discussion: https://developers.google.com/meridian/docs/post-modeling/scenario-planning-and-future-budget-optimization