Question About Response Curve Issues

[gugu-tech]

Hello.
I have another question. I'm currently analyzing the effectiveness of TV and CATV media data, but these two channels are showing linear response curves, which makes it difficult to build proper budget optimization (since I can't find the saturation point without a curve). I've input the data using rf_media, and the Reach forms a sigmoid function with GRP Points, so there doesn't seem to be a problem with the data itself.

My assumption is that the response curve shows linearity because it only looks at the correlation between sales increment and media advertising costs without reflecting reach characteristics. If these two variables maintain proportionality, wouldn't it naturally show a linear relationship? Is my understanding correct?

Are there any parameter adjustment methods to solve this linear curve problem? Or should I just accept this linear proportional relationship? With the current results, it seems difficult to convince media planners and other stakeholders.

[cpulavarthi]

Hi @gugu-tech,

Thank you for reaching out to us!

In the response curves for R&F data, Meridian holds the frequency at its optimum value. Therefore, it's only reach that's increasing as spend increases. Meridian has a "linear reach assumption" which assumes reach has a linear effect on KPI. This is done as a simplifying assumption so that the model can be identifiable.

An alternative you may want to consider is to look at optimal frequency plots.
Another potential option is to also look at the response curves holding reach constant instead of holding frequency constant. This can be done with by_reach argument in response_curves. This gets the data, but doesn't plot the curve. You will have to plot it.

Feel free to contact us if you have any further queries regarding this.

Thank you

Google Meridian Support Team

[gugu-tech]

Thank you @cpulavarthi for your quick response.
However, I think my question wasn't clear enough, so I'm inquiring again. The problem is that, as shown in the attached image, the three rf_media graphs form "straight lines" rather than "curves," making it impossible to find the optimal cost level for advertising. This creates an issue where sales appear to increase endlessly as advertising costs increase.

Could this phenomenon be due to a data input error? Or did something get missed during the use of Meridian? Or is it a possible mutual phenomenon?

[cpulavarthi]

Hello @gugu-tech,

Thank you for sharing the visualization and elaborating your query!

For channels with R&F data, a response curve is actually two-dimensional (changing either R (Reach) or F (Frequency) can change incremental outcome). Meridian projects it into a 2D curve by holding frequency constant. That means only the reach changes on the x-axis. And since KPI is modeled as linearly dependent on Reach, the curve is linear.

We have other options as mentioned in our previous response.

Feel free to reach out for any further questions.

Thank you

Google Meridian Support Team

[gugu-tech]

Hello @cpulavarthi .
Thank you for your response. It was helpful in utilizing Meridian.
Based on this, I would like to share some information about MMM and ask a final additional question.

** 1. MMM results when entering Legacy Media as RF_Media **

Meridian seems to be designed based on digital media, but we tested the possibility of using Legacy media (TV, CATV) by utilizing reach and frequency.

In Meridian, reach must be entered as the number of people, but Legacy is usually estimated as a percentage, so we estimated the number of people reached by multiplying the target population of the area by the reach percentage.

The resulting Response Curve was expressed as a straight line as shown below. As advised by the Google Meridian Team, it was confirmed that it can be supplemented and used through other charts (optimal frequency graph, Reach-Spend graph). However, it is limited in that it is difficult to immediately use the optimal cost per media in the Budget Scenario.

** 2. MMM results when entering Legacy Media as Media **

To solve the problem in 1. (calculating the optimal cost per media in the Budget Scenario), the effect of Legacy media was converted to Impression and entered. (The existing reach and frequency were multiplied to calculate the Impression value). In other words, the effect of Legacy Media was converted to the effect in terms of Impression.

The media results obtained through this are as follows, and although there may be differences from the actual effect, valid results were obtained to some extent. The calculated Impression value reflected the decrease in reach due to advertising costs.

I think both have their own meaning. However, the current modeling of reach and frequency is overly simplified in setting the relationship with KPIs, and it is clearly regrettable that additional calculations are required to reflect and interpret them in the Budget Scenario. In the current situation, it is difficult to understand why modeling reach and frequency is necessary for prediction or modeling just by looking at the results.

The final additional questions are as follows:

1. What are the potential problems with using Meridian in the way described above?
   (In particular, there is concern that there may be a problem with calculating the reach/frequency media results by converting them into impressions.)

2. Is the reason for inputting reach and frequency as data to achieve a more accurate MMM? Is it more effective in actual calculations as in the paper? Judging from the time it takes to get actual results, it seems that when RF_Media is included as an input, the computer calculation speed takes several times longer, indicating higher computational complexity.

[cpulavarthi]

Hello @gugu-tech,

Thank you for sharing the context of your initial query! It helps us better understand your use case.

The advantage of the R&F model is that it provides estimates of optimal frequency and incremental outcome under any counterfactual R&F combination. The R&F model is NOT necessarily more accurate than a model with a single metric like spend or impressions (its accuracy depends on the plausibility of the linear reach assumption and accurate R&F data).

R&F also adds information to the model. The model is getting some more information about how the impressions were executed. Consequently (as you have observed), it does make the model more computationally expensive.

R&F modeling methodology assumes that reach has a linear relationship with the target KPI when holding frequency fixed which implicitly assumes that reach across different audiences (i.e., groups with different responsiveness to advertising) increases or decreases proportionally. We restrict the effect to be linear instead of using a non-linear transformation like hill function to avoid model overparameterization and non-identifiability. The poor identifiability makes it challenging to estimate the parameters well with any statistical method, and could lead to non-convergence in Bayesian framework. You may check the research paper on Bayesian Hierarchical Media Mix Model Incorporating Reach and Frequency Data to better understand the assumptions made and the reasoning behind these choices.

For Budget optimization and making marketing decisions, we suggest using response curves with varying spend as that will allow you to find the saturation point, which I believe is what you are looking for. Alternatively, as mentioned earlier, you can also explore plotting response curves with frequency by holding reach constant (by_reach argument in response_curves method allows you to get the data for the response curves which you may plot manually). Your method of finding the response curves using the product of reach and frequency will result in yet another interpretation for the visualization. In this case, the response curves' saturation points will tell you the number of impressions beyond which the incremental revenue increases aren't as high (depending on your threshold). You may choose any one of these methods for your analysis. You would need to interpret the results accordingly and ensure they make business sense for your use case and your decision making process.

Your queries show your interest in understanding the modeling decisions in detail, I therefore highly recommend that you take a look at the research paper mentioned above as I believe that will help you better understand the need for these simplifying assumptions.

Feel free to reach out if you need any further clarifications regarding this.

Thank you

Google Meridian Support Team