Multiple MMMs Enhance Paid Media Decision-Making

Utilizing multiple marketing mix models (MMMs) is crucial for making more informed and reliable paid media decisions, particularly when allocating significant budgets. While a single MMM can generate charts showing channel decomposition, response curves, R-squared values, and reallocation recommendations, these outputs are heavily influenced by the model's inherent assumptions. Factors such as adstock and decay windows dictate the duration of a channel's impact, while saturation curves determine the rate at which diminishing returns set in, directly affecting reallocation suggestions. Furthermore, prior beliefs encoded through Bayesian or ridge regularization, along with seasonality and control variables, shape how much lift is attributed to external factors versus specific marketing channels. Altering these parameters can lead to divergent narratives from the same historical data, positioning a single model's output as a preliminary opinion rather than a definitive conclusion.

Running multiple MMMs on identical input data allows for the identification of uncertainties and the validation of recommendations by exposing how different modeling assumptions alter the final suggestions. This comparative approach is particularly valuable before committing to substantial budget allocations, such as seven-figure investments. The process aims to surface the range of possible outcomes dictated by varying assumptions, thereby increasing confidence in the chosen strategy. The article advocates for a multi-model MMM comparison as a vital step in the decision-making workflow.

When comparing MMMs, it is beneficial to distinguish between model validation and experimental validation. Incrementality tests, such as geographic lift, holdout, or on/off tests, offer a more robust causal verification by directly measuring whether a specific channel generated additional outcomes. However, these tests are typically limited to one channel at a time, incur higher costs, and necessitate planned variations in media spend. MMMs, conversely, operate at a broader level, inferring causal contributions across all channels simultaneously, including those that are difficult to test directly. They can also be rerun quickly and cost-effectively to validate results within days or weeks. The primary limitation of MMMs lies in their complete reliance on their own internal assumptions.

Ideally, MMMs and incrementality tests should complement each other. MMMs can generate and prioritize hypotheses regarding channel effectiveness, while experiments serve to confirm which of these hypotheses hold true. The confirmed findings from experiments can then be integrated back into the MMMs as priors, refining future model runs. Given that it is not feasible to conduct tests for every channel on a continuous basis, employing second and third MMMs becomes the subsequent line of defense for validating insights and ensuring the robustness of paid media strategies. This layered approach, combining the broad inference of multiple MMMs with targeted experimental validation, leads to more dependable and effective paid media decision-making.