MIM: Concepts and Overview
Getting Started
The Measured Incrementality Model (MIM) is the engine behind the insights that Measured provides into your marketing performance. The foundation is your media mix model, triangulates multiple sources including:
- Your marketing performance data
- Your business's transactional data
- Results from your incrementality tests
By combining these factors, you get a comprehensive view of how effective your media spending is across your entire portfolio.
They inform your MIM so it can define the incrementality adjustment factor we use to correct metrics reported by your integrated platforms. This adjustment factor is used for for forecasting and optimization throughout Measured.
Go to our product video hub for brief overviews on the Measured features that MIM powers.
What's New in MIM
The new version of MIM uses your media mix model to measure your entire media portfolio, instead of just your tactics that are not able to be tested like the previous version. All of your integrated marketing channels will now be covered by this comprehensive measurement method.
Since MIM can now cover testable tactics, the results from tests you run on your tactics will be automatically used to calibrate your specific model. This version of the model will integrate positive results from any holdout tests that have been completed. Future iterations will expand the range of tests that can be drawn upon for calibration.
Additionally, ad stock effects are now incorporated to provide more realistic reporting on your channels. This means that the gradual decrease in impact from your campaigns that have ended will be taken into account, instead of them having an immediate drop-off.
Finally, over-attribution at the bottom of the funnel is prevented by using industry-established insights as prior knowledge in the model. This leads to more accurate attribution for any previous touchpoints before your customer finally converted.
Enabling MIM
Reach out to your customer success partner to review the latest model results, then let them know when you’re ready to enable it. In addition to the benefits that were previously mentioned, MIM will be a key part of upcoming Measured updates.
For instance, we will soon be showing visualizations of the diminishing returns stemming from what you are spending on your marketing. MIM will be necessary for these calculations.
What is MIM?
MIM provides full-funnel, causal measurements of your media spending. This information becomes the suite of Measured applications, from the metrics on your Homepage to the recommended budget plans in Media Plan Optimizer.
- Each week, your media mix model will for each of your conversion events. It will draw upon up to three years of your marketing data, plus your business data and experiments. It will comprehensively measure all of your channels and tactics. The more tests you run, the more calibrated and accurate your media mix model gets.
- Then, MIM will calculate the ratio of incremental orders from your media mix model to platform-reported orders from the last 52 weeks for every tactic in your portfolio. This ratio is used as the adjustment factor, which powers your reporting, optimizations, and forecasting throughout Measured.
- Finally, for any of your tactics with automatic weekly updates enabled, the new adjustment factor will be applied across all Measured features every Thursday morning. It will replace the prior adjustment factor with the latest model results.
For tactics using on-demand updates, you can view their recent incremental performance with the most up-to-date adjustment factors in the MIM Explorer. You can then request that they are applied to your reporting through your customer success partner whenever you see fit.
This multi-step approach spans both testable channels like social, search, and email, as well as non-testable channels like affiliate, radio, and podcasts. As a result, your entire portfolio can be measured granularly on the channel, tactic, and campaign levels.
How Is MIM Calibrated?
With the latest version of the Measured Incrementality Model, many of your geo tests will now automatically calibrate your media mix model. The effect of each test on the model depends primarily on the confidence in the test's results, how recent the test was, and the strength of other competing signals in the model (such as your tests on other tactics).
This version of the model will use positive results from any holdout tests for calibration. Future iterations will expand the range of tests that can be drawn upon to inform your model. The result of model calibration will be an updated adjustment factor for reporting and updated diminishing return curves.
This calibration process also applies to the results of multi-tactic tests. These tests have an adjusted method to determine the most accurate results for all tactics involved. Learn more about multi-tactic testing here.
After a geo test finishes and your model is calibrated, you can apply the latest adjustment factor(s) to your reporting with either the next automatic weekly update or when you next make an on-demand update request, depending on the mode of update you have chosen.
MIM In Action
Here's a common example of how the above factors are used to calibrate an incrementality model for a specific company:
- A brand uses their media mix model-powered MIM to make informed investment decisions, including where to run a geo test and what type of test it should be.
- Based on their data and business objectives, they plan and launch a channel geo test for Facebook, and the results show the incremental performance, or how effective the channel truly is.
- While useful, channel-level results aren't helpful with tactical optimizations, so the brand uses their MIM to get tactic-level insights out of the Facebook channel test.
- The brand then applies those learnings to their platform reporting through updated adjustment factors. These updates power the incremental metrics they see in Measured, and help the brand make optimization decisions with confidence.
Automatic & On-Demand Updates
Your calibrated MIM and the media mix model that power it will update each week on Thursday at approximately 8 AM EST.
However, you can configure how these updates affect your Measured experience. MIM gives you control over when and where your latest model results are applied for each tactic:
- Automatic weekly updates will take any new changes to your adjustment factor and apply them across your account each week. These will ensure you are using the latest model results, including up-to-date return curves, recent test results, and new marketing channels.
- On-demand updates let you determine your own update cadence. You can use the MIM Explorer to review your most recent model results and manually request updates to your adjustment factor. You can also specify how your latest adjustment factor will be applied retroactively.
Note: These options can be applied on a tactic-by-tactic basis. It is very common to have some tactics updated automatically and others strictly upon request.
If you have questions about which of these methods is right for you, your customer success partner can help you explore your options and switch between them as necessary.
Why Adjustment Factors Change
The adjustment factor represents the accuracy of the reporting from your integrated platforms. It is defined as the ratio of incremental orders from your media mix model to the amount of orders originally reported by your marketing channels.
Each week, your media mix model runs using the last three years of data (if available). This replaces the oldest week in the model with the latest week, and will add any new test results and new marketing channels.
However, the biggest driver of change to your adjustment factor is in your integrated ad platforms, where the accuracy can change significantly over time. This is why many marketers use a combination of automatic weekly updates and on-demand updates to their adjustment factors.
Note: If you see an incrementality percentage of 0%, this means the vendor reported no conversions during your chosen timeframe.
FAQs
Q: Will the Measured Incrementality Model be responsive to changes to my marketing?
A: MIM leverages media mix modeling to find the incremental contribution to your sales, and then uses reporting from your ad platforms to highlight ideal short-term optimizations. This makes MIM responsive to changes in your performance, while ensuring your decisions have a solid foundation in incrementality.
Q: What conversion events are included in the media mix model?
A: All conversion events are included in the media mix model except for “roll-up” events. Those “roll-up” conversion events aggregate the spend and incremental orders from the component conversion events in Measured's reporting.
Q: Is MIM a weighted average of multiple inputs? Can I see those separate inputs?
A: No, it is not a weighted average. The only inputs are your tests, your media mix model and the reporting from your ad platforms. MIM uses your media mix model, calibrated by your tests, to measure the incremental contribution of your tactics to sales.
Q: Are you incorporating diminishing returns, seasonality and pricing in my media mix model?
A: Yes. Seasonality, pricing, and diminishing returns are included in your media mix model. Each week, a baseline conversion value is derived from the model to account for seasonality and pricing, while diminishing returns are imposed on your model.
Q: With on-demand MIM updates, when should I apply the latest model results?
A: For any tactics you have chosen manual MIM updates for, we recommend updating the adjustment factors at least once a quarter. It is also best practice to update them after new geo test results, adding new channels or tactics, and when a new version of MIM is released.
Q: How does MIM deal with the "cold start" problem - tactics with limited or no spend history?
A: The Measured Incrementality Model has a special feature for new tactics that leverages media mix models and testing insights from other brands to estimate initial incrementality while we accumulate signal and refine the answer.
Q: How are test results used to calibrate the media mix model?
A: Your test results factor into your media mix model as a definitive signal of your tactics' performance. Since we directly observed your results happening due to real world conditions, we can say that these are the most accurate tactic measurements to factor in.
Q: How are you measuring the accuracy of the media mix model?
A: The media mix model uses standard statistical accuracy metrics such as VIF, R^2, and MSE.
