Back to Courses
How to Calculate Incrementality by Channel and Tactic
As a recap, incrementaility measurement consists of experiments designed to systematically withhold media/ad exposure to a representative subset of users (the control group) while maintaining normal media/ad exposure to the broader user set (the test group).
By using targeting parameters available within publisher platforms to configure ad campaigns that serve a control treatment and a campaign treatment to similar audiences, publisher tracking and reporting can tell us the conversion rates for both the control audience and the campaign (test) audience. The difference in those conversion rates can be interpreted as incrementality.
The control group needs to represent a minimum percentage of the total test and control reach to be scientifically valid, but that number can vary based on a number of factors. The test group will receive your ad in question; the control group will not receive the ad message. There are multiple ways to build a control audience. Some examples are:
- Flight a placebo ad or PSA ad (PSA is Public Service Announcement ad or a generic ad irrelevant to your business) to an exact replica or mirror of the exposed audience.
- Hold out a subset of the target audience and suppress from the ad exposure.
- Programmatic executions have counterfactual bid loss data for the target audience that did not win the media auction. That cohort of target audience that didn’t win the auction can be used with the control group for testing.
To reveal incrementality, calculate the conversion difference between the test group vs the control.
Incrementality Calculation Formula
(Test Conversion – Control Conversion) / (Test Conversion) = Incrementality
Incrementality Test Calculation Example
Your test group saw 1.5% conversion whereas your control group saw a 0.5% conversion. The control data suggests that without any media exposure, you would have seen a 0.5% conversion. (Keep in mind that conversion could mean leads, sales, profit, or whatever metric that is important to your business.)
So (1.5% – 0.5%) / 1.5% = 66.7% incrementality in conversions
Incrementality is volatile and it is important to look at it longitudinally over time within business context. Statistical significance has two primary variables: difference in conversion rates (%CR) and sample sizes. The larger the difference in conversion rates, the smaller the sample you need. As a general rule, we like to see a control reach about 10% of the total test and control reach.
Incrementality Testing + CLV = Richer Insights
As digital spending increases, so does the cost to acquire customers, and ROI will not be positive from day one. The real value comes in after the customer converts, as the cost to re-engage a customer is very low and their future purchases make for positive ROI. So it is very important to understand the Customer Lifetime Value.
CLV is defined as the sum of the total profits over the lifetime of your customer or the discounted value of future profits generated by the customer. To determine CLV we typically look at both historical and future profits from the customer while accounting for seasonality.
Just like incrementality varies by channel, the CLV too varies with the intent and influence of these channels. The ability to apply incrementality to CLV and to break it down into baseline and incremental CLV is very useful. When the data is layered with channel level ROI/ ROAS, it can guide high-quality budget allocation and optimization decisions.
About Measured
Measured helps brands grow by measuring incremental media contribution to maximize performance results. Through a transparent experimentation approach that is always learning, Measured delivers ongoing actionable insights for marketers to increase efficiency and scale media for maximum growth. Experiments are powered by Measured’s privacy compliant Marketing Data Warehouse, which is purpose-built for analytics with data the customer owns. For more information, visit www.measured.com.