Uplift modelling

Measuring uplift

The uplift of a marketing campaign is usually defined as the difference in response rate between a treated group and a randomized control group. This allows a marketing team to isolate the effect of a marketing action and measure the effectiveness or otherwise of that individual marketing action. Honest marketing teams will only take credit for the incremental effect of their campaign.

However, many marketers define lift (rather than uplift) as the difference in response rate between treatment and control, so uplift modeling can be defined as improving (upping) lift through predictive modeling.

The table below shows the details of a campaign showing the number of responses and calculated response rate for a hypothetical marketing campaign. This campaign would be defined as having a response rate uplift of 5%. It has created 50,000 incremental responses (100,000 − 50,000).

Traditional response modelling

Traditional response modelling typically takes a group of treated customers and attempts to build a predictive model that separates the likely responders from the non-responders using one of a number of predictive modelling techniques, such as decision trees or regression analysis.

This model uses only the treated customers to build the model.

In contrast uplift modeling uses both the treated and control customers to build a predictive model that focuses on the incremental response. To understand this type of model it is proposed that there is a fundamental segmentation that separates customers into the following groups (their names were suggested by N. Radcliffe and explained in ):

• The Persuadables: customers who only respond to the marketing action because they were targeted
• The Sure Things: customers who would have responded whether they were targeted or not
• The Lost Causes: customers who will not respond irrespective of whether or not they are targeted
• The Do Not Disturbs or Sleeping Dogs: customers who are less likely to respond because they were targeted

The only segment that provides true incremental responses is the Persuadables.

Uplift modelling provides a scoring technique that attempts to separate customers into these groups.

Traditional response modelling often targets the Sure Things, being unable to distinguish them from the Persuadables.

Return on investment

Because uplift modelling focuses on incremental responses only, it provides very strong return-on-investment cases when applied to traditional demand generation and retention activities. For example, by only targeting the persuadable customers in an outbound marketing campaign, the contact costs and hence the return per unit spend can be dramatically improved.

Removal of negative effects

One of the most effective uses of uplift modelling is in removing negative effects from retention campaigns. In telecommunications and financial services industries, retention campaigns can trigger customers to cancel a contract or policy. Uplift modelling allows these customers — the Do Not Disturbs — to be removed from the campaign.

Application to A/B and multivariate testing

It is rarely the case that there is a single treatment and control group. Often the "treatment" can be a variety of simple message variations or a multi-stage contact strategy that is classed as a single treatment. In the case of A/B or multivariate testing, uplift modelling can help determine whether the variations in tests provide any significant uplift compared to other targeting criteria such as behavioural or demographic indicators.

Advertising-incrementality application

In the field of digital advertising, uplift modelling is increasingly used as part of incrementality measurement, which aims to estimate the causal effect of a campaign — that is, the change in outcomes attributable to the marketing treatment — rather than simply predicting response or conversion likelihood. In this context, uplift models may be used either (a) to predict which customers are most likely to generate incremental lift if treated, or (b) to calibrate or validate results from experimental hold-out designs in which a randomly selected control group is withheld from treatment, allowing the true causal lift to be measured.

History of uplift modelling

The first appearance of true response modelling appears to be in the work of Radcliffe and Surry.

Victor Lo also published on this topic in The True Lift Model (2002), and later Radcliffe again with Using Control Groups to Target on Predicted Lift: Building and Assessing Uplift Models (2007).

Radcliffe also provides a frequently asked questions (FAQ) section on his website, Scientific Marketer. Lo (2008) provides a more general framework, from program design to predictive modeling to optimization, along with future research areas.

Independently uplift modelling has been studied by Piotr Rzepakowski. Together with Szymon Jaroszewicz he adapted information theory to build multi-class uplift decision trees and published the paper in 2010. And later in 2011 they extended the algorithm to the multiple-treatment case.

Similar approaches have been explored in personalised medicine.

Szymon Jaroszewicz and Piotr Rzepakowski (2014) designed uplift methodology for survival analysis and applied it to randomized controlled trial analysis.

Yong (2015) combined a mathematical optimization algorithm via dynamic programming with machine learning methods to optimally stratify patients.

Uplift modelling is a special case of the older psychology concept of differential prediction.

Uplift modeling has been recently extended into diverse machine learning approaches, including inductive logic programming, Bayesian networks, Statistical relational learning, Support-vector machines, Survival analysis, and Ensemble learning.

Even though uplift modeling is widely applied in marketing practice (along with political elections), it has rarely appeared in marketing literature. Kane, Lo and Zheng (2014) published a thorough analysis of three data sets using multiple methods in a marketing journal and provided evidence that a newer approach (the "Four-Quadrant Method") performed well in practice.

Lo and Pachamanova (2015) extended uplift modeling to prescriptive analytics for multiple treatment situations and proposed algorithms to solve large deterministic and stochastic optimization problems.

Recent research analyses the performance of various state-of-the-art uplift models in benchmark studies using large data amounts.

A detailed description of uplift modeling, its history, uplift-specific evaluation techniques, software comparisons, and different economic scenarios can be found in:

Implementations

In Python

• CausalML, implementation of causal inference and uplift algorithms
• DoubleML, implements Chernozhukov et al.'s double/debiased ML framework
• EconML, tools for heterogeneous treatment effect estimation
• UpliftML, scalable uplift modeling from experiments
• PyLift (archived)
• scikit-uplift, sklearn-style uplift modelling

In R

• DoubleML, same framework
• uplift package (removed from CRAN in 2022)

Other languages

• JMP by SAS
• Portrait Uplift by Pitney Bowes
• Uplift node for KNIME by Dymatrix
• Uplift Modelling in Miró by Stochastic Solutions

Datasets

• Hillstrom Email Marketing dataset
• Criteo Uplift Prediction dataset
• Lenta Uplift Modeling Dataset
• X5 RetailHero Uplift Modeling Dataset
• MegaFon Uplift Competition Dataset