Forecasting customer churn: Comparing the performance of statistical methods on more than just accuracy

Abstract

There is a consensus that the best way to forecast customer churn is by statistical methods. It is, however, unclear when which statistical method is more appropriate. This study aims to provide a set of guidelines to data scientists and researchers who are interested in optimizing statistical methods. A systematic literature review revealed six most promising methods for churn forecasting and a selection of metrics which can be used to evaluate the performance of the methods. The six statistical methods are evaluated on five metrics of performance. The best-worst method (BWM), a multi-criteria decision-making method, is used to elicit the relative importance of the performance metrics. Based on the relative importance of the metrics and the performance of the methods, using additive value function, we find an overall value for each method based on which the forecasting methods can be ranked. Experimental analysis reveals that finding an overall value for each statistical analysis leads to a different ranking than when we use a single performance metric like accuracy or AUC. We argue that relying on an aggregated value, like the one we propose in this study, is more reliable than considering only one metric (a common practice).