PREDICTING CHURN
Chaitanya Sagar
Sinna Muthiah
Jagadish Singh Naik
What is Churn?
Customer churn is the loss of clients or customers (Wikipedia)
Why focus on Churn?
Customers
2x
Year 3
• 2% improvement in Churn can lead to 10% savings in costs
• Cost of acquisition of customers is 6-7 times higher than that of retaining customers
Source Matt Dancho
http://bit.ly/2uO723P
What Churn Analysis can give you Causes Timing
Customer Retention Strategy Segments
Source
Useful Machine Learning Algorithms & Packages
Logistic Regression Random Forest LightGBM
XGBoost
CatBoost
ANNs
Scikit Learn
Numpy
Pandas
Matplotlib/Seaborn
Keras
About KKBox
A Cloud-based music streaming service provider, with a ‘freemium’ pricing strategy
10 Million+ Users, 50 Million Songs, 1600 Artists
▪ Personalized Music Recommendation
▪ Ability to listen with friends and chat
▪ Streaming Live interviews and concerts
▪ Lyric Screening
Data
Data about 1 Mn. customers (approx. 32 GB)
It spans across 12 years & has 30 attributes in total New registrations by time (sampled data):
Data
Membership Data
Customer ID, Joining Date, Age, Gender, City etc.
Transactional Data
Is Transaction Auto Renewable, Is Transaction Cancelled, Validity of Subscription, Subscription Price, Actual
Amount Paid, Discounts etc.
User Log Data
Time spent, No. of unique songs heard, No. of songs heard, %length of songs heard,, Length of Playlist etc.
Exploratory Analysis: Insights
Registration Channel Auto Renewal Subscription Plan Price Time spent on platform
Feature Engineering
Why is ‘feature engineering’ important?
Methodology: (Motivated by insights from EDA)
• Contrasting abilities of ‘Transactional’ & ‘User-Log’ data patterns to capture churn -> We’ll use separate feature sets & ML models
• Dynamic customer behavior with time -> We’ll use temporal features
• Helps us capture customer behavior patterns with time
• Amplifies the distinction among churn & non churn customers
Feature Generation 1 Membership 3 User-Log 4 Transactional Cutoff Date Week 1 Week 2 Week 3 Week 4 Month 2 Month 5 • 8 Feature Sets • 260 Features • Age of user • Days as user • Gender • Auto Renew Ratio • Avg. Plan Price • Avg. Discounts • # Songs Heard • # Hours Spent • # Unique Songs
Modelling Methodology
Reasons to choose Ensemble approach?
• To bring together all the distinct patterns to predict churn
• Data Imbalance
1 2 3 5 8 9 10 4 6 7 10 Base Classifiers XGBoost LightGBM ANN Logistic Regression Model No. Feature Set Feature Set Type Algorithm 1 FS1 Membership LightGBM 2 FS2 User-Log 1 LightGBM 3 FS3 Transactional 1 LightGBM 4 FS3 Transactional 1 Neural Network 5 FS4 User-Log 2 LightGBM 6 FS5 User-Log 3 XGBoost 7 FS6 Transactional 2 Logistic Regression 8 FS6 Transactional 2 LightGBM 9 FS7 Transactional 3 LightGBM 10 FS8 Transactional 4 LightGBM
Model Architecture
Input
Train Data
Data Feature Generation
Demographics, Transactions, User history logs Data. Cleaned & Pre-processed
Test
Feature Generation Churn Prediction Model
Model Results
Model Number Model Type Feature Set Category Accuracy (%) F1 – Score (%) 01 LightGBM Membership 80.0 34.0 02 LightGBM User-Log 1 59.7 26.3 03 LightGBM Transactional 1 85.6 55.8 04 Neural Network Transactional 1 88.2 41.6 05 LightGBM User-Log 2 90.3 46.5 06 XGBoost User-Log 3 73.7 16.9 07 Logistic Regression Transactional 2 85.0 50.4 08 LightGBM Transactional 2 81.8 50.3 09 LightGBM Transactional 3 95.0 85.1 10 LightGBM Transactional 4 86.2 59.6 E3 Final Ensemble Ensemble data 96.0 86.5
Top 20 Important Features
Feature Name
Is Cancel ratio month 5- week 8
Is Cancel ratio week 8 - week 4
Is Cancel ratio week 4 - week 1
Is Cancel ratio month 1
Is cancel ration entire history
Auto renew ratio m5 - w8
Auto renew ratio w8 – w4
Auto renew ratio w4 – w1
Auto renew ratio w1
Auto renew ratio m1
Auto renew ratio entire history
Pay ratio
Length of first transaction
whether latest transaction canceled
User’s payment status in last month
Does last transaction has auto renew
Payment method of last transaction
Length of last transaction
Min days bw trans & cutoff dates m5-w8
Min days bw trans & cutoff dates w8-w4
Min days bw trans & cutoff dates w4-w1
4 Transactional
Insights to Business actions
Insight: A user is more likely to churn if she has more cancelled transactions.
Action: We can guide users to suitable, appropriate subscription plans during onboarding to reduce cancelations
Insight: A user is less likely to churn is she has more transaction frequency or opted for Auto-renew Option
Action: Encourage users to select auto renew option with some plan benefit and improve transactional experience by making transaction process hassle free.