An end-to-end machine learning and data engineering framework designed to systematically uncover user attrition risks and statistically validate targeted retention campaigns.
Built an end-to-end data science pipeline analyzing 7,043 customer accounts to preemptively predict customer churn and mathematically validate customer retention campaigns. Rather than stopping at traditional classification accuracy, this project implements a production-grade infrastructure that blends predictive machine learning with statistical experimentation (A/B testing) to provide reliable, data-driven business interventions.
Safely handles multi-type data schemas, coerces corrupt total expenditure features, maps binary classes, and constructs automated feature matrices.
Optimizes classifier ensembles using a 5-fold cross-validated grid search, prioritizing recall boundaries to catch maximum true anomalies.
Isolates high-risk populations by applying explicit validation thresholds across model-derived probability output arrays.
Measures proactive treatment effects directly against untreated customer variations to calculate true baseline conversion lift.
Developed isolated data engineering pipelines using Scikit-Learn’s ColumnTransformer and Pipeline paradigms. Evaluated Logistic Regression, Random Forest, and Gradient Boosting algorithms. Isolated a Tuned Random Forest Classifier via grid search, yielding a 0.8436 ROC-AUC and a 0.6331 F1-score, optimizing for recall to capture 77% of genuine churn incidents.
Leveraged the finalized ensemble model's predict_proba distribution layer to evaluate all active account records. Extracted an experimental sub-cohort of 1,496 high-priority profiles whose structural churn probability scored 70%, routing them directly into the downstream experimentation layer.
Using NumPy's random number generation frameworks, the isolated high-risk cohort was divided into a randomized controlled experiment:
• Control Group (A; n=757): Received no specialized interaction. Retained a baseline survival rate of 28.67%.
• Treatment Group (B; n=739): Offered a strategic retention incentive. Achieved a survival rate of 36.14%.
This automated setup actively eliminates assignment bias, allowing true extraction of marketing campaign utility.
Computed a two-proportion Z-test over the resulting experiment outcomes, revealing an absolute retention lift of +7.46%. The calculation yielded a score of p = 0.00203, falling safely below the alpha line (alpha = 0.05). This statistically proves that the intervention prevented 112 customer dropouts, converting model inference into concrete business value.