Telco Customer Churn

01 — Overview

Customer churn is a binary-classification problem in which the goal is to identify predictive patterns associated with accounts labeled Churn = Yes.

This study analyzes 7,043 customer accounts using 19 model features covering customer profile, tenure, subscribed services, contract terms, billing, payment method and charges. customerID is treated only as an identifier and is excluded from modeling.

The objective is predictive rather than causal: the study asks whether the observed customer-account characteristics contain useful signal for distinguishing churn from non-churn. It does not establish that changing an individual feature would change future customer behavior.

The workflow was designed as a reproducible scientific study, covering source verification, deterministic preparation, holdout definition, train-only exploratory analysis, model-family comparison, predefined model selection, final holdout evaluation and reproducibility evidence.

The training partition used for target-aware analysis contains 4,930 accounts, with 3,622 observations belonging to Churn = No and 1,308 to Churn = Yes. This corresponds to approximately 26.5% positive examples and helps motivate the use of metrics that emphasize retrieval of the minority class, particularly Average Precision.

02 — What the Data Revealed

Target-aware exploratory analysis is performed on the training partition only, containing 4,930 customer accounts.

One of the clearest patterns concerns contract type. Observed churn differs substantially across contract categories:

Month-to-month — 43.15%
One year — 10.60%
Two year — 2.94%

Month-to-month accounts show considerably higher observed churn than accounts with longer contracts. This is predictive evidence, not proof that moving a customer to a longer contract would independently prevent churn.

Tenure shows another strong pattern. Across the tenure groups analyzed by the study, observed churn falls from approximately 59.5% among the shortest-tenure accounts to approximately 3.5% among the longest-tenure accounts.

Tenure contains substantial predictive signal, but it is also mechanically related to how long an account has already remained active. The relationship therefore should not be interpreted as evidence of a causal retention effect.

Contract type, technical support, online security, internet service and payment method are among the strongest categorical associations with churn.

Within the training data, the strongest categorical associations include Contract (Cramér’s V 0.417), TechSupport (0.345), OnlineSecurity (0.340), InternetService (0.315), and PaymentMethod (0.308).

These relationships are used as exploratory evidence only. The ranking is not used for feature selection: all 19 predictors defined by the scientific feature contract are retained for modeling.

03 — Modeling & Result

The study compares a prior-only Dummy baseline with Logistic Regression, Decision Tree, Random Forest and HistGradientBoostingClassifier.

Because Churn = Yes represents approximately 26.5% of the observations, Average Precision is used as the primary model-selection metric. Candidate models undergo hyperparameter search using five-fold stratified cross-validation on the training partition and are then evaluated once on validation data.

The search policy evaluates 24 Logistic Regression configurations, 48 Decision Tree configurations, 40 Random Forest configurations and 40 HistGradientBoosting configurations, for a total of 152 candidate configurations across the four searched model families.

A candidate is eligible only when its validation Average Precision exceeds the Dummy baseline by more than 0.01.

HistGradientBoosting, Logistic Regression and Random Forest form a three-way practical tie under the study’s predefined rule: in addition to remaining within 0.01 of the validation Average Precision leader, their approximate cross-validation performance intervals satisfy the protocol’s overlap criterion.

The tie is resolved by the first predefined tie-breaker: lower validation Brier score. HistGradientBoosting records a Brier score of 0.133203, compared with 0.133898 for Logistic Regression and 0.159317 for Random Forest.

This selects HistGradientBoostingClassifier as the final model.

The selected scientific configuration uses:

• learning_rate = 0.03
• max_iter = 200
• max_depth = 3
• max_leaf_nodes = 7
• min_samples_leaf = 40
• l2_regularization = 1.0

After model and threshold decisions are frozen, the selected pipeline is fitted once on train + validation, totaling 5,986 observations, and evaluated once on the reserved test partition of 1,057 accounts.

The final scientific results are:

Average Precision — 0.6413
ROC-AUC — 0.8402
Brier score — 0.1394
Log loss — 0.4207

The results show that the available customer-account characteristics contain useful predictive signal for ranking accounts associated with higher and lower churn probability within this evaluation protocol.

The study also selects an educational threshold of approximately 0.2578 using validation data only, designed to maximize precision while maintaining recall of at least 80%. This value belongs to the scientific analysis and should not be interpreted as an operational or economically optimized customer-retention threshold.

04 — Key Takeaways & Limitations

The study shows that customer-account information already present in the dataset — particularly contract type, tenure, service and support characteristics, billing and charges — contains relevant predictive information for churn classification.

The model comparison also shows why the scientific conclusion should not be reduced to the claim that one algorithm substantially outperformed all alternatives. HistGradientBoosting, Logistic Regression and Random Forest are sufficiently close to form a practical tie under the predefined protocol, with HistGradientBoosting selected through the Brier-score tie-break.

The result should nevertheless be interpreted within the limits of the experiment. Evaluation uses a stratified random snapshot rather than a temporal or prospective holdout, so future-time generalization is not established. External representativeness and future distribution stability are also not demonstrated.

The exploratory associations are not causal effects, and the study does not establish that changing a contract, service or customer characteristic would reduce churn.

The study also does not define a production retention policy or an economically optimal decision threshold. Its validation-selected threshold is explicitly educational rather than an operational business rule.

A superseded version of the study performed target-aware exploratory analysis on the full dataset before the current holdout protocol was established. The current workflow isolates the test partition procedurally and does not use those rows for the present model-selection decisions, but they cannot be described as data never previously seen by the author.

The study also does not establish prospective temporal validity, external representativeness, future distribution stability, subgroup fairness, retention-intervention effectiveness or the safety of automated customer-retention decisions.

This portfolio post presents the main scientific findings. The study repository contains the notebooks, source and environment contracts, model-selection evidence, reproducibility artifacts and detailed methodology.

Atlas DataFlow maintains scientific-reproduction evidence separately from its own Atlas-native training and release lineage. This separation preserves the distinction between the original scientific-study results, their independent reproduction and evaluations performed under Atlas-owned contracts.