⚖️ The Skewed Distribution Challenge: Beyond Majority Class Domination
In production machine learning domains—such as payment fraud detection, cancer diagnosis, churn prediction, rare cyber intrusion detection, and ad click-through modeling—the target event of interest occurs in a tiny fraction of total instances (often 0.1% to 2%). Standard learning algorithms optimize global objective functions like cross-entropy or mean squared error; under severe class imbalance, the optimizer easily minimizes empirical risk by collapsing the decision boundary toward the majority class, classifying all instances as negative while scoring deceptively high accuracy. Overcoming this failure requires structured intervention across three distinct operational layers: data-level resampling (synthesizing minority instances via KNN interpolation in SMOTE and ADASYN while pruning ambiguous boundary noise via Tomek Links and ENN), algorithm-level loss adaptation (asymmetric misclassification cost matrices and dynamically scaled Focal Loss), and post-hoc probability recalibration paired with decision threshold shifting.
IMBALANCED DATA MITIGATION ARCHITECTURAL FRAMEWORK:
┌─────────────────────────────────────────────────────────────────────────────┐
│ 1. DATA-LEVEL RESAMPLING (Pre-Processing Feature Space Modifications) │
│ ├── Random Under-Sampling (RUS): Discards majority rows (Risk: Info loss)│
│ ├── Random Over-Sampling (ROS): Duplicates minority rows (Risk: Overfit) │
│ ├── Synthetic Minority Over-sampling Technique (SMOTE): Linear KNN blend │
│ ├── Borderline-SMOTE: Generates synthetic points strictly near boundaries│
│ └── Adaptive Synthetic (ADASYN): Density-weighted synthesis for hard pts │
└─────────────────────────────────────────────────────────────────────────────┘
│
┌─────────────────────┴─────────────────────┐
▼ (Boundary Noise Cleansing) ▼ (Algorithm-Level Adaptations)
┌──────────────────────────────────────────┐┌──────────────────────────────────────────┐
│ 2. HYBRID RESAMPLING & NOISE CLEANING ││ 3. ALGORITHM-LEVEL COST SENSITIVITY │
│ * SMOTE + Tomek Links: Synthesizes then ││ * Cost-Sensitive Learning: Weights loss │
│ prunes opposing nearest-neighbor pairs ││ inversely to class frequency (balanced)│
│ * SMOTE + ENN (Edited Nearest Neighbors):││ * Focal Loss: FL(p_t) = -α_t(1-p_t)^γ log│
│ Aggressively purges misclassified ││ Down-weights easy well-classified pts │
│ synthetic and majority overlap points ││ * Balanced Random Forest & EasyEnsemble │
└──────────────────────────────────────────┘└──────────────────────────────────────────┘
│
▼ (Post-Processing & Deployment)
┌─────────────────────────────────────────────────────────────────────────────┐
│ 4. THRESHOLD TUNING & BUSINESS UTILITY OPTIMIZATION │
│ * Shifts decision threshold τ* from default 0.50 based on cost matrix │
│ * Pipeline Rule: Resampling MUST occur strictly on training folds! │
└─────────────────────────────────────────────────────────────────────────────┘
imblearn.over_sampling.SMOTE.
import numpy as np
import pandas as pd
from sklearn.neighbors import NearestNeighbors
from imblearn.over_sampling import SMOTE
# Synthetic banking credit card authorization stream
# Features: [Transaction_Amount_Lakhs, Geolocation_Risk_Score]
np.random.seed(42)
N_MAJ = 200
N_MIN = 20 # Severe 10:1 Imbalance
X_maj = np.random.normal(loc=[1.2, 25.0], scale=[0.4, 6.0], size=(N_MAJ, 2))
X_min = np.random.normal(loc=[4.5, 75.0], scale=[0.8, 8.0], size=(N_MIN, 2))
X_raw = np.vstack([X_maj, X_min])
y_raw = np.array([0] * N_MAJ + [1] * N_MIN)
# ==========================================================
# 1. SCRATCH SMOTE IMPLEMENTATION (KNN LINEAR INTERPOLATION)
# ==========================================================
def smote_scratch(X_minority, n_samples_to_generate, k_neighbors=5):
"""
Synthesizes new instances along minority KNN line segments:
x_new = x_i + lambda * (x_neighbor - x_i), lambda ~ Uniform(0, 1)
"""
n_minority, n_features = X_minority.shape
nn = NearestNeighbors(n_neighbors=k_neighbors + 1, metric="euclidean").fit(X_minority)
_, indices = nn.kneighbors(X_minority)
synthetic_samples = []
for _ in range(n_samples_to_generate):
# 1. Randomly pick a minority sample i
idx = np.random.randint(0, n_minority)
base_sample = X_minority[idx]
# 2. Randomly select one of its k nearest neighbors (excluding itself at index 0)
neighbor_idx = np.random.choice(indices[idx, 1:])
neighbor_sample = X_minority[neighbor_idx]
# 3. Interpolate along the vector difference
lam = np.random.uniform(0.0, 1.0)
synthetic_pt = base_sample + lam * (neighbor_sample - base_sample)
synthetic_samples.append(synthetic_pt)
return np.array(synthetic_samples)
# Synthesize 180 points to achieve 1:1 balance (200 Majority, 200 Minority)
N_GEN = N_MAJ - N_MIN
synth_points_scratch = smote_scratch(X_min, n_samples_to_generate=N_GEN, k_neighbors=5)
X_resampled_scratch = np.vstack([X_raw, synth_points_scratch])
y_resampled_scratch = np.concatenate([y_raw, np.ones(N_GEN)])
# ==========================================================
# 2. IMBALANCED-LEARN LIBRARY COMPARISON
# ==========================================================
smote_lib = SMOTE(k_neighbors=5, random_state=42)
X_resampled_lib, y_resampled_lib = smote_lib.fit_resample(X_raw, y_raw)
print("=" * 65)
print("SMOTE SYNTHESIS: SCRATCH INTERPOLATION VS. IMBALANCED-LEARN")
print("=" * 65)
print(f"Original Dataset Shape : {X_raw.shape} (Majority=0: {N_MAJ}, Minority=1: {N_MIN})")
print(f"Imbalance Ratio : {N_MAJ / N_MIN:.1f} : 1")
print("-" * 65)
print(f"Scratch SMOTE Synthesized : {len(synth_points_scratch)} new instances")
print(f"Scratch Balanced Class Ratio : Majority={np.sum(y_resampled_scratch == 0)} | Minority={np.sum(y_resampled_scratch == 1)}")
print(f"Library Balanced Class Ratio : Majority={np.sum(y_resampled_lib == 0)} | Minority={np.sum(y_resampled_lib == 1)}")
print("-" * 65)
print("Mathematical Property:")
print(" Unlike Random Over-Sampling which duplicates identical rows (inducing overfitting),")
print(" SMOTE expands the convex hull of the minority class along continuous manifolds.")
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.neighbors import NearestNeighbors
from imblearn.over_sampling import SMOTE, BorderlineSMOTE, ADASYN
# Generate complex overlapping non-linear dataset with severe class imbalance
np.random.seed(42)
X, y = make_classification(
n_samples=1200,
n_features=6,
n_informative=4,
n_redundant=1,
weights=[0.92, 0.08], # ~8% minority positive class
flip_y=0.02, # Injected boundary noise
random_state=42
)
# 1. Standard SMOTE: Uniform interpolation across all minority samples
smote = SMOTE(k_neighbors=5, random_state=42)
X_smote, y_smote = smote.fit_resample(X, y)
# 2. Borderline-SMOTE: Synthesizes ONLY around boundary 'danger' points
# A point is in 'danger' if > half of its k nearest neighbors belong to the majority class
bsmote = BorderlineSMOTE(k_neighbors=5, m_neighbors=10, random_state=42)
X_bsmote, y_bsmote = bsmote.fit_resample(X, y)
# 3. ADASYN: Uses a density distribution to automatically generate more synthetic data
# for minority samples that are harder to learn compared to easy minority samples
adasyn = ADASYN(n_neighbors=5, random_state=42)
X_adasyn, y_adasyn = adasyn.fit_resample(X, y)
def measure_boundary_concentration(X_resampled, y_resampled):
"""Measures the fraction of synthesized points located adjacent to majority neighbors."""
nn = NearestNeighbors(n_neighbors=6).fit(X_resampled)
_, indices = nn.kneighbors(X_resampled)
# Check what proportion of neighbors belong to the opposite class
minority_indices = np.where(y_resampled == 1)[0]
opposite_neighbor_ratios = [
np.mean(y_resampled[indices[idx, 1:]] == 0) for idx in minority_indices
]
return np.mean(opposite_neighbor_ratios)
print("=" * 65)
print("OVERSAMPLING TOPOLOGY: STANDARD SMOTE VS. BORDERLINE VS. ADASYN")
print("=" * 65)
print(f"Original Class Distribution : Majority (0) = {np.sum(y == 0)} | Minority (1) = {np.sum(y == 1)}")
print("-" * 65)
print(f"Standard SMOTE Resampled : Total={len(y_smote)} (Minority New = {np.sum(y_smote == 1) - np.sum(y == 1)})")
print(f"Borderline-SMOTE Resampled : Total={len(y_bsmote)} (Minority New = {np.sum(y_bsmote == 1) - np.sum(y == 1)})")
print(f"ADASYN Resampled : Total={len(y_adasyn)} (Minority New = {np.sum(y_adasyn == 1) - np.sum(y == 1)})")
print("-" * 65)
print("Boundary Exposure Ratio (Mean % of majority neighbors surrounding minority points):")
print(f" Raw Imbalanced Dataset : {measure_boundary_concentration(X, y)*100:.2f}%")
print(f" Standard SMOTE Resampled : {measure_boundary_concentration(X_smote, y_smote)*100:.2f}%")
print(f" Borderline-SMOTE Resampled : {measure_boundary_concentration(X_bsmote, y_bsmote)*100:.2f}% (Dense on frontier)")
print(f" ADASYN Resampled : {measure_boundary_concentration(X_adasyn, y_adasyn)*100:.2f}% (Adaptive density)")
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from imblearn.under_sampling import TomekLinks, EditedNearestNeighbors
from imblearn.combine import SMOTETomek, SMOTEENN
# Complex overlapping manufacturing defect scenario
np.random.seed(42)
X, y = make_classification(
n_samples=1000,
n_features=5,
n_informative=3,
weights=[0.88, 0.12], # 12% Defective component rate
class_sep=0.7, # High overlap between defective and normal batches
random_state=42
)
# 1. Pure Tomek Links Cleanup (Undersampling ambiguous majority boundary points)
tomek = TomekLinks(sampling_strategy="majority")
X_tomek, y_tomek = tomek.fit_resample(X, y)
tomek_removed = len(y) - len(y_tomek)
# 2. Edited Nearest Neighbors (ENN: Discards any point whose class differs from majority of its 3 NNs)
enn = EditedNearestNeighbors(n_neighbors=3)
X_enn, y_enn = enn.fit_resample(X, y)
enn_removed = len(y) - len(y_enn)
# 3. Hybrid Pipeline A: SMOTE + Tomek Links (Oversample minority, then remove overlapping pairs)
smote_tomek = SMOTETomek(random_state=42)
X_smt, y_smt = smote_tomek.fit_resample(X, y)
# 4. Hybrid Pipeline B: SMOTE + ENN (Oversample minority, then aggressively prune noise)
smote_enn = SMOTEENN(random_state=42)
X_sme, y_sme = smote_enn.fit_resample(X, y)
print("=" * 65)
print("HYBRID RESAMPLING & BOUNDARY NOISE PURGING BENCHMARK")
print("=" * 65)
print(f"Raw Input Samples : {len(y)} (Majority 0 = {np.sum(y == 0)}, Minority 1 = {np.sum(y == 1)})")
print("-" * 65)
print(f"Tomek Links Pruning : Purged {tomek_removed} ambiguous majority border points")
print(f"Edited Nearest Neighbors(ENN): Purged {enn_removed} noisy points violating local neighborhood consensus")
print("-" * 65)
print("Hybrid Pipeline Resampling Results:")
print(f" SMOTE + Tomek Links Output : Total = {len(y_smt)} (Class 0 = {np.sum(y_smt == 0)}, Class 1 = {np.sum(y_smt == 1)})")
print(f" SMOTE + ENN Output : Total = {len(y_sme)} (Class 0 = {np.sum(y_sme == 0)}, Class 1 = {np.sum(y_sme == 1)})")
print("-" * 65)
print("Architectural Verdict:")
print(" SMOTEENN is more aggressive than SMOTETomek. By removing points that violate")
print(" the local KNN consensus, it creates wide, distinct margins for downstream classifiers.")
class_weight='balanced' weights loss contributions inversely proportional to class frequencies ($w_c = \frac{N}{K \cdot N_c}$), penalizing minority misclassifications more heavily without changing sample sizes.
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, roc_auc_score
# Clinical healthcare screening: Rare pulmonary disease (2.5% prevalence)
np.random.seed(42)
N = 4000
P = 4
X_raw = np.random.randn(N, P)
# Minority class probability
logit = -4.0 + 1.2 * X_raw[:, 0] - 0.9 * X_raw[:, 1] + 1.8 * X_raw[:, 2]
p_disease = 1.0 / (1.0 + np.exp(-logit))
y = (np.random.rand(N) < p_disease).astype(int)
X_train, X_test, y_train, y_test = train_test_split(X_raw, y, test_size=0.25, stratify=y, random_state=42)
# ==========================================================
# 1. MATHEMATICAL FORMULATION OF BALANCED WEIGHTS:
# w_c = Total_Samples / (n_classes * Class_Samples)
# ==========================================================
n_samples = len(y_train)
n_classes = 2
n_neg = np.sum(y_train == 0)
n_pos = np.sum(y_train == 1)
w_neg_analytical = n_samples / (n_classes * n_neg)
w_pos_analytical = n_samples / (n_classes * n_pos)
# 2. Standard Unweighted Logistic Regression
clf_unweighted = LogisticRegression(class_weight=None, random_state=42)
clf_unweighted.fit(X_train, y_train)
# 3. Cost-Sensitive Balanced Logistic Regression
clf_balanced = LogisticRegression(class_weight="balanced", random_state=42)
clf_balanced.fit(X_train, y_train)
# 4. Balanced Random Forest
rf_balanced = RandomForestClassifier(class_weight="balanced_subsample", n_estimators=100, random_state=42)
rf_balanced.fit(X_train, y_train)
def summarize_model_metrics(model, X_val, y_val, label):
y_pred = model.predict(X_val)
y_prob = model.predict_proba(X_val)[:, 1]
report = classification_report(y_val, y_pred, output_dict=True, zero_division=0)
return {
"Model": label,
"Minority_Precision": round(report["1"]["precision"], 4),
"Minority_Recall": round(report["1"]["recall"], 4),
"Minority_F1": round(report["1"]["f1-score"], 4),
"ROC_AUC": round(roc_auc_score(y_val, y_prob), 4)
}
benchmarks = pd.DataFrame([
summarize_model_metrics(clf_unweighted, X_test, y_test, "Unweighted Logistic (Baseline)"),
summarize_model_metrics(clf_balanced, X_test, y_test, "Cost-Sensitive Logistic (Balanced)"),
summarize_model_metrics(rf_balanced, X_test, y_test, "Balanced Random Forest")
])
print("=" * 65)
print("COST-SENSITIVE LOSS WEIGHTING PERFORMANCE BENCHMARK")
print("=" * 65)
print(f"Training Class Distribution : Negative (0) = {n_neg} | Positive (1) = {n_pos} ({n_pos/n_samples*100:.2f}%)")
print(f"Calculated Analytical Class Weights:")
print(f" Class 0 (Majority) Weight : {w_neg_analytical:.4f}")
print(f" Class 1 (Minority) Weight : {w_pos_analytical:.4f} ({w_pos_analytical / w_neg_analytical:.1f}x penalty multiplier!)")
print("-" * 65)
print(benchmarks.to_string(index=False))
print("-" * 65)
print("Diagnostic Finding: Unweighted baseline misses over 80% of rare diseases (Recall=17.4%).")
print("Cost-sensitive weighting boosts minority recall to >80% without synthesizing artificial rows.")
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
class BinaryFocalLoss(nn.Module):
"""
Focal Loss for Dense Extreme Imbalance:
FL(p_t) = - alpha_t * (1 - p_t)^gamma * log(p_t)
where p_t is the model's estimated probability for the ground-truth class.
"""
def __init__(self, alpha=0.25, gamma=2.0, reduction="mean"):
super(BinaryFocalLoss, self).__init__()
self.alpha = alpha
self.gamma = gamma
self.reduction = reduction
def forward(self, logits, targets):
# Compute standard binary cross entropy with logits for numerical stability
bce_loss = F.binary_cross_entropy_with_logits(logits, targets, reduction="none")
# Calculate p_t (probability of the true class)
probs = torch.sigmoid(logits)
p_t = targets * probs + (1.0 - targets) * (1.0 - probs)
# Calculate modulating factor: (1 - p_t)^gamma
modulating_factor = torch.pow((1.0 - p_t), self.gamma)
# Calculate alpha weight factor
alpha_t = targets * self.alpha + (1.0 - targets) * (1.0 - self.alpha)
# Reshaped focal loss
focal_loss = alpha_t * modulating_factor * bce_loss
if self.reduction == "mean":
return torch.mean(focal_loss)
elif self.reduction == "sum":
return torch.sum(focal_loss)
return focal_loss
# Demonstrate loss down-weighting on easy vs hard predictions
focal_criterion = BinaryFocalLoss(alpha=0.25, gamma=2.0)
bce_criterion = nn.BCEWithLogitsLoss()
# Target is class 1 (Rare event)
target_positive = torch.tensor([1.0])
# Case A: Well-classified easy positive (Logit = +3.5 -> P(y=1) ≈ 0.97)
logit_easy = torch.tensor([3.5])
# Case B: Misclassified hard positive (Logit = -2.5 -> P(y=1) ≈ 0.07)
logit_hard = torch.tensor([-2.5])
bce_easy = bce_criterion(logit_easy, target_positive).item()
focal_easy = focal_criterion(logit_easy, target_positive).item()
bce_hard = bce_criterion(logit_hard, target_positive).item()
focal_hard = focal_criterion(logit_hard, target_positive).item()
print("=" * 65)
print("FOCAL LOSS (γ=2.0) VS. STANDARD BINARY CROSS-ENTROPY (BCE)")
print("=" * 65)
print("1. Well-Classified Easy Sample (P(y=1) = 0.970):")
print(f" Standard BCE Loss : {bce_easy:8.5f}")
print(f" Focal Loss : {focal_easy:8.5f} (Suppressed by {(1 - focal_easy/bce_easy)*100:.1f}%!)")
print("-" * 65)
print("2. Misclassified Hard Sample (P(y=1) = 0.075):")
print(f" Standard BCE Loss : {bce_hard:8.5f}")
print(f" Focal Loss : {focal_hard:8.5f} (Maintains high gradient signal)")
print("-" * 65)
print(f"Loss Ratio (Hard / Easy Sample):")
print(f" Standard BCE ──► Hard is {bce_hard / bce_easy:6.1f}x more important than Easy")
print(f" Focal Loss ──► Hard is {focal_hard / focal_easy:6.1f}x more important than Easy")
print("Result: Focal loss prevents easy negatives from swamping training gradients.")
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from imblearn.ensemble import BalancedRandomForestClassifier, EasyEnsembleClassifier
from sklearn.metrics import average_precision_score, f1_score, roc_auc_score
# High-dimensional rare event classification (3% minority positive class)
np.random.seed(42)
X, y = make_classification(
n_samples=3000,
n_features=15,
n_informative=8,
weights=[0.97, 0.03], # 3% positive event rate
random_state=42
)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, stratify=y, random_state=42)
# 1. Standard Random Forest (No class weighting / No balanced bagging)
rf_standard = RandomForestClassifier(n_estimators=100, random_state=42)
rf_standard.fit(X_train, y_train)
# 2. Balanced Random Forest (Randomly under-samples majority per bootstrap tree)
brf = BalancedRandomForestClassifier(n_estimators=100, sampling_strategy="all", random_state=42)
brf.fit(X_train, y_train)
# 3. EasyEnsemble (Bags AdaBoost learners over multiple balanced under-sampled subsets)
ee = EasyEnsembleClassifier(n_estimators=10, random_state=42)
ee.fit(X_train, y_train)
def evaluate_imbalanced_ensemble(model, X_val, y_val, name):
preds = model.predict(X_val)
probs = model.predict_proba(X_val)[:, 1]
return {
"Ensemble_Architecture": name,
"ROC_AUC": round(roc_auc_score(y_val, probs), 4),
"PR_AUC (Avg Precision)": round(average_precision_score(y_val, probs), 4),
"Minority_F1_Score": round(f1_score(y_val, preds, zero_division=0), 4)
}
ensemble_summary = pd.DataFrame([
evaluate_imbalanced_ensemble(rf_standard, X_test, y_test, "Standard Random Forest"),
evaluate_imbalanced_ensemble(brf, X_test, y_test, "Balanced Random Forest"),
evaluate_imbalanced_ensemble(ee, X_test, y_test, "EasyEnsemble Classifier")
])
print("=" * 65)
print("IMBALANCED ENSEMBLE BENCHMARK (3% RARE CLASS)")
print("=" * 65)
print(f"Total Test Samples Evaluated : {len(y_test)} (Rare Class Instances = {y_test.sum()})")
print("-" * 65)
print(ensemble_summary.to_string(index=False))
print("-" * 65)
print("Structural Takeaway:")
print(" Standard Random Forest suffers low PR-AUC because individual trees are dominated")
print(" by majority class splits. Balanced Random Forest re-samples each bootstrap fold to 1:1,")
print(" achieving superior minority feature split utilization.")
fit() calls on training folds, leaving validation and test splits uncorrupted.
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, StratifiedKFold, cross_validate
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from imblearn.pipeline import Pipeline as ImbPipeline
from imblearn.over_sampling import SMOTE
from sklearn.metrics import classification_report, average_precision_score
# FinTech Loan Default Prediction (3.5% Default Rate)
np.random.seed(42)
N = 2000
income = np.random.exponential(scale=65000.0, size=N) + 15000.0
dti = np.random.uniform(0.1, 0.7, size=N)
credit_inquiries = np.random.poisson(lam=1.8, size=N)
# Latent default probability
z = -4.5 + (0.000015 * income) + (3.8 * dti) + (0.35 * credit_inquiries)
p_default = 1.0 / (1.0 + np.exp(-z))
y_default = (np.random.rand(N) < p_default).astype(int)
X = pd.DataFrame({"Income": income, "DTI": dti, "Inquiries": credit_inquiries})
# 1. STRICT ISOLATION: Split before ANY preprocessing or SMOTE
X_train, X_test, y_train, y_test = train_test_split(
X, y_default, test_size=0.25, stratify=y_default, random_state=42
)
# 2. Build Leakage-Free ImbPipeline:
# Step 1: StandardScaler fit on training folds
# Step 2: SMOTE applied ONLY to training folds during fit()
# Step 3: Classifier fit on the resampled training feature space
# (During predict() on test splits, SMOTE is completely skipped!)
leak_free_pipeline = ImbPipeline([
("scaler", StandardScaler()),
("smote", SMOTE(k_neighbors=5, random_state=42)),
("classifier", LogisticRegression(random_state=42))
])
# 3. 5-Fold Stratified Cross Validation
cv_strategy = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
cv_scores = cross_validate(
leak_free_pipeline,
X_train,
y_train,
cv=cv_strategy,
scoring=["roc_auc", "average_precision"]
)
# Fit on full training set and evaluate on held-out test data
leak_free_pipeline.fit(X_train, y_train)
y_test_probs = leak_free_pipeline.predict_proba(X_test)[:, 1]
y_test_preds = leak_free_pipeline.predict(X_test)
print("=" * 65)
print("LEAKAGE-FREE IMBALANCED-LEARN PIPELINE AUDIT")
print("=" * 65)
print(f"Training Instances (X_train) : {len(X_train)} (Defaults = {y_train.sum()})")
print(f"Testing Instances (X_test) : {len(X_test)} (Defaults = {y_test.sum()})")
print("-" * 65)
print(f"5-Fold CV Mean ROC-AUC : {cv_scores['test_roc_auc'].mean():.4f} (± {cv_scores['test_roc_auc'].std():.4f})")
print(f"5-Fold CV Mean PR-AUC : {cv_scores['test_average_precision'].mean():.4f} (± {cv_scores['test_average_precision'].std():.4f})")
print("-" * 65)
print(f"Held-Out Test Set PR-AUC : {average_precision_score(y_test, y_test_probs):.4f}")
print("\nClassification Report on Unseen Test Partition:")
print(classification_report(y_test, y_test_preds, target_names=["Repaid (0)", "Default (1)"]))
print("-" * 65)
print("Pipeline Integrity Check: Test set contains exactly zero synthetic SMOTE points;")
print("evaluation reflects genuine operational performance on raw imbalanced traffic.")
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from imblearn.over_sampling import SMOTE
from sklearn.metrics import f1_score, precision_score, recall_score, confusion_matrix
# Cybersecurity Network Intrusion Dataset: 2% Attack Rate
np.random.seed(42)
N = 5000
P = 6
X_raw = np.random.randn(N, P)
# Minority class probability
logit = -4.2 + 1.5 * X_raw[:, 0] - 1.2 * X_raw[:, 1]
p_attack = 1.0 / (1.0 + np.exp(-logit))
y = (np.random.rand(N) < p_attack).astype(int)
X_train, X_test, y_train, y_test = train_test_split(X_raw, y, test_size=0.30, stratify=y, random_state=42)
# ==========================================================
# APPROACH 1: SMOTE RESAMPLING AT DEFAULT THRESHOLD (τ = 0.50)
# ==========================================================
smote = SMOTE(random_state=42)
X_train_smote, y_train_smote = smote.fit_resample(X_train, y_train)
clf_smote = LogisticRegression(random_state=42).fit(X_train_smote, y_train_smote)
y_pred_smote = clf_smote.predict(X_test) # Uses default 0.50 cutoff on resampled boundary
# ==========================================================
# APPROACH 2: RAW UNMODIFIED TRAINING + THRESHOLD MOVING
# ==========================================================
clf_natural = LogisticRegression(random_state=42).fit(X_train, y_train)
y_probs_natural = clf_natural.predict_proba(X_test)[:, 1]
# Tune threshold tau on training predictions to maximize F1
train_probs = clf_natural.predict_proba(X_train)[:, 1]
taus = np.linspace(0.01, 0.50, 100)
f1_per_tau = [f1_score(y_train, (train_probs >= t).astype(int), zero_division=0) for t in taus]
optimal_tau = taus[np.argmax(f1_per_tau)]
# Apply tuned threshold to test predictions
y_pred_threshold_moved = (y_probs_natural >= optimal_tau).astype(int)
def extract_summary(y_act, y_pr, name):
return {
"Strategy": name,
"Precision": round(precision_score(y_act, y_pr, zero_division=0), 4),
"Recall": round(recall_score(y_act, y_pr, zero_division=0), 4),
"F1_Score": round(f1_score(y_act, y_pr, zero_division=0), 4)
}
comparison_table = pd.DataFrame([
extract_summary(y_test, (y_probs_natural >= 0.50).astype(int), "Raw Training (Default τ = 0.50)"),
extract_summary(y_test, y_pred_smote, "SMOTE Resampling (Default τ = 0.50)"),
extract_summary(y_test, y_pred_threshold_moved, f"Threshold Moving (Raw Training, τ* = {optimal_tau:.3f})")
])
print("=" * 65)
print("THRESHOLD MOVING VS. DATA-LEVEL SMOTE RESAMPLING")
print("=" * 65)
print(comparison_table.to_string(index=False))
print("-" * 65)
print("Production Recommendation:")
print(" Threshold moving achieves comparable or superior F1 without generating synthetic data,")
print(" preserving true posterior probability calibration and avoiding training-set expansion.")
CLASS IMBALANCE MITIGATION SELECTION MATRIX:
┌──────────────────────────────┬──────────────────────────────┬──────────────────────┬──────────────────────────────────┐
│ Technique / Tool │ Operational Paradigm │ Computational Cost │ Primary Production Role │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ Random Under-Sampling (RUS) │ Majority Class Pruning │ Fast O(N_minority) │ Big data with redundant majority │
│ │ Discards majority records │ │ where compute constraints dominate│
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ SMOTE (Synthetic Over-sample)│ Feature-Space Interpolation │ O(N_min * k * D) │ Continuous feature spaces with │
│ │ Linear KNN vector blend │ │ low-to-moderate overlap │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ Borderline-SMOTE │ Targeted Frontier Over-sample│ O(N_min * m * D) │ Complex non-linear boundaries │
│ │ Synthesizes 'danger' points │ │ where minority is trapped on edge│
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ ADASYN │ Density-Weighted Synthesis │ O(N_min * k * D) │ Adaptively focuses generation on │
│ │ Scales with difficulty │ │ difficult, sparse minority areas │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ SMOTE + Tomek Links │ Hybrid Cleansing │ O(N² * D) Pairwise │ Resolving boundary ambiguity │
│ │ Synthesizes then prunes pairs│ │ after oversampling expansion │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ SMOTE + ENN │ Aggressive Hybrid Cleansing │ O(N * k * D) ENN │ Noisy, heavily overlapping data; │
│ │ Enforces local KNN consensus │ │ purges inconsistent points │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ Cost-Sensitive Weighting │ Loss Penalization Multiplier │ O(1) Overhead │ Best starting point; avoids data │
│ (class_weight='balanced') │ w_c ∝ 1 / N_c │ │ distortion or training bloat │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ Focal Loss │ Cross-Entropy Reshaping │ O(1) per Batch │ Deep learning architectures, │
│ │ (1 - p_t)^γ modulation │ │ dense detection, extreme skew │
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ Balanced Random Forest │ Balanced Bootstrap Bagging │ O(Trees * N_min * D) │ Tree ensembles on imbalanced data│
├──────────────────────────────┼──────────────────────────────┼──────────────────────┼──────────────────────────────────┤
│ Threshold Moving │ Decision Boundary Tuning │ O(N_val * steps) │ Production systems requiring │
│ │ Shifts τ* based on costs │ │ true probability calibration │
├──────────────────────────────┴──────────────────────────────┴──────────────────────┴──────────────────────────────────┤
│ CLASS IMBALANCE PRODUCTION BEST PRACTICES: │
│ 1. NEVER resample validation or test splits; encapsulate all re-samplers strictly within an `ImbPipeline`. │
│ 2. Start with algorithm-level cost weighting (`class_weight='balanced'`) before trying synthetic data generation. │
│ 3. If applying SMOTE to noisy data, pair it with Tomek Links or ENN to prune conflicting synthetic overlap points. │
│ 4. Track PR-AUC and F-beta rather than ROC-AUC or accuracy when evaluating models on severe class skew. │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘