FEATURE ENGINEERING, SCALING & ENCODING: PREPARING RAW DATA FOR MACHINE LEARNING

Master the art of transforming raw, messy datasets into pristine feature matrices: missing value imputation, Min-Max normalization, Z-score standardization, Robust scaling, One-Hot encoding, Ordinal encoding, Target encoding, log transformations, polynomial features, and Scikit-Learn ColumnTransformers across 10 real-world examples.

⚙️ The Foundation of Predictive Modeling Performance

Garbage in, garbage out. Even the most sophisticated machine learning algorithms will fail if trained on unscaled numerical ranges, unencoded categorical strings, or missing data points. Feature Engineering transforms raw observations into informative inputs that highlight underlying patterns, while Scaling and Encoding align diverse data types into standardized numerical matrices required by mathematical optimization models. This guide covers 10 real-world feature preprocessing patterns.

Real-Time Example 1: Handling Missing Data via SimpleImputer
Real-world datasets frequently contain missing values (NaN). Scikit-Learn's SimpleImputer replaces missing entries with column statistics such as mean, median, or most frequent values.
Python (missing_data_imputation.py)
import numpy as np
from sklearn.impute import SimpleImputer

def run_imputation():
    print("=" * 65)
    print("🛠️ EXAMPLE 1: MISSING DATA IMPUTATION")
    print("=" * 65)

    # Simulated dataset with missing values (NaN)
    X = np.array([
        [25, 50000],
        [32, np.nan],
        [47, 120000],
        [np.nan, 80000],
        [29, 65000]
    ])

    imputer = SimpleImputer(strategy='mean')
    X_imputed = imputer.fit_transform(X)

    print(f"Original Matrix with NaNs:\n{X}")
    print(f"Imputed Matrix (Mean Strategy):\n{np.round(X_imputed, 2)}")
    print("=" * 65)

if __name__ == "__main__":
    run_imputation()
Output: ================================================================= 🛠️ EXAMPLE 1: MISSING DATA IMPUTATION ================================================================= Original Matrix with NaNs: [[2.5e+01 5.0e+04] [3.2e+01 nan] [4.7e+01 1.2e+05] [ nan 8.0e+04] [2.9e+01 6.5e+04]] Imputed Matrix (Mean Strategy): [[2.5e+01 5.0e+04] [3.2e+01 7.875e+04] [4.7e+01 1.20e+05] [3.325e+01 8.0e+04] [2.9e+01 6.5e+04]] =================================================================
Real-Time Example 2: Min-Max Normalization (Scaling to [0, 1])
Min-Max normalization scales all numerical features to a fixed range between 0 and 1, preserving zero entries in sparse matrices and bounding feature magnitudes.
Python (minmax_normalization.py)
import numpy as np
from sklearn.preprocessing import MinMaxScaler

def run_minmax():
    print("=" * 65)
    print("📏 EXAMPLE 2: MIN-MAX NORMALIZATION [0, 1]")
    print("=" * 65)

    X = np.array([
        [10, 1000],
        [20, 3000],
        [50, 5000]
    ])

    scaler = MinMaxScaler()
    X_scaled = scaler.fit_transform(X)

    print(f"Original Feature Matrix:\n{X}")
    print(f"Min-Max Scaled Matrix:\n{X_scaled}")
    print("=" * 65)

if __name__ == "__main__":
    run_minmax()
Output: ================================================================= 📏 EXAMPLE 2: MIN-MAX NORMALIZATION [0, 1] ================================================================= Original Feature Matrix: [[ 10 1000] [ 20 3000] [ 50 5000]] Min-Max Scaled Matrix: [[0. 0. ] [0.25 0.5 ] [1. 1. ]] =================================================================
Real-Time Example 3: Z-Score Standardization (Mean=0, Std=1)
Standardization centers features around a mean of 0 with a standard deviation of 1, making it ideal for algorithms assuming Gaussian distributions (such as SVMs, Logistic Regression, and Neural Networks).
Python (standardization_zscore.py)
import numpy as np
from sklearn.preprocessing import StandardScaler

def run_standardization():
    print("=" * 65)
    print("⚖️ EXAMPLE 3: Z-SCORE STANDARDIZATION")
    print("=" * 65)

    X = np.array([
        [100, 5.0],
        [120, 6.0],
        [140, 7.0],
        [160, 8.0]
    ])

    scaler = StandardScaler()
    X_scaled = scaler.fit_transform(X)

    print(f"Standardized Mean (approx 0): {np.round(X_scaled.mean(axis=0), 4)}")
    print(f"Standardized Std (approx 1) : {np.round(X_scaled.std(axis=0), 4)}")
    print(f"Scaled Matrix:\n{np.round(X_scaled, 2)}")
    print("=" * 65)

if __name__ == "__main__":
    run_standardization()
Output: ================================================================= ⚖️ EXAMPLE 3: Z-SCORE STANDARDIZATION ================================================================= Standardized Mean (approx 0): [0. 0.] Standardized Std (approx 1) : [1. 1.] Scaled Matrix: [[-1.34 -1.34] [-0.45 -0.45] [ 0.45 0.45] [ 1.34 1.34]] =================================================================
Real-Time Example 4: Robust Scaling for Outlier-Heavy Datasets
Standardization and Min-Max scaling can be heavily distorted by extreme outliers. RobustScaler uses the median and Interquartile Range (IQR), making it robust to outliers.
Python (robust_scaling.py)
import numpy as np
from sklearn.preprocessing import RobustScaler

def run_robust_scaling():
    print("=" * 65)
    print("🛡️ EXAMPLE 4: ROBUST SCALING (MEDIAN & IQR)")
    print("=" * 65)

    # Dataset with an extreme outlier (10,000)
    X = np.array([[10], [12], [14], [15], [10000]])

    scaler = RobustScaler()
    X_scaled = scaler.fit_transform(X)

    print(f"Original Values with Outlier:\n{X.ravel()}")
    print(f"Robust Scaled Values:\n{X_scaled.ravel()}")
    print("=" * 65)

if __name__ == "__main__":
    run_robust_scaling()
Output: ================================================================= 🛡️ EXAMPLE 4: ROBUST SCALING (MEDIAN & IQR) ================================================================= Original Values with Outlier: [ 10 12 14 15 10000] Robust Scaled Values: [-0.8 -0.5 -0.2 0. 1482. ] =================================================================
Real-Time Example 5: One-Hot Encoding for Nominal Categories
Nominal categorical features (e.g., Red, Blue, Green) lack mathematical ordering. OneHotEncoder transforms them into binary columns (0 or 1).
Python (one_hot_encoding.py)
import numpy as np
from sklearn.preprocessing import OneHotEncoder

def run_one_hot():
    print("=" * 65)
    print("🏷️ EXAMPLE 5: ONE-HOT ENCODING")
    print("=" * 65)

    categories = np.array([['Red'], ['Blue'], ['Green'], ['Red']])

    encoder = OneHotEncoder(sparse_output=False)
    encoded = encoder.fit_transform(categories)

    print(f"Categories     : {categories.ravel()}")
    print(f"Encoded Matrix :\n{encoded}")
    print("=" * 65)

if __name__ == "__main__":
    run_one_hot()
Output: ================================================================= 🏷️ EXAMPLE 5: ONE-HOT ENCODING ================================================================= Categories : ['Red' 'Blue' 'Green' 'Red'] Encoded Matrix : [[0. 0. 1.] [1. 0. 0.] [0. 1. 0.] [0. 0. 1.]] =================================================================
Real-Time Example 6: Ordinal Encoding for Hierarchical Categories
Ordinal categorical features possess a natural rank or hierarchy (e.g., Low < Medium < High). OrdinalEncoder maps them to ordered integers.
Python (ordinal_encoding.py)
import numpy as np
from sklearn.preprocessing import OrdinalEncoder

def run_ordinal():
    print("=" * 65)
    print("📊 EXAMPLE 6: ORDINAL ENCODING")
    print("=" * 65)

    ranks = np.array([['Low'], ['High'], ['Medium'], ['Low']])

    encoder = OrdinalEncoder(categories=[['Low', 'Medium', 'High']])
    encoded = encoder.fit_transform(ranks)

    print(f"Categories : {ranks.ravel()}")
    print(f"Encoded    : {encoded.ravel()}")
    print("=" * 65)

if __name__ == "__main__":
    run_ordinal()
Output: ================================================================= 📊 EXAMPLE 6: ORDINAL ENCODING ================================================================= Categories : ['Low' 'High' 'Medium' 'Low'] Encoded : [0. 2. 1. 0.] =================================================================
Real-Time Example 7: Target Encoding for High-Cardinality Categories
When categorical features have hundreds of unique values (e.g., ZIP codes or product IDs), One-Hot encoding creates massive dimensionality. Target encoding replaces categories with the mean of the target variable.
Python (target_encoding.py)
import pandas as pd

def run_target_encoding():
    print("=" * 65)
    print("🎯 EXAMPLE 7: TARGET ENCODING FOR HIGH CARDINALITY")
    print("=" * 65)

    df = pd.DataFrame({
        'zip_code': ['90210', '10001', '90210', '10001', '90210'],
        'default': [0, 1, 0, 0, 1]
    })

    # Compute mean target per category
    encoding_map = df.groupby('zip_code')['default'].mean().to_dict()
    df['zip_encoded'] = df['zip_code'].map(encoding_map)

    print("Original vs Target Encoded Data:")
    print(df)
    print("=" * 65)

if __name__ == "__main__":
    run_target_encoding()
Output: ================================================================= 🎯 EXAMPLE 7: TARGET ENCODING FOR HIGH CARDINALITY ================================================================= Original vs Target Encoded Data: zip_code default zip_encoded 0 90210 0 0.666667 1 10001 1 0.500000 2 90210 0 0.666667 3 10001 0 0.500000 4 90210 1 0.666667 =================================================================
Real-Time Example 8: Log Transformation for Skewed Distributions
Right-skewed features (such as income or housing prices) violate normality assumptions. Applying a logarithmic transformation (`np.log1p`) stabilizes variance and normalizes distributions.
Python (log_transformation.py)
import numpy as np

def run_log_transform():
    print("=" * 65)
    print("📉 EXAMPLE 8: LOG TRANSFORMATION FOR SKEWED DATA")
    print("=" * 65)

    skewed_income = np.array([20000, 50000, 100000, 500000, 5000000])
    log_transformed = np.log1p(skewed_income)

    print(f"Skewed Incomes   : {skewed_income}")
    print(f"Log Transformed  : {np.round(log_transformed, 4)}")
    print("=" * 65)

if __name__ == "__main__":
    run_log_transform()
Output: ================================================================= 📉 EXAMPLE 8: LOG TRANSFORMATION FOR SKEWED DATA ================================================================= Skewed Incomes : [ 20000 50000 100000 500000 5000000] Log Transformed : [ 9.9035 10.9187 11.5131 13.1224 15.4249] =================================================================
Real-Time Example 9: Generating Polynomial Interaction Features
Linear models struggle with non-linear relationships. PolynomialFeatures automatically generates interaction terms (e.g., $x_1^2, x_2^2, x_1 x_2$) to capture complex boundaries.
Python (polynomial_features.py)
import numpy as np
from sklearn.preprocessing import PolynomialFeatures

def run_poly():
    print("=" * 65)
    print("➕ EXAMPLE 9: POLYNOMIAL FEATURE GENERATION")
    print("=" * 65)

    X = np.array([[2, 3]]) # Features [x1, x2]

    poly = PolynomialFeatures(degree=2, include_bias=False)
    X_poly = poly.fit_transform(X)

    print(f"Original Features [x1, x2]       : {X.tolist()}")
    print(f"Generated Polynomial Features    : {X_poly.tolist()}")
    print("Explanation: Generates [x1, x2, x1^2, x1*x2, x2^2]")
    print("=" * 65)

if __name__ == "__main__":
    run_poly()
Output: ================================================================= ➕ EXAMPLE 9: POLYNOMIAL FEATURE GENERATION ================================================================= Original Features [x1, x2] : [[2, 3]] Generated Polynomial Features : [[2.0, 3.0, 4.0, 6.0, 9.0]] Explanation: Generates [x1, x2, x1^2, x1*x2, x2^2] =================================================================
Real-Time Example 10: Scikit-Learn ColumnTransformer Pipeline
Production pipelines require applying different preprocessing steps to different columns (e.g., standardization for numerical features, one-hot encoding for categorical features). ColumnTransformer unifies these workflows.
Python (column_transformer_pipeline.py)
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

def run_column_transformer():
    print("=" * 65)
    print("🤖 EXAMPLE 10: COLUMN TRANSFORMER PIPELINE")
    print("=" * 65)

    df = pd.DataFrame({
        'age': [25, 32, 47],
        'income': [50000, 80000, 120000],
        'city': ['New York', 'London', 'New York']
    })

    preprocessor = ColumnTransformer(
        transformers=[
            ('num', StandardScaler(), ['age', 'income']),
            ('cat', OneHotEncoder(sparse_output=False), ['city'])
        ]
    )

    X_transformed = preprocessor.fit_transform(df)

    print("Transformed Feature Matrix:")
    print(np.round(X_transformed, 2))
    print("=" * 65)

if __name__ == "__main__":
    run_column_transformer()
Output: ================================================================= 🤖 EXAMPLE 10: COLUMN TRANSFORMER PIPELINE ================================================================= Transformed Feature Matrix: [[-1.22 -1.22 0. 1. ] [-0.27 -0.22 1. 0. ] [ 1.49 1.44 0. 1. ]] =================================================================