← Back to the chapter · Previous detail · Next detail →
Technical detail · Feature Scaling Lab: Statistical Analysis
Contrasting leakage examples
The first example leaks test statistics before a random split; optimistic evaluation is possible. The seasonal examples assume a chronological winter-training/summer-test split; random splitting does not guarantee those periods. load_sensor_data() is illustrative and must be supplied.
# WRONG: Normalize first, split second
data = load_sensor_data() # 10,000 samples
normalized = MinMaxScaler().fit_transform(data) # Uses ALL data stats!
train, test = train_test_split(normalized, test_size=0.2)
# The test set's min/max values influenced the scaling parameters!
# Model evaluation may now be too optimistic.
# Calculate on ALL data (winter + summer)
overall_min, overall_max = 15, 35 # degrees C
# Normalize training data using overall stats
train_normalized = (train - 15) / (35 - 15)
# Result: Training temps (15-25°C) map to 0.0-0.5
# Normalize test data using SAME overall stats
test_normalized = (test - 15) / (35 - 15)
# Result: Test temps (20-35°C) map to 0.25-1.0
# Model trained on 0.0-0.5 range, tested on 0.25-1.0 range
# Test performance may appear optimistic because normalization used held-out summer data.
# CORRECT: Split first, normalize second
data = load_sensor_data()
train, test = train_test_split(data, test_size=0.2, shuffle=False) # Chronological split FIRST
# Fit normalization on training data ONLY
scaler = MinMaxScaler()
scaler.fit(train) # Learns min=15, max=25 from training (winter)
# Apply fitted scaler to both train and test
train_normalized = scaler.transform(train)
test_normalized = scaler.transform(test) # Summer data (20-35°C) may exceed [0,1]!
# This is CORRECT - test data reflecting real-world distribution