← Back to the chapter · Previous detail

Technical detail · Feature Scaling Lab: Statistical Analysis

A split-first scaling template

The first split reserves 30% of the data; the second divides that remainder equally between validation and test. Only X_train reaches fit. Both reported scores use the same fitted scaler.

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler

X_train, X_temp, y_train, y_temp = train_test_split(
    X, y, test_size=0.3
)
X_val, X_test, y_val, y_test = train_test_split(
    X_temp, y_temp, test_size=0.5
)
scaler = MinMaxScaler()
scaler.fit(X_train)
X_train_norm = scaler.transform(X_train)
X_val_norm = scaler.transform(X_val)
X_test_norm = scaler.transform(X_test)
model.fit(X_train_norm, y_train)
val_score = model.score(X_val_norm, y_val)
test_score = model.score(X_test_norm, y_test)

The template assumes X, y, and model are supplied. The important evidence is when fitting occurs and which rows it can see.

← Back to the chapter · Previous detail