← Back to the chapter · Previous detail
Technical detail · Feature Engineering for ML
Under the Hood: Leakage, drift, and feature contracts
Feature engineering often fails because a model learns a shortcut that will not hold after deployment. The shortcut may be explicit leakage, such as a post-event field inside the feature table. It may also be a quiet proxy, such as a device ID that stands in for a site, a timestamp that stands in for maintenance schedule, or a missing-value pattern that stands in for a broken collection process.
Temporal leakage uses values, labels, repairs, alarms, or operator actions after prediction time. Population leakage lets the same device, user, room, asset, or site appear in training and validation when deployment requires generalization.
Processing drift changes sampling, normalization, clipping, imputation, encoding, or missing-data labels compared with training. Meaning drift keeps a feature name after sensors, firmware, labels, usage, environment, or operating policy changes its meaning.
The strongest defense is a feature contract: a versioned recipe, an availability check at prediction time, a leakage audit, a held-out validation split that matches deployment, and monitors for input range, missingness, drift, and feature calculation failures. Feature stores can help when they preserve this contract, but they do not remove the need to review feature meaning.
Under the hood, feature selection should also be treated as a reliability decision. Dropping a feature can reduce cost, but it may remove the only signal that catches a rare failure mode. Adding a feature can improve a metric, but it may increase latency, power, privacy risk, or false confidence. The accepted feature set should state what it optimizes and what it no longer claims to detect.