32 Feature Engineering for ML

32.4 Features Preserve Signal Meaning

Feature engineering turns raw IoT observations into model inputs that preserve the meaning needed for a decision. The useful feature is not simply a statistic or embedding. It is a repeatable measurement of the signal, context, label boundary, and time window that the model will see again in deployment.

For a vibration classifier, a feature may describe energy in a frequency band, peak behavior, or a change from normal baseline. For an occupancy model, a feature may combine motion events, door state, time of day, and recent absence. For a battery-risk model, a feature may tie voltage, temperature, current draw, firmware state, and communication retries to a service decision.

Worked example: a temperature-alert model might turn one-second sensor samples into a 60-second window with mean, maximum, slope, missing-sample count, and last-calibration age. Those features are only trustworthy if the same window boundary, unit conversion, calibration state, and missing-data rule are used during training, validation, and deployment. If the edge device later reports every 10 seconds, the feature recipe has changed even when the feature names stay the same.

The feature-pipeline diagram in Figure 32.1 makes the deployable recipe visible before individual statistics are judged.

Follow Figure 32.1 from raw sensor values into a defined window, through statistic extraction, and finally normalization. The window fixes sample membership and latency; statistics encode the chosen signal behavior; normalization uses training-derived parameters to place features on the expected scale. Every arrow must execute identically after deployment. This carries forward the chapter’s running narrative: a useful feature is not just predictive in a notebook—it is physically meaningful, leakage-controlled, affordable, versioned, and available with the same semantics at inference time.

What Makes a Feature Set Useful

A feature set is useful when the chosen dimensions make the groups easier to distinguish together than they would be alone. Color by itself may overlap, weight by itself may overlap, but color plus weight may separate the examples well enough for a simple model. Even a simple image question such as whether a fruit is more red or orange needs a recipe: maximum channel value, mean channel value, channel ratio, and histogram shape can support different decisions. A weak feature is not "wrong" because it is ugly. It is weak when it is irrelevant, unstable, too expensive to compute, unavailable at inference time, or unable to separate the decision groups it is supposed to support.

Real sensor preprocessing is application-dependent. Filtering, denoising, normalization, clipping, and windowing should be chosen because they preserve the decision signal, not because they are standard notebook steps. For a gesture classifier on a Particle-class device, the pipeline might collect IMU motion on the device, compute compact window features locally, train or configure the model from a phone workflow, display the predicted character on the phone, and keep raw logs for Python plots so the team can see whether four in-air gestures are actually separable.

32.5 Practitioner: Write a Feature Recipe

A feature recipe is the contract between training and inference. It states exactly how a feature is computed, what assumptions it uses, how missing and out-of-range values are handled, and what evidence proves it still means the same thing when deployed.

Keep feature selection reviewable. Removing redundant features can make a model cheaper and easier to monitor, but the retained features should still explain the operational claim. If a selected feature is a proxy for user identity, protected status, site ID, maintenance shift, or a post-decision workflow, the review should check whether it is meaningful, fair, legal, and available at inference.

Common selection methods ask different questions. A variance threshold keeps features that actually spread out across examples, so a nearly constant column does not pretend to add evidence. Univariate tests score one feature at a time against the label and can reveal a single strong separator, but they can miss combinations that only work together. Principal component analysis (PCA) transforms the feature space toward directions with high variation and can reduce dimensionality, but the projected components still need scaling, provenance, and deployment-time reproducibility.

Frequency-Domain Comparison as a Feature Contract

When two sensors observe the same motion through different mechanisms, compare more than their raw traces. Welch power spectral density estimation divides a window into overlapping segments, windows each segment, estimates a periodogram, and averages the results. The averaging makes the estimate less erratic than one periodogram, at the cost of frequency resolution and added latency. The feature contract must record sample rate, synchronization, segment length, overlap, window function, detrending, filtering, normalization, and the frequency band used for the decision; otherwise two curves can differ because the pipelines differ rather than because the sensors disagree.

For example, an event camera and a balance board may both observe postural sway while measuring different physical quantities. Agreement of band power, dominant-frequency region, or spectral shape within a predeclared sway band supports the claim that both preserve relevant dynamics. It does not prove that their amplitudes are interchangeable or that either is clinically valid. Check coherence only when synchronization and signal definitions support it, preserve participant and stance context, and investigate spectral disagreement as possible timing, geometry, calibration, filtering, noise-floor, or measurand evidence. This turns a PSD plot into a reproducible validation feature rather than a visual similarity claim.

32.6 Leakage, Drift, Feature Contracts

Feature engineering often fails because a model learns a shortcut that will not hold after deployment. The shortcut may be explicit leakage, such as a post-event field inside the feature table. It may also be a quiet proxy, such as a device ID that stands in for a site, a timestamp that stands in for maintenance schedule, or a missing-value pattern that stands in for a broken collection process.

A practical leakage test asks whether the value would exist, with the same precision and delay, at the moment the model must act. If the answer depends on a future repair ticket, a later human label, a batch backfill, or a post-alarm state, the feature contract is invalid for real-time inference.

The strongest defense is a feature contract: a versioned recipe, an availability check at prediction time, a leakage audit, a held-out validation split that matches deployment, and monitors for input range, missingness, drift, and feature calculation failures. Feature stores can help when they preserve this contract, but they do not remove the need to review feature meaning.

Under the hood, feature selection should also be treated as a reliability decision. Dropping a feature can reduce cost, but it may remove the only signal that catches a rare failure mode. Adding a feature can improve a metric, but it may increase latency, power, privacy risk, or false confidence. The accepted feature set should state what it optimizes and what it no longer claims to detect.