39 Time-Series Anomalies: Baselines and Residuals

39.6 Time-Series Residual Scoring

Time-series anomaly detection asks whether a reading is unusual for its temporal context. A value can be normal at one hour, season, operating mode, or cycle stage and abnormal at another. The usual pattern is to forecast or decompose the expected value, compute a residual, and alert only when the residual is large enough to matter.

This makes time-series methods a natural fit for contextual anomalies in IoT: load profiles, temperature cycles, vibration windows, occupancy patterns, battery discharge, and watermarks in stream processing. The detector must record the forecast window, seasonal context, residual, threshold, missing-data handling, and latency budget.

When normal depends on time, inspect Figure 39.1 before setting an alert threshold; it separates modeling the expected signal from scoring what remains.

Follow Figure 39.1 from temporal structure to residual. ARIMA-style forecasting models trend and repeating cycles, exponential smoothing adapts the recent level, and STL separates trend and seasonality explicitly. Each path ends by comparing observed and expected behavior, not by thresholding the raw value. That order advances the chapter's evidence chain: preserve the context, forecast or decomposition, residual, and decision threshold together so two identical readings can legitimately receive different decisions.

Worked example: a freezer temperature of 6 deg C may be normal during a scheduled defrost cycle and abnormal during a steady cooling period. A raw threshold cannot tell those cases apart. A time-aware detector compares the reading with the expected value for that minute, operating mode, and recent history, then stores the residual and context that made the decision reviewable.

This is also why two identical readings can produce different decisions. A 13 kW residual at 02:00 may be suspicious if the site is normally idle, while the same raw load during a shift change may be expected. The alert should preserve the forecast horizon, context fields, residual band, data freshness, and whether the score came from a complete or provisional window. That evidence lets reviewers compare the alert with the baseline that actually existed at decision time.

Overview Knowledge Check

39.7 Choose Temporal Baseline First

Start by asking what makes the signal predictable. If the signal has daily or weekly cycles, use a seasonal baseline or STL residual. If it has autocorrelation and stable sampling, use a forecasting residual such as ARIMA or a simpler rolling model. If it changes slowly but has sudden level shifts, use EWMA or Holt-Winters-style smoothing. If anomalies are window shapes, score windows rather than single points.

Worked example: forecast-residual alert
sensor: building power demand
time context: weekday, 02:00
forecast from seasonal baseline: 18 kW
expected residual band: +/- 4 kW
observed reading: 31 kW

residual:
31 - 18 = 13 kW

candidate rule:
flag when abs(residual) > 4 kW

alert rule:
raise an alert only if the reading is fresh, the meter health is good,
and either two consecutive residuals exceed the band or an occupancy
signal confirms the abnormal context.

Choose the method from the shape and deadline. If samples arrive every minute but the operations team can wait five minutes, a short persistence rule can suppress one-off meter noise. If a motor trip must be caught within one sample, use an edge EWMA or threshold residual and send a richer window to the gateway for review. The design should state which layer owns the first alert and which layer owns the later explanation.

Practitioner Knowledge Check

39.8 Sampling, Drift, and Latency

Time-series detectors rely on timestamp quality. Irregular sampling, delayed packets, clock drift, duplicated readings, and gaps change the residual. Stream deployments also need explicit watermarks or late-data policies: a detector cannot safely score a window if the window may still receive late readings. The alert record should say whether the data was complete, late, imputed, or scored in degraded mode.

Drift control is equally important. Seasonal baselines become stale when equipment, firmware, occupancy, weather, or operating policy changes. Adaptive methods can reduce false alerts, but they can also absorb a slow fault into the baseline. Production systems should separate drift monitoring from incident scoring and require baseline review after known site changes.

For a five-minute window sampled every 30 seconds, the detector expects 10 readings. If only eight have arrived when the one-minute watermark expires, the system can either score a provisional residual, wait and violate the latency budget, or mark the window as incomplete and route a sensor-health review. The chosen policy changes both alert quality and response time, so it belongs in the incident record.

Gap handling needs the same precision. Carrying the last value forward through six missing samples may hide a pump shutdown, while dropping the window may hide a short overload. A robust implementation records the gap count, imputation method, clock source, model version, and baseline id beside the residual. Reviewers can then distinguish a true contextual anomaly from an artifact of timing or missing evidence.

Latency budgets should be checked against the method. A 30-minute STL window cannot support a two-minute safety response unless an edge rule raises a provisional alert first. Conversely, a one-sample edge residual may be fast but weak, so the gateway or cloud path should attach the later context review to the same incident id instead of creating a second, disconnected alert.

Under-the-Hood Knowledge Check