Analytics & ML · Study deck
Time-Series Anomalies: Baselines and Residuals
A freezer can warm during defrost and still be healthy.
Data Dora is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- choose a temporal baseline that preserves seasonality and drift
- compute a residual score and keep sampling latency visible
- Explain: A fixed alarm line may flag every warm afternoon, even though the pattern is normal.
- Explain: Under the Hood shows how sample rate, delay, and drift can change the result.
Major section
Start With the Story
A fixed alarm line may flag every warm afternoon, even though the pattern is normal.
- The useful question is whether today differs from the pattern expected at this time.
- An unusual value is not proof of a fault.
- A normal-looking value is not proof of health if the baseline or sampling has changed.
Major section
Start With the Story (continued)
Under the Hood shows how sample rate, delay, and drift can change the result.
- This makes a late but accurate warning easy to spot.
- Reopen the baseline after a repair, season change, software change, or new work pattern.
- A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.
Major section
Time-Series Residual Scoring
A value can be normal at one hour, season, operating mode, or cycle stage and abnormal at another.
- The usual pattern is to forecast or decompose the expected value, compute a residual, and alert only when the residual is large enough to matter.
- A raw threshold cannot tell those cases apart.
Major section
Time-Series Residual Scoring (continued)
This makes time-series methods a natural fit for contextual anomalies in IoT: load profiles, temperature cycles, vibration windows, occupancy patterns, battery discharge, and watermarks in stream processing.
- The detector must record the forecast window, seasonal context, residual, threshold, missing-data handling, and latency budget.
- ARIMA-style forecasting models trend and repeating cycles, exponential smoothing adapts the recent level, and STL separates trend and seasonality explicitly.
- That order advances the chapter's evidence chain: preserve the context, forecast or decomposition, residual, and decision threshold together so two identical readings can legitimately receive different decisions.
Major section
Time-Series Residual Scoring (continued)
Worked example: a freezer temperature of 6 deg C may be normal during a scheduled defrost cycle and abnormal during a steady cooling period.
- A time-aware detector compares the reading with the expected value for that minute, operating mode, and recent history, then stores the residual and context that made the decision reviewable.
- A 13 kW residual at 02:00 may be suspicious if the site is normally idle, while the same raw load during a shift change may be expected.
- That evidence lets reviewers compare the alert with the baseline that actually existed at decision time.
Major section
Choose Temporal Baseline First
If the signal has daily or weekly cycles, use a seasonal baseline or STL residual.
- If it changes slowly but has sudden level shifts, use EWMA or Holt-Winters-style smoothing.
- If anomalies are window shapes, score windows rather than single points.
- Missing timestamps, daylight-saving changes, holiday modes, and new schedules.
Major section
Choose Temporal Baseline First (continued)
If samples arrive every minute but the operations team can wait five minutes, a short persistence rule can suppress one-off meter noise.
- If a motor trip must be caught within one sample, use an edge EWMA or threshold residual and send a richer window to the gateway for review.
- The design should state which layer owns the first alert and which layer owns the later explanation.
- Non-stationary drift, abrupt maintenance changes, and unstable sampling intervals.
Major section
Sampling, Drift, and Latency
Drift control is equally important.
- Irregular sampling, delayed packets, clock drift, duplicated readings, and gaps change the residual.
- Stream deployments also need explicit watermarks or late-data policies: a detector cannot safely score a window if the window may still receive late readings.
- Seasonal baselines become stale when equipment, firmware, occupancy, weather, or operating policy changes.
Major section
Sampling, Drift, and Latency (continued)
The alert record should say whether the data was complete, late, imputed, or scored in degraded mode.
- Production systems should separate drift monitoring from incident scoring and require baseline review after known site changes.
- The chosen policy changes both alert quality and response time, so it belongs in the incident record.
- Gap handling needs the same precision.
Major section
Sampling, Drift, and Latency (continued)
Reviewers can then distinguish a true contextual anomaly from an artifact of timing or missing evidence.
- Carrying the last value forward through six missing samples may hide a pump shutdown, while dropping the window may hide a short overload.
- Latency budgets should be checked against the method.
- Temporal context is missing or the seasonal period is wrong.
Major section
Sampling, Drift, and Latency (continued)
Watermark Defines when a time window is complete enough to score despite late or out-of-order events.
- A robust implementation records the gap count, imputation method, clock source, model version, and baseline id beside the residual.
- A 30-minute STL window cannot support a two-minute safety response unless an edge rule raises a provisional alert first.
- Gap Policy Records whether missing samples are dropped, imputed, held, or treated as sensor-health evidence.
Major section
Sampling, Drift, and Latency (continued)
Conversely, a one-sample edge residual may be fast but weak, so the gateway or cloud path should attach the later context review to the same incident id instead of creating a second, disconnected alert.
- Baseline Review Triggers retesting after maintenance, firmware updates, schedule changes, or sustained drift.
- A partial window looks abnormal, then becomes normal after delayed readings arrive.
- A slow degradation stops alerting after the adaptive baseline catches up.
Deck summary
Key takeaways
A fixed alarm line may flag every warm afternoon, even though the pattern is normal.
- Under the Hood shows how sample rate, delay, and drift can change the result.
- A value can be normal at one hour, season, operating mode, or cycle stage and abnormal at another.
- This makes time-series methods a natural fit for contextual anomalies in IoT: load profiles, temperature cycles, vibration windows, occupancy patterns, battery discharge, and watermarks in stream processing.
- Worked example: a freezer temperature of 6 deg C may be normal during a scheduled defrost cycle and abnormal during a steady cooling period.
Retrieval practice
Recall check 1 of 3
Data Dora says: answer from memory, then check your reasoning.
Q1Why should a seasonal IoT signal usually be scored on residuals instead of raw readings?
Show answer
Answer: B Time-series anomaly detection compares observations with temporal expectations.
Retrieval practice
Recall check 2 of 3
Data Dora says: answer from memory, then check your reasoning.
Q2A site has strong weekday/weekend and hour-of-day load cycles. What is the safest first detector?
Show answer
Answer: A Seasonal and contextual signals should be scored against the matching temporal baseline.
Retrieval practice
Recall check 3 of 3
Data Dora says: answer from memory, then check your reasoning.
Q3Why does a streaming time-series detector need a late-data or watermark policy?
Show answer
Answer: C Time-series anomaly decisions depend on whether the relevant window and context are complete.
Print reference
Answers
Answer key.
- B · Time-series anomaly detection compares observations with temporal expectations.
- A · Seasonal and contextual signals should be scored against the matching temporal baseline.
- C · Time-series anomaly decisions depend on whether the relevant window and context are complete.