Analytics & ML · Study deck

Time-Series Anomalies: Evaluation and Types

A detector may catch nine faults and still waste the operator's day with false alarms.

Data Dora is your guide for this deck.

anomalytimeseries
Data Dora, the module guide, in a scene from this chapter.
iotclass.org

After studying this chapter

Learning objectives

You will be able to:

  • Explain: A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.
  • Explain: Point anomalies are unusual single values, contextual anomalies are unusual only under specific conditions, and collective anomalies are unusual patterns across windows or groups.
  • Explain: In: Collective Anomaly, Sensor 1 through Sensor 4 are individually plausible while the labelled: Pattern: Increasing variance is abnormal only as a group.
  • Explain: Sensor faults should remain a separate boundary so operations teams do not confuse measurement failure with process failure.
iotclass.org

Major section

Start With the Decision · Anomaly Detector Evaluation

A detector may catch nine faults and still waste the operator's day with false alarms.

  • Confusion counts and anomaly type show whether the model is useful.
  • Twelve point to real faults, while eight real fault windows are missed.
  • Latency means the time from enough evidence existing to the reviewed action that matters.

Key terms

Anomaly detection
Anomaly detection is usually a rare-event problem.
True positives
True positives are real incidents that were alerted.

Why it matters

If 12 alerts match real incidents and 28 are normal defrost cycles, the sample-level accuracy still looks high because most windows are normal.

Evaluation is a threshold decision, not a single score: precision, recall, F1, confusion matrix evidence, and operating cost all feed threshold selection.
Evaluation is a threshold decision, not a single score: precision, recall, F1, confusion matrix evidence, and operating cost all feed threshold selection.
iotclass.org

Major section

Point, Contextual, Collective Anomalies

That may be a bad point.

  • Yet the same value may be normal while the door is open for loading.
  • A slow rise across many readings may matter even when no single value crosses a limit.
  • The alert should show why this evidence set was used.
  • A wide group can reveal drift, but it may react late.

Key terms

Pattern: Increasing variance
Pattern: Increasing variance is abnormal only as a group.

Why it matters

The detector and alert record must therefore retain the matching boundary—sample, context, or window—because that choice controls the baseline, retest, and operator response used next.

Anomaly type defines the evidence boundary: one extreme value, one value interpreted in context, or a group pattern that is only abnormal when read together.
Anomaly type defines the evidence boundary: one extreme value, one value interpreted in context, or a group pattern that is only abnormal when read together.
iotclass.org

Major section

Point, Contextual, Collective Anomalies (continued)

In: Collective Anomaly, Sensor 1 through Sensor 4 are individually plausible while the labelled: Pattern: Increasing variance is abnormal only as a group.

  • A tight rule can catch a clear fault fast, but it may raise false alarms.
  • This first pass does not choose a maths method or prove the cause.
  • Anomaly type is an engineering decision.
iotclass.org

Major section

Point, Contextual, Collective Anomalies (continued)

A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.

  • Point anomalies are unusual single values, contextual anomalies are unusual only under specific conditions, and collective anomalies are unusual patterns across windows or groups.
  • Sensor faults should remain a separate boundary so operations teams do not confuse measurement failure with process failure.
  • Classify the anomaly type first, then select the detector, evidence record, deployment tier, and retest trigger.
  • Statistical Methods Use simple statistical scores for point anomalies and edge-friendly candidate alerts.
iotclass.org

Deck summary

Key takeaways

A detector may catch nine faults and still waste the operator's day with false alarms.

  • That may be a bad point.
  • In: Collective Anomaly, Sensor 1 through Sensor 4 are individually plausible while the labelled: Pattern: Increasing variance is abnormal only as a group.
  • A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.
iotclass.org

Retrieval practice

Recall check 1 of 6

Data Dora says: answer from memory, then check your reasoning.

Q1Why can ordinary accuracy mislead an IoT anomaly-detection review?

ABecause normal windows usually dominate
BBecause accuracy needs probability scores.
CBecause false alarms outweigh missed incidents.
DBecause precision and recall do not require labels.
Show answer

Answer: A Class imbalance can make accuracy look strong even when anomaly detection is operationally weak.

iotclass.org

Retrieval practice

Recall check 2 of 6

Data Dora says: answer from memory, then check your reasoning.

Q2A validation set has 20 real incidents, 30 alerts, 12 true positives, and 18 false positives. What does precision measure?

A12 / 30 = 0.40, meaning 40% of alerts were real reviewed incidents.
B12 / 20 = 0.60, meaning 60% of real incidents were caught.
C18 / 9,980 = about 0.18%, meaning normal-window false positive rate.
D9,962 / 10,000 = about 99.62%, meaning the detector is operationally strong.
Show answer

Answer: A Precision is TP divided by all alerts, while recall is TP divided by all real incidents.

iotclass.org

Retrieval practice

Recall check 3 of 6

Data Dora says: answer from memory, then check your reasoning.

Q3Why should an anomaly-metrics pipeline define event matching and duplicate suppression before computing precision and recall?

ABecause repeated alerts should be matched to separate operator visits, so an incident that needs more review contributes more true positives to the recall calculation.
BBecause one incident can create many samples or alerts, and counting each duplicate as a separate event distorts both alert quality and missed-event counts.
CBecause event matching should replace operator labels by assigning each alert to a detector threshold, even when maintenance evidence is missing.
DBecause precision and recall should be calculated only from true negatives once alerts have been grouped into site-level operating windows.
Show answer

Answer: B IoT anomalies often span windows and repeated alerts, so metrics need an explicit mapping from alerts to incidents.

iotclass.org

Retrieval practice

Recall check 4 of 6

Data Dora says: answer from memory, then check your reasoning.

Q4Twelve conveyor-speed sensors all stay inside their normal individual bands, but every sensor drifts upward together over two hours. Which anomaly type best describes the evidence?

AA collective anomaly, because the evidence is the shared drift pattern across sensors.
BA point anomaly, because each conveyor sample can be checked against its band.
CA contextual anomaly, because the two-hour interval changes the expected speed.
DNo anomaly, because the individual limit checks pass across the two-hour window.
Show answer

Answer: A Anomaly type follows the evidence: single value, context-conditioned value, or group pattern.

iotclass.org

Retrieval practice

Recall check 5 of 6

Data Dora says: answer from memory, then check your reasoning.

Q5Why is a plain point threshold a poor first choice for a contextual anomaly problem?

AThe same reading can be normal or abnormal under different modes, loads, or nearby signals.
BA point threshold reacts too slowly, so the first change should be to increase the sampling rate.
CContext needs a deep learning model, so threshold review should wait until that model has been trained.
DSensor-health evidence should be ignored once a contextual score is available.
Show answer

Answer: A Detector choice follows the type of evidence required by the anomaly.

iotclass.org

Retrieval practice

Recall check 6 of 6

Data Dora says: answer from memory, then check your reasoning.

Q6Why should the anomaly type be stored with the alert record in a production IoT system?

AIt explains what evidence was evaluated and how reviewers should retest the alert.
BIt replaces the need to store values, scores, windows, or context fields.
CIt proves all anomaly types should use one baseline and one metric.
DIt prevents concept drift automatically once the label is attached.
Show answer

Answer: A A production alert needs reviewable evidence, and anomaly type defines the evidence boundary.

iotclass.org

Print reference

Answers 1 of 2

Answer key.

  1. A · Class imbalance can make accuracy look strong even when anomaly detection is operationally weak.
  2. A · Precision is TP divided by all alerts, while recall is TP divided by all real incidents.
  3. B · IoT anomalies often span windows and repeated alerts, so metrics need an explicit mapping from alerts to incidents.
  4. A · Anomaly type follows the evidence: single value, context-conditioned value, or group pattern.
  5. A · Detector choice follows the type of evidence required by the anomaly.
iotclass.org

Print reference

Answers 2 of 2

Answer key.

  1. A · A production alert needs reviewable evidence, and anomaly type defines the evidence boundary.
iotclass.org