← Back to the chapter · Next detail →

Technical detail · Time-Series Anomalies: Evaluation and Types

Metric labels, windows, and latency

Detector metrics depend on how events are matched to labels. A vibration spike, a battery brownout, or a connectivity outage may last many samples but represent one incident. Evaluation should define the event window, the allowed detection delay, duplicate-alert suppression, and whether a late alert counts as success, partial success, or a miss.

Streaming systems add timing constraints. Kafka, Spark, Flink, or a gateway pipeline may score event-time windows, and those windows may close before late packets arrive. Metrics should record whether an alert was based on complete data, a provisional early window, imputed samples, or degraded sensor-health evidence. Otherwise, late data and missing samples can change both the label and the score after the dashboard has already reported a metric.

Choose one metric grain before comparing detectors. If the grain is samples, a 20-minute outage with four five-minute windows can create four positives. If the grain is incidents, those four windows may map to one outage. A detector that alerts on all four windows looks strong at sample recall but noisy at operator workload. The incident-grain record should keep first-detection time, last-alert time, duplicate count, and final disposition so precision and recall do not change when someone re-aggregates the same evidence later.

Late data also needs a declared rule. If a gateway sends an alert after three minutes using partial data and the cloud receives the missing packet five minutes later, the metric should preserve both the early alert state and the completed-window state. Otherwise teams may tune a detector on corrected historical data while operators experience provisional edge decisions in production. Keep that distinction in the release note so historical replays do not overwrite what the operator actually saw.

Event Matching

Map many samples or alerts to one incident using a declared window and duplicate-suppression policy.

Label Source

Separate confirmed maintenance records, operator labels, synthetic fault tests, and uncertain weak labels.

Latency Budget

Measure when the detector had enough evidence, when it alerted, and when review completed.

Segment Drift

Track metrics by site, sensor type, firmware version, season, and operating mode before averaging.

← Back to the chapter · Next detail →