โ† Back to the chapter

Technical detail ยท Edge Processing: Stream Window Contracts

Window state and incremental aggregation

For a 100 Hz vibration stream, a one-minute tumbling average needs a running sum and count, with perhaps a minimum/maximum pair. A ten-minute sliding window with a one-second slide spans 600 seconds: a naive implementation can retain 60,000 samples per sensor.

A ring buffer of smaller per-second buckets can subtract expired buckets as the window advances. Sums, counts, and averages support incremental updates as events arrive and leave, avoiding full recomputation. Percentiles and distinct counts are harder and often use approximate sketches. Shared incremental state can reduce the cost of maintaining separate aggregates for every overlapping window.

For a ten-minute window advancing every ten seconds, width divided by step gives about 60 overlapping windows. Separate state per window can therefore be expensive. Advancing once every ten minutes removes the overlap, and each event contributes to one window instead. Neither schedule requires an infinite window lifetime.

Multiply the chosen state by the number of sensors and keyed streams, including worst-case device counts and reconnect bursts. Add per-device session timeouts and partial aggregates where needed. Select the least state that still answers the operating question: tumbling for periodic reports, sliding for responsive trends, or session for natural activity bursts.

โ† Back to the chapter