Analytics & ML · Study deck
Edge Processing: Stream Window Contracts
Two edge services can report the same average from different time windows.
Data Dora is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: A practical edge design stores smaller per-second buckets in a ring buffer and subtracts expired buckets as the window advances, but that still multiplies memory by the number of sensors and keyed streams.
- Explain: Worked example: same stream, three answers: Pick by the question: "how many per minute?" is tumbling; "what is the current smoothed rate?" is sliding; "what happened during one burst of activity?" is session.
- Explain: None of those answers is wrong; each reflects a different operating question and a different amount of state held by the gateway.
Major section
Edge Stream Window Contracts
A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.
- Edge Processing Patterns compares filter, aggregate, infer, and store-forward designs.
- Batch analytics runs over a finite dataset that sits still.
- Intuition: a window is a stopwatch strategy.
Major section
Edge Stream Window Contracts (continued)
Those labels prevent a maintainer from comparing incompatible numbers.
- The three fundamental window types -- tumbling, sliding, and session -- answer different questions, and choosing the wrong one gives numbers that are technically correct but answer a question you did not ask.
- Same stream, three very different reports.
- On a constrained gateway, that memory and CPU cost is real.
Major section
Edge Stream Window Contracts (continued)
Worked example: a cold-chain gateway receives one temperature event every second from each trailer.
- If operations wants a compliance report, a tumbling one-hour window can produce one non-overlapping min/max/average for each trailer.
- If dispatch wants early warning that a trailer is warming, a sliding ten-minute window updated every minute gives a fresher trend.
- The wrong window can make alerts late, double-count events, or merge two separate operating periods.
Major section
Edge Stream Window Contracts (continued)
None of those answers is wrong; each reflects a different operating question and a different amount of state held by the gateway.
- If the trailer sleeps between trips, a session window can group each active delivery run and close after a quiet gap.
- The raw stream is identical in all three cases; the window changes the business question answered by the same events.
- Percentiles and distinct-counts are harder and often use approximate sketches.
Major section
Edge Stream Window Contracts (continued)
Choosing the gap is a trade-off between splitting one activity and merging two.
- A downstream dashboard that says "average temperature" is ambiguous unless it also says whether the value came from a clock-aligned bucket, a rolling lookback, or one activity session.
- At the edge, where memory is scarce, the practicalities of maintaining windows matter as much as their semantics.
- Bounded state or it grows: Every open window is memory.
Major section
Edge Stream Window Contracts (continued)
Worked example: same stream, three answers: Pick by the question: "how many per minute?" is tumbling; "what is the current smoothed rate?" is sliding; "what happened during one burst of activity?" is session.
- Incremental aggregation: Sums, counts, and averages can be updated incrementally as events arrive and leave a window, avoiding a full recompute.
- Session gaps need a timeout: A session window cannot close until a gap of inactivity passes, so it holds state until the timeout fires.
- So window choice is both a semantic and a resource decision.
Major section
Edge Stream Window Contracts (continued)
Streaming systems must expire and emit windows promptly; a stuck or ever-growing window is a classic cause of edge memory exhaustion.
- Tumbling is cheapest and cleanest for periodic aggregates; sliding gives smooth, responsive trends at higher cost; session captures natural bursts of activity but must hold state until a quiet gap.
- On the edge, the right window is the one that answers the question with the least state held open.
- If the gateway also tracks per-device sessions, each active device keeps its own timeout and partial aggregate.
- Edge Processing Patterns for the filter, aggregate, infer, and store-forward choices that consume these windows.
Major section
Edge Stream Window Contracts (continued)
The engineering decision is not just accuracy; it is whether the available memory, CPU, and battery budget can support the state implied by the chosen window.
- For a 100 Hz vibration stream, a tumbling one-minute average only needs a running sum, a count, and perhaps a min/max pair for the current minute.
- A sliding ten-minute window with a one-second slide covers 600 seconds of recent data, so a naive design can drift toward 60,000 retained samples per sensor.
- Those operational details are easy to hide in code, but they decide whether edge results are repeatable during outages.
Major section
Edge Stream Window Contracts (continued)
Edge Patterns Placement and Cost Contracts for deciding where windowed processing should run.
- A practical edge design stores smaller per-second buckets in a ring buffer and subtracts expired buckets as the window advances, but that still multiplies memory by the number of sensors and keyed streams.
- Late events need a cutoff rule, clock drift needs a timestamp policy, and reboot recovery needs a decision about whether partial windows are replayed, discarded, or emitted with a quality flag.
- Sliding windows cost more: Because events belong to many overlapping windows, sliding windows keep more in-flight state and recompute more often than tumbling ones.
Deck summary
Key takeaways
A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.
- Those labels prevent a maintainer from comparing incompatible numbers.
- Worked example: a cold-chain gateway receives one temperature event every second from each trailer.
- None of those answers is wrong; each reflects a different operating question and a different amount of state held by the gateway.
- Choosing the gap is a trade-off between splitting one activity and merging two.
Retrieval practice
Recall check 1 of 5
Data Dora says: answer from memory, then check your reasoning.
Q1Why does streaming analytics compute over windows rather than the whole stream at once?
Show answer
Answer: A Windows make an infinite stream tractable by defining finite scopes for computation.
Retrieval practice
Recall check 2 of 5
Data Dora says: answer from memory, then check your reasoning.
Q2You need a non-overlapping count of events for each clock minute, with every event counted exactly once. Which window type fits?
Show answer
Answer: A Tumbling windows partition the stream so each event is counted once.
Retrieval practice
Recall check 3 of 5
Data Dora says: answer from memory, then check your reasoning.
Q3On a memory-constrained gateway, why can a 10-minute sliding window that advances every 10 seconds be expensive?
Show answer
Answer: A Fine-grained overlap multiplies open windows and state, which is costly on constrained hardware.
Retrieval practice
Recall check 4 of 5
Data Dora says: answer from memory, then check your reasoning.
Q4A smart greenhouse has 200 humidity sensors, each sending a reading every second. The greenhouse only needs to know when humidity drops below 40% or rises above 80%. Which edge processing pattern should the gateway use to minimize bandwidth?
Show answer
Answer: B The Filter pattern is ideal when only exceptions (threshold violations) matter.
Retrieval practice
Recall check 5 of 5
Data Dora says: answer from memory, then check your reasoning.
Q5A fleet of delivery trucks uses satellite connectivity that costs $5 per MB and is available only 30% of the time. Each truck has GPS, temperature, and vibration sensors generating 500 KB per hour. Which edge processing pattern combination is most appropriate?
Show answer
Answer: C With expensive, intermittent satellite connectivity, the best approach combines Store-Forward (buffer during the 70% offline time) with Aggregate (reduce 500 KB/hour to small summaries) to minimize both data loss and satellite costs.
Print reference
Answers 1 of 2
Answer key.
- A · Windows make an infinite stream tractable by defining finite scopes for computation.
- A · Tumbling windows partition the stream so each event is counted once.
- A · Fine-grained overlap multiplies open windows and state, which is costly on constrained hardware.
- B · The Filter pattern is ideal when only exceptions (threshold violations) matter.
Print reference
Answers 2 of 2
Answer key.
- C · With expensive, intermittent satellite connectivity, the best approach combines Store-Forward (buffer during the 70% offline time) with Aggregate (reduce 500 KB/hour to small summaries) to minimize both data loss and satellite costs.