Analytics & ML · Study deck
Edge Processing: Stream Window Contracts
Two edge services can report the same average from different time windows.
Data Dora is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: Worked example: same stream, three answers: Pick by the question: "how many per minute?" is tumbling; "what is the current smoothed rate?" is sliding; "what happened during one burst of activity?" is session.
- Explain: The three fundamental window types -- tumbling, sliding, and session -- answer different questions, and choosing the wrong one gives numbers that are technically correct but answer a question you did not ask.
- Explain: Session gaps need a timeout: A session window cannot close until a gap of inactivity passes, so it holds state until the timeout fires.
Major section
Edge Stream Window Contracts
A stream window must retain event time, late arrivals, and the rule that closed it before its output can trigger action.
- Edge Processing Patterns compares filter, aggregate, infer, and store-forward designs.
- Batch analytics runs over a finite dataset that sits still.
- Intuition: a window is a stopwatch strategy.
- Same stream, three very different reports.
Major section
Edge Stream Window Contracts (continued)
The three fundamental window types -- tumbling, sliding, and session -- answer different questions, and choosing the wrong one gives numbers that are technically correct but answer a question you did not ask.
- Worked example: a cold-chain gateway receives one temperature event every second from each trailer.
- If operations wants a compliance report, a tumbling one-hour window can produce one non-overlapping min/max/average for each trailer.
- Those labels prevent a maintainer from comparing incompatible numbers.
Major section
Edge Stream Window Contracts (continued)
If dispatch wants early warning that a trailer is warming, a sliding ten-minute window updated every minute gives a fresher trend.
- If the trailer sleeps between trips, a session window can group each active delivery run and close after a quiet gap.
- The raw stream is identical in all three cases; the window changes the business question answered by the same events.
- On a constrained gateway, that memory and CPU cost is real.
Major section
Edge Stream Window Contracts (continued)
At the edge, where memory is scarce, the practicalities of maintaining windows matter as much as their semantics.
- A downstream dashboard that says "average temperature" is ambiguous unless it also says whether the value came from a clock-aligned bucket, a rolling lookback, or one activity session.
- The wrong window can make alerts late, double-count events, or merge two separate operating periods.
- Percentiles and distinct-counts are harder and often use approximate sketches.
Major section
Edge Stream Window Contracts (continued)
Worked example: same stream, three answers: Pick by the question: "how many per minute?" is tumbling; "what is the current smoothed rate?" is sliding; "what happened during one burst of activity?" is session.
- None of those answers is wrong; each reflects a different operating question and a different amount of state held by the gateway.
- Incremental aggregation: Sums, counts, and averages can be updated incrementally as events arrive and leave a window, avoiding a full recompute.
- Choosing the gap is a trade-off between splitting one activity and merging two.
Major section
Edge Stream Window Contracts (continued)
Session gaps need a timeout: A session window cannot close until a gap of inactivity passes, so it holds state until the timeout fires.
- Bounded state or it grows: Every open window is memory.
- Streaming systems must expire and emit windows promptly; a stuck or ever-growing window is a classic cause of edge memory exhaustion.
- So window choice is both a semantic and a resource decision.
Major section
Edge Stream Window Contracts (continued)
Tumbling is cheapest and cleanest for periodic aggregates; sliding gives smooth, responsive trends at higher cost; session captures natural bursts of activity but must hold state until a quiet gap.
- On the edge, the right window is the one that answers the question with the least state held open.
- If the gateway also tracks per-device sessions, each active device keeps its own timeout and partial aggregate.
- The engineering decision is not just accuracy; it is whether the available memory, CPU, and battery budget can support the state implied by the chosen window.
Major section
Edge Stream Window Contracts (continued)
For a 100 Hz vibration stream, a tumbling one-minute average only needs a running sum, a count, and perhaps a min/max pair for the current minute.
- A sliding ten-minute window with a one-second slide covers 600 seconds of recent data, so a naive design can drift toward 60,000 retained samples per sensor.
- A practical edge design stores smaller per-second buckets in a ring buffer and subtracts expired buckets as the window advances, but that still multiplies memory by the number of sensors and keyed streams.
- Edge Patterns Placement and Cost Contracts for deciding where windowed processing should run.
Major section
Edge Stream Window Contracts (continued)
Late events need a cutoff rule, clock drift needs a timestamp policy, and reboot recovery needs a decision about whether partial windows are replayed, discarded, or emitted with a quality flag.
- Those operational details are easy to hide in code, but they decide whether edge results are repeatable during outages.
- A good review therefore checks three items: the window semantics match the question, the state has a bounded maximum, and the emitted result records the window start, window end, key, and any lateness or recovery caveat.
- Edge Processing Patterns for the filter, aggregate, infer, and store-forward choices that consume these windows.
Deck summary
Key takeaways
A stream window must retain event time, late arrivals, and the rule that closed it before its output can trigger action.
- The three fundamental window types -- tumbling, sliding, and session -- answer different questions, and choosing the wrong one gives numbers that are technically correct but answer a question you did not ask.
- If dispatch wants early warning that a trailer is warming, a sliding ten-minute window updated every minute gives a fresher trend.
- At the edge, where memory is scarce, the practicalities of maintaining windows matter as much as their semantics.
Retrieval practice
Recall check 1 of 6
Data Dora says: answer from memory, then check your reasoning.
Q1Why does streaming analytics compute over windows rather than the whole stream at once?
Show answer
Answer: A Windows define finite scopes so counts and averages can be emitted as events arrive without waiting for all future events.
Retrieval practice
Recall check 2 of 6
Data Dora says: answer from memory, then check your reasoning.
Q2After deduplicating events, you need a non-overlapping event-time count for each clock minute, with each accepted event assigned to one minute. Which window type fits?
Show answer
Answer: C One-minute tumbling event-time windows assign each deduplicated event to one clock-minute interval; deduplication is a separate prerequisite.
Retrieval practice
Recall check 3 of 6
Data Dora says: answer from memory, then check your reasoning.
Q3A gateway restarts while a stream window is open. What should its emitted metric record explain?
Show answer
Answer: D These rules explain what data the metric includes after restart.
Retrieval practice
Recall check 4 of 6
Data Dora says: answer from memory, then check your reasoning.
Q4On a memory-constrained gateway, why can a 10-minute sliding window that advances every 10 seconds be expensive?
Show answer
Answer: B With separate state per open window, this 10-minute width and 10-second step overlap about 60 windows.
Retrieval practice
Recall check 5 of 6
Data Dora says: answer from memory, then check your reasoning.
Q5A smart greenhouse has 200 humidity sensors, each sending a reading every second. The greenhouse only needs to know when humidity drops below 40% or rises above 80%. Which edge processing pattern should the gateway use to minimize bandwidth?
Show answer
Answer: D When only threshold transitions matter, the gateway can filter routine readings and send crossing events.
Retrieval practice
Recall check 6 of 6
Data Dora says: answer from memory, then check your reasoning.
Q6A fleet of delivery trucks uses satellite connectivity that costs $5 per decimal MB and is available only 30% of the time. Each truck generates 500 decimal KB per hour from GPS, temperature, and vibration sensors. Dispatchers need hour-level trends after outages, and no local history is kept unless the chosen pattern buffers it. Which pattern combination fits?
Show answer
Answer: C Store-and-forward preserves hour-level history while disconnected; aggregation sends compact trend summaries when the link returns.
Print reference
Answers 1 of 2
Answer key.
- A · Windows define finite scopes so counts and averages can be emitted as events arrive without waiting for all future events.
- C · One-minute tumbling event-time windows assign each deduplicated event to one clock-minute interval; deduplication is a separate prerequisite.
- D · These rules explain what data the metric includes after restart.
- B · With separate state per open window, this 10-minute width and 10-second step overlap about 60 windows.
Print reference
Answers 2 of 2
Answer key.
- D · When only threshold transitions matter, the gateway can filter routine readings and send crossing events.
- C · Store-and-forward preserves hour-level history while disconnected; aggregation sends compact trend summaries when the link returns.