13 Edge Processing: Aggregation Decisions

13.1 Start With the Decision

This section: Start With the Decision — A threshold is a cutoff used to keep or discard readings.

A threshold is a cutoff used to keep or discard readings. It can save bandwidth, the amount of data a link can carry in a set time, but it can also erase a short alarm. Test the filter rule before an edge node, a computer near the sensor, groups samples into shorter summary reports.

13.2 Route Overview

This section: Route Overview — This is part 2 of 3.

This is part 2 of 3. Review Edge Processing: Filtering Patterns for the preceding evidence.

13.3 Learning Objectives

This section: Learning Objectives

• Tune filter thresholds with false-alarm evidence.
• Select an edge aggregation window from latency and bandwidth needs.

13.4 Chapter Roadmap

This section: Chapter Roadmap — Aggregation should reduce traffic without erasing the field story.

• Try It: Filter Threshold Simulator
• Pattern 2: Aggregate at Edge
• Putting Numbers to It
• Pattern 3: Infer at Edge
• Try It: Edge ML vs Cloud ML Comparison
• Pattern 4: Store and Forward
• Try It: Store-Forward Buffer Simulator
• Checkpoint: Run the Four Contracts
• Trade-off Analysis
• Edge ML Inference vs Cloud ML Inference
• Edge ML vs Cloud ML Inference
• Batch Processing vs Real-Time Streaming
• Batch vs Streaming at Edge
• When Edge Saves Money
• Edge Does Not Always Save Cost
• Edge vs Cloud Breakeven Calculator
• Checkpoint: Price the Trade-off
• Illustrative Supermarket Fleet
• Checkpoint: Validate the Fleet Mix
• Interactive Quiz: Match Concepts
• Interactive Quiz: Sequence the Steps
• Label the Diagram
• Continue: Edge Stream Window Contracts

Aggregation should reduce traffic without erasing the field story. Figure 13.1 follows soil, air, and leaf readings through alignment, combination, outage buffering, and an irrigation decision.

+

At ALIGN, Figure 13.1 allows at most 2 s clock skew inside the 60 s event window; COMBINE takes the median of redundant probes and retains quality flag Q1. When BUFFER stores 1,080 rows during the 18-minute outage, ordered replay and the raw-lineage link keep aggregation from silently changing the irrigation evidence.

13.5 Pattern 2: Aggregate at Edge

This section: Pattern 2: Aggregate at Edge

The aggregate pattern computes statistics locally and sends summaries:

# Example: Statistical aggregation
import statistics

def aggregate_window(readings, window_minutes=60):
    return {
        "min": min(readings),
        "max": max(readings),
        "avg": sum(readings) / len(readings),
        "stddev": statistics.stdev(readings),
        "count": len(readings),
        "window_end": now()
    }

Best for:

• Trend analysis where individual readings are less important than patterns
• Environmental monitoring (temperature, humidity, air quality)
• Capacity planning and historical analysis

13.6 Pattern 3: Infer at Edge

This section: Pattern 3: Infer at Edge

The infer pattern runs machine learning models locally:

# Example: Anomaly detection at edge
model = load_tflite_model("anomaly_detector.tflite")

def infer_anomaly(sensor_data):
    prediction = model.predict(sensor_data)
    if prediction["anomaly_score"] > 0.85:
        return {"anomaly": True, "score": prediction["anomaly_score"],
                "features": sensor_data}
    return None  # Normal - don't transmit

Best for:

• Visual inspection (defect detection in manufacturing)
• Predictive maintenance (vibration analysis)
• Safety systems requiring immediate response

13.7 Pattern 4: Store and Forward

This section: Pattern 4: Store and Forward — Read the Pattern 4: Store and Forward material as a decision path rather than as isolated entries.

Read the Pattern 4: Store and Forward material as a decision path rather than as isolated entries. First identify the operating condition in each entry and keep its units, timing, source, and assumed system state attached to it. Next compare the entries at the point where responsibility changes between device, gateway, network, analytic service, and operator; that hand-off is where apparently similar choices often produce different outcomes. Then follow the failure case: ask what becomes stale, delayed, unavailable, or unsafe, who detects it, and what evidence permits recovery. Finally connect the result to the chapter’s running design record by naming the selected behavior, the rejected alternative, the measurement that justifies the choice, and the condition that forces a recheck. That order turns the examples or comparison into an auditable engineering argument.

The store-and-forward pattern handles intermittent connectivity:

# Example: Store-and-forward buffer
class EdgeBuffer:
    def __init__(self, max_size_mb=100):
        self.buffer = []
        self.max_size = max_size_mb * 1024 * 1024

    def store(self, reading):
        self.buffer.append(reading)
        self.compact_if_needed()

    def forward_when_connected(self):
        if network_available():
            batch = self.buffer.copy()
            if upload_batch(batch):
                self.buffer.clear()

Best for:

• Remote sites (oil rigs, agricultural fields, offshore platforms)
• Mobile assets (vehicles, shipping containers, drones)
• Any deployment with unreliable connectivity

+Checkpoint: Run the Four Contracts

• A threshold filter sends exceptions and discards routine readings.
• A windowed aggregate compresses many samples into min, max, avg, stddev, count, and window metadata.
• An inference step sends anomaly evidence, while Store-Forward keeps buffered readings until the network returns.

Next, decide which trade-off the deployment can defend when latency, accuracy, power, connectivity, and cost conflict.

13.8 Trade-off Analysis

This section: Trade-off Analysis

13.9 Edge ML Inference vs Cloud ML Inference

This section: Edge ML Inference vs Cloud ML Inference

13.10 Batch Processing vs Real-Time Streaming

This section: Batch Processing vs Real-Time Streaming

13.11 When Edge Saves Money

This section: When Edge Saves Money

13.12 Illustrative Supermarket Fleet

This section: Illustrative Supermarket Fleet — Scenario: Assume a fleet of 40,000 supermarkets.

Scenario: Assume a fleet of 40,000 supermarkets. Each store has 12 display cases with 3 sensors each (temperature, defrost status, compressor current), reporting every 30 seconds.

Given:

• 40,000 stores x 12 cases x 3 sensors = 1,440,000 sensors
• 30-second intervals = 86,400 s / 30 s = 2,880 readings/sensor/day
• Raw payload: 8 bytes per reading (timestamp + value)
• Cellular backhaul at EUR 0.50/MB

Step 1: Calculate raw data volume

Step 2: Apply edge patterns per sensor type

Step 3: Calculate optimized volumes

Each sensor type accounts for one-third of the 1,440,000 sensors (480,000 sensors each). The combined volume and cost below are modeled using the stated record-size assumptions; compressor bytes depend on timestamp, RMS, and peak encoding:

Result for this illustrative fleet: Edge processing reduces modeled daily cellular costs from EUR 16,600 to about EUR 722.69 — a 95.6% reduction by record count. In this illustrative fleet, the gateway investment (40,000 gateways x EUR 120 = EUR 4.8 million) recovers in about 10 months at the assumed daily cellular savings of about EUR 15,877.31.

Key Insight: The three sensor types in the same deployment use three different edge patterns. Temperature uses Filter (only exceptions matter), defrost uses Filter (only state changes matter), and compressor current uses Aggregate (trend data matters). Pattern selection is per-sensor, not per-deployment.

13.13 Continue: Edge Stream Window Contracts

This section: Continue: Edge Stream Window Contracts — The four processing patterns above still need precise stream semantics before a gateway emits reliable metrics.

The four processing patterns above still need precise stream semantics before a gateway emits reliable metrics. Continue to Edge Stream Window Contracts for tumbling, sliding, and session window rules, state budgets, late-event policy, and replay metadata.

The level map in Figure 13.2 locates the processing patterns in this chapter between bit transport and durable storage.

+

In Figure 13.2, Transform near the source names the Level 3 work: filter noise, normalize units, detect thresholds, and retain provenance. Its output then reaches Turn streams into queryable records at Level 4, preventing filter, aggregate, and infer patterns from being confused with the storage system that retains their results.

13.14 Continue to the Next Part

This section: Continue to the Next Part — Carry this evidence into Edge Processing: Stream Window Contracts, which begins with Edge Stream Window Contracts.

Carry this evidence into Edge Processing: Stream Window Contracts, which begins with Edge Stream Window Contracts.