12 Edge Processing: Filtering Patterns

12.1 Start With the Decision

A vibration node can send every sample or remove noise near the machine. The right filter cuts traffic without hiding a real fault.

12.2 Route Overview

This is part 1 of 3. Continue with Edge Processing: Aggregation Decisions.

12.3 Part Objectives

• Choose edge filters from signal and fault evidence.
• Compare raw and filtered traffic costs.

12.4 Start With the Story

Decide What Must Happen Near the Sensor

Picture a freezer sensor sending one reading each second. The site needs a fast alarm, but the central service needs only a short trend and a record of unusual events. Sending every raw value may waste power and make an outage harder to manage.

Edge processing means changing data on or near the device before it travels farther. Start with the job. Filter values that have no use. Combine a group into a clear summary. Make a local guess only when the model and its limits are known. Store results during an outage and send them later in the right order.

Choose one pattern from the failure you must prevent. A fast alarm favors local action. A costly link favors a smaller summary. An unreliable link needs a safe store-and-send path. Keep the original time, rule version, and quality note so later users can explain the result.

First review

• Name the input.
• Name the useful output.
• Set the response time.
• Set the size limit.
• Keep the source time.
• Keep the rule version.
• Mark missing values.
• Mark low trust.
• Test a lost link.
• Test a full store.
• Test a restart.
• Compare the final result.

The edge is not always cheaper or safer. It adds power, code, updates, and local failure.

Use Practitioner to place and test each pattern. Use Under the Hood for timing, models, cost, and buffer limits.

Picture an IoT team using the ideas in Edge Processing Patterns during a live operations review. A device has produced messy evidence, an analytic step is about to change an alert or control decision, and someone has to explain why the result should be trusted.

Read this page as that path from sensor evidence to accountable action. Start with what the system observes, keep the model or data treatment visible, and finish with the check that would convince an operator, maintainer, or auditor to act.

12.5 Learning Objectives

By the end of this chapter, you will be able to:

• Apply Four Edge Processing Patterns: Implement Filter, Aggregate, Infer, and Store-Forward patterns for different IoT scenarios
• Select Optimal Patterns: Match processing patterns to application requirements based on latency, bandwidth, and reliability needs
• Evaluate Trade-offs: Compare edge ML inference versus cloud ML, and batch processing versus real-time streaming
• Avoid Common Mistakes: Understand when edge processing saves costs versus when it adds unnecessary complexity

The mathematical gist. The chapter’s 1 Hz temperature stream has a 0.5 Hz Nyquist ceiling, while its 1 kHz vibration stream has a 500 Hz ceiling. An illustrative 650 Hz bearing harmonic therefore folds to $|650-1000|=350$ Hz. Summarising 3,600 readings as four statistics gives the stated $900\times$ compression and 99.89% reduction, but those statistics preserve the fake 350 Hz signature too. Aggregation saves bandwidth; it cannot restore information that aliasing removed.

Math Bridge · guided foundationsWhy can't aggregation repair aliasing?Let Data Dora fold one vibration tone, then keep its bandwidth saving honest. Learn the maths

12.6 Prerequisites

Before diving into this chapter, you should be familiar with:

• Edge Compute Patterns Overview: Introduction to edge computing concepts
• IoT Reference Model: Understanding the seven-level architecture and Level 3 processing

Use those prerequisites as the frame: what stays local, and what evidence moves upstream?

12.7 Edge Processing Patterns

Edge computing employs four primary data processing patterns, but choosing among them requires seeing what each one changes in the data path. The diagram in Figure 12.1 provides that comparison before the chapter turns to placement and timing.

Read Figure 12.1 from left to right. Filter decides which observations deserve to continue; Aggregate replaces many readings with a summary; Infer converts a feature window into a local decision; and Store-Forward changes when preserved data crosses an unreliable link. The first three alter the information or its representation, while store-forward preserves the payload and delays delivery. That distinction carries the running narrative forward: edge placement earns its cost only when it meets a named bandwidth, latency, privacy, or outage requirement.

Latency-Based Processing Location

This view helps decide whether a processing step belongs on the edge device, an intermediate fog gateway, or the cloud before selecting the specific pattern:

Pattern selection starts with a placement bound. The decision tree in Figure 12.2 compares edge, fog, and cloud capacity against the response time the workload can tolerate.

The Latency Requirement? branch in the tree Figure 12.2 sends < 10ms work to Edge Processing, whose 32KB-256KB RAM budget suits simple thresholds and local filtering. The 10ms-1s branch reaches Fog Processing with 1-4GB RAM for pattern detection and regional aggregation, while >1s OK allows Cloud Processing to use global correlation, historical analysis, and ML training. The final Hybrid Approach combines tiers rather than erasing their limits. Choose Filter, Aggregate, Infer, or Store-Forward only after this placement decision has named latency, memory, and context needs.

Match latency, compute, and bandwidth constraints to the processing location, then choose Filter, Aggregate, Infer, or Store-Forward inside that tier.

Edge Processing Timeline

This view shows how data flows through edge processing over time:

Once placement is chosen, the next question is how much information each local step removes. The timeline in Figure 12.3 quantifies the reduction and its timing before asynchronous upload.

At t=0, the timeline Figure 12.3 labels the Raw volume as 5.2 TB. Filtered data falls to 3.1 TB at t=1ms, Aggregated data reaches 1.2 TB at t=5ms, and Compressed output is 522 GB at t=10ms before Transmitted at t=50ms. The annotation summarises a 90% bandwidth saving, but each reduction also discards or changes representation. That is why the selected pattern must record what was filtered, how windows were aggregated, which codec was used, and whether the remaining stream still supports the cloud decision.

Edge processing pipelines execute in milliseconds while cloud sync happens asynchronously.

12.8 Pattern Selection Guide

Read the Pattern Selection Guide material as a decision path rather than as isolated entries. First identify the operating condition in each entry and keep its units, timing, source, and assumed system state attached to it. Next compare the entries at the point where responsibility changes between device, gateway, network, analytic service, and operator; that hand-off is where apparently similar choices often produce different outcomes. Then follow the failure case: ask what becomes stale, delayed, unavailable, or unsafe, who detects it, and what evidence permits recovery. Finally connect the result to the chapter’s running design record by naming the selected behavior, the rejected alternative, the measurement that justifies the choice, and the condition that forces a recheck. That order turns the examples or comparison into an auditable engineering argument.

Checkpoint: Choose the Pattern

• Filter and Aggregate both aim for 99%+ bandwidth savings, but Filter preserves exception timing while Aggregate preserves trends.
• Infer fits local ML decisions when edge latency matters more than cloud accuracy.
• Store-Forward is the reliability pattern when the link is intermittent rather than merely expensive.

Read each pattern as a contract: what the gateway keeps, forwards, and can reconstruct.

12.9 Pattern 1: Filter at Edge

Read the Pattern 1: Filter at Edge material as a decision path rather than as isolated entries. First identify the operating condition in each entry and keep its units, timing, source, and assumed system state attached to it. Next compare the entries at the point where responsibility changes between device, gateway, network, analytic service, and operator; that hand-off is where apparently similar choices often produce different outcomes. Then follow the failure case: ask what becomes stale, delayed, unavailable, or unsafe, who detects it, and what evidence permits recovery. Finally connect the result to the chapter’s running design record by naming the selected behavior, the rejected alternative, the measurement that justifies the choice, and the condition that forces a recheck. That order turns the examples or comparison into an auditable engineering argument.

The filter pattern applies simple threshold checks to determine which data warrants transmission:

# Example: Temperature threshold filter
def filter_reading(temp_celsius, threshold=80.0):
    if temp_celsius > threshold:
        return {"alert": True, "value": temp_celsius, "timestamp": now()}
    return None  # Don't transmit normal readings

Best for:

• Alarm/alert systems where only exceptions matter
• High-frequency sensors where most readings are routine
• Bandwidth-constrained links where every byte counts

12.10 Continue to the Next Part

Carry this evidence into Edge Processing: Aggregation Decisions, which begins with Try It: Filter Threshold Simulator.