%% fig-alt: "Decision tree for selecting edge data compression algorithms based on downstream analytics requirements. The tree starts with the question of what matters most for analytics. If exact values are required for compliance or audit purposes, use GZIP or DEFLATE lossless compression achieving 3-4:1 ratios. If trends and statistics are sufficient, the choice depends on sampling rate: for sensors below 1 Hz like temperature and humidity, use window aggregation with 60-300 second windows achieving 10-50:1 compression; for sensors above 10 Hz like vibration and audio, use FFT compression keeping top 10-20 components achieving 50-500:1 compression. If only events matter such as alerts and state changes, use semantic compression with event extraction achieving 100-1000:1 compression ratios that vary by activity level."
%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#2C3E50', 'primaryTextColor': '#fff', 'primaryBorderColor': '#16A085', 'lineColor': '#16A085', 'secondaryColor': '#E67E22', 'tertiaryColor': '#ecf0f1'}}}%%
flowchart TD
Q[What matters most for<br/>downstream analytics?]
Q --> Exact[Exact values required<br/>Compliance, Audit]
Q --> Trends[Trends and statistics<br/>sufficient]
Q --> Events[Only events matter<br/>Alerts, State changes]
Exact --> GZIP[GZIP/DEFLATE<br/>Lossless<br/>Compression: 3-4:1]
Trends --> Rate{Sensor<br/>Sampling Rate?}
Rate -->|"< 1 Hz<br/>Temperature, Humidity"| Window[Window Aggregation<br/>60-300 sec windows<br/>Compression: 10-50:1]
Rate -->|"> 10 Hz<br/>Vibration, Audio"| FFT[FFT Compression<br/>Keep top 10-20 components<br/>Compression: 50-500:1]
Events --> Semantic[Semantic Compression<br/>Event extraction<br/>Compression: 100-1000:1]
style Q fill:#2C3E50,stroke:#16A085,color:#fff
style Exact fill:#E67E22,stroke:#2C3E50,color:#fff
style Trends fill:#E67E22,stroke:#2C3E50,color:#fff
style Events fill:#E67E22,stroke:#2C3E50,color:#fff
style GZIP fill:#16A085,stroke:#2C3E50,color:#fff
style Rate fill:#7F8C8D,stroke:#2C3E50,color:#fff
style Window fill:#16A085,stroke:#2C3E50,color:#fff
style FFT fill:#16A085,stroke:#2C3E50,color:#fff
style Semantic fill:#16A085,stroke:#2C3E50,color:#fff
1331 Edge Data Acquisition: Sampling and Compression
1331.1 Learning Objectives
By the end of this chapter, you will be able to:
- Apply Nyquist Theorem: Calculate appropriate sampling rates for different sensor types
- Implement Data Reduction Techniques: Use aggregation, compression, event-based reporting, and delta encoding
- Select Compression Algorithms: Choose optimal algorithms based on data type and edge device constraints
- Avoid Common Pitfalls: Prevent sampling aliasing, buffer overflow, and rate mismatch errors
1331.2 Prerequisites
Before diving into this chapter, you should be familiar with:
- Edge Data Acquisition: Architecture: Understanding device categories and data generation patterns
- Basic signal processing concepts: Familiarity with frequency and time-domain representations
- Python programming: Code examples use Python for data processing
TipMinimum Viable Understanding: Data Reduction at the Edge
Core Concept: Transform raw sensor data into actionable information at the source - send summaries, statistics, and alerts rather than every reading.
Why It Matters: Transmitting data costs 10-100x more energy than processing it locally. A sensor sending 1000 samples/minute to the cloud uses 100x more bandwidth than one sending minute-averages - with identical analytical value for most applications.
Key Takeaway: Apply the 90% rule - if 90% of your data is “normal” readings that will never be analyzed individually, aggregate them locally. Send statistical summaries (min, max, mean, std) at lower frequency, and only transmit raw data when anomalies are detected. This extends battery life from days to years.
1331.3 Nyquist Sampling Rate
To accurately capture a signal, the sampling rate must be at least twice the highest frequency component of interest:
\[f_{sample} \geq 2 \times f_{max}\]
Practical examples:
| Signal Type | Max Frequency | Min Sample Rate | Typical Rate |
|---|---|---|---|
| Temperature | 0.1 Hz (slow changes) | 0.2 Hz | 1 sample/minute |
| Vibration | 500 Hz | 1 kHz | 2-5 kHz |
| Audio | 20 kHz | 40 kHz | 44.1 kHz |
| Motion (IMU) | 50 Hz | 100 Hz | 100-200 Hz |
WarningCommon Pitfall: Sampling Aliasing
The mistake: Sampling signals below the Nyquist rate (2x the highest frequency), causing phantom patterns (aliasing) that don’t exist in the original signal.
Symptoms:
- Vibration analysis shows unexpected low-frequency patterns
- Motor speed readings fluctuate despite constant RPM
- Temperature data shows oscillations that don’t match physical reality
- Frequency analysis reveals false peaks at wrong frequencies
- Bearing fault detection produces false positives
Why it happens: Engineers apply “common sense” sampling rates without frequency analysis. Underestimating signal bandwidth - a 60 Hz motor generates harmonics at 120 Hz, 180 Hz, etc. Cost pressure drives lower sampling rates. Copy-pasting configurations between different sensor types.
The fix: Always sample at >2x the highest frequency of interest:
| Signal Type | Max Frequency | Minimum Sample Rate | Recommended |
|---|---|---|---|
| Room temperature | 0.01 Hz | 0.02 Hz | 1/minute |
| HVAC response | 0.1 Hz | 0.2 Hz | 1/second |
| Motor vibration | 500 Hz | 1 kHz | 2.5 kHz |
| Bearing analysis | 5 kHz | 10 kHz | 25 kHz |
Prevention: Perform frequency analysis on representative signals before deployment. Use anti-aliasing filters (low-pass hardware filters) before the ADC. When in doubt, oversample then downsample digitally with proper filtering.
1331.4 Edge Data Reduction Techniques
Before transmitting data to the cloud, edge devices can apply several reduction strategies:
- Aggregation: Compute statistics over time windows (mean, min, max, variance)
- Compression: Apply lossless (ZIP) or lossy (threshold-based) compression
- Event-based reporting: Only transmit when values exceed thresholds
- Delta encoding: Send only changes from previous values
# Example: Edge aggregation for temperature sensor
class EdgeAggregator:
def __init__(self, window_size=60): # 60 samples = 1 minute at 1 Hz
self.window_size = window_size
self.buffer = []
def add_sample(self, value):
self.buffer.append(value)
if len(self.buffer) >= self.window_size:
return self.compute_summary()
return None
def compute_summary(self):
summary = {
"min": min(self.buffer),
"max": max(self.buffer),
"mean": sum(self.buffer) / len(self.buffer),
"samples": len(self.buffer)
}
self.buffer = []
return summary
# Usage: Send 1 summary per minute instead of 60 raw samples
aggregator = EdgeAggregator(window_size=60)
for temp_reading in sensor_stream:
summary = aggregator.add_sample(temp_reading)
if summary:
transmit_to_cloud(summary) # 60x bandwidth reduction
WarningCommon Pitfall: Sampling Rate Mismatch
The mistake: Combining data from sensors with different sampling rates without proper resampling, leading to incorrect correlations and temporal misalignment.
Symptoms:
- Correlation analysis shows unexpected null or spurious relationships
- Merged datasets have many NaN/missing values at certain timestamps
- Time-series plots show “jagged” or misaligned signals
- ML models perform poorly despite good individual sensor data
Why it happens: Teams often assume all sensors operate at the same rate. A temperature sensor at 1 Hz combined with a vibration sensor at 100 Hz creates 99 missing values per temperature reading. Naive timestamp matching drops 99% of vibration data.
The fix: Use proper resampling/interpolation before combining:
# Resample high-frequency data to match low-frequency
vibration_1hz = vibration_100hz.resample('1S').mean()
# Or upsample low-frequency with interpolation
temp_100hz = temp_1hz.resample('10ms').interpolate(method='linear')
# Then merge on aligned timestamps
merged = pd.merge_asof(vibration_1hz, temp_1hz, on='timestamp', tolerance=pd.Timedelta('500ms'))Prevention: Document sampling rates in sensor metadata. Create a data alignment layer that resamples all sources to a common time base before analysis.
WarningCommon Pitfall: Edge Buffer Overflow
The mistake: Configuring edge device buffers without considering worst-case scenarios, causing data loss during network outages or traffic spikes.
Symptoms:
- Gaps in time-series data after network recovery
- “Buffer full, dropping oldest data” warnings in device logs
- Critical events missing during high-activity periods
- Inconsistent data counts between edge and cloud
- Post-incident analysis reveals missing sensor readings
Why it happens: Buffer sizes calculated for average conditions, not peak loads. Network outage duration underestimated. Sensor burst rates during events (motion, vibration) exceed steady-state assumptions. Memory constraints on edge devices force small buffers.
The fix: Size buffers for worst-case, not average:
# Buffer sizing calculation
samples_per_second = 10
max_outage_duration_seconds = 3600 # 1 hour
safety_margin = 1.5
min_buffer_size = samples_per_second * max_outage_duration_seconds * safety_margin
# = 10 * 3600 * 1.5 = 54,000 samples
# If memory-constrained, implement tiered retention:
# - Last 5 minutes: Full resolution
# - 5-60 minutes: 10x downsampled
# - Beyond 60 minutes: Statistical summary onlyPrevention: Monitor buffer utilization as a health metric. Alert at 70% capacity. Implement graceful degradation (reduce resolution before dropping data). Test with simulated network outages lasting 2x your expected maximum.
1331.5 Compression Algorithms Deep Dive
NoteDeep Dive: Edge Data Compression Algorithms Comparison
Edge devices face a fundamental trade-off: transmit less data (save power, bandwidth, cost) while preserving information needed for downstream analytics. This deep dive compares compression techniques across three dimensions: compression ratio, computational cost, and information preservation.
1331.5.1 Compression Algorithm Categories
| Category | Compression Ratio | CPU Cost | Information Loss | Best For |
|---|---|---|---|---|
| Lossless | 2:1 - 4:1 | Medium | None | Critical data, audit logs |
| Lossy Statistical | 10:1 - 100:1 | Low | Controlled | Trend analysis, dashboards |
| Lossy Transform | 50:1 - 500:1 | High | Controlled | Pattern detection, ML features |
| Semantic | 100:1 - 1000:1 | Very High | Significant | Event detection, alerts |
1331.5.2 Lossless Compression: DEFLATE/GZIP
Standard lossless compression works well for structured IoT data:
import gzip
import json
def compress_batch(readings: list[dict]) -> bytes:
"""
Compress a batch of sensor readings losslessly.
Typical compression: 3-5x for JSON sensor data.
"""
json_str = json.dumps(readings)
compressed = gzip.compress(json_str.encode('utf-8'), compresslevel=6)
return compressed
# Example: 100 temperature readings
readings = [{"ts": 1704067200 + i, "v": 22.5 + (i % 10) * 0.1} for i in range(100)]
raw_size = len(json.dumps(readings).encode()) # ~4,500 bytes
compressed_size = len(compress_batch(readings)) # ~1,200 bytes
# Compression ratio: 3.75:1Performance characteristics:
| Metric | GZIP Level 1 | GZIP Level 6 | GZIP Level 9 |
|---|---|---|---|
| Compression ratio | 2.5:1 | 3.5:1 | 4:1 |
| Compress speed | 50 MB/s | 20 MB/s | 5 MB/s |
| Decompress speed | 100 MB/s | 100 MB/s | 100 MB/s |
| Edge CPU impact | Low | Medium | High |
When to use: Audit trails, compliance data, any data that may be queried in original form. Do not use for real-time streams on constrained MCUs.
1331.5.3 Lossy Statistical: Aggregation Windows
Compute statistics over time windows, discard raw samples:
import statistics
from dataclasses import dataclass
from typing import Optional
@dataclass
class AggregatedWindow:
timestamp: int # Window start
count: int # Number of samples
mean: float
min_val: float
max_val: float
std_dev: float
p95: Optional[float] = None # Optional percentile
class WindowAggregator:
def __init__(self, window_seconds: int = 60):
self.window_seconds = window_seconds
self.buffer: list[float] = []
self.window_start: Optional[int] = None
def add_sample(self, timestamp: int, value: float) -> Optional[AggregatedWindow]:
if self.window_start is None:
self.window_start = timestamp
# Check if window complete
if timestamp - self.window_start >= self.window_seconds:
result = self._compute_aggregate()
self.buffer = [value]
self.window_start = timestamp
return result
self.buffer.append(value)
return None
def _compute_aggregate(self) -> AggregatedWindow:
sorted_buffer = sorted(self.buffer)
p95_idx = int(len(sorted_buffer) * 0.95)
return AggregatedWindow(
timestamp=self.window_start,
count=len(self.buffer),
mean=statistics.mean(self.buffer),
min_val=min(self.buffer),
max_val=max(self.buffer),
std_dev=statistics.stdev(self.buffer) if len(self.buffer) > 1 else 0,
p95=sorted_buffer[p95_idx] if p95_idx < len(sorted_buffer) else None
)
# Example: 1 Hz sensor, 60-second windows
# Input: 60 samples x 8 bytes = 480 bytes
# Output: 1 aggregate x 48 bytes = 48 bytes
# Compression ratio: 10:1
# Preserved: Trend (mean), anomaly detection (min/max/std), health (count)Information loss analysis:
| What’s Preserved | What’s Lost |
|---|---|
| Average value (trend) | Individual sample timing |
| Min/max (bounds) | Exact sequence of values |
| Standard deviation (stability) | Sub-window patterns |
| Sample count (health) | Correlation with other sensors at sample level |
When to use: Temperature, humidity, air quality - any slowly changing signal where trends matter more than exact samples.
1331.5.4 Lossy Transform: FFT-Based Compression
Transform to frequency domain, keep only significant components:
import numpy as np
from dataclasses import dataclass
@dataclass
class FFTCompressed:
timestamp: int
sample_rate: float
duration: float
frequencies: list[float] # Top N frequency components
magnitudes: list[float] # Corresponding magnitudes
phases: list[float] # Phase angles for reconstruction
def fft_compress(samples: np.ndarray, sample_rate: float,
timestamp: int, top_n: int = 10) -> FFTCompressed:
"""
Compress time-series data using FFT, keeping top N frequency components.
Typical compression: 100:1 to 500:1 depending on signal complexity.
Best for: Vibration, audio, periodic signals.
"""
# Compute FFT
fft_result = np.fft.rfft(samples)
freqs = np.fft.rfftfreq(len(samples), 1/sample_rate)
# Get magnitudes and find top N (excluding DC component)
magnitudes = np.abs(fft_result[1:]) # Skip DC
phases = np.angle(fft_result[1:])
freqs = freqs[1:]
# Select top N by magnitude
top_indices = np.argsort(magnitudes)[-top_n:]
return FFTCompressed(
timestamp=timestamp,
sample_rate=sample_rate,
duration=len(samples) / sample_rate,
frequencies=freqs[top_indices].tolist(),
magnitudes=magnitudes[top_indices].tolist(),
phases=phases[top_indices].tolist()
)
def fft_decompress(compressed: FFTCompressed, num_samples: int) -> np.ndarray:
"""
Reconstruct signal from FFT components (lossy reconstruction).
"""
t = np.linspace(0, compressed.duration, num_samples)
signal = np.zeros(num_samples)
for freq, mag, phase in zip(compressed.frequencies,
compressed.magnitudes,
compressed.phases):
signal += mag * np.cos(2 * np.pi * freq * t + phase)
return signal
# Example: Vibration sensor, 1 second at 1000 Hz
samples = np.sin(2*np.pi*50*np.linspace(0, 1, 1000)) # 50 Hz signal
samples += 0.3 * np.sin(2*np.pi*150*np.linspace(0, 1, 1000)) # 150 Hz harmonic
# Input: 1000 samples x 4 bytes = 4000 bytes
# Output: 10 freq-mag-phase tuples x 12 bytes = 120 bytes + 20 bytes metadata
# Compression ratio: ~30:1
compressed = fft_compress(samples, 1000.0, 1704067200, top_n=10)
reconstructed = fft_decompress(compressed, 1000)
# Reconstruction error for this example: ~5% RMS
# Bearing fault detection: Still works (frequency peaks preserved)When to use: Vibration analysis, acoustic monitoring, any signal where frequency content matters more than exact waveform.
1331.5.5 Semantic Compression: Event Extraction
Highest compression, but requires domain knowledge:
from dataclasses import dataclass
from enum import Enum
from typing import Optional
class EventType(Enum):
THRESHOLD_EXCEEDED = "threshold_exceeded"
ANOMALY_DETECTED = "anomaly_detected"
STATE_CHANGE = "state_change"
PERIODIC_SUMMARY = "periodic_summary"
@dataclass
class SemanticEvent:
timestamp: int
device_id: str
event_type: EventType
value: float
context: dict # Additional info (threshold, previous state, etc.)
class SemanticCompressor:
def __init__(self, device_id: str, threshold_high: float,
threshold_low: float, anomaly_std_factor: float = 3.0):
self.device_id = device_id
self.threshold_high = threshold_high
self.threshold_low = threshold_low
self.anomaly_std_factor = anomaly_std_factor
self.history: list[float] = []
self.last_state: Optional[str] = None
self.summary_count = 0
self.summary_sum = 0.0
def process_sample(self, timestamp: int, value: float) -> list[SemanticEvent]:
"""
Process a sample and return events (if any).
Most samples produce NO events - that's the compression.
"""
events = []
# Update history for anomaly detection
self.history.append(value)
if len(self.history) > 100:
self.history.pop(0)
# Track for periodic summary
self.summary_count += 1
self.summary_sum += value
# Check threshold crossing
current_state = "normal"
if value > self.threshold_high:
current_state = "high"
elif value < self.threshold_low:
current_state = "low"
if current_state != self.last_state and self.last_state is not None:
events.append(SemanticEvent(
timestamp=timestamp,
device_id=self.device_id,
event_type=EventType.STATE_CHANGE,
value=value,
context={
"previous_state": self.last_state,
"new_state": current_state
}
))
self.last_state = current_state
# Check for statistical anomaly
if len(self.history) >= 20:
mean = sum(self.history) / len(self.history)
std = (sum((x - mean)**2 for x in self.history) / len(self.history)) ** 0.5
if std > 0 and abs(value - mean) > self.anomaly_std_factor * std:
events.append(SemanticEvent(
timestamp=timestamp,
device_id=self.device_id,
event_type=EventType.ANOMALY_DETECTED,
value=value,
context={
"mean": mean,
"std": std,
"z_score": (value - mean) / std
}
))
return events
def get_periodic_summary(self, timestamp: int) -> SemanticEvent:
"""Call every N minutes to send a heartbeat/summary."""
avg = self.summary_sum / self.summary_count if self.summary_count > 0 else 0
event = SemanticEvent(
timestamp=timestamp,
device_id=self.device_id,
event_type=EventType.PERIODIC_SUMMARY,
value=avg,
context={
"sample_count": self.summary_count,
"period_seconds": 300 # 5 minutes
}
)
self.summary_count = 0
self.summary_sum = 0.0
return event
# Example: Temperature sensor, 1 sample/second
# Normal operation: 0 events per sample
# State change: 1 event (~100 bytes)
# 5-minute summary: 1 event (~80 bytes)
#
# Input: 300 samples x 8 bytes = 2400 bytes per 5 minutes
# Output: 1 summary + maybe 0-2 events = 80-280 bytes
# Compression ratio: 10:1 to 30:1 (varies by activity)When to use: Monitoring systems where “nothing happening” is the common case. Alarm systems, threshold monitoring, sparse event streams.
1331.5.6 Algorithm Selection Decision Tree
1331.5.7 Benchmark Results: ESP32 Edge Device
Real measurements on ESP32-WROOM-32 (240 MHz, 520KB RAM):
| Algorithm | 1000 Samples | Compress Time | Output Size | Power (mJ) |
|---|---|---|---|---|
| Raw JSON | - | 15 ms (serialize) | 28,000 bytes | 2.4 |
| GZIP-6 | 28,000 bytes | 85 ms | 8,200 bytes | 8.5 |
| Window Agg | 8 bytes/sample | 2 ms | 48 bytes | 0.4 |
| FFT Top-10 | 4 bytes/sample | 45 ms | 140 bytes | 5.0 |
| Semantic | 8 bytes/sample | 3 ms | 0-100 bytes | 0.5 |
Key insight: For battery-powered edge devices, window aggregation offers the best power efficiency. FFT is valuable when frequency content matters, but the CPU cost is significant. Semantic compression is ideal for sparse event streams.
1331.5.8 Memory Constraints on Edge Devices
Compression algorithms have memory overhead. Consider carefully on constrained devices:
| Algorithm | RAM Required | Notes |
|---|---|---|
| GZIP | 32-64 KB | Sliding window + Huffman tables |
| Window Agg | <1 KB | Just buffer for current window |
| FFT (1024 pt) | 16 KB | Complex float buffer + twiddle factors |
| FFT (4096 pt) | 64 KB | May not fit on small MCUs |
| Semantic | 2-4 KB | History buffer + state |
ESP32 recommendation: Use window aggregation or semantic compression as primary strategy. Reserve FFT for specific signals where frequency analysis is required.
1331.6 Common Compression Pitfalls
CautionPitfall: Over-Aggressive Lossy Compression
The Mistake: Applying high compression ratios uniformly across all sensor data without understanding which information is critical for downstream analytics, permanently destroying signals needed for root cause analysis.
Why It Happens: Bandwidth costs drive aggressive compression targets. Teams optimize for average case without considering anomaly detection requirements. Compression algorithms are chosen based on benchmark performance rather than domain-specific information preservation. The “we can always collect more data later” assumption fails for non-reproducible events.
The Fix: Profile your analytics requirements before choosing compression. For predictive maintenance, preserve frequency-domain information (use FFT compression, not just statistics). For threshold alerting, min/max preservation is critical. For trend analysis, mean and standard deviation suffice. Implement tiered compression: full resolution for anomalies detected locally, heavy compression for steady-state readings. Always retain enough information to answer “why did this alert trigger?” after the fact.
CautionPitfall: Compression Without Metadata
The Mistake: Compressing sensor data without preserving the metadata needed to decompress or interpret it correctly, creating files that cannot be decoded weeks or months later.
Why It Happens: Metadata seems redundant during development when context is fresh. Schema documentation is maintained separately and drifts over time. Edge device memory constraints pressure developers to strip every unnecessary byte. Compression parameters are hardcoded rather than embedded in output.
The Fix: Always include compression metadata in the payload or use self-describing formats. For FFT compression, include sample rate, window size, and which frequency bins are transmitted. For statistical aggregation, include sample count, window duration, and timestamp precision. Use envelope formats that version the compression scheme: {"compression": "fft-v2", "params": {...}, "data": [...]}. Maintain a compression schema registry that maps version identifiers to decompression algorithms.
CautionPitfall: Ignoring Compression Computation Cost
The Mistake: Selecting compression algorithms based purely on compression ratio without accounting for CPU time and energy cost on battery-powered edge devices, resulting in net-negative energy savings.
Why It Happens: Compression benchmarks on desktop hardware show impressive ratios with negligible CPU time. The 1000x difference in computational efficiency between an ESP32 and a laptop is underestimated. Energy cost of computation versus transmission varies by network type (Wi-Fi is cheap to transmit, LoRa is expensive). Algorithm selection copied from cloud/server contexts.
The Fix: Measure end-to-end energy consumption: E_total = E_compute + E_transmit. For LoRaWAN devices where transmission costs 100+ mJ per packet, aggressive compression (even expensive algorithms) saves energy. For Wi-Fi devices where transmission costs 1-5 mJ per packet, simple aggregation beats complex compression. Profile specific algorithms on your target MCU: GZIP on ESP32 consumes 8.5 mJ for 1000 samples versus 0.4 mJ for window aggregation. Choose the algorithm that minimizes total energy, not just bytes transmitted.
1331.7 Understanding Check: Industrial Edge Data Pipeline Design
1331.8 Knowledge Check
1331.9 Practice Exercises
NoteExercise 1: Sampling Rate Selection
Objective: Determine optimal sampling rates for different sensor types.
Tasks:
- Identify signal characteristics for 4 sensors: temperature (max 0.1 Hz), vibration (max 500 Hz), audio (max 20 kHz), motion IMU (max 50 Hz)
- Apply Nyquist theorem: calculate minimum sampling rates
- Implement with margin and measure data rate impact
Expected Outcome: Understand the relationship between signal bandwidth and sampling requirements.
NoteExercise 2: Edge Aggregation Implementation
Objective: Implement multiple data reduction techniques and compare bandwidth savings.
Tasks:
- Collect 1-minute of high-rate sensor data (100 Hz = 6,000 samples)
- Apply 4 reduction strategies: downsampling, statistical aggregation, delta encoding, event-based
- Transmit reduced data and compare bandwidth
- Validate: can you detect a 1C temperature spike with each method?
Expected Outcome: Understand trade-offs between compression ratio and information preservation.
1331.10 Summary
Edge data acquisition requires careful balance between data fidelity and resource constraints:
- Nyquist compliance: Sample at 2x or higher than your highest frequency of interest to avoid aliasing
- Reduction techniques: Aggregation (10-50x), FFT compression (50-500x), and semantic extraction (100-1000x) each serve different use cases
- Algorithm selection: Match compression to downstream analytics needs - lossless for audit, statistical for trends, FFT for vibration, semantic for events
- Resource awareness: Consider CPU time and memory on constrained edge devices, not just compression ratio
1331.11 What’s Next
The next chapter, Edge Data Acquisition: Power and Gateways, covers power management strategies, duty cycling calculations, and gateway functions for non-IP device integration.
NoteRelated Chapters
- Edge Data Acquisition: Architecture - Device categories and data generation patterns
- Edge Data Acquisition: Power and Gateways - Power management and gateway functions
- Edge Compute Patterns - Processing patterns at the edge
- Multi-Sensor Data Fusion - Combining data from multiple sources