65 Quantization and Digital Filtering

65.1 Learning Objectives

By the end of this chapter, you will be able to:

Understand Quantization: Calculate ADC resolution and its impact on measurement accuracy
Select ADC Resolution: Choose appropriate bit depth for your application
Apply Digital Filters: Implement moving average, median, and low-pass filters
Choose Appropriate Filters: Match filter type to noise characteristics

65.2 ADC Resolution: How Precisely to Measure?

Time: ~11 min | Level: Intermediate | Unit: P02.C05.U04

Resolution determines measurement precision:

8-bit ADC (0-255):

Voltage range: 0-3.3V
Step size: 3.3V / 256 = 12.9 mV
Reading 127 means: 127 x 12.9mV = 1.638V +/- 6.4mV

12-bit ADC (0-4095):

Voltage range: 0-3.3V
Step size: 3.3V / 4096 = 0.8 mV
Reading 2048 means: 2048 x 0.8mV = 1.638V +/- 0.4mV

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor':'#2C3E50','primaryTextColor':'#fff','primaryBorderColor':'#16A085','lineColor':'#16A085','secondaryColor':'#E67E22','tertiaryColor':'#7F8C8D','clusterBkg':'#ECF0F1','clusterBorder':'#16A085','edgeLabelBackground':'#ECF0F1'}}}%%
graph LR
    subgraph bit8["8-bit ADC"]
        A1["256 Levels<br/>0-255"]
        A2["Step Size<br/>12.9 mV<br/>(for 3.3V range)"]
        A3["Precision<br/>+/-0.4%"]
    end

    subgraph bit12["12-bit ADC"]
        B1["4,096 Levels<br/>0-4095"]
        B2["Step Size<br/>0.8 mV<br/>(for 3.3V range)"]
        B3["Precision<br/>+/-0.025%"]
    end

    subgraph bit16["16-bit ADC"]
        C1["65,536 Levels<br/>0-65535"]
        C2["Step Size<br/>0.05 mV<br/>(for 3.3V range)"]
        C3["Precision<br/>+/-0.0015%"]
    end

    A1 --> A2 --> A3
    B1 --> B2 --> B3
    C1 --> C2 --> C3

    style bit8 fill:#E67E22,stroke:#2C3E50,stroke-width:3px,color:#000
    style bit12 fill:#16A085,stroke:#2C3E50,stroke-width:3px,color:#fff
    style bit16 fill:#2C3E50,stroke:#16A085,stroke-width:3px,color:#fff
    style A1 fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style A2 fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style A3 fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style B1 fill:#ECF0F1,stroke:#16A085,stroke-width:2px,color:#000
    style B2 fill:#ECF0F1,stroke:#16A085,stroke-width:2px,color:#000
    style B3 fill:#ECF0F1,stroke:#16A085,stroke-width:2px,color:#000
    style C1 fill:#ECF0F1,stroke:#2C3E50,stroke-width:2px,color:#000
    style C2 fill:#ECF0F1,stroke:#2C3E50,stroke-width:2px,color:#000
    style C3 fill:#ECF0F1,stroke:#2C3E50,stroke-width:2px,color:#000

Figure 65.1: ADC Resolution Comparison: 8-bit, 12-bit, and 16-bit Quantization Levels

Quantization Levels Comparison: Higher bit depth provides more levels and finer resolution. 8-bit (orange) is coarse with 256 steps. 12-bit (teal) is typical for IoT with 4096 steps. 16-bit (navy) provides medical-grade precision with 65536 steps at the cost of power and speed. {fig-alt=“Three-column comparison of ADC quantization levels: 8-bit (256 levels, 12.9 mV steps, 0.4% precision), 12-bit (4096 levels, 0.8 mV steps, 0.025% precision), and 16-bit (65536 levels, 0.05 mV steps, 0.0015% precision). Each shows number of levels, voltage step size for 3.3V range, and measurement precision percentage.”}

Trade-offs:

Resolution	Levels	Precision	Cost	Power	Speed
8-bit	256	~0.4%	Low	Low	Fast
10-bit	1024	~0.1%	Medium	Medium	Medium
12-bit	4096	~0.025%	Medium	Medium	Medium
16-bit	65536	~0.0015%	High	High	Slow

Rule of thumb for IoT: - 8-bit: Simple applications, on/off detection - 10-12-bit: Most sensor applications (temperature, light, pressure) - 16-bit: High-precision measurements (audio, medical, scientific)

65.2.1 ADC Selection Decision Tree

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor':'#2C3E50','primaryTextColor':'#fff','primaryBorderColor':'#16A085','lineColor':'#16A085','secondaryColor':'#E67E22','tertiaryColor':'#7F8C8D','clusterBkg':'#ECF0F1','clusterBorder':'#16A085','edgeLabelBackground':'#ECF0F1'}}}%%
flowchart TD
    A["ADC Selection<br/>Decision Process"] --> B{"Signal Bandwidth?"}

    B -->|"< 1 kHz<br/>(Temperature,<br/>Pressure)"| C["Low Speed OK"]
    B -->|"1-100 kHz<br/>(Audio,<br/>Vibration)"| D["Medium Speed"]
    B -->|"> 100 kHz<br/>(RF, High-Speed<br/>Signals)"| E["High Speed<br/>Required"]

    C --> F{"Required Precision?"}
    D --> F
    E --> F

    F -->|"Simple<br/>On/Off"| G["8-bit ADC<br/>Low Cost<br/>Fast<br/>Low Power"]

    F -->|"Typical<br/>Sensor"| H["10-12-bit ADC<br/>Good Balance<br/>Most IoT Apps<br/>Moderate Power"]

    F -->|"High<br/>Precision"| I["16-bit ADC<br/>Higher Cost<br/>Slower<br/>More Power"]

    G --> J{"Power Budget?"}
    H --> J
    I --> J

    J -->|"Battery<br/>Powered"| K["Optimize:<br/>- Duty cycle ADC<br/>- Lower resolution<br/>- Reduce sample rate"]

    J -->|"Mains<br/>Powered"| L["Flexible:<br/>- Higher resolution OK<br/>- Faster sampling OK<br/>- Better performance"]

    style A fill:#2C3E50,stroke:#16A085,stroke-width:3px,color:#fff
    style B fill:#7F8C8D,stroke:#2C3E50,stroke-width:2px,color:#fff
    style F fill:#7F8C8D,stroke:#2C3E50,stroke-width:2px,color:#fff
    style J fill:#7F8C8D,stroke:#2C3E50,stroke-width:2px,color:#fff
    style G fill:#E67E22,stroke:#2C3E50,stroke-width:2px,color:#fff
    style H fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
    style I fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
    style K fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style L fill:#ECF0F1,stroke:#16A085,stroke-width:2px,color:#000

Figure 65.2: ADC Selection Decision Tree Based on Bandwidth, Precision, and Power

ADC Selection Decision Tree: Choose ADC based on three key factors: signal bandwidth (sampling speed), required precision (bit depth), and power budget. Most IoT applications use 10-12-bit ADCs as they balance precision, speed, and power consumption. {fig-alt=“Decision tree flowchart for ADC selection. Starts with signal bandwidth decision (low/medium/high speed), then precision requirements (8-bit simple, 10-12-bit typical, 16-bit high precision), finally power budget (battery vs mains powered). Each path shows trade-offs in cost, speed, and power consumption.”}

65.3 Digital Filtering

Time: ~10 min | Level: Advanced | Unit: P02.C05.U05

Problem: ADC readings are noisy due to electrical interference, sensor limitations, and quantization.

Solution: Apply digital filters in firmware.

65.3.1 Moving Average Filter

Simplest filter: Average the last N samples.

Readings: [23.4, 23.6, 23.5, 23.7, 23.5]
5-point average: (23.4+23.6+23.5+23.7+23.5)/5 = 23.54C

Pros: Simple, smooths noise Cons: Slow response to rapid changes

65.3.2 Median Filter

Better for removing spikes: Take the middle value of last N samples.

Readings: [23.5, 23.6, 99.9, 23.7, 23.5]  (99.9 is a spike/outlier)
5-point median: Sort -> [23.5, 23.5, 23.6, 23.7, 99.9]
Median = 23.6C  (spike removed!)

Pros: Removes outliers effectively Cons: More complex than moving average

65.3.3 Low-Pass Filter (Simple RC)

Removes high-frequency noise, keeps slow changes:

// Simple exponential moving average
float filtered_value = 0.9 * previous_value + 0.1 * new_reading;

Weight factor (0.9) determines how much history to keep: - High weight (0.95): Very smooth, slow response - Low weight (0.5): Faster response, less smoothing

65.3.4 Filter Comparison

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor':'#2C3E50','primaryTextColor':'#fff','primaryBorderColor':'#16A085','lineColor':'#16A085','secondaryColor':'#E67E22','tertiaryColor':'#7F8C8D','clusterBkg':'#ECF0F1','clusterBorder':'#16A085','edgeLabelBackground':'#ECF0F1'}}}%%
graph TB
    subgraph input["Noisy Input Signal"]
        A["Temperature Readings:<br/>23.4, 23.6, 99.9, 23.5, 23.7, 23.5C<br/>(Contains spike at 99.9)"]
    end

    A --> B["No Filter"]
    A --> C["Moving Average<br/>(5-point)"]
    A --> D["Median Filter<br/>(5-point)"]
    A --> E["Low-Pass Filter<br/>(a = 0.9)"]

    B --> B1["Output:<br/>23.4, 23.6, 99.9, 23.5, 23.7<br/>Spike remains<br/>Noise visible"]

    C --> C1["Output:<br/>(23.4+23.6+99.9+23.5+23.7)/5<br/>= 38.8C<br/>Spike corrupts average<br/>Smooths gradual noise"]

    D --> D1["Output:<br/>Sort: 23.4, 23.5, 23.6, 23.7, 99.9<br/>Median = 23.6C<br/>Spike removed!<br/>Best for outliers"]

    E --> E1["Output:<br/>0.9xprev + 0.1xnew<br/>Gradual approach to true value<br/>Smooth<br/>Slow response"]

    style input fill:#E67E22,stroke:#2C3E50,stroke-width:3px,color:#fff
    style A fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style B fill:#7F8C8D,stroke:#2C3E50,stroke-width:2px,color:#fff
    style C fill:#7F8C8D,stroke:#2C3E50,stroke-width:2px,color:#fff
    style D fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
    style E fill:#7F8C8D,stroke:#2C3E50,stroke-width:2px,color:#fff
    style B1 fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style C1 fill:#ECF0F1,stroke:#E67E22,stroke-width:2px,color:#000
    style D1 fill:#ECF0F1,stroke:#16A085,stroke-width:2px,color:#000
    style E1 fill:#ECF0F1,stroke:#7F8C8D,stroke-width:2px,color:#000

Figure 65.3: Digital Filter Comparison: No Filter, Moving Average, Median, and Low-Pass

{fig-alt=“Four-way comparison of digital filtering techniques applied to noisy temperature sensor data containing outlier spike at 99.9C. Input shows readings 23.4, 23.6, 99.9, 23.5, 23.7, 23.5C. No filter preserves all noise and spikes (red X). Moving average calculates mean 38.8C, corrupted by outlier (orange X). Median filter sorts values and selects middle 23.6C, successfully removing spike (teal checkmark, recommended). Low-pass exponential filter with alpha 0.9 gradually approaches true value, smooth but slow response (gray warning). Median filter highlighted as best approach for spike and outlier removal.”}

Digital Filter Comparison: Four approaches to handling noisy temperature data with outlier spike (99.9C). No filter keeps noise and spikes. Moving average is corrupted by outliers. Median filter (teal) effectively removes spikes by selecting middle value. Low-pass filter smooths gradually but responds slowly to changes.

Alternative View: Filter Selection Decision Tree

This variant presents filtering as a decision-making process based on your specific noise problems:

%% fig-alt: "Decision tree for selecting the right digital filter for IoT sensor data. Start with the question: What is your main noise problem? If the problem is sudden spikes or outliers (like EMI, loose connections, or sensor glitches), choose Median Filter - it removes spikes while preserving real signal changes. If the problem is gradual noise or jitter (like thermal noise, electrical hum, or quantization), choose Low-Pass Filter - it smooths variations but slows response time. If you need both (spikes AND gradual noise), use Median first to remove spikes, then Low-Pass to smooth. If memory is limited (less than 32 bytes available), use Exponential Moving Average (EMA) - it only stores one value. Real-world tip: start with median filter, then add smoothing if needed."
%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#2C3E50', 'primaryTextColor': '#fff', 'primaryBorderColor': '#16A085', 'lineColor': '#16A085', 'secondaryColor': '#E67E22', 'tertiaryColor': '#7F8C8D'}}}%%
flowchart TD
    START["What's your<br/>noise problem?"] --> Q1{"Sudden spikes<br/>or outliers?"}

    Q1 -->|"Yes"| MED["MEDIAN FILTER<br/>Removes spikes<br/>Preserves real changes<br/>Good for: EMI, glitches"]

    Q1 -->|"No"| Q2{"Gradual noise<br/>or jitter?"}

    Q2 -->|"Yes"| LP["LOW-PASS FILTER<br/>Smooths variations<br/>Slows response<br/>Good for: thermal noise"]

    Q2 -->|"Both!"| BOTH["COMBO: Median then Low-Pass<br/>1. Remove spikes first<br/>2. Then smooth result"]

    Q1 -->|"Memory limited?"| EMA["EMA (alpha filter)<br/>Stores only 1 value<br/>0.9xold + 0.1xnew<br/>Good for: tiny MCUs"]

    MED --> TIP["Start with median,<br/>add smoothing if needed"]
    LP --> TIP
    BOTH --> TIP
    EMA --> TIP

    style START fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
    style Q1 fill:#7F8C8D,stroke:#2C3E50,stroke-width:1px,color:#fff
    style Q2 fill:#7F8C8D,stroke:#2C3E50,stroke-width:1px,color:#fff
    style MED fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
    style LP fill:#E67E22,stroke:#2C3E50,stroke-width:2px,color:#fff
    style BOTH fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
    style EMA fill:#7F8C8D,stroke:#E67E22,stroke-width:2px,color:#fff
    style TIP fill:#ECF0F1,stroke:#16A085,stroke-width:1px,color:#000

Figure 65.4: Filter selection decision tree: choose based on noise type, then optimize

Why this variant helps: The original compares all filters side-by-side. This decision tree helps students answer the practical question: “Which filter should I use for MY problem?” It turns knowledge into actionable decisions by matching filter type to noise characteristics.

65.4 Practical Quiz: ADC Configuration

Quiz: ADC Configuration for Water Flow Sensor

Scenario: You’re designing a smart irrigation system using a turbine water flow sensor. The sensor outputs: - Voltage range: 0.5V (no flow) to 4.5V (maximum flow 10 L/min) - Signal characteristics: Pulsed output, 1-50 Hz depending on flow rate - Noise: +/-50mV from pump electrical interference

Your microcontroller options: - Option A: 8-bit ADC, 100 ksps (samples/second), 10uA power - Option B: 12-bit ADC, 10 ksps, 100uA power - Option C: 16-bit ADC, 1 ksps, 500uA power

Think about: 1. What sampling rate do you need to accurately measure 50 Hz pulses? (Nyquist) 2. What ADC resolution do you need to measure +/-0.1 L/min precision (assuming linear sensor)? 3. Which ADC option provides the best balance of precision, speed, and power?

Key Insights:

Sampling rate requirement: - Maximum frequency: 50 Hz - Nyquist requirement: > 100 samples/second - Practical target: 250-500 samples/second for safety - All options meet this (slowest is 1000 sps)

Resolution requirement: - Voltage span: 4.5V - 0.5V = 4.0V - Flow span: 10 L/min - 0 L/min = 10 L/min - Sensitivity: 4.0V / 10 L/min = 0.4V per L/min - Desired precision: +/-0.1 L/min = +/-0.04V

8-bit ADC: - Step size: 5V / 256 = 19.5 mV - Precision: 19.5mV / 400mV per L/min = +/-0.05 L/min (Just barely meets requirement!)

12-bit ADC: - Step size: 5V / 4096 = 1.2 mV - Precision: 1.2mV / 400mV per L/min = +/-0.003 L/min (Excellent!)

Noise consideration: - Noise: +/-50mV - 8-bit: Noise = +/-2.5 steps (poor SNR) - 12-bit: Noise = +/-40 steps (good SNR, averaging helps)

Best choice: Option B - 12-bit ADC - Plenty of resolution (0.003 L/min vs. 0.1 L/min requirement) - Adequate sampling rate (10,000 sps >> 100 sps needed) - Reasonable power (100uA vs. 500uA for 16-bit) - Good SNR with noise (40 steps margin)

Option A (8-bit) would work but has minimal margin for noise. Option C (16-bit) is overkill and wastes 5x more power for precision you don’t need.

Real-world lesson: Don’t over-specify. Match ADC resolution to actual sensor precision and application requirements. The 16-bit ADC costs more, draws 5x power, and provides 400x the precision - but irrigation doesn’t need microliter accuracy!

65.5 Summary

Key concepts from this chapter:

ADC Resolution: Higher bits = more precision, but also more power and cost
Step Size: V_ref / 2^bits determines measurement granularity
Match Resolution to Sensor: Don’t use 16-bit ADC for a sensor that’s only 1% accurate
Digital Filters:
- Moving average: Smooths noise, but corrupted by outliers
- Median filter: Best for spike/outlier removal
- Low-pass (EMA): Smooths gradual noise, slow response
- Combination: Median first (remove spikes), then low-pass (smooth)

Decision Framework: 1. Identify signal bandwidth -> choose sampling rate 2. Identify precision needs -> choose ADC resolution 3. Identify noise type -> choose filter strategy 4. Consider power budget -> optimize all choices

65.6 What’s Next

Continue to Voice Compression for IoT to learn about audio signal processing techniques for bandwidth-constrained devices, or proceed to Sensor Dynamics and Response to understand how sensors behave over time.

Continue to Voice Compression ->