17 Sensor Processing: Sampling and Signals

17.1 Start With the Decision

This section: Start With the Decision — A clean chart can hide aliasing, clipping, or lost detail.

A clean chart can hide aliasing, clipping, or lost detail. Sensor processing starts with the signal band, sample rate, and step size.

17.2 Route Overview

This section: Route Overview — This is part 1 of 2.

This is part 1 of 2. Continue with Sensor Processing: Filter Selection and Validation.

17.3 Part Objectives

This section: Part Objectives

• Choose sample rate and resolution from signal limits.
• Detect aliasing, quantization, clipping, and noise.

17.4 Start With the Measurement Story

This section: Start With the Measurement Story — An analog-to-digital converter is a part that turns a voltage into a number.

Keep the Raw Reading Beside the Claim

An analog-to-digital converter is a part that turns a voltage into a number. It is often called an ADC. Bandwidth is the range of change a signal or link can carry. Picture a cold-room sensor that jumps once when a motor starts. Smoothing may hide the jump, but it may also hide a real door-open event.

Write the physical claim before choosing a data step. Record units, valid range, expected speed of change, sample time, and known sources of noise. Keep the raw value, quality state, and corrected value together so a reviewer can trace the result.

Test steady input, a real fast change, a single false spike, a missing sample, and a shifted reference. Change the order of checks only when the evidence supports it. Reject bad input before it can pull later values away from truth.

No filter can recover facts that were never measured. The deeper sections compare smoothing, middle-value checks, calibration, and stored settings. They show which fault each method can reduce and which limit must remain visible.

A reading becomes information only after noise, drift, outliers, scaling, and timing are handled. Start with the claim the processed value must support, then choose the smallest processing step that makes that claim reliable.

Keep one concrete record beside the chapter: name the raw observable and units, expected physical bandwidth, valid range, sample interval, known interference, and the decision threshold that consumes the result. A median filter is defensible when rare spikes are impossible physical events; a moving average is defensible when uncorrelated jitter is faster than the real process; calibration is required when a stable bias or scale error remains. Record the order too, because rejecting an invalid sample before smoothing is not equivalent to averaging it into later values, and converting ADC counts into engineering units before the reference is known only gives a more polished wrong number. Each later method should be justified against this same measurement story.

• Start With the Measurement Story
• In 60 Seconds
• Prerequisites
• Key Concepts
• Quick Check: Processing Purpose
• Introduction
• Sensor Data Processing
• Making Sensors Tell Truth
• Moving Average Noise Reduction
• Checkpoint: Measurement Meaning
• Filtering Noisy Sensor Data
• Folded Sampling And Aliasing Notes
• Phoebe’s Field Notes: Where Vibration Nyquist Meets the Sensor’s Own Step Size

17.5 Learning Objectives

This section: Learning Objectives

By the end of this chapter, you will be able to:

• Implement and compare moving average, Kalman, and median filters to reduce sensor noise
• Execute two-point calibration procedures to correct sensor offset and gain errors
• Design data validation pipelines that detect anomalies and reject outliers
• Select and justify appropriate filtering strategies based on signal characteristics and resource constraints
• Persist calibration coefficients in EEPROM and restore them reliably across power cycles

17.6 In 60 Seconds

This section: In 60 Seconds

17.7 Prerequisites

This section: Prerequisites

Before diving into this chapter, you should be familiar with:

• Sensor Fundamentals: Understanding of sensor types and output characteristics
• ADC Fundamentals: How analog signals are converted to digital values
• Electricity Basics: Basic understanding of voltage, resistance, and signal levels
• Basic Statistics (general knowledge): Mean, median, and variance concepts

17.8 Key Concepts

This section: Key Concepts

• Noise: Random variations in sensor readings caused by electrical interference, temperature fluctuations, or quantization errors.
• Filtering: Mathematical techniques to remove noise while preserving the true signal.
• Calibration: Process of adjusting sensor output to match known reference values.
• Offset Error: Constant shift in readings across the entire measurement range (zero-point shift).
• Gain Error: Proportional error that increases with measured value (sensitivity error).
• Drift: Gradual change in sensor accuracy over time due to aging or environmental factors.
• EEPROM: Non-volatile memory for storing calibration data that persists across power cycles.

17.9 Quick Check: Processing PurposeCheck your understanding: Processing Purpose

This section: Quick Check: Processing Purpose

17.10 Introduction

This section: Introduction — Raw sensor data is rarely perfect.

Raw sensor data is rarely perfect. Environmental noise, electrical interference, and manufacturing variations all affect measurement accuracy. This chapter covers the essential techniques for transforming noisy, uncalibrated sensor readings into reliable, accurate measurements.

17.11 Sensor Data Processing

This section: Sensor Data Processing

17.12 Making Sensors Tell Truth

This section: Making Sensors Tell Truth

17.13 Moving Average Noise Reduction

This section: Moving Average Noise Reduction — An average is useful when several readings observe essentially the same physical condition.

An average is useful when several readings observe essentially the same physical condition. In the example below, five temperatures straddle 25°C, so the average estimates their shared centre. The gain comes from letting independent random deviations partly cancel.

The square-root noise rule assumes uncorrelated noise and a signal that remains sufficiently steady over the window. Five samples from a drifting reference do not become unbiased by being averaged. Likewise, a real temperature step is spread across successive windows, delaying the reported change. Compare the window’s duration with the fastest event the controller must detect before choosing its size. Use the calculation as a noise estimate under those assumptions, not as a guarantee of absolute accuracy.

17.14 Filtering Noisy Sensor Data

This section: Filtering Noisy Sensor Data — Sensor noise comes from many sources: electrical interference, quantization errors, and environmental factors.

Sensor noise comes from many sources: electrical interference, quantization errors, and environmental factors. Filters remove this noise while preserving the true signal.

The roadmap now shifts from “why process?” to “which process?” Start with the noise pattern you can observe, because the wrong filter can make a clean-looking but misleading signal.

17.15 Folded Sampling And Aliasing Notes

This section: Folded Sampling And Aliasing Notes — Filtering starts before the ADC.

Filtering starts before the ADC. If a signal is sampled below twice the highest frequency of interest, high-frequency content can appear as a false low-frequency pattern. Diagnose aliasing by naming the highest signal component, comparing it with the sample rate, changing the sample rate during a test, or checking the real waveform with faster capture.

Use the Nyquist limit as a minimum, not a complete design. Practical systems often sample faster than two times the highest frequency because analog filters roll off gradually. Place an anti-aliasing low-pass filter before the ADC when unwanted higher-frequency energy could enter the sampled data, then choose the digital filter after the sampled record preserves the signal you actually care about.

Math Bridge · guided foundationsWhat can a 100 Hz accelerometer really reveal?Let Phoebe connect Nyquist, aliasing, and the ADXL345 quantisation floor. Learn the maths

17.15.1 Sensor Data Processing Pipeline

The following diagram shows the complete sensor data processing pipeline from raw readings to calibrated output:

Before choosing a filter, inspect Figure 17.1 to locate filtering among validation, calibration, and conversion to engineering units.

Shows: Figure 17.1: Sensor data processing pipeline from raw readings to calibrated output

+

Show text version

Read Figure 17.1 from raw ADC input through outlier rejection and smoothing, then follow calibration into clean output. The order shows why noise reduction cannot repair bias and why calibrated values still require invalid-sample handling.

// Moving average filter (template-based, stack-allocated)
// Template parameter N sets window size at compile time —
// avoids heap allocation, which is unsafe on memory-constrained MCUs.
template<int N>
class MovingAverageFilter {
  private:
    float buffer[N];  // Stack-allocated, size known at compile time
    int index;
    int count;         // Tracks samples received (for cold-start)
    float sum;

  public:
    MovingAverageFilter() {
      index = 0;
      count = 0;
      sum = 0;

      for(int i = 0; i < N; i++) {
        buffer[i] = 0;
      }
    }

    float filter(float value) {
      sum -= buffer[index];
      buffer[index] = value;
      sum += value;

      index = (index + 1) % N;

      if (count < N) count++;  // Track fill level

      return sum / count;  // Divide by actual samples, not window size
    }
};

// Kalman filter (simple 1D implementation)
class KalmanFilter {
  private:
    float q;  // Process noise covariance
    float r;  // Measurement noise covariance
    float x;  // Estimated value
    float p;  // Estimation error covariance
    float k;  // Kalman gain

  public:
    KalmanFilter(float processNoise, float measurementNoise, float initialValue) {
      q = processNoise;
      r = measurementNoise;
      x = initialValue;
      p = 1;
    }

    float filter(float measurement) {
      // Prediction
      p = p + q;

      // Update
      k = p / (p + r);
      x = x + k * (measurement - x);
      p = (1 - k) * p;

      return x;
    }
};

// Usage example
MovingAverageFilter<10> maFilter;  // 10-sample window (stack-allocated)
KalmanFilter kFilter(0.01, 0.1, 25.0);  // Process noise, measurement noise, initial value

void loop() {
  float rawTemp = readTemperature();

  float filteredMA = maFilter.filter(rawTemp);
  float filteredKalman = kFilter.filter(rawTemp);

  Serial.print("Raw: ");
  Serial.print(rawTemp);
  Serial.print(" | Moving Avg: ");
  Serial.print(filteredMA);
  Serial.print(" | Kalman: ");
  Serial.println(filteredKalman);

  delay(100);
}

// Median filter with fixed-size buffer (no VLA — portable C++)
const int MAX_MEDIAN_SIZE = 16;

float medianFilter(float* buffer, int size) {
    // Safety check: clamp to max supported window
    if (size > MAX_MEDIAN_SIZE) size = MAX_MEDIAN_SIZE;

    float sorted[MAX_MEDIAN_SIZE];
    memcpy(sorted, buffer, size * sizeof(float));

    // Bubble sort — O(N^2), but acceptable for small N typical
    // in sensor filtering (N=3 to N=9). For larger windows,
    // consider insertion sort or std::nth_element for O(N) median.
    for (int i = 0; i < size - 1; i++) {
        for (int j = 0; j < size - i - 1; j++) {
            if (sorted[j] > sorted[j+1]) {
                float temp = sorted[j];
                sorted[j] = sorted[j+1];
                sorted[j+1] = temp;
            }
        }
    }

    // For even N, returns upper-median (standard median averages
    // the two middle values, but single-value is simpler for embedded)
    return sorted[size / 2];
}

// Example: [22, 55, 23] -> sorted: [22, 23, 55] -> median: 23
// The spike (55) is completely ignored!

17.15.10 Filter Behavior Comparison

The following diagram compares moving average and median filter responses to the same noisy input with a spike. Kalman filter response varies based on Q and R parameters and is explored in the interactive calculator below.

The noisy trace and isolated spike in Figure 17.2 reveal how each filter changes both noise and transient response.

Shows: Figure 17.2: Filter comparison showing response to noisy input with spike

+

Show text version

Read Figure 17.2 from the raw series to the moving-average and median outputs. The moving average spreads a spike across its window, so several later samples remain displaced after the raw event has passed. The median instead rejects an isolated outlier while preserving the surrounding level. That difference connects the expected noise shape to filter choice: use averaging for distributed random variation, but prefer a median when occasional impulses would contaminate a windowed mean.

// Exponential Moving Average — one previous output; implementation storage is additional
class EMAFilter {
  private:
    float filtered;
    float alpha;
    bool initialized;

  public:
    EMAFilter(float smoothingFactor) {
      alpha = smoothingFactor;
      filtered = 0;
      initialized = false;
    }

    float filter(float newValue) {
      if (!initialized) {
        filtered = newValue;  // First sample: no smoothing
        initialized = true;
      } else {
        filtered = alpha * newValue + (1.0 - alpha) * filtered;
      }
      return filtered;
    }
};

// Usage: alpha=0.1 for heavy smoothing, alpha=0.5 for fast response
EMAFilter emaFilter(0.1);

void loop() {
  float raw = readTemperature();
  float smoothed = emaFilter.filter(raw);  // O(1) time; implementation storage is additional
  delay(100);
}

17.16 Continue to the Next Part

This section: Continue to the Next Part — Carry this evidence into Sensor Processing: Filter Selection and Validation, which begins with Vibration Filter Selection.

Carry this evidence into Sensor Processing: Filter Selection and Validation, which begins with Vibration Filter Selection.