17 Sensor Processing: Sampling and Signals

17.1 Start With the Decision

A clean chart can hide aliasing, clipping, or lost detail. Sensor processing starts with the signal band, sample rate, and step size.

17.2 Route Overview

This is part 1 of 2. Continue with Sensor Processing: Filter Selection and Validation.

17.3 Part Objectives

• Choose sample rate and resolution from signal limits.
• Detect aliasing, quantization, clipping, and noise.

17.4 Start With the Measurement Story

Keep the Raw Reading Beside the Claim

An analog-to-digital converter is a part that turns a voltage into a number. It is often called an ADC. Bandwidth is the range of change a signal or link can carry. Picture a cold-room sensor that jumps once when a motor starts. Smoothing may hide the jump, but it may also hide a real door-open event.

Write the physical claim before choosing a data step. Record units, valid range, expected speed of change, sample time, and known sources of noise. Keep the raw value, quality state, and corrected value together so a reviewer can trace the result.

Test steady input, a real fast change, a single false spike, a missing sample, and a shifted reference. Change the order of checks only when the evidence supports it. Reject bad input before it can pull later values away from truth.

No filter can recover facts that were never measured. The deeper sections compare smoothing, middle-value checks, calibration, and stored settings. They show which fault each method can reduce and which limit must remain visible.

A reading becomes information only after noise, drift, outliers, scaling, and timing are handled. Start with the claim the processed value must support, then choose the smallest processing step that makes that claim reliable.

Keep one concrete record beside the chapter: name the raw observable and units, expected physical bandwidth, valid range, sample interval, known interference, and the decision threshold that consumes the result. A median filter is defensible when rare spikes are impossible physical events; a moving average is defensible when uncorrelated jitter is faster than the real process; calibration is required when a stable bias or scale error remains. Record the order too, because rejecting an invalid sample before smoothing is not equivalent to averaging it into later values, and converting ADC counts into engineering units before the reference is known only gives a more polished wrong number. Each later method should be justified against this same measurement story.

17.5 Learning Objectives

By the end of this chapter, you will be able to:

• Implement and compare moving average, Kalman, and median filters to reduce sensor noise
• Execute two-point calibration procedures to correct sensor offset and gain errors
• Design data validation pipelines that detect anomalies and reject outliers
• Select and justify appropriate filtering strategies based on signal characteristics and resource constraints
• Persist calibration coefficients in EEPROM and restore them reliably across power cycles

17.6 In 60 Seconds

17.7 Prerequisites

Before diving into this chapter, you should be familiar with:

• Sensor Fundamentals: Understanding of sensor types and output characteristics
• ADC Fundamentals: How analog signals are converted to digital values
• Electricity Basics: Basic understanding of voltage, resistance, and signal levels
• Basic Statistics (general knowledge): Mean, median, and variance concepts

17.8 Key Concepts

Start with Noise: Random variations in sensor readings caused by electrical interference, temperature fluctuations, or quantization errors. Then Filtering: Mathematical techniques to remove noise while preserving the true signal. Next Calibration: Process of adjusting sensor output to match known reference values. After that Offset Error: Constant shift in readings across the entire measurement range (zero-point shift). Continue by Gain Error: Proportional error that increases with measured value (sensitivity error). Continue by Drift: Gradual change in sensor accuracy over time due to aging or environmental factors. Finally EEPROM: Non-volatile memory for storing calibration data that persists across power cycles.

17.9 Quick Check: Processing Purpose

17.10 Introduction

Raw sensor data is rarely perfect. Environmental noise, electrical interference, and manufacturing variations all affect measurement accuracy. This chapter covers the essential techniques for transforming noisy, uncalibrated sensor readings into reliable, accurate measurements.

17.11 Sensor Data Processing

17.12 Making Sensors Tell Truth

17.13 Moving Average Noise Reduction

17.14 Filtering Noisy Sensor Data

Sensor noise comes from many sources: electrical interference, quantization errors, and environmental factors. Filters remove this noise while preserving the true signal.

The roadmap now shifts from “why process?” to “which process?” Start with the noise pattern you can observe, because the wrong filter can make a clean-looking but misleading signal.

17.15 Folded Sampling And Aliasing Notes

Filtering starts before the ADC. If a signal is sampled below twice the highest frequency of interest, high-frequency content can appear as a false low-frequency pattern. Diagnose aliasing by naming the highest signal component, comparing it with the sample rate, changing the sample rate during a test, or checking the real waveform with faster capture.

Use the Nyquist limit as a minimum, not a complete design. Practical systems often sample faster than two times the highest frequency because analog filters roll off gradually. Place an anti-aliasing low-pass filter before the ADC when unwanted higher-frequency energy could enter the sampled data, then choose the digital filter after the sampled record preserves the signal you actually care about.

Phoebe’s Field Notes: Where Vibration Nyquist Meets the Sensor’s Own Step Size

The mathematical gist. The chapter’s 100 Hz ADXL345 stream has a 50 Hz Nyquist ceiling: higher vibration components fold to false lower frequencies before a median or moving-average filter can see them. Its 3.9 mg/LSB step has a 0.00113 g RMS quantisation floor, far below the chapter’s 0.3 g spikes and 0.15 g drift, so the selected filters are fighting real acquisition noise rather than ADC rounding.

Math Bridge · guided foundationsWhat can a 100 Hz accelerometer really reveal?Let Phoebe connect Nyquist, aliasing, and the ADXL345 quantisation floor. Learn the maths

17.15.1 Sensor Data Processing Pipeline

The following diagram shows the complete sensor data processing pipeline from raw readings to calibrated output:

Before choosing a filter, inspect Figure 17.1 to locate filtering among validation, calibration, and conversion to engineering units.

Read Figure 17.1 from raw ADC input through outlier rejection and smoothing, then follow calibration into clean output. The order shows why noise reduction cannot repair bias and why calibrated values still require invalid-sample handling.

// Moving average filter (template-based, stack-allocated)
// Template parameter N sets window size at compile time —
// avoids heap allocation, which is unsafe on memory-constrained MCUs.
template<int N>
class MovingAverageFilter {
  private:
    float buffer[N];  // Stack-allocated, size known at compile time
    int index;
    int count;         // Tracks samples received (for cold-start)
    float sum;

  public:
    MovingAverageFilter() {
      index = 0;
      count = 0;
      sum = 0;

      for(int i = 0; i < N; i++) {
        buffer[i] = 0;
      }
    }

    float filter(float value) {
      sum -= buffer[index];
      buffer[index] = value;
      sum += value;

      index = (index + 1) % N;

      if (count < N) count++;  // Track fill level

      return sum / count;  // Divide by actual samples, not window size
    }
};

// Kalman filter (simple 1D implementation)
class KalmanFilter {
  private:
    float q;  // Process noise covariance
    float r;  // Measurement noise covariance
    float x;  // Estimated value
    float p;  // Estimation error covariance
    float k;  // Kalman gain

  public:
    KalmanFilter(float processNoise, float measurementNoise, float initialValue) {
      q = processNoise;
      r = measurementNoise;
      x = initialValue;
      p = 1;
    }

    float filter(float measurement) {
      // Prediction
      p = p + q;

      // Update
      k = p / (p + r);
      x = x + k * (measurement - x);
      p = (1 - k) * p;

      return x;
    }
};

// Usage example
MovingAverageFilter<10> maFilter;  // 10-sample window (stack-allocated)
KalmanFilter kFilter(0.01, 0.1, 25.0);  // Process noise, measurement noise, initial value

void loop() {
  float rawTemp = readTemperature();

  float filteredMA = maFilter.filter(rawTemp);
  float filteredKalman = kFilter.filter(rawTemp);

  Serial.print("Raw: ");
  Serial.print(rawTemp);
  Serial.print(" | Moving Avg: ");
  Serial.print(filteredMA);
  Serial.print(" | Kalman: ");
  Serial.println(filteredKalman);

  delay(100);
}

// Median filter with fixed-size buffer (no VLA — portable C++)
const int MAX_MEDIAN_SIZE = 16;

float medianFilter(float* buffer, int size) {
    // Safety check: clamp to max supported window
    if (size > MAX_MEDIAN_SIZE) size = MAX_MEDIAN_SIZE;

    float sorted[MAX_MEDIAN_SIZE];
    memcpy(sorted, buffer, size * sizeof(float));

    // Bubble sort — O(N^2), but acceptable for small N typical
    // in sensor filtering (N=3 to N=9). For larger windows,
    // consider insertion sort or std::nth_element for O(N) median.
    for (int i = 0; i < size - 1; i++) {
        for (int j = 0; j < size - i - 1; j++) {
            if (sorted[j] > sorted[j+1]) {
                float temp = sorted[j];
                sorted[j] = sorted[j+1];
                sorted[j+1] = temp;
            }
        }
    }

    // For even N, returns upper-median (standard median averages
    // the two middle values, but single-value is simpler for embedded)
    return sorted[size / 2];
}

// Example: [22, 55, 23] -> sorted: [22, 23, 55] -> median: 23
// The spike (55) is completely ignored!

17.15.10 Filter Behavior Comparison

The following diagram compares moving average and median filter responses to the same noisy input with a spike. Kalman filter response varies based on Q and R parameters and is explored in the interactive calculator below.

Before choosing a smoothing method, inspect Figure 17.2 to compare how the same noisy trace and isolated spike pass through different filters.

Read Figure 17.2 from the raw series to the moving-average and median outputs. The moving average spreads a spike across its window, while the median rejects an isolated outlier, connecting signal shape to filter choice.

// Exponential Moving Average — only 4 bytes of RAM
class EMAFilter {
  private:
    float filtered;
    float alpha;
    bool initialized;

  public:
    EMAFilter(float smoothingFactor) {
      alpha = smoothingFactor;
      filtered = 0;
      initialized = false;
    }

    float filter(float newValue) {
      if (!initialized) {
        filtered = newValue;  // First sample: no smoothing
        initialized = true;
      } else {
        filtered = alpha * newValue + (1.0 - alpha) * filtered;
      }
      return filtered;
    }
};

// Usage: alpha=0.1 for heavy smoothing, alpha=0.5 for fast response
EMAFilter emaFilter(0.1);

void loop() {
  float raw = readTemperature();
  float smoothed = emaFilter.filter(raw);  // O(1) time, 4 bytes RAM
  delay(100);
}

17.16 Continue to the Next Part

Carry this evidence into Sensor Processing: Filter Selection and Validation, which begins with Vibration Filter Selection.