21 Quantization and Digital Filtering

Match ADC Codes and Filters to Real Sensor Evidence

fundamentals

signal

processing

quantization

21.1 In 60 Seconds

Quantization and filtering decide whether ADC samples become evidence or noise. First fit the ADC range and bit depth so one code step is smaller than a meaningful signal change but not far below the measured noise floor. Then choose the lightest filter that removes the failure pattern without hiding real events or adding too much delay.

Once a sample has been taken, two jobs remain before the number can be trusted. Quantization decides how finely the analog voltage is rounded into discrete codes, and digital filtering decides how much noise to remove from the stream of codes. Both can be done well or badly: too coarse a code step hides real change, too fine a step digitizes noise, too little filtering leaves a jittery reading, and too much filtering erases the events you wanted to see.

21.2 Start With the Story

Start with a physical signal that is noisy, delayed, sampled, quantized, calibrated, filtered, packed, and finally sent as a number someone will trust. The core idea in Quantization and Digital Filtering is simple: signal processing is the bridge between the physical world and digital evidence, so every sampling, ADC, filter, and calibration choice changes the value that leaves the device. This page focuses that idea on Signal-processing ADC code-step fit, reference and range choices, quantization error, filter selection, filter order. In everyday IoT, temperature drift, vibration spikes, audio snippets, and lab traces all become decisions only after their limits and uncertainty are made visible. Start simple: trace one measurement through the chain, keep the raw-to-processed evidence, and move advanced math into the deeper review only when the simple chain no longer explains the result.

Phoebe’s Field Notes: Why Averaging Buys Extra Bits

Phoebe the physics guide

Phoebe’s Why

Quantization error is not noise in the everyday sense – it is a rounding artifact, and for a perfectly steady input it can repeat the exact same code forever. Real quantization-noise theory quietly assumes the input wobbles a little from sample to sample, which is exactly this page’s own 2 mV of pressure-sensor noise doing its job. When that holds, each sample’s rounding error is close to independent of the last one, so averaging several samples lets the errors partly cancel while the slowly changing pressure signal does not. That is the physics behind this page’s own advice to “consider averaging instead of more resolution” – and how much it buys is a short piece of arithmetic.

The Derivation

One sample’s quantization noise power:

\[\sigma_q^2 = \frac{q^2}{12}, \qquad q = \text{LSB}\]

Averaging \(N\) independent samples divides the noise power by \(N\):

\[\sigma_{avg}^2 = \frac{\sigma_q^2}{N} \quad\Longrightarrow\quad \sigma_{avg} = \frac{q}{\sqrt{12N}}\]

So the signal-to-noise improvement from averaging is

\[\Delta\mathrm{SNR}_{dB} = 10\log_{10}N\]

and since each ADC bit is worth about \(6.02\) dB, that is an equivalent bit gain of

\[\Delta N_{bits} = \frac{10\log_{10}N}{6.02} = 0.5\log_2 N\]

Worked Numbers: This Page’s Own Pressure Sensor

This page’s 12-bit, 3.3 V ADC: \(q = 3.3/4096 = 0.806\) mV, its own quoted figure.
The page’s own 2 mV RMS input noise is \(2/0.806 = 2.48\) LSB. Averaging brings it below one LSB once \(N > (2/0.806)^2 = 6.16\), so \(N = 7\): \(10\log_{10}7 = 8.45\) dB gain, \(+1.40\) effective bits, leaving \(2/\sqrt{7} = 0.756\) mV of residual noise – just under the 0.806 mV step.
Round up to a power of two for a cheap running-sum filter: \(N = 8\) gives exactly \(10\log_{10}8 = 9.03\) dB \(= 1.50\) bits.
\(N = 16\) gives exactly \(10\log_{10}16 = 12.0\) dB \(= 2.00\) extra bits – turning this page’s 12-bit part into an effective ~14-bit reading, at the cost of 16 sample periods of added lag before the average settles.

21.3 Rounding to Lines on a Ruler

Quantization is like reading a value against a ruler that only has so many tick marks. The true value falls somewhere between two ticks, and the ADC reports the nearest tick. If the ticks are far apart, small real changes vanish between them. If the ticks are absurdly fine, you start measuring the unsteadiness of your own hand rather than the object, because the smallest tick is finer than the noise.

Filtering is the second job. A raw reading often shakes from electrical noise, so a filter smooths it. But smoothing always costs something: a smoother line responds more slowly and can hide a genuine fast change. The skill is removing noise without erasing signal.

If you only need the intuition, this layer is enough: choose a code step small enough to see meaningful changes but not so small that you only resolve noise, and filter enough to calm the jitter without hiding real events. Both choices are trade-offs, not “more is better.”

The size of one tick is called the least significant bit, or LSB: the smallest voltage difference the ADC can distinguish. It is set by the input range and the number of bits, and it is the unit in which quantization error is measured.

A pressure sensor that outputs 0.5 to 2.5 V into a 12-bit, 3.3 V ADC has about 0.806 mV per code across the ADC reference, but the useful question is not the ADC number alone. If the signal-conditioning circuit leaves 2 mV RMS noise on the input, several adjacent codes can represent the same pressure state. Averaging may make the displayed pressure steadier, but a long window can also hide a short pressure pulse from a pump or valve. The right design compares three quantities together: the smallest pressure change that matters, the voltage represented by one code, and the measured noise plus required response time.

Code-step fit: useful span, noise floor, reference range, and one-code resolution must be reviewed together.

flowchart TD
  A["Sensor voltage range"] --> B["ADC reference and bit depth"]
  B --> C["Compute one LSB"]
  D["Smallest meaningful change"] --> E{"LSB small enough?"}
  C --> E
  E -- "no" --> F["Adjust range or resolution"]
  E -- "yes" --> G["Measure noise in LSBs"]
  G --> H{"Noise dominates low bits?"}
  H -- "yes" --> I["Reduce noise or average samples"]
  H -- "no" --> J["Choose the lightest filter that preserves events"]

The One-Minute View

Code step sets the detail

One LSB is the smallest change the ADC can show. It must be finer than the change you care about.

Noise sets the useful floor

Resolution finer than the noise adds digits, not information; it measures the wobble more precisely.

Filtering is a trade

Smoothing reduces noise but adds lag and can hide real fast events. Match the filter to the signal.

Beginner Examples

A coarse code step might report room temperature only in whole degrees, hiding a slow half-degree drift.
A reading that jitters in its last digit is often noise near the LSB, not real change.
A heavily smoothed reading looks clean but reacts slowly, so a brief spike can be averaged away.

LSB Fit Knowledge Check

If this gives you the idea, you can stop here. Continue to Practitioner when you need to fit the code step and choose a filter for a real stream.

21.4 Apply It: Fit the Code Step, Then Filter

The practical order is range and resolution first, filtering second. Fit the input range so the signal uses most of the codes without clipping, confirm the code step is finer than the meaningful change, then add only as much filtering as the noise requires.

Walkthrough: From Voltage to a Clean Stream

Fit the range. Scale the sensor output so it spans most of the ADC input range, avoiding clipping at the top and wasted codes at the bottom.
Check the code step. Confirm one LSB is smaller than the smallest change you must resolve.
Compare to noise. If the noise is much larger than one LSB, extra bits are wasted; consider averaging instead of more resolution.
Choose a filter. Match the filter type to the noise: averaging for random noise, a median for spikes, a low-pass for high-frequency content.
Validate. Apply a known input and confirm the filtered stream still reaches real values quickly enough for the application.

ADC fit review: extra bits help only after signal span, noise floor, and headroom have earned them.

Worked Example: A 12-Bit ADC at 3.3 Volts

A 12-bit ADC with a 3.3-volt reference has 4096 codes, so one code step is:

LSB = V_ref / 2^N = 3.3 V / 4096 = about 0.806 mV

If the smallest meaningful change is 5 mV, the 0.806 mV step resolves it with room to spare. If the input noise is around 3 mV, however, then most of the lowest bits only describe noise, and a smoother result comes from averaging several samples rather than from a higher-resolution part.

Choosing a Filter

Filter

Good For

Cost

Notes

Moving average

Random, zero-mean noise.

Adds delay proportional to the window.

Simple; a single spike still shifts the average.

Exponential average

Continuous smoothing with low memory.

Adds lag set by the smoothing factor.

Cheap on small devices; one stored value.

Median

Isolated spikes and outliers.

More computation than averaging.

Rejects spikes while preserving the real level.

Low-pass (FIR or IIR)

Specific high-frequency noise bands.

Higher order means more lag and compute.

Cannot undo aliasing already in the samples.

High-pass (FIR or IIR)

Slow drift or baseline wander when only rapid change matters.

Can remove real slow behavior if that level is part of the evidence.

Record the cutoff and prove the retained event is still meaningful.

Feature Extraction After Filtering

A repetitive sensor trace often needs a feature before it needs a classifier. Peak intervals, step counts, zero crossings, and simple energy windows can turn an accelerometer, EOG, or vibration stream into compact evidence for activity recognition, interface state, or device condition. Those features are trustworthy only when the sampling and filtering choices preserve the event being counted. A larger median window may reject jitter, for example, but it can also shift zero crossings, flatten a short pulse, or change the interval estimate. Keep the raw trace, filtered trace, window size, feature rule, and a validation event together so the smoother signal still proves the intended behavior.

Start by checking the baseline. A noisy waveform whose mean is not zero can make a zero-crossing counter, threshold rule, or peak detector report the offset instead of the event. Subtracting the measured mean, or otherwise removing a documented baseline, recenters the trace before median filtering and feature extraction are compared. The review record should keep the original trace, the de-meaned trace, and the filtered trace separate so later readers can see whether the processing removed bias, removed spikes, or accidentally changed the event timing.

Frequency evidence is the other useful cross-check. A time-domain trace may hide that it is really a slow useful component plus a faster interference component, such as a 1 kHz behavior riding with a 15 kHz tone. An FFT or spectrum view makes those components visible before a low-pass cutoff is chosen. The low-pass claim is defensible only when the record names the sample rate, frequency peak or band to keep, high-frequency content to reject, cutoff, and before/after trace that proves the useful feature survived.

Match the digital filter to the failure pattern - spikes, drift, broadband noise, or a known tone - rather than applying a generic cleanup step.

Incremental Practice

Beginner

For a 10-bit ADC with a 5-volt reference, compute one LSB and decide whether it resolves a 10 mV change.

Intermediate

A reading jitters by several codes. Decide whether to add resolution or to average, and justify the choice from the noise level.

Advanced

A signal has both random noise and rare spikes. Design a two-stage filter and state the order in which it should run.

Spike Filter Knowledge Check

If you can fit a code step and pick a filter, you can stop here. Continue to Under the Hood for the quantization math and the resolution-versus-noise limits.

21.5 Under the Hood: Quantization Math and Noise Limits

Quantization and filtering both have clean formulas. They reveal why extra bits stop helping past the noise floor, and why averaging buys resolution only under the right conditions.

Code Step and Quantization Error

For an ideal ADC with reference voltage V_ref and N bits, one code step and the ideal error bound are:

LSB = V_ref / 2^N

ideal quantization error = within plus or minus one half LSB

For a 12-bit, 3.3-volt ADC, the LSB is about 0.806 mV, so the ideal rounding error is within about plus or minus 0.4 mV. The ideal signal-to-quantization-noise ratio grows with bits:

SNR_ideal = 6.02 N + 1.76 dB

Each added bit improves the ideal ratio by about 6 dB. Real parts reach fewer effective bits because input noise, reference drift, and timing error all add uncertainty.

Resolution Versus Noise

If the input noise is larger than one LSB, the lowest code bits only describe that noise, so buying more resolution adds digits, not information. The useful measure is the effective number of bits, which can be well below the nominal count. The remedy is often to reduce noise or to average, not to specify a larger converter.

Averaging and Oversampling

Averaging N independent samples of a steady value with random, zero-mean noise reduces the noise standard deviation by the square root of N:

noise reduction factor = square root of N

So averaging four samples halves the noise, and averaging sixteen quarters it. Oversampling and averaging can even recover effective resolution, gaining about one bit for every fourfold increase in samples, but only when the noise is random and at least around one LSB, which is what makes the averaging effective. The cost is delay and computation, the same trade as any smoothing.

Filter Order and the Aliasing Boundary

A higher-order filter has a sharper cutoff but adds more delay, more ringing, and more computation. The choice balances how cleanly you must separate signal from noise against how much lag the application tolerates. One limit is absolute: a digital filter operates on samples that already exist, so it cannot undo aliasing. Content that folded into the band during sampling looks like real signal, which is why anti-alias filtering must be analog and must happen before the ADC.

Coefficient Filters Are Sliding Evidence Tests

Many digital filters are just a short coefficient list sliding across a sampled signal. At each position, the firmware multiplies nearby samples by the coefficients and adds the products. That operation is discrete convolution. Written as a practical rule, a filtered sample is:

y[i] = sum over j of x[i - j + 1] * coeff[j]

The coefficient pattern decides what the filter hears. Positive taps that average neighboring samples behave like a low-pass smoother because fast wiggles cancel into a calmer line. Difference-style taps, or taps whose sum is near zero, behave like a high-pass filter because steady baseline cancels while rapid changes remain. For a simple unnormalized example, x = [1, 2, 3, 4, ...] and coeff = [1, 2, 3] produce a convolution output that begins [1, 4, 10, 16, ...] as the three taps slide across the input. A production smoother would usually normalize the taps, but the review question is the same: keep the raw trace, coefficient list, cutoff or response goal, and before/after evidence together.

Filter-order pipeline: preserve raw evidence, check the ADC boundary, reject spikes, smooth jitter, downsample after the limit, and validate the event.

Symptom

Likely Cause

Evidence to Collect

First Fix to Try

Small changes invisible

Code step larger than the change.

LSB versus smallest meaningful change.

Fit the range or increase resolution.

Last digits jitter

Noise larger than the LSB.

Noise level in LSB units.

Reduce noise or average samples.

Reading reacts too slowly

Filter window too long.

Filter delay versus required freshness.

Shorten the window or lower the order.

Filter will not remove a pattern

Pattern is an alias already in the data.

Compare pre-ADC signal with samples.

Add analog anti-alias filtering before sampling.

Common Pitfalls

Buying bits past the noise. Extra resolution below the noise floor only digitizes noise more precisely.
Wasting range. A signal that uses only a small part of the input range throws away available codes.
Over-filtering. Too much smoothing hides real fast events and adds latency.
Expecting a digital filter to fix aliasing. Folded content cannot be separated after sampling; prevent it with an analog filter.

Averaging Noise Knowledge Check

At this depth, quantization and filtering are paired budgets: fit the range and code step to the signal, stop adding resolution at the noise floor, average or filter only as much as the noise demands, and remember that anti-alias work belongs in the analog domain before the sample is ever taken.

21.6 Summary

Quantization rounds the analog input to discrete codes; one LSB equals V_ref divided by 2 to the power N and sets the finest resolvable change.
The ideal quantization error is within plus or minus half an LSB, and ideal SNR rises about 6 dB per added bit.
Resolution finer than the noise floor adds digits, not information; reduce noise or average instead of buying bits.
Averaging N independent samples reduces random noise by the square root of N, at the cost of delay.
Filter choice depends on the noise: averaging for random noise, a median for spikes, a low-pass for high-frequency bands.
A coefficient filter applies discrete convolution: slide the taps across the samples, multiply, sum, and keep the coefficient list with the validation record.
Higher filter order means a sharper cutoff but more lag, and no digital filter can undo aliasing already in the samples.
Simple features such as peaks, intervals, and zero crossings should be computed from a filtered trace only when the filter window preserves the event being counted.
De-mean before offset-sensitive features, and use frequency-domain evidence before claiming a low-pass filter preserves the useful component.

Key Takeaway

Fit the code step to the signal and the filter to the noise. Stop adding resolution at the noise floor, smooth only as much as freshness allows, and keep anti-alias filtering in the analog domain where it still works.