25 Quantization and Digital Filtering

25.1 In 60 Seconds

Keep the Pressure Spike After Rounding and Smoothing

Picture a brief pressure spike that is visible at the sensor but missing from the stored samples. An analog-to-digital converter is the circuit that turns a measured voltage into a digital number. It is also called an ADC.

Name the input range, code step, noise floor, sample timing, filter, and event limit. Test a steady value, noise, a short spike, and a slow change while altering one choice at a time.

Keep raw voltage, digital codes, filtered output, settings, and delay. Latency means the delay between input and usable output. This proves one chain preserves the chosen events, not every signal; the deeper sections derive quantization error and compare filters, order, noise, and latency.

Quantization and filtering decide whether ADC samples become evidence or noise. First fit the ADC range and bit depth so one code step is smaller than a meaningful signal change but not far below the measured noise floor. Then choose the lightest filter that removes the failure pattern without hiding real events or adding too much delay.

Once a sample has been taken, two jobs remain before the number can be trusted. Quantization decides how finely the analog voltage is rounded into discrete codes, and digital filtering decides how much noise to remove from the stream of codes. Both can be done well or badly: too coarse a code step hides real change, too fine a step digitizes noise, too little filtering leaves a jittery reading, and too much filtering erases the events you wanted to see.

25.2 Start With the Story

You will check whether ADC steps resolve useful changes and choose filtering that addresses the observed noise. Start by comparing the sensor span, reference range, noise floor, and smallest change that matters.

Follow the pressure trace across four beats to see why code-step fit and filtering must preserve the same physical evidence.

The mathematical gist. When successive rounding errors are independent, averaging $N$ samples cuts root-mean-square noise by $\sqrt N$, improves signal-to-noise ratio by $10\log_{10}N$ dB, and gains $0.5\log_2N$ effective bits. For this chapter’s 2 mV noise and 0.806 mV code step, seven samples leave 0.756 mV; eight samples give 9.03 dB or 1.50 effective bits, while sixteen give 12.0 dB or 2.00 bits at the cost of more lag.

Math Bridge · guided foundationsWhy does averaging eight samples buy 1.5 bits?Let Phoebe unpack square roots, logarithms, residual noise, and the chapter's 2 mV pressure-sensor case. Learn the maths

25.3 Rounding to Lines on a Ruler

Quantization is like reading a value against a ruler that only has so many tick marks. The true value falls somewhere between two ticks, and the ADC reports the nearest tick. If the ticks are far apart, small real changes vanish between them. If the ticks are absurdly fine, you start measuring the unsteadiness of your own hand rather than the object, because the smallest tick is finer than the noise.

Filtering is the second job. A raw reading often shakes from electrical noise, so a filter smooths it. But smoothing always costs something: a smoother line responds more slowly and can hide a genuine fast change. The skill is removing noise without erasing signal.

The size of one tick is called the least significant bit, or LSB: the smallest voltage difference the ADC can distinguish. It is set by the input range and the number of bits, and it is the unit in which quantization error is measured.

A pressure sensor that outputs 0.5 to 2.5 V into a 12-bit, 3.3 V ADC has about 0.806 mV per code across the ADC reference, but the useful question is not the ADC number alone. If the signal-conditioning circuit leaves 2 mV RMS noise on the input, several adjacent codes can represent the same pressure state. Averaging may make the displayed pressure steadier, but a long window can also hide a short pressure pulse from a pump or valve. The right design compares three quantities together: the smallest pressure change that matters, the voltage represented by one code, and the measured noise plus required response time.

Pause at the figure Figure 25.1 before applying Rounding to Lines on a Ruler. Its Quantization Code-Step Fit and ADC range · 4096 codes labels show why Code-step fit: useful span, noise floor, reference range, and one-code resolution must be reviewed together needs an evidence check for Rounding to Lines on a Ruler here.

Start with the useful span inside the full ADC range in Figure 25.1, then inspect the enlarged code steps below. The noise covers several adjacent codes, while the meaningful change is larger; finer resolution alone does not remove that noise.

The One-Minute View

Beginner Examples

• A coarse code step might report room temperature only in whole degrees, hiding a slow half-degree drift.
• A reading that jitters in its last digit is often noise near the LSB, not real change.
• A heavily smoothed reading looks clean but reacts slowly, so a brief spike can be averaged away.

LSB Fit Knowledge Check

If this gives you the idea, you can stop here. Continue to Practitioner when you need to fit the code step and choose a filter for a real stream.

25.4 Apply It: Fit the Code Step, Then Filter

The practical order is range and resolution first, filtering second. Fit the input range so the signal uses most of the codes without clipping, confirm the code step is finer than the meaningful change, then add only as much filtering as the noise requires.

Walkthrough: From Voltage to a Clean Stream

1. Fit the range. Scale the sensor output so it spans most of the ADC input range, avoiding clipping at the top and wasted codes at the bottom.
2. Check the code step. Confirm one LSB is smaller than the smallest change you must resolve.
3. Compare to noise. If the noise is much larger than one LSB, extra bits are wasted; consider averaging instead of more resolution.
4. Choose a filter. Match the filter type to the noise: averaging for random noise, a median for spikes, a low-pass for high-frequency content.
5. Validate. Apply a known input and confirm the filtered stream still reaches real values quickly enough for the application.

Challenge Walkthrough: From Voltage to a Clean Stream with the visual in Figure 25.2: Signal Span names one checkpoint and Code Step names another. Their relationship frames ADC fit review: extra bits help only after signal span, noise floor, and headroom have earned them.

For Walkthrough: From Voltage to a Clean Stream, the visual sequence in Figure 25.2 opens with Signal Span, where it highlights Signal Span. Code Step follows to show how it highlights Code Step; Noise Floor then highlights Noise Floor. That progression connects ADC fit review: extra bits help only after signal span, noise floor, and headroom have earned them to the next Walkthrough: From Voltage to a Clean Stream check.

Worked Example: A 12-Bit ADC at 3.3 Volts

A 12-bit ADC with a 3.3-volt reference has 4096 codes, so one code step is:

If the smallest meaningful change is 5 mV, the 0.806 mV step resolves it with room to spare. If the input noise is around 3 mV, however, then most of the lowest bits only describe noise, and a smoother result comes from averaging several samples rather than from a higher-resolution part.

Choosing a Filter

Feature Extraction After Filtering

A repetitive sensor trace often needs a feature before it needs a classifier. Peak intervals, step counts, zero crossings, and simple energy windows can turn an accelerometer, EOG, or vibration stream into compact evidence for activity recognition, interface state, or device condition. Those features are trustworthy only when the sampling and filtering choices preserve the event being counted. A larger median window may reject jitter, for example, but it can also shift zero crossings, flatten a short pulse, or change the interval estimate. Keep the raw trace, filtered trace, window size, feature rule, and a validation event together so the smoother signal still proves the intended behavior.

Start by checking the baseline. A noisy waveform whose mean is not zero can make a zero-crossing counter, threshold rule, or peak detector report the offset instead of the event. Subtracting the measured mean, or otherwise removing a documented baseline, recenters the trace before median filtering and feature extraction are compared. The review record should keep the original trace, the de-meaned trace, and the filtered trace separate so later readers can see whether the processing removed bias, removed spikes, or accidentally changed the event timing.

Frequency evidence is the other useful cross-check. A time-domain trace may hide that it is really a slow useful component plus a faster interference component, such as a 1 kHz behavior riding with a 15 kHz tone. An FFT or spectrum view makes those components visible before a low-pass cutoff is chosen. The low-pass claim is defensible only when the record names the sample rate, frequency peak or band to keep, high-frequency content to reject, cutoff, and before/after trace that proves the useful feature survived.

Use the figure Figure 25.3 to test Feature Extraction After Filtering against the depicted system. Filter Selection Guide and Smooth sensor readings expose the two named boundaries behind Match the digital filter to the failure pattern - spikes, drift, broadband noise, or a known tone - rather than applying a generic cleanup step.

Read downward from the noise-pattern question in Figure 25.3 to the response sketch for each filter. Compare what passes with what is rejected before selecting a method that preserves the feature you need.

Incremental Practice

Spike Filter Knowledge Check

If you can fit a code step and pick a filter, you can stop here. Continue to Under the Hood for the quantization math and the resolution-versus-noise limits.

25.5 Under the Hood: Quantization Math and Noise Limits

Quantization and filtering both have clean formulas. They reveal why extra bits stop helping past the noise floor, and why averaging buys resolution only under the right conditions.

Code Step and Quantization Error

For an ideal ADC with reference voltage V_ref and N bits, one code step and the ideal error bound are:

For a 12-bit, 3.3-volt ADC, the LSB is about 0.806 mV, so the ideal rounding error is within about plus or minus 0.4 mV. The ideal signal-to-quantization-noise ratio grows with bits:

Each added bit improves the ideal ratio by about 6 dB. Real parts reach fewer effective bits because input noise, reference drift, and timing error all add uncertainty.

Resolution Versus Noise

If the input noise is larger than one LSB, the lowest code bits only describe that noise, so buying more resolution adds digits, not information. The useful measure is the effective number of bits, which can be well below the nominal count. The remedy is often to reduce noise or to average, not to specify a larger converter.

The front-end-noise preset below connects that statement to bandwidth. Keep the nominal converter at 16 bits and change only the upper band edge from 1010 Hz to 110 Hz. The integrated analog-noise floor falls, SNR and effective bits improve, and quantization becomes the larger remaining floor without any change to converter resolution.

Use the shaded integration band and noise ledger as the narrative for filter choice. Narrowing bandwidth is useful only when the removed frequencies are outside the signal and response-time requirements.

Averaging and Oversampling

Averaging N independent samples of a steady value with random, zero-mean noise reduces the noise standard deviation by the square root of N:

So averaging four samples halves the noise, and averaging sixteen quarters it. Oversampling and averaging can even recover effective resolution, gaining about one bit for every fourfold increase in samples, but only when the noise is random and at least around one LSB, which is what makes the averaging effective. The cost is delay and computation, the same trade as any smoothing.

Filter Order and the Aliasing Boundary

A higher-order filter has a sharper cutoff but adds more delay, more ringing, and more computation. The choice balances how cleanly you must separate signal from noise against how much lag the application tolerates. One limit is absolute: a digital filter operates on samples that already exist, so it cannot undo aliasing. Content that folded into the band during sampling looks like real signal, which is why anti-alias filtering must be analog and must happen before the ADC.

Coefficient Filters Are Sliding Evidence Tests

Many digital filters are just a short coefficient list sliding across a sampled signal. At each position, the firmware multiplies nearby samples by the coefficients and adds the products. That operation is discrete convolution. Written as a practical rule, a filtered sample is:

The coefficient pattern decides what the filter hears. Positive taps that average neighboring samples behave like a low-pass smoother because fast wiggles cancel into a calmer line. Difference-style taps, or taps whose sum is near zero, behave like a high-pass filter because steady baseline cancels while rapid changes remain. For a simple unnormalized example, `x = [1, 2, 3, 4, ...]` and `coeff = [1, 2, 3]` produce a convolution output that begins `[1, 4, 10, 16, ...]` as the three taps slide across the input. A production smoother would usually normalize the taps, but the review question is the same: keep the raw trace, coefficient list, cutoff or response goal, and before/after evidence together.

Make Coefficient Filters Are Sliding Evidence Tests traceable: inspect Figure 25.4 for Raw. Focus next on Boundary, the companion label anchoring Filter-order pipeline: preserve raw evidence, check the ADC boundary, reject spikes, smooth jitter, downsample after the limit, and validate the event.

In the diagram Figure 25.4, begin at Raw, which highlights Raw. Shift next to Boundary because it highlights Boundary, and close on Reject, which uses Reject to mark a decision point. This route carries Coefficient Filters Are Sliding Evidence Tests from Filter-order pipeline: preserve raw evidence, check the ADC boundary, reject spikes, smooth jitter, downsample after the limit, and validate the event into the project record.

Common Pitfalls

1. Buying bits past the noise. Extra resolution below the noise floor only digitizes noise more precisely.
2. Wasting range. A signal that uses only a small part of the input range throws away available codes.
3. Over-filtering. Too much smoothing hides real fast events and adds latency.
4. Expecting a digital filter to fix aliasing. Folded content cannot be separated after sampling; prevent it with an analog filter.

Averaging Noise Knowledge Check

At this depth, quantization and filtering are paired budgets: fit the range and code step to the signal, stop adding resolution at the noise floor, average or filter only as much as the noise demands, and remember that anti-alias work belongs in the analog domain before the sample is ever taken.

25.6 Summary

• Quantization rounds the analog input to discrete codes; one LSB equals V_ref divided by 2 to the power N and sets the finest resolvable change.
• The ideal quantization error is within plus or minus half an LSB, and ideal SNR rises about 6 dB per added bit.
• Resolution finer than the noise floor adds digits, not information; reduce noise or average instead of buying bits.
• Averaging N independent samples reduces random noise by the square root of N, at the cost of delay.
• Filter choice depends on the noise: averaging for random noise, a median for spikes, a low-pass for high-frequency bands.
• A coefficient filter applies discrete convolution: slide the taps across the samples, multiply, sum, and keep the coefficient list with the validation record.
• Higher filter order means a sharper cutoff but more lag, and no digital filter can undo aliasing already in the samples.
• Simple features such as peaks, intervals, and zero crossings should be computed from a filtered trace only when the filter window preserves the event being counted.
• De-mean before offset-sensitive features, and use frequency-domain evidence before claiming a low-pass filter preserves the useful component.

25.7 See Also