Quantization and Filtering Calculation Audit
Quantization and Filtering Calculation Audit
Ada checks LSB size, measured noise, averaging, and coefficient filter work
ADA · CALCULATION AUDIT
Quantization and Filtering Calculation Audit
A clean ADC number is not just a big bit count. The voltage step, measured noise, and filter window decide whether firmware receives evidence or a smoother-looking guess.
A pressure sensor outputs 0.5 to 2.5 V into a 12-bit, 3.3 V ADC, giving a code step of about 0.806 mV — fine enough for a 5 mV meaningful change, but the chapter warns that 3 mV of input noise leaves most of the lowest bits describing noise rather than signal. This audit asks the question that gap invites: how many code steps does that 5 mV signal really span once the 3 mV noise floor is counted, and does a median filter actually recover the difference?
Companion to the chapter Quantization and Digital Filtering — every number here comes from that chapter.
See the relationship before changing it
The figure reads from left to right. The blue card is input noise. The middle card applies this page's rule. The green card is noise span. Walk the arrows once: set the input, apply the rule, then read the result with its unit.
The retained audit below checks several chapter fixtures. This model keeps those stated values fixed and changes only input noise, so the numeric fixture does not switch without explanation.
Derive the baseline in four named moves
- 1
Name the input. The chapter baseline is 3 mV.
- 2
Name the relationship. noise codes = input noise / 0.805664 mV per code
- 3
Substitute with units. 3 mV / 0.805664 mV per code = 3.72 codes
- 4
Read the result. Keep the unit beside the value. Use it only inside the technical boundary on this page.
Predict, then change input noise
Try Predict the direction of noise codes = input noise / 0.805664 mV per code. Test another input noise, then compare noise span.
Observe More input noise spans more ADC codes even though the converter resolution does not change. Reset input noise to 3 and compare noise span.
Explain More input noise spans more ADC codes even though the converter resolution does not change.
Check yourself
What should you do before trusting a moved-control result?
What does this small model leave out?
With Vref = 3.3 V and 12 bits, express the stated 3 mV noise in LSB, then average windows of 4 and 16 samples.
The raw noise spans 3.72 codes; averaging reduces it to 1.86 LSB for 4 samples and 0.93 LSB for 16 under the stated random-noise assumption.
Uncorrelated zero-mean noise cancels statistically at the square-root rate, but filtering only reshapes samples already captured and cannot undo pre-ADC aliasing.
1. The chapter's 12-bit example has a 0.806 mV code step
The worked example uses the existing 3.3 V reference and 4096 possible codes:
Rounded at the end, one code is 0.806 mV. The ideal rounding uncertainty is half a code: 0.805664 / 2 = 0.403 mV. A 5 mV meaningful change spans 5 / 0.805664 = 6.21 codes, so the ADC can represent that change before real noise is considered.
2. The 3 mV noise example is bigger than one LSB
The chapter says input noise can be around 3 mV. In code units:
That is why extra nominal resolution is not automatically useful. The physics and electronics are moving the input by several code steps, so the lowest bits mostly describe uncertainty unless the noise source is reduced or averaged under the right assumptions.
3. Averaging helps only with the stated random, zero-mean noise case
The chapter's rule is square-root reduction, so the arithmetic is:
| Averaging window | Arithmetic shown | Audit result |
| 4 samples | sqrt(4) = 2; 3 mV / 2 = 1.5 mV; 1.5 / 0.805664 | 1.86 LSB noise, with a 4-sample window |
| 16 samples | sqrt(16) = 4; 3 mV / 4 = 0.75 mV; 0.75 / 0.805664 | 0.93 LSB noise, with a 16-sample window |
| Ideal 12-bit SNR | 6.02 × 12 + 1.76 | 74.0 dB before reference, timing, and input-noise losses |
4. A coefficient filter is arithmetic sliding over evidence
For the chapter's unnormalized example, x = [1, 2, 3, 4, ...] and coeff = [1, 2, 3] produce these first outputs:
What the audit buys you: amplitude resolution, random-noise averaging, and filter coefficients are separate review gates. Filtering can calm jitter after sampling, but it cannot recover information that aliased before the ADC; that physics boundary still belongs in the analog path.
Every number above is taken from this chapter's own worked example and re-derived step by step.
Technical boundaries: This ideal quantisation-and-average model excludes correlated drift, reference noise, ADC nonlinearity, filter startup, coefficient scaling, overflow, anti-alias response, and lost event bandwidth.