Resolution and Sample Rate as Separate Gates

Resolution and Sample Rate as Separate Gates

Ada re-derives this chapter’s own numbers step by step, at full precision

foundations
math-foundations
calculation-audit
electronics
Ada ADA · CALCULATION AUDIT

Resolution and Sample Rate as Separate Gates

An ESP32 vibration monitor has to catch a 5 kHz bearing defect to within ±0.01 g, and the design rests on three numbers: a 12-bit step, an accuracy chain that rounds up to 10 bits, and a 12,500 Hz sample rate. It is tempting to treat those as one requirement. This audit re-derives the step size, the accuracy chain, and the sampling margin separately, and asks whether resolution and sample rate are really independent gates that must each pass on their own.

Companion to the chapter Analog vs. Digital Signals — every number here comes from that chapter.

Try

Use the displayed 5 kHz vibration signal, +/-0.01 g target, 12-bit conversion, and 12500 Hz sampling as a fixed case; press Calculate.

Observe

The resolution gate and sample-rate gate pass separately: the code step meets amplitude needs and 12500 Hz clears Nyquist.

Explain

Bit depth limits amplitude discrimination while sample rate limits temporal bandwidth; satisfying only 1 of those gates still produces misleading data.

See the relationship before changing it

The figure reads from left to right. The blue input is adc bits. The middle card names the page’s rule. The green output is step size. The arrow matters: change the input, apply the rule once, then read the result with its unit.

ADC Bits changes step size A three-part teaching diagram connects adc bits, the rule step = 3300 mV / (2^bits - 1), and step size. INPUT ADC bits APPLY THE RULE predict calculate check units OUTPUT RESULT
Walk the arrow. Each added bit doubles the level count, so the voltage step almost halves.

Derive the baseline in four named moves

  1. 1

    Name the input. The chapter baseline is 12 bits.

  2. 2

    Name the relationship. step = 3300 mV / (2^bits - 1)

  3. 3

    Substitute with units. 3300 / (2^12 - 1) = 3300 / 4095 = 0.8059 mV/step

  4. 4

    Read the result. Keep the unit beside the value, then use the result only inside the technical boundary below.

Predict, then change adc bits

Try Predict how step size responds when adc bits moves. Calculate adc bits; compare step size with that prediction.

12 bits
Chapter baseline
Step size

Observe Return to 12 bits. Recheck step size with adc bits at its chapter value.

Explain Each added bit doubles the level count, so the voltage step almost halves.

Check yourself

What should you do before trusting a moved-slider result?
Answer: Predict its direction, apply the displayed relationship, keep the units, and compare the reset value with the chapter’s worked baseline.
What does this small model leave out?
Answer: Only adc bits moves here. The step size calculation excludes field effects listed below.

Honesty note. The step size result holds adc bits as the only changing input. It is not a field prediction.

Technical boundaries

For the vibration contract, excluded from this fixed arithmetic are analogue noise spectra, anti-alias filter roll-off, ADC aperture jitter, reference drift, or sensor bandwidth limits.

Ada: The vibration design above rests on three numbers: a 12-bit step size, an accuracy chain that lands on 10 bits, and a 12.5 kHz sample rate. Resolution and sampling answer two different questions, so let me audit them separately before trusting the design.

  • Step size. A 12-bit ESP32 on a 3.3 V reference resolves 3.3 V / 4095 = 0.00080586 V = 0.8059 mV/step. Against a 10-bit part the level count rises 4095 / 1023 = 4.003x — the “4x better” the chapter claims.
  • Accuracy chain. 0.8059 mV/step / 40 mV/g = 0.020147 g/step, so quantization alone is +/- 0.010073 g, just inside the +/- 0.01 g target. And resolving 1,000 distinct steps needs log2(1000) = 9.9658, which rounds up to 10 bits.
  • Sampling. A 5 kHz defect needs 2 x 5,000 = 10,000 Hz minimum; the chapter’s 2.5x margin gives 12,500 Hz, a 1 / 12,500 = 0.00008 s = 80 us sample period — well under the ESP32’s 83 kHz ceiling.
  • One reading. At 2.0 g the sensor outputs 1.65 V + 2.0 g x 0.040 V/g = 1.73 V, digitized as floor(1.73 / 3.3 x 4095) = floor(2146.77) = 2146.

Resolution and sample rate are independent gates: 12 bits fixes how finely each sample is measured, 12,500 Hz fixes how fast the samples arrive, and the vibration monitor only passes because both clear their own requirement at the same time. Fix one and ignore the other and the data still lies.

Every number above is taken from the chapter’s own material and re-derived step by step.