Resolution and Sample Rate as Separate Gates

Ada re-derives this chapter’s own numbers step by step, at full precision

ADA · CALCULATION AUDIT

Resolution and Sample Rate as Separate Gates

An ESP32 vibration monitor has to catch a 5 kHz bearing defect to within ±0.01 g, and the design rests on three numbers: a 12-bit step, an accuracy chain that rounds up to 10 bits, and a 12,500 Hz sample rate. It is tempting to treat those as one requirement. This audit re-derives the step size, the accuracy chain, and the sampling margin separately, and asks whether resolution and sample rate are really independent gates that must each pass on their own.

Companion to the chapter Analog vs. Digital Signals — every number here comes from that chapter.

Ada: The vibration design above rests on three numbers: a 12-bit step size, an accuracy chain that lands on 10 bits, and a 12.5 kHz sample rate. Resolution and sampling answer two different questions, so let me audit them separately before trusting the design.

• Step size. A 12-bit ESP32 on a 3.3 V reference resolves 3.3 V / 4095 = 0.00080586 V = 0.8059 mV/step. Against a 10-bit part the level count rises 4095 / 1023 = 4.003x — the “4x better” the chapter claims.
• Accuracy chain. 0.8059 mV/step / 40 mV/g = 0.020147 g/step, so quantization alone is +/- 0.010073 g, just inside the +/- 0.01 g target. And resolving 1,000 distinct steps needs log2(1000) = 9.9658, which rounds up to 10 bits.
• Sampling. A 5 kHz defect needs 2 x 5,000 = 10,000 Hz minimum; the chapter’s 2.5x margin gives 12,500 Hz, a 1 / 12,500 = 0.00008 s = 80 us sample period — well under the ESP32’s 83 kHz ceiling.
• One reading. At 2.0 g the sensor outputs 1.65 V + 2.0 g x 0.040 V/g = 1.73 V, digitized as floor(1.73 / 3.3 x 4095) = floor(2146.77) = 2146.

Resolution and sample rate are independent gates: 12 bits fixes how finely each sample is measured, 12,500 Hz fixes how fast the samples arrive, and the vibration monitor only passes because both clear their own requirement at the same time. Fix one and ignore the other and the data still lies.

Every number above is taken from the chapter’s own material and re-derived step by step.