The Narrow-Span Lever Arm

Ada re-derives this chapter’s own numbers step by step, at full precision

ADA · CALCULATION AUDIT

The Narrow-Span Lever Arm

A soil moisture sensor is calibrated at just 45% and 55% moisture — only 10% apart on a 0-1000 ADC scale (raw readings 450 and 550) — and after deployment a 10-count slip at the high reference turns the true 0% point into 4.1% and the true 100% point into 95.0%. This audit asks the question that lever-arm invites: carried to full precision, how much of that swing is really a 9.09% gain shift, and how much shrinks away once the calibration span is widened?

Companion to the chapter Calibration Span Error and Validation — every number here comes from that chapter.

Ada: The panel above says a 10-count slip at the high reference turns a true 0% into 4.1% and a true 100% into 95.0%. Let me carry every digit and check what a narrow span really costs.

• Correct fit through (450, 45%) and (550, 55%): gain (55 - 45) / (550 - 450) = 10 / 100 = 0.100000, offset 45 - 0.1 x 450 = 0.
• Slip the high raw by +10 counts to 560: gain 10 / 110 = 0.0909091, offset 45 - 0.0909091 x 450 = 4.090909.
• True 0% (raw 0) now reads 0.0909091 x 0 + 4.090909 = 4.09%.
• True 100% (raw 1000) now reads 0.0909091 x 1000 + 4.090909 = 95.00%.
• The gain shifted by (0.1 - 0.0909091) / 0.1 = 9.09%. The chapter’s quick “error / span” rule (10 / 100 = 10%) is the first-order estimate; carried exactly it is 9.09%. Widen the span to 800 counts and the same 10-count slip becomes 10 / 810 = 1.23% (the chapter’s 1.25% shorthand).

The short reference span acts as a lever arm: the fit pivots on the low point, so a fixed reference error swings the far end in proportion to how short the baseline is. Bracket the operating range and the lever shortens; crowd the two references together and a single-count reference mistake is amplified into a several-percent field error at the extremes.

Every number above is taken from the chapter’s own material and re-derived step by step.