Carry the Gain in Full, Round Only the Reading
Carry the Gain in Full, Round Only the Reading
Ada re-derives this chapter’s own numbers step by step, at full precision
ADA · CALCULATION AUDIT
Carry the Gain in Full, Round Only the Reading
A load cell for beehive monitoring reads 410 raw ADC counts at 0 kg and 3685 counts at 50 kg on a 12-bit ADC spanning 0-4095, and the chapter’s own verification check lands the high point at 50.01 kg instead of exactly 50. This audit asks the question that stray 0.01 kg invites: is that gap a property of the load cell, or does it disappear once the gain is carried at full precision instead of the chapter’s rounded 0.01527 kg/count?
Companion to the chapter Lab: Sensor Calibration — every number here comes from that chapter.
See the relationship before changing it
The figure reads from left to right. The blue card is raw adc reading. The middle card applies this page's rule. The green card is calibrated load. Walk the arrows once: set the input, apply the rule, then read the result with its unit.
The retained audit below checks several chapter fixtures. This model keeps those stated values fixed and changes only raw adc reading, so the numeric fixture does not switch without explanation.
Derive the baseline in four named moves
- 1
Name the input. The chapter baseline is 3685 counts.
- 2
Name the relationship. load = raw x (50 / 3,275) - 6.259542
- 3
Substitute with units. 3,685 x 0.01526718 - 6.259542 = 50.000 kg
- 4
Read the result. Keep the unit beside the value. Use it only inside the technical boundary on this page.
Predict, then change raw adc reading
Try Predict the direction of load = raw x (50 / 3,275) - 6.259542. Test another raw adc reading, then compare calibrated load.
Observe Full-precision gain returns the high reference exactly; early rounding creates false drift. Reset raw adc reading to 3685 and compare calibrated load.
Explain Full-precision gain returns the high reference exactly; early rounding creates false drift.
Check yourself
What should you do before trusting a moved-control result?
What does this small model leave out?
Calibrate from raw 410 at 0 kg and 3685 at 50 kg; retain 0.01526718 kg/count until Check audit.
The offset resolves to -6.259542 kg and full-scale reconstruction returns 50.000 kg; early gain rounding produces a visible 0.01 kg drift.
Coefficient rounding propagates through every count, and using only 3275 of 4096 codes makes resolution 1.25x coarser than the ideal span.
Ada: The load-cell example maps 410 counts to 0 kg and 3685 counts to 50 kg, then verifies the high point at 50.01 kg. That 0.01 kg is not the sensor – it is rounding. Let me carry the gain in full and watch it vanish.
- Gain:
50 / (3685 - 410) = 50 / 3275 = 0.01526718 kg/count(the chapter rounds this to 0.01527). - Offset:
0 - 410 x 0.01526718 = -6.259542 kg(chapter: -6.26). - Verify the high point with the unrounded coefficients:
3685 x 0.01526718 - 6.259542 = 50.0000 kgexactly, because(3685 x 50 - 410 x 50) / 3275 = 50. The chapter’s 50.01 kg comes only from multiplying 3685 by the truncated 0.01527. - Resolution equals the gain:
0.01526718 kg = 15.27 g/count; a perfect 12-bit span would give50 / 4096 = 12.21 g/count.
The 15.27 g is coarser than the 12.21 g ideal by exactly 4096 / 3275 = 1.25x, because the load cell drives only 3275 / 4096 = 80% of the ADC range and wastes the other 20%. Two lessons sit in one example: resolution is set by how much of the converter you actually fill, and coefficients must be carried at full precision through the whole calculation – round the gain early, as the truncated 0.01527 does, and you inject a 0.01 kg error at full scale into every reading it touches.
Every number above is taken from the chapter’s own material and re-derived step by step.
Technical boundaries: The two-point linear fit omits load-cell nonlinearity, hysteresis, creep, temperature drift, ADC noise, excitation variation, mechanical preload, and calibration-mass uncertainty.