Math Bridge: Calibration Versus Quantisation

← Back to Sensor Calibration
Math BridgeSensorsStruggle-friendly runway

Is the calibration miss just a digital step?

One thread from a 0.100 °C code step to the chapter's 1.68 °C validation residual.

Phoebe, the physics guidePhoebe guides
The one targetCompare residual with the ADC floor.
The chapter case0.100 °C, scale 1.0817, residual 1.68 °C.
What it buys youRecognise real nonlinearity.

See the relationship before changing it

The figure reads from left to right. The blue card is raw quantization step. The middle card applies this page's rule. The green card is scale-corrected step. Walk the arrows once: set the input, apply the rule, then read the result with its unit.

The retained audit below checks several chapter fixtures. This model keeps those stated values fixed and changes only raw quantization step, so the numeric fixture does not switch without explanation.

Raw quantization step changes scale-corrected step An input card leads through the rule corrected step = raw step x 1.0817 to the scale-corrected step result. INPUT PAGE INPUT APPLY THE RULE predict calculate check units OUTPUT RESULT
Walk the arrows. Calibration rescales each code step but does not remove residual error.

Derive the baseline in four named moves

  1. 1

    Name the input. The chapter baseline is 0.1 degrees C.

  2. 2

    Name the relationship. corrected step = raw step x 1.0817

  3. 3

    Substitute with units. 0.1000 x 1.0817 = 0.1082 degrees C

  4. 4

    Read the result. Keep the unit beside the value. Use it only inside the technical boundary on this page.

Predict, then change raw quantization step

Try Predict the direction of corrected step = raw step x 1.0817. Test another raw quantization step, then compare scale-corrected step.

0.1 degrees C
Chapter baseline
Scale-corrected step

Observe Calibration rescales each code step but does not remove residual error. Reset raw quantization step to 0.1 and compare scale-corrected step.

Explain Calibration rescales each code step but does not remove residual error.

Check yourself

What should you do before trusting a moved-control result?
Answer: Predict its direction, apply the shown relationship, keep the units, and reset to the worked baseline.
What does this small model leave out?
Answer: Only raw quantization step moves here. Field effects named in the technical boundary stay fixed.

Honesty note. This teaching model changes only raw quantization step. It is not a field prediction.

1. Separate calibration from resolution

Calibration changes the line used to interpret a reading. Quantisation has already rounded the raw measurement to one of a finite set of steps.

Phoebe: Multiplying a rounded number can move its rungs. It cannot create the missing values between them.

2. Derive the rounding floor

1

One stepThe DHT22 temperature code changes by q=0.100 °C.

2

Error intervalIdeal rounding error lies uniformly between −q/2 and +q/2.

3

RMS floorσq=q/√12=0.0289 °C.

3. Apply the chapter's straight line

The two-point correction is corrected=raw×scale+offset. Its scale of 1.0817 stretches each 0.100 °C raw rung to 0.108 °C, but the ladder is still discrete.

qcorrected=qraw×1.0817=0.108 °C

4. Try a validation residual

σq=q/√12; qcorrected=q×scale; diagnostic ratio=residual/σq

TryMove the validation miss while the chapter's DHT22 resolution, two-point scale, and 0.5 °C HVAC tolerance stay fixed.

Raw code step
Quantisation RMS
Corrected code step
Residual / RMS floor
HVAC tolerance / floor
Outside 0.5 °C tolerance?

ObserveThe chapter's 1.68 °C residual is about 58.2× the unrounded 0.02887 °C RMS floor and exceeds the 0.5 °C application tolerance.

ExplainA discrepancy tens of quantisation floors wide is evidence of model error, nonlinearity, drift, reference error, or conditions—not the last ADC rounding step.

Technical boundaries.

q/√12 assumes ideal uniformly distributed rounding with enough signal variation.

It does not include DHT22 accuracy limits, repeatability, hysteresis, reference uncertainty, temperature gradients, self-heating, interpolation error, coefficient rounding, drift, or correlated noise
Needs separate evidence

Use field evidence or a deeper model before release.

5. Work the chapter's scale

scale=(45.0−0.0)/(43.80−2.20)=1.0817
0.100 °C×1.0817=0.108 °C per corrected rung

6. Diagnose the residual

1.68/0.02887=58.2×
0.500/0.02887=17.3×

The quantisation floor is comfortably below the HVAC tolerance. The straight-line model, not code resolution, is the limiting evidence in this validation case.

7. Check yourself

Does two-point calibration create extra raw resolution?
Answer: No. It remaps existing codes with scale and offset.
Why is 1.68 °C not explained by 0.100 °C steps?
Answer: It is about 58 RMS quantisation floors wide and far beyond the 0.5 °C application tolerance.
What does the large residual prove?
Answer: It proves quantisation alone is not enough; further evidence is needed to distinguish nonlinearity, drift, reference error, or test conditions.
Honesty boundary.

These are the chapter inputs, worked results, and named teaching assumptions.

the raw 2.20/43.80 °C points
Temperature or angle value
0/45 °C references
Temperature or angle value
scale 1.0817
Chapter input or worked result
1.68 °C residual
Temperature or angle value
0.5 °C tolerance
Temperature or angle value
0.100 °C DHT22 resolution is catalog-typical
Named teaching assumption

Using the unrounded RMS gives 58.2×; the chapter's 58.1× uses its displayed 0.0289 °C rounding.

Return to the chapter with the maths unpacked →