See the relationship before changing it
The figure reads from left to right. The blue card is high calibration raw count. The middle card applies this page's rule. The green card is raw span per known percent. Walk the arrows once: set the input, apply the rule, then read the result with its unit.
The retained audit below checks several chapter fixtures. This model keeps those stated values fixed and changes only high calibration raw count, so the numeric fixture does not switch without explanation.
Derive the baseline in four named moves
- 1
Name the input. The chapter baseline is 550 counts.
- 2
Name the relationship. span leverage = (high raw - 450) / 10 percent
- 3
Substitute with units. (550 - 450) / 10 = 10.00 counts/%
- 4
Read the result. Keep the unit beside the value. Use it only inside the technical boundary on this page.
Predict, then change high calibration raw count
Try Predict the direction of span leverage = (high raw - 450) / 10 percent. Test another high calibration raw count, then compare raw span per known percent.
Observe Moving the high point changes fitted span while the low fixture stays fixed. Reset high calibration raw count to 550 and compare raw span per known percent.
Explain Moving the high point changes fitted span while the low fixture stays fixed.
Check yourself
What should you do before trusting a moved-control result?
What does this small model leave out?
1. A calibration line has two jobs
The rule y=mx+b converts a raw ADC count x into a reported percentage y. The slope m says how much percentage one count represents. The offset b says what the line reports at raw zero.
2. Fit the correct points
Subtract known valuesΔy=55−45=10 percentage points.
Subtract raw countsΔx=550−450=100 counts.
Divide for slopem=Δy/Δx=10/100=0.1000% per count.
Solve the offsetb=45−0.1000(450)=0.
3. Move only the high raw point
A 10-count error changes the measured high point from 550 to 560. The known reference is still labelled 55%.
New raw span560−450=110 counts.
New slopem=10/110=0.0909% per count.
New offsetb=45−0.0909(450)=4.09%.
4. Try the reference error
TryMove the high-reference error from 0 to 20 counts and watch both endpoints move.
ObserveAt 10 bad counts, the slope becomes 0.0909%/count; raw 0 reads 4.09% and raw 1,000 reads 95.00%.
ExplainThe widget refits m and b from the same two-point equations shown above, then evaluates that line at both endpoints.
This is a straight-line, one-point-error example.
- reference uncertainty
- Needs separate evidence
- repeat readings
- Needs separate evidence
- independent holdout points
- Needs separate evidence
- nonlinearity checks
- Needs separate evidence
- an allowed operating range
- Needs separate evidence
Use field evidence or a deeper model before release.
5. Evaluate the endpoints
At raw zeroy=0.0909(0)+4.09=4.09%, which the chapter rounds to 4.1%.
At raw 1,000y=0.0909(1,000)+4.09=95.00%.
The fit is exact at the two points it was told to trust, yet wrong far away. That is why a fit coefficient is not release evidence.
6. Widen the baseline
The same 10-count error is 10% of a 100-count span but only 1.25% of an 800-count span. References near 10% and 90% bracket the chapter's 20–80% operating range, so the system interpolates instead of extrapolating from the middle.
7. Check yourself
What is the correct slope through 450→45% and 550→55%?
Why does a 10-count error make the new slope smaller?
Why add holdout points?
These are the chapter inputs, worked results, and named teaching assumptions.
- 45%
- Percentage, ratio, or gain
- 55%
- Percentage, ratio, or gain
- 450
- Chapter input or worked result
- 550
- Chapter input or worked result
- 10-count error
- Sensor scale, pressure, or digital result
- 0–1,000 raw scale
- Chapter input or worked result
- 4.1%
- Percentage, ratio, or gain
- 95.0%
- Percentage, ratio, or gain
- 10% relative narrow-span error
- Sensor scale, pressure, or digital result
- 1.25% wide-span comparison come from the chapter
- Sensor scale, pressure, or digital result
The page assumes one erroneous high raw point and an otherwise linear sensor; it does not claim two-point calibration removes curvature.
Phoebe guides