A field team faces an unresolved physical question: Why does 0.236 become code 60? They must answer it before changing maximum absolute calibrated weight on the real device. Predict the direction first.
See the relationship before changing it
The figure reads from left to right. The blue card is maximum absolute calibrated weight. The middle card applies this page's relationship. The green card is rms floor. Walk the arrows once: set the input, apply the rule, then read the result with its unit.
The retained audit below checks several chapter fixtures. This added model holds every other chapter fixture fixed, so the numeric fixture does not switch without explanation.
Derive the baseline in four named moves
- 1
Name the input. The chapter baseline for maximum absolute calibrated weight is 0.5.
- 2
Name the relationship. s=max|w|/127; code=round(w/s); ŵ=codes; error=|ŵ-w|
- 3
Substitute the chapter fixture. Set maximum absolute calibrated weight to 0.5. The page ledger gives rms floor as 0.001.
- 4
Read the result. Keep the stated output unit beside the value. Use it only inside the technical boundary on this page.
Predict, then change maximum absolute calibrated weight
Try Predict the direction of rms floor. Move one control, calculate, then check your prediction.
Observe The range is shared across many values, so protecting a large outlier spends resolution on every ordinary weight. Calibration chooses that trade; validation decides whether it is acceptable. Reset the control to 0.5 and compare rms floor.
Explain Only maximum absolute calibrated weight moves here. The other chapter fixtures remain fixed.
Check yourself
What should you do before trusting a moved-control result?
What does this small model leave out?
1. Draw the int8 ruler
A symmetric int8 quantiser maps zero to zero and the chosen maximum magnitude to code 127. The scale is the gap between adjacent ruler marks. Each weight moves to its nearest available mark unless it lies outside the calibrated range.
2. Name each move
Set the scales = max|w|/127.
Normalize the weightraw code = w/s.
Round onceInteger code = round(raw code).
Reconstructŵ = integer code × s.
Bound ideal error|ŵ − w| ≤ s/2; σ = s/√12 across uniform positions.
3. Work 0.236 exactly
Reconstruction gives 60s = 0.236220. The absolute error is 0.000220, below the half-step bound 0.001969. The ideal per-weight RMS floor is 0.0011365 and the ideal eight-bit ceiling is 49.9 dB.
4. Try one controlled change
TryMove only the calibrated maximum magnitude. The example weight, code count, and eight-bit ideal ceiling stay fixed.
ObserveAt max |w| = 0.50, code 60 reconstructs 0.236220 with 0.000220 error. A wider range makes the scale and error bound wider, although the chosen code may change in steps.
ExplainThe range is shared across many values, so protecting a large outlier spends resolution on every ordinary weight. Calibration chooses that trade; validation decides whether it is acceptable.
This is one symmetric per-tensor weight quantiser with no clipping.
- Granularity
- Per-channel scales may preserve narrow channels better
- Activations
- They need their own calibration, zero point, clipping, and error analysis
- Network
- Layer interactions can amplify or cancel local weight errors
Use representative calibration data and compare the complete target artifact against the baseline task gates.
5. Read the half-step bound correctly
It applies to an in-range value rounded to the nearest uniform code. A clipped value can miss by much more. The RMS expression describes many positions under an ideal uniform-error assumption, not this single weight's exact error.
6. Carry a trustworthy optimization record
Record range selection, granularity, calibration set, clipped fraction, representative layer errors, full-model metrics, thresholds, latency, memory, runtime support, artifact hash, and rollback rule.
7. Check yourself
Why divide 0.50 by 127 rather than 255?
Why is the reconstructed value not exactly 0.236?
Does the 0.000220 error approve the optimized model?
The page reproduces one chapter weight and exposes what that example cannot prove.
- 0.50
- Chapter range maximum
- 60
- Nearest code for 0.236
- 0.000220
- One local reconstruction error
Go deeper in the chapter, then calibrate and validate the entire deployed model.
Edge Eddie guides