← Back to the chapter · Next detail →
Technical detail · Model Optimization for Edge AI
Why 0.236 becomes int8 code 60
12.7 Small Quantization Example
For weights in [-0.50, 0.50], symmetric signed int8 can use . A weight of 0.236 becomes code 60 and reconstructs as approximately 0.236220, an error of 0.000220. The half-step error bound is approximately , and the quantization-noise estimate is .
Using the coarsely rounded scale 0.00394 instead gives round(0.236 / 0.00394) = 60 and 60 x 0.00394 = 0.2364. The different reconstructed value comes from rounding the scale. Preserve sufficient precision when comparing results.
The affine rule also explains a simple exact example: with scale 0.01 and zero point 0, weight 0.53 maps to code 53 and reconstructs as 0.53. A small change in one weight never replaces calibration or task-level validation: many small changes can alter a borderline decision.