Why int8 Compression Falls Short of 4x
Why int8 Compression Falls Short of 4x
Ada re-derives this chapter’s own numbers step by step, at full precision
ADA · CALCULATION AUDIT
Why int8 Compression Falls Short of 4x
The chapter’s model has a 12 → 8 → 4 shape, and the intuition is that int8 stores one byte instead of four for a 4x saving. But the actual sizes are 560 bytes in float and 184 bytes in int8 — only 3.04x. This audit computes what the model really saves to show why int8 compression falls short of 4x: the un-quantized biases and scales are a fixed tax.
Companion to the chapter Lab: TinyML Gesture Classification — every number here comes from that chapter.
See the relationship before changing it
The figure reads from left to right. The blue card is quantized weights. The middle card applies the page rule. The green card is whole-model shrink. Walk the arrows once: set the input, apply the rule, then read the result with its unit.
Derive the baseline in four named moves
- 1
Name the input. The chapter baseline is 128 weights.
- 2
Name the relationship. shrink = 4 x (weights + 12 biases) / (weights + 48 bias bytes + 8 scale bytes)
- 3
Substitute with units. 4 x (128 + 12) / (128 + 48 + 8) = 3.04 times
- 4
Read the result. Keep the unit beside the value. Use it only inside the technical boundary on this page.
Predict, then change quantized weights
Try Predict the direction of shrink = 4 x (weights + 12 biases) / (weights + 48 bias bytes + 8 scale bytes). Test another quantized weights, then compare whole-model shrink.
Observe Fixed bias and scale bytes matter most when the model is small. Reset quantized weights to 128 and compare whole-model shrink.
Explain Fixed bias and scale bytes matter most when the model is small.
Check yourself
What should you do before trusting a moved-control result?
What does this small model leave out?
Ada: The “Model Size Formula” note gives the float and int8-style byte counts, and the intuition everyone carries is “int8 is one byte instead of four, so a 4x saving.” Let me compute what this model actually saves and find the gap.
The parameter counts are fixed by the shape 12 -> 8 -> 4:
- Weights:
(12 x 8) + (8 x 4) = 96 + 32 = 128 - Biases:
8 + 4 = 12
Now the two storage sizes, exactly as the sketch computes them:
- Float:
(128 + 12) x 4 = 560 bytes - int8-style:
128 x 1 + 12 x 4 + 2 x 4 = 128 + 48 + 8 = 184 bytes
The realized reduction is 560 / 184 = 3.04x, not the 4x the “one byte instead of four” story promises. A true 4x would be 560 / 4 = 140 bytes; this model lands at 184. The 44-byte gap is the part that stayed in float: the 12 biases and 2 scale factors total 12 x 4 + 2 x 4 = 56 bytes, which is 56 / 184 = 30% of the compressed model. The weights alone do hit the full ratio — 512 bytes -> 128 bytes = 4x — but they no longer dominate a model this small.
The design meaning is that quantization’s advertised per-weight ratio is an upper bound the whole model rarely reaches: the un-quantized remainder — biases, scales, and any float headers — is a fixed tax that eats a larger share the smaller the model gets, so the memory gate must be checked on the real converted artifact, never assumed from “4 bytes became 1.”
The storage calculation deliberately does not simulate tensor alignment, operator metadata, activation arenas, allocator overhead, or quantisation accuracy; it counts the stated weights, biases, and scale values only.
Work the audit first, then check the displayed derivation.
Every number above is taken from the chapter’s own material and re-derived step by step.