Why int8 Compression Falls Short of 4x

Ada re-derives this chapter’s own numbers step by step, at full precision

foundations
math-foundations
calculation-audit
edge-fog
Ada ADA · CALCULATION AUDIT

Why int8 Compression Falls Short of 4x

The chapter’s model has a 12 → 8 → 4 shape, and the intuition is that int8 stores one byte instead of four for a 4x saving. But the actual sizes are 560 bytes in float and 184 bytes in int8 — only 3.04x. This audit computes what the model really saves to show why int8 compression falls short of 4x: the un-quantized biases and scales are a fixed tax.

Companion to the chapter Lab: TinyML Gesture Classification — every number here comes from that chapter.

Ada: The “Model Size Formula” note gives the float and int8-style byte counts, and the intuition everyone carries is “int8 is one byte instead of four, so a 4x saving.” Let me compute what this model actually saves and find the gap.

The parameter counts are fixed by the shape 12 -> 8 -> 4:

  • Weights: (12 x 8) + (8 x 4) = 96 + 32 = 128
  • Biases: 8 + 4 = 12

Now the two storage sizes, exactly as the sketch computes them:

  • Float: (128 + 12) x 4 = 560 bytes
  • int8-style: 128 x 1 + 12 x 4 + 2 x 4 = 128 + 48 + 8 = 184 bytes

The realized reduction is 560 / 184 = 3.04x, not the 4x the “one byte instead of four” story promises. A true 4x would be 560 / 4 = 140 bytes; this model lands at 184. The 44-byte gap is the part that stayed in float: the 12 biases and 2 scale factors total 12 x 4 + 2 x 4 = 56 bytes, which is 56 / 184 = 30% of the compressed model. The weights alone do hit the full ratio — 512 bytes -> 128 bytes = 4x — but they no longer dominate a model this small.

The design meaning is that quantization’s advertised per-weight ratio is an upper bound the whole model rarely reaches: the un-quantized remainder — biases, scales, and any float headers — is a fixed tax that eats a larger share the smaller the model gets, so the memory gate must be checked on the real converted artifact, never assumed from “4 bytes became 1.”

Every number above is taken from the chapter’s own material and re-derived step by step.