← Back to the chapter · Previous detail · Next detail →

Technical detail · Edge AI Fundamentals: Why and When

Budget the artifact and its buffers

A sizing check keeps the release honest. A model with 5 million weights is about 20 MB if each weight is stored as FP32 and about 5 MB after INT8 quantization, before runtime overhead. On a gateway with 16 MB available for the inference process, that can leave room for a 4 MB tensor arena, a 2 MB input buffer, and logging. On a smaller microcontroller, the same artifact may still be too large, so the design must move to a smaller architecture, a shorter input window, or a different target.

← Back to the chapter · Previous detail · Next detail →