← Back to the chapter · Previous detail · Next detail →

Technical detail · TinyML on Microcontrollers

Runtime allocation and kernel fit

Runtime Piece Role Fit Evidence
Model FlatBuffer Stores the trained graph and quantized weights in flash. Converted artifact size, checksum, operator list, and version metadata.
Interpreter Walks the graph and calls each operator kernel in order. Exact runtime build, compiler flags, and target-board initialization result.
Tensor arena Pre-allocated SRAM used for tensors and intermediate activations. Smallest successful arena plus headroom for stack, logs, buffers, and radio state.
Operator resolver Registers only the kernels the model needs. Required kernels included; unused kernels excluded to save flash.

Under the hood, flash and SRAM are separate release gates. An INT8 model with 80,000 weights needs roughly 80 kB of flash for those weights because each weight is one byte. Peak live activations drive the tensor arena in SRAM; that peak is not the same as the weight count. Pruning or quantizing weights can reduce flash pressure while leaving the activation peak unchanged. Changing an input window, layer shape, or feature buffer can increase RAM even when the model file gets smaller.

On Arm Cortex-M devices, CMSIS-NN kernels can accelerate supported INT8 operators by using the core’s SIMD and multiply-accumulate instructions. That benefit is not automatic: the converted model must use supported operators, the resolver must include the right kernels, and the measured firmware build must still fit flash and SRAM with the sensor path enabled.

yes

no

yes

no

Converted INT8 model

Code, constants, and weights in flash

Target firmware build

Register required operators

Interpreter initialization

Pre-allocated tensor arena in SRAM

AllocateTensors succeeds?

Run inference with peak activations in arena

Revise model, operators, buffers, or board

Measure flash, peak SRAM, latency, and energy

Headroom remains?

Pilot deployment

← Back to the chapter · Previous detail · Next detail →