Edge & Fog Computing · Study deck

TinyML on Microcontrollers

Picture a small motor monitor that should spot an unusual sound before damage grows.

Edge Eddie is your guide for this deck.

tinyml

Edge Eddie, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

Build a TinyML deployment budget that separates flash, SRAM, latency, energy, and update headroom.
Explain how TensorFlow Lite Micro, vendor runtimes, CMSIS-NN kernels, and Edge Impulse-style workflows fit into the deployment path.
Decide whether an application is suitable for microcontroller inference or should move to a gateway, accelerator, or cloud tier.
Identify common failures caused by unsupported operators, activation memory, sensor mismatch, and unmeasured power policy.

Major section

In 60 Seconds

A sampling rate is the number of measurements taken in a set time.

An analog-to-digital converter is a part that turns a measured voltage into a number; it is often shortened to ADC.
A microcontroller is a small computer built to control a device.
Firmware is the code stored on that device.

Major section

In 60 Seconds (continued)

Latency means the delay from an event to a useful result.

A safe design should expose low confidence or a late result rather than presenting every answer as timely and certain.
One test set cannot represent every machine or season.
The design problem is not simply shrinking a model.

Major section

Phoebe's Field Notes: Why The Audio Buffer Cannot Hold Raw Sound

An 8 kB buffer therefore holds only 4096 samples, or 256 ms, even though the 8 kHz Nyquist limit easily covers 4 kHz speech.

Sampling, precision, and memory must fit together before inference begins.

Major section

Minimum Viable Understanding

TinyML is constrained inference: Most microcontroller deployments run a pre-trained model; training and retraining normally happen off-device.

Flash and RAM are separate budgets: Weights usually live in flash, while activations, input buffers, stacks, and framework memory consume SRAM.
The sensor pipeline is part of the model: Sampling rate, ADC resolution, microphone front end, feature extraction, and windowing must match training data.
Runtime support decides feasibility: A model that converts successfully can still fail because an operator, tensor shape, or memory plan is unsupported on the target runtime.

Major section

TinyML Fit Gate

Task fit: The inference output is local, narrow, and stable enough for a compact model: wake word, gesture, anomaly score, simple classification, or thresholded event detection.

Memory fit: Model weights, firmware, runtime code, tensor arena, feature buffers, stack, logs, and update staging all fit with deliberate headroom.

Major section

What Fits on a Microcontroller?

Microcontrollers vary by family and board configuration, so always check the exact datasheet and firmware build.

The patterns below are useful starting points.
Simple anomaly detection, gesture windows, and low-rate sensor classification often fit when the model, feature buffer, and tensor arena are intentionally small.
Image tasks are possible on selected MCUs, but input resolution, color format, camera buffer, and activation memory usually dominate the budget.

Major section

Deep Dive: Runtime and Memory Fit

Under the hood, flash and SRAM are separate release gates.

TinyML is machine-learning inference on microcontroller-class devices: kilobytes to a few megabytes of memory, milliwatt power budgets, often no operating system, and sometimes no floating-point unit.
On an MCU, the model must be self-contained with its runtime, sensor buffers, tensor arena, firmware stack, and update path.
Pruning or quantizing weights can reduce flash pressure while leaving the activation peak unchanged.

Major section

Deep Dive: Runtime and Memory Fit (continued)

That deployment is plausible; doubling the feature buffer requires a new fit check even though the model file did not change.

On a board with 256 kB flash and 64 kB SRAM, a 60,000-weight INT8 model consumes about 60 kB of flash.
If the runtime and required operators take 92 kB, the application and radio stack take 74 kB, and update metadata takes 12 kB, the flash total is 238 kB, leaving 18 kB of headroom.
Peak live activations drive the tensor arena in SRAM; that peak is not the same as the weight count.

Major section

Deep Dive: Runtime and Memory Fit (continued)

SRAM is separate: an 18 kB tensor arena plus an 8 kB audio feature buffer and 12 kB for stack, logs, and radio state uses 38 kB, leaving 26 kB.

Changing an input window, layer shape, or feature buffer can increase RAM even when the model file gets smaller.
On Arm Cortex-M devices, CMSIS-NN kernels can accelerate supported INT8 operators by using the core's SIMD and multiply-accumulate instructions.
An INT8 model with 80,000 weights needs roughly 80 kB of flash for those weights because each weight is one byte.

Major section

TinyML Development Loop

Firmware integrates the runtime before Measure exercises the target device; only then does Deploy use a signed update.

The instruction to measure before every pilot or update is the loop's control point, keeping model, firmware, and observed device behaviour aligned.

Major section

Sensor Data Quality

A model trained on clean lab data can fail when deployed with a different microphone, ADC, accelerometer mounting, enclosure, vibration profile, or temperature range.

If training uses one FFT window, mel filter bank, normalization rule, image crop, or sensor axis order, firmware must implement the same rule.
Small models cannot repair inconsistent labeling.
Confidence thresholds that work in a notebook may fail when quantized, duty-cycled, or exposed to field noise.

Deck summary

Key takeaways

A sampling rate is the number of measurements taken in a set time.

Latency means the delay from an event to a useful result.
An 8 kB buffer therefore holds only 4096 samples, or 256 ms, even though the 8 kHz Nyquist limit easily covers 4 kHz speech.
TinyML is constrained inference: Most microcontroller deployments run a pre-trained model; training and retraining normally happen off-device.
Task fit: The inference output is local, narrow, and stable enough for a compact model: wake word, gesture, anomaly score, simple classification, or thresholded event detection.

Retrieval practice

Recall check 1 of 5

Edge Eddie says: answer from memory, then check your reasoning.

Q1A team has a quantized model that fits in flash, but the tensor arena plus audio feature buffer leaves almost no SRAM for stack, logs, and firmware state. What should the team do before approving the TinyML deployment?

AApprove the deployment because the model weights fit in flash, even if activation buffers leave no runtime headroom

BMove input buffers into the cloud while still claiming the device performs the full local inference path

CIncrease the confidence threshold, because decision logic after inference reduces the SRAM required to run tensors

DReject or revise the deployment until peak SRAM is measured with enough headroom on the target firmware

Show answer

Answer: D TinyML memory approval requires peak SRAM headroom, not only a model file that fits in flash.

Retrieval practice

Recall check 2 of 5

Edge Eddie says: answer from memory, then check your reasoning.

Q2A TinyML model has 80,000 INT8 weights. Where do the weights live, and what separately bounds SRAM use?

AWeights live in SRAM; flash only stores confidence thresholds and labels

BWeights and activations stream from the cloud for every inference window

CWeight count alone bounds SRAM, so activation peaks do not need review

DWeights live in flash; the tensor arena and other runtime buffers bound SRAM use

Show answer

Answer: D Flash stores the quantized weights, while SRAM is bounded by the tensor arena, sensor buffers, stack, heap, and runtime state.

Retrieval practice

Recall check 3 of 5

Edge Eddie says: answer from memory, then check your reasoning.

Q3A quantized model converts to TensorFlow Lite, but the microcontroller build fails because one operator is not registered in the TinyML runtime. What is the best response?

AKeep the converted model and disable the missing-operator check, since successful conversion confirms that the model graph is valid.

BRegister the needed operator, replace the unsupported layer, or choose a runtime that supports the model before approval

CIncrease the sampling rate because more input data compensates for the missing executable model code.

DStore the model in SRAM instead of flash, because storage location adds missing operator implementations

Show answer

Answer: B TinyML feasibility includes runtime operator support, not only model conversion.

Retrieval practice

Recall check 4 of 5

Edge Eddie says: answer from memory, then check your reasoning.

Q4Which workload is the best TinyML candidate?

AA camera that must run large open-vocabulary object detection on high-resolution video with changing labels

BA low-power sensor that detects a fixed vibration anomaly and sends only an alert when confidence crosses a validated threshold

CA device that must retrain a large model every hour using new field labels and broad context

DA gateway that already runs Linux and has an accelerator available for large perception models and dashboards

Show answer

Answer: B The task is local, compact, stable, and benefits from avoiding continuous raw-data transmission.

Retrieval practice

Recall check 5 of 5

Edge Eddie says: answer from memory, then check your reasoning.

Q5What immediate bounded response makes low-latency on-device inference useful?

ASensor and feature pipeline

BMCU inference runtime

CLocal action or alert

DCloud training and update path

Show answer

Answer: C The local action or alert belongs beside the device decision because the value of low-latency inference is an immediate bounded response.

Q6Complete the simplified TensorFlow Lite Micro inference sequence:

Acopy_features_to_tensor(input, features);

Bstore_model_in_stack(input, features);

Cdelete_tensor_arena(input);

Dconvert_flash_to_sram(input);

Show answer

Answer: A A TinyML inference call copies preprocessed features into the input tensor, invokes the interpreter, checks the status, and reads the output tensor.

Print reference

Answers 1 of 2

Answer key.

D · TinyML memory approval requires peak SRAM headroom, not only a model file that fits in flash.
D · Flash stores the quantized weights, while SRAM is bounded by the tensor arena, sensor buffers, stack, heap, and runtime state.
B · TinyML feasibility includes runtime operator support, not only model conversion.
B · The task is local, compact, stable, and benefits from avoiding continuous raw-data transmission.

Print reference

Answers 2 of 2

Answer key.

C · The local action or alert belongs beside the device decision because the value of low-latency inference is an immediate bounded response.
A · A TinyML inference call copies preprocessed features into the input tensor, invokes the interpreter, checks the status, and reads the output tensor.