20 OTA Update Architecture for IoT

Signed Artifacts, Manifests, Readiness, Boot Confirmation, and Rollback Evidence

testing

validation

cicd

iot

Keywords

IoT OTA update, firmware update CI/CD, signed firmware manifest, A/B dual-slot update, staged rollout, canary release, boot confirmation, rollback evidence

Start With the Device You Cannot Touch

Imagine an update reaching a device on a roof, in a field, or inside a wall. No one can press a reset button, plug in a debugger, or swap the firmware by hand. OTA architecture exists for that moment: the device must know which signed artifact it is allowed to install, whether it is ready, whether the new image really booted, and which previous image can recover it if the update fails.

Overview: An Update Is a Controlled Release, Not a File Copy

It is tempting to picture an over-the-air update as sending a file to a device. That picture is the source of most OTA disasters. A safe update is a controlled release path: a CI/CD system prepares and signs an artifact, a manifest states exactly what that artifact is allowed to update, the device decides whether it can attempt the update at all, the artifact is verified before it is used, the new image is confirmed to have actually booted, and the previous working image stays recoverable until that confirmation arrives.

The stakes are higher than for a server because the target is a remote device you may never touch again. A web rollout that goes wrong is rolled back centrally; a firmware update that goes wrong can leave a fleet bricked in the field. That is why the durable OTA practices are not about transfer speed but about safety: signed artifacts so a device only runs authorized code, staged rollout and canary cohorts so a bad release reaches few devices, a dual-slot layout so an update is atomic, and rollback so a failed boot is survivable.

If you only need the intuition, this layer is enough: receiving the bytes is the easy part. The update is not done when the file arrives, and it is not done when the file is written. It is done when the device has verified the artifact, booted the new image, and confirmed it — and even then the old image is kept until that confirmation is in hand.

The OTA gate path separates release-side evidence from device-side evidence so a failure can be located, not guessed.

The One-Minute View

Signed and described

A signed artifact plus a manifest that names the target, version, integrity material, and recovery expectation.

Verify before, confirm after

Check integrity and authorization before installing, then confirm the new image actually booted before trusting it.

Always recoverable

Keep the previous working image, roll out in stages, and define a rollback path before the first device updates.

Beginner Examples

A device finishes downloading new firmware but the signature does not verify, so it refuses to install — the transfer succeeded, the update did not.
A new image is written to a spare slot; if it fails to confirm a healthy boot, the bootloader switches back to the old slot automatically.
"Downloaded" is not "installed," and "installed" is not "confirmed running" — three different states that a good release record never collapses into one.

Overview Knowledge Check

If you can explain why arrival is not completion, you have the core idea. Continue to Practitioner for the release-side and device-side evidence.

Practitioner: Release-Side and Device-Side Evidence

Validation work for OTA is about preserving evidence on both sides of the boundary so a reviewer can trace one decision from a change record all the way to a confirmed boot. Keep the two sides separate: the release side proves what was built and what it is allowed to do; the device side proves what the device decided and what actually happened.

Release-Side: Change, Artifact, Manifest

The change record ties the update candidate to a reviewed change, naming the affected component, the intended target group, and the rollback assumptions. The artifact is the build output devices may receive; record its build reference, target hardware or boot profile, a hash for integrity, and a signature for authorization, so there is no ambiguity about which binary passed review. The manifest is the contract between the release system and the device's update logic, and it should be short and testable.

A manifest should name the artifact identity, the target identity, the version intent, any required boot or runtime condition, the integrity and authorization material, the install mode or slot expectation, the recovery expectation, and the fields the device reports back after completion or failure. If a device is allowed to defer an update, the valid reason categories belong here too — policy should not hide in prose.

Device-Side: Readiness, Verify, Confirm, Recover

Before attempting anything, the device records a readiness decision: its current firmware identity, target compatibility, storage and connection state, and a reason to attempt, defer, or reject. This matters most when a device does not update — without a reason, a skipped update looks identical to a transfer fault. Then it keeps transfer evidence (the artifact was received) strictly separate from verification evidence (integrity and authorization checks passed). Only then does it install and record a boot confirmation that the new image reached a defined checkpoint, while the previous image is retained until that confirmation. If any gate fails, a recovery record names the failed gate, the previous working image, and the next allowed action.

A compact OTA review record makes the update decision reproducible from artifact identity through recovery.

Roll Out in Stages

Do not ship to the whole fleet at once. A staged rollout releases to a small canary cohort first, watches health telemetry, and expands by waves only while the signals stay clean; a regression halts the rollout and triggers rollback for the affected cohort. This bounds the blast radius of a bad release to a fraction of the fleet, which is the difference between a contained incident and a mass bricking.

Gate

What It Proves

Evidence to Keep

Failure Mode If Weak

Signed artifact

The binary is the one that passed review and is authorized.

Build reference, hash, and signature reference.

An ambiguous name lets the wrong image ship.

Manifest

What the artifact may update and on which target.

Target identity, version intent, install mode, reporting fields.

A target too broad to explain why a device accepted it.

Readiness

Why the device attempts, defers, or rejects now.

Current state and a specific defer or reject reason.

A skipped update looks like a transfer failure.

Transfer + verify

The artifact arrived and was validated before use.

Transfer status separate from integrity and authorization results.

Collapsing the two trusts unverified bytes.

Boot confirmation

The new image actually ran to a defined checkpoint.

The observed confirmation condition, not just "installed."

The release cannot tell if the new image ran.

Recovery

A failed update is survivable.

Failed gate, retained image, next action, retest trigger.

"Rollback happened" with no failed gate named.

Practitioner Knowledge Check

If you can build the manifest, separate transfer from verification, and stage the rollout with recovery in place, you can stop here. Continue to Under the Hood for the three states an update passes through and how recovery actually works.

Under the Hood: Three States, Atomic Slots, and Real Recovery

The deeper layer is about the mechanisms that make an update survivable. Most OTA failures trace back to one of three things: treating distinct states as the same, writing the update in a way that is not atomic, or trusting an image that never confirmed it booted.

Downloaded, Installed, Confirmed Are Three Different States

A device that downloaded an artifact, a device that installed it, and a device that confirmed the new image booted have reached three different evidence states, and collapsing them hides risk. Transfer completion says bytes arrived. Installation says they were written to a target. Confirmation says the new image ran far enough to assert it is healthy. Only the third state should mark an update successful, and the release system needs each state reported separately to know where a stuck device actually stands.

Integrity, Authorization, and Anti-Rollback

Two cryptographic checks do different jobs. A hash proves the artifact is intact and was not corrupted in transfer. A signature proves it came from an authorized source and was not tampered with; both must be verified on the device, before install, not assumed from a successful download. A separate protection is anti-rollback: refusing to install an older firmware version than the device already trusts, so an attacker cannot push a known-vulnerable old build. Note the word "rollback" carries two meanings here — recovery rollback returns to the last working image after a failed boot, while anti-rollback prevents downgrading to an old version on purpose. Keep them distinct in any review.

Atomic Updates and the Dual-Slot Layout

An update must never leave the device in a half-written, unbootable state if power is lost mid-write. The standard answer is a dual-slot (A/B) layout: the new image is written to the inactive slot while the running image stays untouched, the boot target is switched only after the write and verification complete, and the previous slot is retained for rollback. This makes the update atomic from the bootloader's point of view — either the new slot is selected and confirmed, or the device falls back to the slot that already works. Overwriting the running image in place to save storage is how single-image devices brick on an interrupted update.

Boot Confirmation and Automatic Rollback

The link that ties it together is a confirmation handshake. After switching to the new slot, the device must reach a defined checkpoint — application started, connectivity restored, a self-test passed — and explicitly mark the image confirmed. If it does not confirm within a bounded window, often enforced by a watchdog, the bootloader treats the new image as failed and reverts to the retained previous slot on the next boot. Without this handshake, an image that boot-loops before the application runs is never detected as failed, and a device can sit broken while reporting "installed."

Recovery evidence attaches to a specific gate: which check failed, which image was retained, and what triggers retest.

Mechanisms and Failure Modes

Mechanism

What It Guarantees

Evidence to Request

Failure Mode If Weak

Signed + hashed artifact

Only intact, authorized code is installed.

On-device verify result before install, not after download.

A tampered or corrupt image is run as trusted.

Dual-slot atomic write

An interrupted update cannot brick the device.

Inactive-slot write with the running image untouched.

In-place overwrite bricks on power loss.

Boot confirmation

Only a verified healthy boot counts as success.

An observed checkpoint, not an "installed" flag.

A boot-looping image reports as installed.

Anti-rollback

Old, vulnerable versions cannot be reinstalled.

A monotonic version check at install time.

An attacker downgrades to a known-bad build.

Staged rollout

A bad release reaches few devices.

Canary cohort, halt criteria, and rollback telemetry.

A single bad build hits the whole fleet at once.

Common Review Findings

Transfer treated as installation. A completed download is recorded as a successful update, hiding verification and boot failures.
Integrity and authorization collapsed. A single vague "verified" status hides whether the signature, the hash, or neither was actually checked.
Missing boot confirmation. The release cannot tell whether the new image ran, only that it was written.
Rollback claimed without a gate. The record says rollback happened but never names which gate triggered it or which image was restored.
No retest trigger. The same accept, defer, or rollback decision cannot be reproduced because the inputs were not preserved.

Under-the-Hood Knowledge Check

At this depth, OTA is a survivability discipline: verify before you install, treat downloaded, installed, and confirmed as distinct states, keep the update atomic with a retained previous image, and let a confirmation handshake trigger automatic rollback. A trustworthy review asks which gate produced each decision and what proves the new image actually ran.

20.1 Summary

An OTA update is a controlled release path, not a file transfer: signed artifact, manifest, device readiness, verification, boot confirmation, and a recoverable previous image.
Keep release-side evidence (change record, artifact identity, hash, signature, manifest) separate from device-side evidence (readiness, transfer, verification, boot confirmation, recovery).
The manifest is the contract: it names artifact identity, target identity, version intent, integrity and authorization material, install mode or slot, recovery expectation, and reporting fields.
Downloaded, installed, and confirmed are three distinct states; only a confirmed healthy boot should mark an update successful.
A hash proves integrity and a signature proves authorization; both must be verified on the device before install, and anti-rollback prevents downgrading to a vulnerable old version.
A dual-slot (A/B) layout makes updates atomic so an interrupted write cannot brick the device, and it retains the previous image for rollback.
A boot-confirmation handshake, often watchdog-enforced, triggers automatic reversion to the previous slot when the new image fails to confirm.
Staged rollout with a canary cohort and halt criteria bounds the blast radius of a bad release; recovery evidence must name the failed gate and the retest trigger.

Key Takeaway

A trustworthy OTA pipeline never confuses arrival with success. Sign and describe the artifact, verify it on the device before install, keep the update atomic with a dual-slot layout, confirm the new image actually booted, and roll out in stages with a rollback that is proven, not promised. The question that grades any OTA review is simple: which gate produced this decision, and what evidence shows the new image is really running?