By the end of this section, you will be able to:
When data travels across a network, bits can get flipped by interference, like static on a phone line garbling words. Error detection methods like CRC and checksums are mathematical tricks that let the receiving device check whether the data arrived intact. Think of it as a self-checking code, like the last digit on a credit card number.
“Wireless signals are noisy,” said Sammy the Sensor. “Electromagnetic interference can flip bits during transmission. Without error detection, my temperature reading of 25°C could arrive as 125°C – and nobody would know it was wrong!”
“Simple checksums are like adding up all the digits,” explained Max the Microcontroller. “If I send ‘1, 2, 3’ with a checksum of 6, the receiver adds them up and checks. If they get 6, the data is probably correct. But checksums miss some errors – like if two numbers get swapped.”
“CRC is much more powerful,” added Lila the LED. “It uses polynomial division to create a mathematical fingerprint of the data. CRC-32 can detect ALL single-bit errors, ALL double-bit errors, and ALL burst errors up to 32 bits long. It is the gold standard for data integrity.”
“The trade-off is simple,” said Bella the Battery. “Checksums use minimal CPU and memory – great for tiny sensors sending routine readings. CRC uses more processing power but catches more errors – essential for critical data like medical sensors or industrial controls. Pick based on how important the data is.”
Before diving into this chapter, you should be familiar with:
Wireless channels are inherently noisy. Electromagnetic interference from motors, microwaves, and other devices can flip bits during transmission. Signal fading in multipath environments corrupts data randomly. Without error detection, your temperature sensor might report 125°C instead of 25°C due to a single bit flip – and your system would have no way to know the data is wrong.
Error detection identifies when data has been corrupted during transmission. The sender computes a mathematical function over the data and appends the result (checksum or CRC). The receiver performs the same computation - if the results differ, corruption occurred.
| Error Type | Description | Example | Detection Difficulty |
|---|---|---|---|
| Single bit | One bit flipped | 0x48 becomes 0x49 | Easy |
| Burst error | Multiple consecutive bits flipped | 0x4845 becomes 0x3912 | Moderate |
| Byte swap | Two bytes exchange positions | 0x1234 becomes 0x3412 | Hard for checksums |
| Insertion | Extra byte added | “AB” becomes “ACB” | Moderate |
| Deletion | Byte removed | “ABC” becomes “AC” | Moderate |
The simplest approach is to sum all bytes in the message. Several variants exist with different trade-offs.
uint8_t calculateChecksum(const uint8_t* data, size_t length) {
uint8_t sum = 0;
for (size_t i = 0; i < length; i++) {
sum += data[i];
}
return sum;
}Advantages:
Disadvantages:
Used in IP, TCP, and UDP headers, this algorithm treats data as 16-bit words and performs one’s complement addition:
uint16_t internetChecksum(const uint16_t* data, size_t wordCount) {
uint32_t sum = 0;
for (size_t i = 0; i < wordCount; i++) {
sum += data[i];
}
// Fold 32-bit sum to 16 bits (add carry bits)
while (sum >> 16) {
sum = (sum & 0xFFFF) + (sum >> 16);
}
return ~sum; // One's complement
}Key property: The checksum of the data plus its checksum equals 0xFFFF (all ones), making verification simple.
Simple checksums cannot detect when bytes are transposed:
Original: 0x12 + 0x34 = 0x46
Swapped: 0x34 + 0x12 = 0x46 (same checksum!)This is why TCP/IP uses checksums for headers (where errors are unlikely) but CRC for link-layer frames (where bit errors are common).
CRC provides much stronger error detection by treating the data as a polynomial and dividing by a fixed generator polynomial. The remainder becomes the CRC value.
#define CRC16_POLYNOMIAL 0x1021 // CRC-CCITT
#define CRC16_INITIAL 0xFFFF
uint16_t calculateCRC16(const uint8_t* data, size_t length) {
uint16_t crc = CRC16_INITIAL;
for (size_t i = 0; i < length; i++) {
crc ^= ((uint16_t)data[i] << 8);
for (int bit = 0; bit < 8; bit++) {
if (crc & 0x8000) {
crc = (crc << 1) ^ CRC16_POLYNOMIAL;
} else {
crc = crc << 1;
}
}
}
return crc;
}| CRC Type | Polynomial | Width | Used In |
|---|---|---|---|
| CRC-8 | 0x07 | 8 bits | I2C, SMBus |
| CRC-16-CCITT | 0x1021 | 16 bits | Bluetooth, USB, X.25 |
| CRC-16-IBM | 0x8005 | 16 bits | Modbus, USB |
| CRC-32 | 0x04C11DB7 | 32 bits | Ethernet, ZIP, PNG |
| Feature | Simple Checksum | CRC-16 | CRC-32 |
|---|---|---|---|
| Computation | Sum of bytes | Polynomial division | Polynomial division |
| Size | 1-2 bytes | 2 bytes | 4 bytes |
| Burst Error Detection | Poor | Excellent (all bursts ≤16 bits) | Excellent (all bursts ≤32 bits) |
| Single Bit Errors | 100% | 100% | 100% |
| Two Bit Errors | ~99.6% | 100% | 100% |
| Byte Swap Detection | 0% | 100% | 100% |
| CPU Cycles (8-bit MCU) | ~2 per byte | ~40 per byte | ~80 per byte |
| Code Size | ~20 bytes | ~100 bytes | ~200 bytes |
CRC is guaranteed to detect:
No error detection is perfect. CRC may miss:
CRC’s error detection probability follows from information theory. For an n-bit CRC:
\[P_{\text{undetected}} = 2^{-n}\]
For CRC-32 (most common):
\[P_{\text{undetected}} = 2^{-32} = \frac{1}{4{,}294{,}967{,}296} \approx 2.3 \times 10^{-10}\]
In a wireless sensor network transmitting 1 billion packets over 10 years:
\[\text{Expected undetected errors} = 10^9 \times 2.3 \times 10^{-10} \approx 0.23\]
This means you’d expect less than one undetected corruption across a billion transmissions. CRC-16 provides \(2^{-16} \approx 1.5 \times 10^{-5}\), which would give approximately 15,000 undetected errors per billion packets – demonstrating why CRC-32 is preferred for critical data.
For CRC-32 with random errors: - Probability of undetected error = 2^(-32) ≈ 0.0000000233% (about 2.3 per 10 billion) - For 1 billion messages, expect ~0.23 undetected corruptions
This is why CRC is combined with other mechanisms (sequence numbers, ACKs) for critical applications.
Many IoT microcontrollers include hardware CRC units that compute CRC in a single clock cycle per byte:
// ESP32 hardware CRC example
#include "esp_rom_crc.h"
uint32_t fastCRC32(const uint8_t* data, size_t length) {
// Hardware-accelerated CRC-32
return esp_rom_crc32_le(0, data, length);
}
uint16_t fastCRC16(const uint8_t* data, size_t length) {
// Hardware-accelerated CRC-16/CCITT
return esp_rom_crc16_le(0, data, length);
}| Method | Cycles per Byte | 100-byte Message |
|---|---|---|
| Software CRC-32 | ~80 | 8000 cycles |
| Hardware CRC-32 | ~1 | 100 cycles |
| Software CRC-16 | ~40 | 4000 cycles |
| Hardware CRC-16 | ~1 | 100 cycles |
Before implementing software CRC, check if your microcontroller has hardware support: - ESP32: Yes (ROM functions) - STM32: Yes (CRC peripheral) - nRF52: Yes (CRC peripheral) - ATmega328: No (software only)
This chapter covered error detection mechanisms for IoT networks:
Key Concepts:
Trade-offs:
| Mechanism | Overhead | Detection | Best For |
|---|---|---|---|
| 8-bit checksum | 1 byte | Weak | Resource-constrained, reliable channels |
| CRC-16 | 2 bytes | Strong | Typical IoT wireless |
| CRC-32 | 4 bytes | Excellent | Critical data, Ethernet |
While not an IoT case, this incident illustrates why data integrity matters in every system with constrained arithmetic. The Patriot missile system tracked time using a 24-bit fixed-point register with 0.1-second resolution. After 100 hours of continuous operation, cumulative rounding error reached 0.34 seconds – enough for the system to miscalculate an incoming Scud missile’s position by 687 meters. The missile struck a military barracks, killing 28 soldiers.
The lesson for IoT: Sensors accumulating readings over long periods (smart meters, environmental monitors) face the same class of error. A 16-bit temperature sensor with 0.01°C resolution accumulates rounding drift of 0.006°C per hour – after 30 days, that is 4.3°C of uncorrected drift. CRC detects transmission corruption but cannot detect computation drift; IoT systems need both error detection on the wire AND periodic calibration validation at the application layer.
NASA’s investigation of Toyota’s electronic throttle control system found that single-event upsets (cosmic ray-induced bit flips in SRAM) could corrupt throttle position variables. The investigation identified insufficient software fault tolerance: safety-critical RAM variables lacked robust integrity verification, and the task scheduler had defects that could cause stack overflow, corrupting memory unpredictably. The system’s integrity checks were inadequate to detect the specific bit-flip patterns that could unlock the throttle.
Quantified impact: 89 deaths attributed to unintended acceleration incidents over the investigation period. Remediation required upgrading integrity verification of safety-critical RAM variables using stronger checksums, adding redundant variable storage with cross-checking (store each value twice and compare), and implementing a hardware watchdog timer – demonstrating that application-layer integrity checks are essential even when lower-level hardware appears reliable.
| Application Class | Example Devices | BER Environment | Recommended | Cost per Device |
|---|---|---|---|---|
| Non-critical telemetry | Room temperature, ambient light | Indoor, low interference (<10-6) | 8-bit checksum | $0.00 (software only) |
| Environmental monitoring | Air quality, soil moisture | Outdoor, moderate interference (~10-5) | CRC-16 | $0.00 (software, +40 cycles/byte) |
| Industrial process control | Valve position, flow meters | Factory floor, high EMI (~10-4) | CRC-32 + hardware accelerator | $0.15 (MCU with CRC peripheral) |
| Safety-critical | Gas leak detectors, medical sensors | Any environment | CRC-32 + redundant channel + application-level validation | $0.50-2.00 (dual MCU or safety-rated SoC) |
Cost insight: The difference between “no detection” and “CRC-32 with hardware acceleration” is essentially zero for modern MCUs – STM32L0 series ($0.89, includes CRC peripheral) computes CRC-32 over a 128-byte sensor payload in 2 microseconds. There is no technical or economic reason to skip CRC on any wireless IoT link.
Different IoT protocols use different error detection algorithms, each optimized for specific constraints. Understanding why each protocol made its choice helps you make the right choice for custom implementations.
| Protocol | Algorithm | Strength | CPU Cost (8-bit MCU) | Why This Choice |
|---|---|---|---|---|
| IEEE 802.15.4 (Zigbee, Thread) | CRC-16 (ITU-T / 0x1021) | All bursts <=16 bits | Hardware offloaded to radio chip | Short frames (127 bytes max), noisy RF channel; radio hardware computes CRC |
| Ethernet | CRC-32 | All bursts <=32 bits | Hardware offloaded | Large frames (1500 bytes), copper/fiber reliability |
| LoRaWAN | CRC-16 (PHY) + MIC-32 (MAC) | CRC: burst detection; MIC: authentication | CRC: ~30 us; MIC: ~500 us (AES) | MIC also prevents tampering, not just corruption |
| TCP/UDP | 16-bit one’s complement checksum | Misses byte-swap, some burst errors | ~5 us per 100 bytes | Designed for 1970s CPUs; fast but weak |
| CoAP | Relies on UDP checksum + DTLS | Weak without DTLS; strong with DTLS | Minimal (delegates to lower layers) | Constrained devices cannot afford redundant checks |
| MQTT | Relies on TCP CRC + TLS | TCP catches transport errors; TLS catches rest | Zero additional overhead | TCP already provides reliable, checked delivery |
| BLE | CRC-24 | All bursts <=24 bits | Hardware offloaded | Short packets (251 bytes max), noisy 2.4 GHz |
If you are designing a custom binary protocol for sensor data (common in industrial IoT), choose your error detection based on three factors:
Factor 1: Bit Error Rate (BER) of the channel
BER < 10^-6 (wired Ethernet, fiber):
Simple checksum is adequate. Errors are extremely rare.
BER 10^-6 to 10^-4 (Wi-Fi, BLE, indoor RF):
CRC-16 minimum. Burst errors from interference are common.
BER > 10^-4 (LoRa at SF12, noisy industrial environments):
CRC-32 plus application-level integrity (HMAC or MIC).
At BER=10^-3, a 100-byte packet has 55% chance of containing an error.Factor 2: Consequence of undetected corruption
Low consequence (ambient temperature, lighting level):
Checksum-16 is sufficient. Worst case: one bad reading displayed.
Medium consequence (HVAC control, inventory tracking):
CRC-16 recommended. Bad data triggers incorrect actions.
High consequence (medical dosing, industrial valve position):
CRC-32 + authenticated integrity check (HMAC-SHA256 or AES-CMAC).
Undetected corruption could cause physical harm.Factor 3: Available CPU and ROM
8-bit MCU, <8 KB ROM (ATtiny85):
Checksum-8 or Fletcher-16 (table-free, ~20 bytes of code)
8-bit MCU, >8 KB ROM (ATmega328P):
CRC-16-CCITT with 256-byte lookup table
32-bit MCU (ESP32, nRF52840):
CRC-32 with hardware acceleration or 1 KB lookup table
HMAC-SHA256 via hardware crypto peripheralPractical example: A custom RS-485 Modbus device sending 16-bit pressure readings over a 200 m cable in a factory with welding machines nearby (high EMI):
Understanding the polynomial division behind CRC error detection:
Polynomial Representation:
CRC-16-CCITT uses polynomial: x^16 + x^12 + x^5 + 1
Binary representation: 1 0001 0000 0010 0001 = 0x11021
(Leading 1 is implicit, so polynomial = 0x1021)Step-by-Step Calculation for “Hi” (0x48 0x69):
Step 1: Append 16 zero bits to message
Message bits: 01001000 01101001 0000000000000000
(0x48 )(0x69 )(appended zeros)
Step 2: Initialize CRC register to 0xFFFF
CRC = 1111111111111111
Step 3: Process first byte (0x48 = 01001000)
XOR message byte into upper 8 bits of CRC:
CRC = 1111111111111111 XOR 0100100000000000
= 1011011111111111
For each bit (8 times):
If MSB = 1:
Shift left, XOR with 0x1021
Else:
Shift left only
Bit 1 (MSB=1): 0110111111111110 XOR 0001000000100001 = 0111111111011111
Bit 2 (MSB=0): 1111111110111110
...
(Continue for all 8 bits)
Step 4: Process second byte (0x69)
XOR into upper 8 bits, repeat shift-XOR for 8 bits
Step 5: Final CRC value
CRC = 0x64E5 (after processing all bits with init=0xFFFF)Why It Works:
Hardware Implementation (Shift Register):
Input → [XOR] → [D15..D12][XOR][D11..D6][XOR][D5..D1][XOR][D0] → Output
↑ ↑ ↑ ↑
| | | |
Polynomial bits: x^16 x^12 x^5 x^0Each shift processes one input bit in constant time, making hardware CRC extremely fast.
Exercise 1: Checksum Weakness Demonstration
Prove that simple checksums miss byte-swap errors:
# checksum_weakness.py
def simple_checksum(data):
return sum(data) & 0xFF
# Original packet: sensor reading
packet1 = bytes([0x19, 0x00, 0xE8, 0x03]) # Temp=25.0°C, Humidity=100.0%
checksum1 = simple_checksum(packet1)
print(f"Original: {packet1.hex()} → Checksum: 0x{checksum1:02X}")
# Corrupted: humidity bytes swapped
packet2 = bytes([0x19, 0x00, 0x03, 0xE8]) # Temp=25.0°C, Humidity=59395% (WRONG!)
checksum2 = simple_checksum(packet2)
print(f"Swapped: {packet2.hex()} → Checksum: 0x{checksum2:02X}")
if checksum1 == checksum2:
print("⚠️ CHECKSUM FAILED TO DETECT ERROR!")
else:
print("✓ Checksum caught the error")
# Now try CRC-32
import zlib
crc1 = zlib.crc32(packet1) & 0xFFFFFFFF
crc2 = zlib.crc32(packet2) & 0xFFFFFFFF
print(f"\nCRC-32 Original: 0x{crc1:08X}")
print(f"CRC-32 Swapped: 0x{crc2:08X}")
print("✓ CRC-32 DETECTS THE SWAP" if crc1 != crc2 else "CRC failed")Expected Output:
Original: 1900e803 → Checksum: 0x04
Swapped: 190003e8 → Checksum: 0x04
⚠️ CHECKSUM FAILED TO DETECT ERROR!
CRC-32 Original: 0x03FBEF57
CRC-32 Swapped: 0x89A9FE80
✓ CRC-32 DETECTS THE SWAPWhat to Observe: Byte-swap preserves the sum but changes the CRC. This demonstrates why IoT protocols use CRC for wireless links where byte-order corruption is possible.
Exercise 2: Burst Error Detection Threshold
Find the maximum burst length CRC-32 can detect:
# burst_error_test.py
import random
import zlib
def inject_burst_error(data, burst_start_bit, burst_length):
"""Flip burst_length consecutive bits starting at burst_start_bit."""
data_bits = bytearray(data)
for i in range(burst_length):
bit_pos = burst_start_bit + i
byte_idx = bit_pos // 8
bit_idx = bit_pos % 8
data_bits[byte_idx] ^= (1 << (7 - bit_idx))
return bytes(data_bits)
def test_burst_detection(burst_length, trials=1000):
detected = 0
for _ in range(trials):
# Generate random 100-byte packet
original = bytes([random.randint(0, 255) for _ in range(100)])
original_crc = zlib.crc32(original) & 0xFFFFFFFF
# Inject burst error at random position
burst_start = random.randint(0, len(original)*8 - burst_length)
corrupted = inject_burst_error(original, burst_start, burst_length)
corrupted_crc = zlib.crc32(corrupted) & 0xFFFFFFFF
if original_crc != corrupted_crc:
detected += 1
return detected / trials * 100
# Test different burst lengths
for burst_len in [1, 2, 5, 10, 16, 17, 20, 32]:
detection_rate = test_burst_detection(burst_len)
status = "GUARANTEED" if burst_len <= 32 else "PROBABLE"
print(f"Burst length {burst_len:2d} bits: {detection_rate:5.1f}% detected ({status})")Expected Results:
Burst length 1 bits: 100.0% detected (GUARANTEED)
Burst length 2 bits: 100.0% detected (GUARANTEED)
Burst length 5 bits: 100.0% detected (GUARANTEED)
Burst length 10 bits: 100.0% detected (GUARANTEED)
Burst length 16 bits: 100.0% detected (GUARANTEED)
Burst length 17 bits: 100.0% detected (GUARANTEED)
Burst length 20 bits: 100.0% detected (GUARANTEED)
Burst length 32 bits: 100.0% detected (GUARANTEED)What to Observe:
Exercise 3: Hardware vs Software CRC Performance
Measure the speed difference on ESP32:
// crc_performance_test.ino (ESP32)
#include "esp_rom_crc.h"
uint8_t test_data[1000];
void setup() {
Serial.begin(115200);
// Generate random test data
for (int i = 0; i < 1000; i++) {
test_data[i] = random(256);
}
}
void loop() {
// Software CRC-16 (table-driven)
unsigned long t1 = micros();
uint16_t crc_sw = crc16_ccitt_software(test_data, 1000);
unsigned long t2 = micros();
Serial.printf("Software CRC-16: 0x%04X in %lu µs\n", crc_sw, t2 - t1);
// Hardware CRC-16 (ESP32 ROM function)
unsigned long t3 = micros();
uint16_t crc_hw = esp_rom_crc16_le(0xFFFF, test_data, 1000);
unsigned long t4 = micros();
Serial.printf("Hardware CRC-16: 0x%04X in %lu µs\n", crc_hw, t4 - t3);
Serial.printf("Speedup: %.1fx\n\n", (float)(t2-t1)/(t4-t3));
delay(5000);
}
// Software implementation (for comparison)
uint16_t crc16_ccitt_software(const uint8_t* data, size_t len) {
uint16_t crc = 0xFFFF;
for (size_t i = 0; i < len; i++) {
crc ^= (data[i] << 8);
for (int j = 0; j < 8; j++) {
if (crc & 0x8000) {
crc = (crc << 1) ^ 0x1021;
} else {
crc <<= 1;
}
}
}
return crc;
}Expected Output (representative timing on ESP32 at 240 MHz):
Software CRC-16: 0xB3F2 in 320 µs
Hardware CRC-16: 0x6D4A in 4 µs
Speedup: 80.0xNote: the CRC values differ because esp_rom_crc16_le uses initial value 0 while the software implementation uses 0xFFFF. Both compute valid CRC-16-CCITT checksums – the key observation is the 80x speedup, not the specific values.
What to Observe: Hardware CRC is typically 50-100x faster. Always use hardware when available (check MCU datasheet for CRC peripheral).
CRC-16 has a 1-in-65,536 chance of failing to detect a random error — acceptable for small 64-byte packets but dangerous for 1 KB firmware images. CRC-32 reduces the false-pass probability to 1-in-4,294,967,296. For firmware integrity verification, OTA image validation, and anything larger than 256 bytes, use CRC-32 (polynomial 0xEDB88320). Never use simple 8-bit checksums (1-in-256 false-pass) for any security-relevant data integrity check.
A common CRC bug: computing the CRC over a buffer that includes the CRC field itself (initialized to 0), then writing the CRC into that field, then transmitting. The receiver computes the CRC over the data including the CRC field — if the CRC polynomial has the right properties this will produce a constant residue (0 or specific value), but many implementations get this wrong. Clearly define in the protocol specification: “CRC is computed over bytes N–M, not including the CRC field.”
Implementing CRC in firmware without testing it with deliberately corrupted data is incomplete. A CRC implementation that always returns 0 or always returns the same value will pass “happy path” tests (uncorrupted data → CRC matches) but fail silently when corruption occurs. Test with: single bit flip (byte XOR 1), byte insertion, byte deletion, and truncation. Verify CRC detection on at least 1,000 randomly corrupted messages.
CRC detects all single-bit errors, all 2-bit errors, and burst errors shorter than the CRC polynomial degree. However, CRC does NOT detect all errors: two specific error patterns that differ by a multiple of the generator polynomial have the same CRC (probability 1/2^n for n-bit CRC). For cryptographic data integrity (firmware, security keys, certificates), use HMAC-SHA256 or SHA-256 hash verification instead of CRC. CRC is for accidental corruption detection, not tamper detection.
| Topic | Chapter | Why Read It |
|---|---|---|
| Retry Mechanisms and Sequence Numbers | reliability-retry-sequencing.html | Implement ARQ and sequence numbering to recover from errors that your CRC or checksum has detected |
| Reliability Overview: All Five Pillars | reliability-error-handling.html | Assess the full reliability framework — detection is only one of five mechanisms needed for robust IoT transport |
| Connection State Lab | reliability-connection-lab.html | Construct a complete reliable transport layer combining all pillars, including error detection, retries, and flow control |
| DTLS and Security | dtls-and-security.html | Analyze how DTLS combines cryptographic integrity (HMAC) with error detection to protect IoT data from both corruption and tampering |
| Transport Fundamentals: TCP vs UDP | transport-fundamentals.html | Compare how TCP’s built-in error detection and retransmission differ from UDP’s minimal overhead approach for IoT workloads |
| Transport Protocol Selection | transport-protocols-selection.html | Apply decision frameworks to select the right transport protocol — and the right error detection strength — for specific IoT application classes |