13  Packet Structure and Framing

In 60 Seconds

Every IoT message rides inside a packet with three core parts: a header for addressing and control, a payload for the actual sensor data, and often a trailer for integrity checks such as a checksum or CRC. Framing tells the receiver where each packet starts and ends, either with explicit lengths or special delimiter bytes. In constrained IoT links, these structural bytes matter: a 4-byte sensor value can become a 17-byte LoRaWAN frame or a 68-byte Ethernet/IP/TCP packet once all protocol overhead is added.

A network packet is like a postal envelope for digital data. It has a header (the address label telling where to deliver it), a payload (the actual letter inside), and a trailer (a seal to verify nothing was tampered with). Every piece of information sent between IoT devices – from a temperature reading to a command to turn on a light – travels inside these structured packets. Understanding their structure helps you appreciate how data moves reliably across networks.

Learning Objectives

By the end of this chapter series, you will be able to:

  • Dissect packet anatomy: Identify and label headers, payloads, and trailers in network packets across IoT protocols
  • Compare framing techniques: Differentiate between length-field and delimiter-based framing, and evaluate when each approach is appropriate
  • Compute error detection values: Calculate checksums, apply CRC polynomial division, and select the appropriate error detection method for a given reliability requirement
  • Interpret protocol specifications: Read frame format diagrams in datasheets and map byte offsets to field definitions
  • Extract packet header fields: Parse binary packet data to retrieve source/destination addresses, sequence numbers, and control flags
  • Evaluate protocol overhead: Quantify header-to-payload ratios for different IoT protocol stacks and justify protocol selection for constrained devices
  • Design custom packet formats: Construct a basic packet structure with appropriate framing, addressing, and error detection for a given IoT application
Key Concepts
  • Core Concept: Fundamental principle underlying Packet Structure and Framing — understanding this enables all downstream design decisions
  • Header: Routing and control bytes that identify source, destination, ports, flags, sequence numbers, or protocol type
  • Payload: The actual application data, such as a temperature reading, command, or telemetry sample
  • Trailer / FCS / MIC: Integrity bytes appended at the end of the frame to detect corruption or tampering
  • Length-Field Framing: A framing method where the header states how many bytes belong to the current packet
  • Delimiter Framing: A framing method using special start or end markers, often paired with byte stuffing to escape those markers in the payload
  • MTU: Maximum Transmission Unit — the largest frame or packet a link can carry without fragmentation
  • Protocol Overhead: Bytes spent on structure rather than user data; often the dominant cost on constrained IoT links
  • CRC: Cyclic Redundancy Check — the most common strong error-detection mechanism for IoT frames and packets

13.1 MVU: Minimum Viable Understanding

If you only have 5 minutes, here’s what you need to know about packet structure for IoT:

  1. Every packet has three parts: Header (addressing/control), Payload (your actual data), and Trailer (error checking)
  2. Framing tells receivers where packets start and end: Either by length fields (“next 50 bytes are mine”) or delimiter bytes (special markers like 0x7E)
  3. Error detection catches corrupted data: CRC is more reliable than checksums - use CRC-16 for most IoT, CRC-32 for safety-critical
  4. Protocol overhead matters for constrained devices: A 10-byte sensor reading can balloon to 60+ bytes after all headers are added

Bottom line: Understanding packet structure helps you choose efficient protocols for bandwidth-limited IoT networks, debug communication failures, and design custom protocols when needed.

Key Takeaway

In one sentence: Network packets wrap your sensor data in headers for routing, payloads for the actual information, and trailers for error detection - all bounded by framing mechanisms that tell receivers where each packet begins and ends.

Remember this rule: Smaller packets = less overhead per message, but more fragmentation for large data. Balance payload size against your protocol’s Maximum Transmission Unit (MTU).

Meet the Data Delivery Team!

Imagine you’re sending a present to your friend. You can’t just throw the toy in the mail - you need to wrap it up properly!

The Three Parts of Every Package (Packet):

  • Header = The address label on your package. It tells the postal service WHERE to deliver it and WHO it’s from
  • Payload = The actual present inside the box - the thing you really wanted to send!
  • Trailer = A seal or sticker that proves nobody opened your package. If the seal is broken, you know something went wrong!

How Does the Post Office Know Where One Package Ends?

  • Length field: Like writing “THIS BOX IS 30 CM LONG” on the package
  • Delimiters: Like putting a bright red ribbon at the start AND end of your present

What If Your Present Gets Damaged?

  • Checksum: Like counting all the jellybeans in a jar - if the count doesn’t match, some are missing!
  • CRC: A super-smart math trick that catches almost ANY damage, even if jellybeans get swapped around!

Fun Fact: When you watch a video online, your computer receives MILLIONS of tiny packets and puts them back together like a jigsaw puzzle!

Learning Resources:

  • Quizzes Hub - Test your packet structure knowledge with MCQs on framing, checksums, and protocol overhead
  • Simulations Hub - Experiment with packet fragmentation and reassembly in interactive demos
  • Videos Hub - Watch Wireshark tutorials dissecting real network packets
  • Knowledge Gaps Hub - Address common misconceptions about MTU, overhead, and error detection
  • Knowledge Map - See how packet structure connects to protocols like MQTT, CoAP, and LoRaWAN

Prerequisites (Read These First):

Fundamentals (Core Concepts):

Networking Foundations:

Protocol-Specific Packet Structures:

  • MQTT - MQTT fixed/variable headers, QoS, and packet types
  • CoAP - CoAP 4-byte header and message format
  • LoRaWAN - LoRa MAC header (MHDR), DevAddr, FCnt, MIC

13.2 Chapter Overview

This chapter series covers the fundamental concepts of packet structure, framing, error detection, and protocol overhead that underpin all network communication. Understanding these concepts is essential for designing efficient IoT systems, debugging network issues, and choosing appropriate protocols.

The content has been organized into four focused chapters:

Packet structure journey diagram showing how a raw sensor reading becomes a payload, is wrapped with header and trailer bytes into a packet, framed for transmission on a specific link, then interpreted by the receiver as packet anatomy, framing, error detection, and protocol overhead
Figure 13.1: Packet structure overview showing the journey from sensor data to network transmission

From Sensor Reading to Packet on the Wire

The same message moves from raw sensor value to payload, framing, integrity checks, and protocol-efficiency decisions.

Step 1

Raw Reading Becomes Payload

A sensor value such as temperature = 23.5 C may take only 2 to 10 bytes depending on the encoding.

Step 2

Packet Anatomy Adds Structure

Header + payload + trailer define what the bytes mean.

Header
Payload
Trailer
Step 3

Framing Defines Packet Boundaries

  • Length field: “read the next N bytes”
  • Delimiters: start/end markers plus escaping
Step 4

Integrity Checks Guard Against Errors

  • CRC-16 / CRC-32: polynomial error detection
  • Checksum: simple sum or XOR of bytes
Step 5

Protocol Overhead and Efficiency

  • Overhead ratio = (packet size - payload size) / packet size
  • Lower overhead means more application data per transmission
On a constrained IoT link, these structural bytes determine reliability, maximum payload size, battery cost, and how much of each transmission is real application data.

Figure 1: Packet structure overview showing the journey from sensor data to network transmission.


13.3 Chapter Series

13.3.1 Packet Anatomy: Headers, Payloads, and Trailers

Read: Packet Anatomy ->

Learn the three fundamental parts of every network packet:

  • Headers: Routing, identification, and control information (10-40 bytes)
  • Payloads: The actual data being transmitted (variable size)
  • Trailers: Error detection values like checksums and CRC (2-4 bytes)

Includes a real-world LoRaWAN temperature sensor packet dissection and the postal letter analogy for beginners.


13.3.2 Frame Delimiters and Boundaries

Read: Frame Delimiters and Boundaries ->

Understand how receivers detect packet boundaries:

  • Length fields: Header specifies payload size (used by IP, TCP, UDP, MQTT)
  • Delimiters: Special marker bytes signal start/end (used by Ethernet, HDLC, PPP)
  • Byte stuffing: Escape sequences handle delimiter conflicts in payload data
  • MTU and fragmentation: How large payloads get split across network limits

Includes an interactive packet fragmentation demo showing 6LoWPAN, IPv4, IPv6, and LoRaWAN fragmentation behavior.


13.3.3 Error Detection: Checksums and CRC

Read: Error Detection ->

Compare methods for ensuring data integrity:

  • Simple checksums: Fast but weak - can miss transposed bytes
  • CRC-16/CRC-32: Polynomial-based detection catching 99.9999% of errors
  • When to use each: Trade-offs between computation and reliability
  • Safety-critical systems: Why industrial protocols use CRC-32

Includes scenario-based practice designing error detection for industrial sensors and debugging corrupted Zigbee packets.


13.3.4 Protocol Overhead and Encapsulation

Read: Protocol Overhead and Encapsulation ->

Analyze and optimize protocol efficiency:

  • Header comparison: BLE (57% overhead) vs MQTT/TCP/IPv6 (86% overhead)
  • Encapsulation: The “Russian doll” model of protocol layering
  • Cost analysis: How overhead impacts bandwidth costs and battery life
  • Protocol selection: Choosing the right stack for constrained IoT devices

Includes overhead calculations for LoRaWAN deployments and TCP vs UDP decision scenarios.


13.4 Quick Reference: Packet Structure Summary

Packet Components

  • Header (10-40 bytes): Routing, control, and identification fields
  • Payload (0-MTU bytes): Actual sensor readings, commands, or telemetry
  • Trailer (2-4 bytes): Error detection such as a checksum, CRC, or MIC

Framing Methods

  • Length field: Best when payload size is known; used by IP, TCP, UDP, MQTT, and CoAP
  • Delimiters: Best for variable data or controlled content; used by Ethernet, HDLC, and serial links
  • Byte stuffing: Escapes delimiter bytes so arbitrary binary payloads can be transmitted safely

Error Detection Choices

  • Checksum (8-bit): Detects all single-bit errors and about 99.6% of random errors; use for legacy, non-critical traffic
  • CRC-16: Detects about 99.998% of errors; use for general IoT links
  • CRC-32: Detects about 99.9999% of errors; use for safety-critical systems
Check Your Understanding: Match Packet Concepts

Match each packet component or technique to its correct description:


13.5 Start Learning

Begin with the fundamentals and progress through the series:

  1. Packet Anatomy - Start here for the basics
  2. Frame Delimiters - Learn boundary detection
  3. Error Detection - Understand data integrity
  4. Protocol Overhead - Master efficiency analysis

Begin with Packet Anatomy ->


13.6 Packet Structure Visualization

The following diagram illustrates how protocol headers stack during encapsulation - the “Russian doll” model where each layer wraps the previous layer’s data:

Vertical encapsulation diagram showing a 10-byte application payload wrapped by a 20-byte TCP header into a 30-byte segment, wrapped by a 20-byte IPv4 header into a 50-byte packet, then wrapped by a 14-byte Ethernet header and 4-byte FCS into a 68-byte Ethernet frame
Figure 13.2: Protocol encapsulation showing how a 10-byte sensor reading becomes a 68-byte transmitted packet

Protocol Encapsulation of One Sensor Reading

The same 10-byte sensor reading is wrapped again and again as it moves down the stack.

Application Payload

10 bytes of real sensor data

This is the data the application actually wants to send.

TCP Segment

Adds a 20-byte TCP header

Port numbers, sequencing, and transport-layer control fields wrap the payload.

IPv4 Packet

Adds a 20-byte IP header

Source and destination IP addressing wrap the TCP segment.

Ethernet Frame

Adds a 14-byte Ethernet header + 4-byte FCS

The data-link layer prepares the packet for transmission on the wire and adds frame-level error detection.

Total on the wire: 68 bytes. A tiny 10-byte reading turns into 58 bytes of structure and only 10 bytes of actual application data.

Figure 2: Protocol encapsulation showing how a 10-byte sensor reading becomes a 68-byte transmitted packet.

Overhead Analysis: A 10-byte sensor reading transmitted over Ethernet/IP/TCP becomes 68 bytes on the wire: Ethernet header (14) + IP header (20) + TCP header (20) + Payload (10) + Ethernet FCS (4) = 68 bytes, representing 85% overhead (58 bytes of overhead out of 68 total). This is why IoT protocols like CoAP/UDP and LoRaWAN are preferred for constrained devices.

Why overhead matters even more for IoT: Unlike traditional networks with 1500-byte Ethernet MTUs, IoT networks face extreme constraints. LoRaWAN has a maximum payload of just 51 bytes at the lowest data rate (SF12), meaning every overhead byte directly displaces sensor data. NB-IoT and other cellular IoT networks charge per byte transmitted, so protocol overhead translates directly to operating cost. Additionally, radio transmission consumes significant power on battery-powered devices — larger packets mean longer transmit times and faster battery drain, though sleep current often dominates total energy consumption over the device lifetime.

Check Your Understanding: Encapsulation Order

When a sensor reading is transmitted over Ethernet/IP/TCP, the protocol layers add headers in a specific order. Arrange these steps from first (innermost) to last (outermost) in the encapsulation process:

Let’s calculate the real impact of protocol overhead on IoT deployments with concrete numbers.

Scenario: 1,000 temperature sensors sending 4-byte readings every 15 minutes over LoRaWAN.

Payload Analysis:

\[ \text{Data per message} = \begin{cases} \text{Sensor value:} & 4 \text{ bytes} \\ \text{LoRaWAN header:} & 13 \text{ bytes} \\ \text{Total transmitted:} & 17 \text{ bytes} \end{cases} \]

Overhead ratio: \(\frac{13}{17} = 76\%\) of every packet is overhead, not sensor data.

Annual transmission volume:

\[ \begin{aligned} \text{Messages per sensor per year} &= \frac{24 \times 365}{15/60} = 35,040 \text{ messages} \\ \text{Total network messages} &= 35,040 \times 1,000 = 35.04 \text{ million/year} \\ \text{Data transmitted} &= 35.04M \times 17 \text{ bytes} = 595.7 \text{ MB/year} \end{aligned} \]

Cost on cellular IoT (NB-IoT at $0.10/MB): \(595.7 \times \$0.10 = \$59.57\) per year for this overhead.

Energy impact (at 20mW transmit power, 250 kbps):

\[ \begin{aligned} \text{Transmit time per packet} &= \frac{17 \text{ bytes} \times 8}{250,000 \text{ bps}} = 0.544 \text{ ms} \\ \text{Energy per transmission} &= 20 \text{ mW} \times 0.544 \text{ ms} = 0.0109 \text{ mWs} \\ \text{Annual energy (per sensor)} &= 35,040 \times 0.0109 \text{ mWs} = 382 \text{ mWs} = 0.106 \text{ mWh} \end{aligned} \]

With a CR2032 battery (220 mAh at 3V = 660 mWh), transmission alone consumes 0.106/660 = 0.016% of battery per year - sensors could theoretically last 6,000+ years on transmission energy alone. This shows why sleep current (not transmission) dominates IoT battery budgets.

Key insight: For IoT, reducing overhead from 76% to 50% matters less for battery life than reducing sleep current from 10 µA to 1 µA.


13.7 Interactive Calculators

Protocol Overhead Calculator

Calculate the overhead percentage for different packet structures.

Example Configurations:

  • LoRaWAN: Header=13, Payload=4, Trailer=4 → 76% overhead
  • MQTT/TCP/IPv4: Header=50, Payload=20, Trailer=4 → 73% overhead
  • CoAP/UDP/IPv6: Header=48, Payload=20, Trailer=0 → 71% overhead
Transmission Time & Energy Calculator

Calculate how long it takes to transmit a packet and the energy consumed.

Annual Energy for 1 message/hour:

  • Messages per year: 8,760
  • Total energy: mWh
  • Battery remaining after 1 year: %
MTU Fragmentation Calculator

See how a payload gets fragmented across different network MTUs.


13.8 Knowledge Check

Test your understanding of packet structure fundamentals before diving into the detailed chapters.

Scenario: A LoRaWAN soil moisture sensor transmits data through a 3-hop network path:

  1. Sensor → LoRaWAN Gateway (LoRaWAN packet)
  2. Gateway → Cloud Broker (MQTT/TCP/IPv4 over cellular)
  3. Cloud → Time-Series Database (HTTP/REST API)

Question: For a 4-byte sensor reading, calculate the total bytes transmitted at each hop and cumulative overhead.

13.8.1 Step 1: Sensor → Gateway (LoRaWAN)

Packet structure:

Component Size Value/Purpose
MHDR 1 byte Message type/direction
DevAddr 4 bytes Device address
FCtrl 1 byte Frame control
FCnt 2 bytes Frame counter
FPort 1 byte Application port
Payload 4 bytes Sensor data: moisture reading
MIC 4 bytes Message integrity code

Total: 1 + 4 + 1 + 2 + 1 + 4 + 4 = 17 bytes on LoRaWAN link

Overhead: 13 bytes header/trailer ÷ 17 bytes total = 76% overhead

13.8.2 Step 2: Gateway → Cloud (MQTT/TCP/IPv4)

The gateway extracts the 4-byte payload and forwards it via MQTT:

Packet structure:

Component Size Value/Purpose
Ethernet header 14 bytes MAC addresses (on local network)
IPv4 header 20 bytes IP addresses, TTL, protocol
TCP header 20 bytes Ports, sequence, acknowledgment
MQTT fixed header 2 bytes Packet type, flags
MQTT variable header 8 bytes Topic name length + “sensors/1”
Payload 4 bytes Sensor data
Ethernet FCS 4 bytes Frame check sequence

Total: 14 + 20 + 20 + 2 + 8 + 4 + 4 = 72 bytes on cellular link

Overhead: 68 bytes ÷ 72 bytes = 94% overhead

13.8.3 Step 3: Cloud → Database (HTTP POST)

The cloud broker stores data in InfluxDB via HTTP API:

Packet structure (simplified HTTP POST):

POST /write?db=sensors HTTP/1.1
Host: influxdb.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 45

moisture,sensor_id=1 value=425 1702834567

Total for this HTTP request:

Component Size Notes
HTTP headers ~120 bytes Method, Host, Content-Type, etc.
Payload 45 bytes Line protocol: “moisture,sensor_id=1 value=425 timestamp”
TCP/IP headers 40 bytes IPv4 + TCP

Total: 120 + 45 + 40 = 205 bytes

Overhead: 160 bytes headers ÷ 205 total = 78% overhead

13.8.4 Step 4: Cumulative Analysis

Bytes transmitted across all hops:

Hop Link Bytes Transmitted Actual Data Overhead Ratio
1 Sensor → Gateway 17 bytes 4 bytes 76%
2 Gateway → Cloud 72 bytes 4 bytes 94%
3 Cloud → Database 205 bytes 4 bytes (45 bytes formatted) 78% (relative to formatted)
Total 294 bytes 4 bytes 98.6% cumulative overhead!

13.8.5 Step 5: Real-World Cost Impact

For 1,000 sensors, each transmitting once per hour:

Hop Annual Data @ $0.10/MB Cost
LoRaWAN (license-free) 149 MB $0 (no cost)
Cellular (gateway → cloud) 631 MB $63
Database writes (cloud → DB) 1,797 MB $0 (internal LAN)

Key insights:

  1. Multi-hop multiplies overhead: Same 4 bytes of data transmitted 3 times with different wrapping
  2. Cellular link is the cost bottleneck: 94% overhead on the paid link
  3. HTTP is inefficient: Line protocol adds 10x overhead vs binary (45 bytes vs 4 bytes)

13.8.6 Step 6: Optimization Strategies

Optimization 1: Batch sensor readings

Instead of sending 1 reading/hour, send 24 readings once/day:

  • Before: 72 bytes × 24 msgs = 1,728 bytes/day
  • After: (68 bytes header + 96 bytes payload) = 164 bytes/day
  • Savings: 90% reduction!

Optimization 2: Use binary database protocol

Instead of HTTP + line protocol, use InfluxDB’s native binary protocol:

  • Before: 205 bytes per write
  • After: ~50 bytes (binary encoding)
  • Savings: 76% reduction

Optimization 3: Compress payloads

Use CBOR instead of MQTT JSON/text payload:

  • Before: MQTT topic “sensors/1” + JSON {"moisture":425} = 8 + 18 = 26 bytes
  • After: CBOR binary encoding = 8 bytes
  • Savings: 69% reduction

13.8.7 Final Optimized Architecture

With batching + binary protocols:

Hop Before After Savings
Sensor → Gateway 17 bytes/hour 17 bytes/day (batched) 96%
Gateway → Cloud 72 bytes/hour 80 bytes/day (batched + binary) 96%
Cloud → Database 205 bytes/hour 60 bytes/day (binary protocol) 97%
Annual cellular cost (1,000 sensors) $63/year $7/year 89% savings

Key takeaway: Understanding packet structure at every hop reveals optimization opportunities. Multi-hop overhead compounds, making batching and binary formats essential for constrained IoT.


Common Pitfalls

Relying on theoretical models without profiling actual behavior leads to designs that miss performance targets by 2-10×. Always measure the dominant bottleneck in your specific deployment environment — hardware variability, interference, and load patterns routinely differ from textbook assumptions.

Optimizing one parameter in isolation (latency, throughput, energy) without considering impact on others creates systems that excel on benchmarks but fail in production. Document the top three trade-offs before finalizing any design decision and verify with realistic workloads.

Most field failures come from edge cases that work in the lab: intermittent connectivity, partial node failure, clock drift, and buffer overflow under peak load. Explicitly design and test failure handling before deployment — retrofitting error recovery after deployment costs 5-10× more than building it in.

13.9 Summary

This chapter introduced the fundamental concepts of packet structure and framing that underlie all network communication:

Packet Anatomy

Every packet combines a header for routing and control, a payload for the actual data, and a trailer for integrity checking.

Framing

Length fields or delimiters mark packet boundaries, while byte stuffing prevents delimiter conflicts inside binary payloads.

Error Detection

Checksums are fast but weak; CRC-16 and CRC-32 provide much stronger corruption detection when reliability matters.

Protocol Overhead

Headers can dominate small payloads, so protocol choice should balance efficiency against the features the application actually needs.

Key Design Decisions for IoT:

  1. Choose error detection based on consequence of corruption: CRC-32 for safety-critical, CRC-16 for general IoT, checksums only for non-critical legacy systems
  2. Minimize overhead for constrained devices: CoAP/UDP instead of HTTP/TCP, header compression (6LoWPAN), payload aggregation
  3. Match MTU to network constraints: Understand fragmentation costs on each link (LoRaWAN: 51-222 bytes, 6LoWPAN: 127 bytes, Ethernet: 1500 bytes)

13.10 What’s Next

Now that you understand packet structure, framing, and error detection, explore these related topics:

Data Formats for IoT How sensor data gets encoded inside the packet payload using JSON, CBOR, MessagePack, and Protocol Buffers. Sensor to Network Pipeline The full journey of a sensor reading from analog signal to transmitted network packet. Networking Fundamentals How packets flow through network layers, switching, and routing infrastructure. Transport Protocols TCP vs UDP trade-offs for IoT, congestion control, and connection management. MQTT MQTT packet types, QoS levels, and how the fixed and variable header structure works in practice. LoRaWAN Overview LoRaWAN MAC layer packet structure, device classes, and adaptive data rate.