49  Packet Structure and Framing

49.1 Learning Objectives

By the end of this chapter series, you will be able to:

  • Understand packet anatomy: Identify headers, payloads, and trailers in network packets
  • Explain framing techniques: Describe how receivers know where packets begin and end
  • Calculate error detection: Compute checksums and understand CRC principles
  • Read protocol specifications: Interpret frame format diagrams in datasheets
  • Parse packet headers: Extract fields from binary packet data
  • Design simple protocols: Create basic packet formats for IoT applications

Prerequisites (Read These First): - Data Representation - Binary, hexadecimal, and bitwise operations for parsing packet fields

Fundamentals (Core Concepts): - Data Formats for IoT - Payload encoding (JSON, CBOR, Protocol Buffers) - Sensor to Network Pipeline - How sensor data becomes network packets

Networking Foundations: - Networking Fundamentals - Network layers and packet flow - Layered Models - OSI and TCP/IP encapsulation - Transport Protocols - TCP/UDP packet headers

Protocol-Specific Packet Structures: - MQTT - MQTT fixed/variable headers, QoS, and packet types - CoAP - CoAP 4-byte header and message format - LoRaWAN - LoRa MAC header (MHDR), DevAddr, FCnt, MIC

49.2 Chapter Overview

This chapter series covers the fundamental concepts of packet structure, framing, error detection, and protocol overhead that underpin all network communication. Understanding these concepts is essential for designing efficient IoT systems, debugging network issues, and choosing appropriate protocols.

The content has been organized into four focused chapters:

49.3 Chapter Series

49.3.1 1. Packet Anatomy: Headers, Payloads, and Trailers

Read: Packet Anatomy ->

Learn the three fundamental parts of every network packet:

  • Headers: Routing, identification, and control information (10-40 bytes)
  • Payloads: The actual data being transmitted (variable size)
  • Trailers: Error detection values like checksums and CRC (2-4 bytes)

Includes a real-world LoRaWAN temperature sensor packet dissection and the postal letter analogy for beginners.

49.3.2 2. Frame Delimiters and Boundaries

Read: Frame Delimiters and Boundaries ->

Understand how receivers detect packet boundaries:

  • Length fields: Header specifies payload size (used by IP, TCP, UDP, MQTT)
  • Delimiters: Special marker bytes signal start/end (used by Ethernet, HDLC, PPP)
  • Byte stuffing: Escape sequences handle delimiter conflicts in payload data
  • MTU and fragmentation: How large payloads get split across network limits

Includes an interactive packet fragmentation demo showing 6LoWPAN, IPv4, IPv6, and LoRaWAN fragmentation behavior.

49.3.3 3. Error Detection: Checksums and CRC

Read: Error Detection ->

Compare methods for ensuring data integrity:

  • Simple checksums: Fast but weak - can miss transposed bytes
  • CRC-16/CRC-32: Polynomial-based detection catching 99.9999% of errors
  • When to use each: Trade-offs between computation and reliability
  • Safety-critical systems: Why industrial protocols use CRC-32

Includes scenario-based practice designing error detection for industrial sensors and debugging corrupted Zigbee packets.

49.3.4 4. Protocol Overhead and Encapsulation

Read: Protocol Overhead and Encapsulation ->

Analyze and optimize protocol efficiency:

  • Header comparison: BLE (57% overhead) vs MQTT/TCP/IPv6 (86% overhead)
  • Encapsulation: The “Russian doll” model of protocol layering
  • Cost analysis: How overhead impacts bandwidth costs and battery life
  • Protocol selection: Choosing the right stack for constrained IoT devices

Includes overhead calculations for LoRaWAN deployments and TCP vs UDP decision scenarios.

49.4 Quick Reference: Packet Structure Summary

Component Purpose Typical Size
Header Routing, control, identification 10-40 bytes
Payload Actual data (sensor readings, commands) Variable (0-MTU)
Trailer Error detection (checksum/CRC) 2-4 bytes
Framing Method When to Use Examples
Length field Known payload size IP, TCP, UDP, MQTT, CoAP
Delimiters Variable data, controlled content Ethernet, HDLC, Serial
Byte stuffing Arbitrary binary data PPP, SLIP, binary protocols
Error Detection Detection Rate Use Case
Checksum ~50% of errors Legacy, non-critical
CRC-16 99.998% General IoT
CRC-32 99.9999% Safety-critical

49.5 Start Learning

Begin with the fundamentals and progress through the series:

  1. Packet Anatomy - Start here for the basics
  2. Frame Delimiters - Learn boundary detection
  3. Error Detection - Understand data integrity
  4. Protocol Overhead - Master efficiency analysis

Begin with Packet Anatomy ->

49.6 What’s Next

After completing this chapter series, continue to Data Formats for IoT to learn how sensor data gets encoded within the packet payload using JSON, CBOR, MessagePack, and Protocol Buffers.