41  App Protocols Overview

In 60 Seconds

CoAP is a RESTful protocol running over UDP designed for direct device queries on constrained networks, while MQTT is a publish-subscribe protocol running over TCP optimized for event-driven telemetry through a central broker. Modern HTTP/2 (multiplexing, header compression) and HTTP/3 (QUIC/UDP transport, 0-RTT) make HTTP viable for IoT gateways but not for battery-powered sensors.

41.1 Learning Objectives

By the end of this chapter, you will be able to:

  • Explain CoAP Architecture: Describe CoAP’s RESTful design, UDP transport, and resource discovery mechanism
  • Explain MQTT Architecture: Describe MQTT’s publish-subscribe pattern, QoS levels, and broker role
  • Compare Protocol Trade-offs: Analyze and distinguish differences in transport, reliability, security, and resource usage across CoAP, MQTT, and HTTP
  • Select a Protocol for Scenarios: Evaluate and justify the choice of HTTP, MQTT, or CoAP based on device constraints and application requirements
  • Assess HTTP/2 and HTTP/3: Analyze modern HTTP improvements and calculate their impact on IoT gateway bandwidth and latency

This overview introduces the main application protocols used in IoT: MQTT for publish-subscribe messaging, CoAP for lightweight web requests, AMQP for reliable message queuing, and HTTP for web-based communication. Each protocol has its strengths, and understanding them helps you pick the right one for your project.

“Let’s line up the whole protocol family!” said Max the Microcontroller. “MQTT is the gossip network – publish once, everyone who cares gets the update. CoAP is the quick question-and-answer – ‘Hey Sammy, what’s the temperature?’ ‘It’s 23 degrees.’ Done.”

“AMQP is the enterprise mail room,” continued Sammy the Sensor. “It has sorting rules, guaranteed delivery, and can route one message to multiple departments based on complex criteria. Overkill for a garden sensor, but perfect for a factory with hundreds of machines.”

“And HTTP is the old reliable,” said Lila the LED. “Every device, every browser, every cloud service speaks HTTP. It’s heavy, but it’s universal. When you need to talk to a regular web server, HTTP is your only choice.”

Bella the Battery wrapped it up: “The overview lesson is simple – no single protocol is best for everything. A smart IoT system often uses two or three protocols. MQTT at the sensor edge, AMQP in the middleware, and HTTP for the user dashboard. Knowing all four gives you the power to choose wisely!”

41.2 Prerequisites

Before diving into this chapter, you should be familiar with:

Key Concepts

  • Protocol Zoo: The diverse landscape of IoT application protocols (MQTT, CoAP, AMQP, DDS, XMPP, HTTP) each with different tradeoffs in overhead, reliability, and feature richness.
  • Interoperability: The ability of devices using different protocols to exchange data, often requiring gateways or translators (MQTT-to-HTTP bridges, CoAP-to-MQTT converters).
  • Protocol Stack Selection: Choosing the appropriate combination of transport (TCP/UDP/BLE) and application protocol (MQTT/CoAP/HTTP) based on device constraints, network characteristics, and application requirements.
  • Payload Format: The data encoding used within application protocol messages: JSON (readable, large), CBOR (compact binary), Protocol Buffers (schema-based), MessagePack.
  • TLS/DTLS Security: Transport security layer wrapping MQTT (uses TLS over TCP) and CoAP (uses DTLS over UDP) to provide encryption and authentication.

41.3 How This Chapter Fits

Chapter Series Navigation:

  1. Introduction and Why Lightweight Protocols Matter
  2. Protocol Overview and Comparison (this chapter)
  3. REST API Design for IoT
  4. Real-time Protocols
  5. Worked Examples

This chapter provides the technical foundation for understanding CoAP and MQTT protocol architectures, enabling informed protocol selection decisions.

41.4 Protocol Overview

41.4.1 CoAP: RESTful IoT Communication

CoAP Quick Summary

Pattern: Request-Response (like HTTP) Transport: UDP (lightweight, connectionless) Model: One-to-One Best For: Direct device queries, resource-constrained networks Header Size: 4 bytes minimum

Key Features:

  • RESTful architecture (GET, POST, PUT, DELETE)
  • UDP-based for minimal overhead
  • Built-in discovery mechanism
  • DTLS security
  • Designed for IEEE 802.15.4 networks

Typical Use Cases:

  • Smart home device control
  • Sensor data retrieval
  • Building automation
  • Direct machine-to-machine queries

Learn more about CoAP →

41.4.2 MQTT: Publish-Subscribe Messaging

MQTT Quick Summary

Pattern: Publish-Subscribe (broker-based) Transport: TCP (reliable, connection-oriented) Model: Many-to-Many Best For: Event-driven systems, telemetry, multiple subscribers Header Size: 2 bytes minimum

Key Features:

  • Publish-subscribe pattern with broker
  • TCP-based for reliability
  • Three QoS levels (0, 1, 2)
  • Last Will and Testament
  • Retained messages

Typical Use Cases:

  • Sensor data collection
  • Remote monitoring dashboards
  • Alert and notification systems
  • Industrial telemetry

Explore how MQTT’s publish-subscribe pattern works with multiple publishers and subscribers communicating through a central broker.

Learn more about MQTT →

41.5 Videos

Application Protocols Overview (from slides)
Application Protocols Overview
Lesson 4 overview — MQTT, CoAP, HTTP, and AMQP positioning and trade-offs.
MQTT vs CoAP Comparison
MQTT vs CoAP Comparison
From slides — request/response vs publish/subscribe and REST implications.

41.6 CoAP vs MQTT: Detailed Comparison

Choosing between CoAP and MQTT is not always straightforward. Both protocols excel in different scenarios. Here’s a comprehensive comparison:

41.6.1 Architecture and Communication Model

Aspect CoAP MQTT
Pattern Request-Response (client-server) Publish-Subscribe (broker-based)
Communication One-to-One Many-to-Many
Coupling Tight (direct connection) Loose (decoupled via broker)
Discovery Built-in resource discovery Topic-based addressing

Table: Detailed CoAP vs MQTT Comparison

FACTOR CoAP MQTT
Main transport protocol UDP TCP
Typical messaging Request/response Publish/subscribe
Effectiveness in LLNs Excellent Low/fair (Implementations pairing UDP with MQTT are better for LLNs.)
Security DTLS SSL/TLS
Communication model One-to-one Many-to-many
Strengths Lightweight and fast, with low overhead, and suitable for constrained networks; uses a RESTful model that is easy to code to; easy to parse and process for constrained devices; support for multicasting; asynchronous and synchronous messages. TCP and multiple QoS options provide robust communications; simple management and scalability using a broker architecture.
Weaknesses Not as reliable as TCP-based MQTT, so the application must ensure reliability. Higher overhead for constrained devices and networks; TCP connections can drain low-power devices; no multicasting support.
Figure 41.1

Comparison table showing CoAP versus MQTT across dimensions including transport protocol (UDP vs TCP), messaging pattern (request-response vs publish-subscribe), effectiveness in LLNs, security mechanisms (DTLS vs SSL/TLS), and relative strengths and weaknesses for IoT applications

CoAP vs MQTT Protocol Comparison
Figure 41.2: Visual comparison of CoAP and MQTT protocol characteristics

41.6.2 Network and Transport

Aspect CoAP MQTT
Transport UDP (User Datagram Protocol) TCP (Transmission Control Protocol)
Reliability Optional confirmable messages TCP ensures delivery
Overhead Very low (4-byte header) Low (2-byte header + TCP)
Connection Connectionless Persistent connection
Latency Lower (no handshake) Slightly higher (TCP handshake)

41.6.3 Quality of Service and Reliability

Aspect CoAP MQTT
Message Types CON (confirmable), NON (non-confirmable) QoS 0, 1, 2
Acknowledgment Optional ACK messages Depends on QoS level
Retransmission Application handles it QoS 1&2 handle automatically
Duplicate Detection Message IDs Packet identifiers

Experiment with MQTT Quality of Service levels to understand the trade-offs between reliability, latency, and overhead.

41.6.4 Security

Aspect CoAP MQTT
Security Layer DTLS (Datagram TLS) TLS/SSL
Authentication Pre-shared keys, certificates Username/password, certificates
Encryption Yes (with DTLS) Yes (with TLS)
Overhead DTLS adds overhead to UDP TLS integrated with TCP

41.6.5 Resource Usage

Aspect CoAP MQTT
Memory Footprint Minimal Small
Power Consumption Very low (UDP, sleep modes) Low (keep-alive messages)
Battery Life Excellent Very good
Bandwidth Very efficient Efficient

41.6.6 Design Philosophy

Aspect CoAP MQTT
Inspired By HTTP (RESTful) MQTT for SCADA
Paradigm Resource-oriented Message-oriented
Standards Body IETF (RFC 7252) OASIS (ISO/IEC 20922)
Target Constrained nodes Telemetry and messaging

41.6.7 Interactive: Protocol Overhead Calculator

Tradeoff: HTTP vs MQTT vs CoAP

Decision context: When selecting an application protocol for IoT communication

Factor HTTP MQTT CoAP
Battery impact High (connection overhead) Medium (persistent TCP) Low (connectionless UDP)
Bandwidth High (verbose headers) Low (2-byte header) Very low (4-byte header)
Latency Medium-High (TCP + headers) Low (persistent connection) Lowest (UDP, no handshake)
Reliability TCP guaranteed QoS 0/1/2 options Optional CON/NON
Complexity Simple (universal) Moderate (broker required) Low (direct communication)
Pattern Request-Response Publish-Subscribe Request-Response + Observe
Ecosystem Universal Strong IoT Growing

Choose HTTP when:

  • Integrating with existing web infrastructure and REST APIs
  • Building mobile/web apps that communicate with gateways or cloud
  • Debugging and development simplicity is priority
  • Bandwidth and battery constraints are not critical (Wi-Fi/Ethernet devices)

Choose MQTT when:

  • Many devices need to publish to or subscribe from a central system
  • Event-driven architecture with multiple consumers per message
  • Devices have intermittent connectivity (store-and-forward via broker)
  • Smart home, telemetry dashboards, industrial monitoring

Choose CoAP when:

  • Constrained devices with limited RAM/flash (8-bit MCUs)
  • Battery-powered sensors on LPWAN (6LoWPAN, Thread)
  • Direct device-to-device or device-to-gateway communication
  • RESTful semantics needed on constrained networks

Default recommendation: MQTT for cloud/broker-based IoT systems, CoAP for constrained edge networks, HTTP only when interfacing with web systems or during prototyping

While HTTP/1.1 is often dismissed as too heavyweight for IoT, the newer HTTP/2 and HTTP/3 protocols offer significant improvements that make HTTP more viable for certain IoT applications. This deep dive explores when and how to leverage these modern HTTP versions.

41.6.8 HTTP/2: Multiplexing and Header Compression

Key improvements over HTTP/1.1:

Feature HTTP/1.1 HTTP/2 IoT Benefit
Connections Multiple TCP connections needed Single connection, multiplexed streams Reduced handshake overhead
Headers Repeated in full each request HPACK compression (90%+ reduction) Smaller packet sizes
Server Push Not supported Server can proactively send resources Efficient firmware distribution
Binary Protocol Text-based (verbose) Binary framing Faster parsing on MCUs
Stream Priority None Priority hints for streams Critical data first

Header compression example:

HTTP/1.1 headers (repeated every request):
Content-Type: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
User-Agent: IoT-Sensor/1.0
Accept: application/json
Host: api.example.com
(~350 bytes per request)

HTTP/2 with HPACK:
First request: ~350 bytes (headers indexed)
Subsequent requests: ~15-30 bytes (index references only)
Savings: 90%+ for repeated headers

HTTP/2 header compression ratio can be quantified using HPACK’s static and dynamic table compression.

For a sensor gateway making 1,000 requests/day with typical headers:

\[\text{Compression Ratio} = \frac{\text{Uncompressed Size}}{\text{Compressed Size}}\]

First request (cold start, no dynamic table): \[CR_1 = \frac{350 \text{ bytes}}{280 \text{ bytes}} = 1.25\]

Subsequent requests (warm cache, headers indexed): \[CR_n = \frac{350 \text{ bytes}}{25 \text{ bytes}} = 14\]

Daily bandwidth savings over HTTP/1.1: \[\text{Savings} = (999 \times 350) - (350 + 999 \times 25) = 349,650 - 25,325 = 324,325 \text{ bytes}\]

For 1,000 sensors, that’s 324 MB/day saved, or ~9.7 GB/month. At $0.10/GB cellular data rates, HTTP/2 saves $1/month per 1,000 sensors compared to HTTP/1.1 — modest but meaningful for large fleets.

Multiplexing benefit for IoT gateways:

Scenario: Gateway aggregating 50 sensors, sending to cloud

HTTP/1.1 approach:
- 50 sequential requests OR 6-8 parallel connections
- Each connection: TCP handshake (1.5 RTT) + TLS (2 RTT)
- Total overhead: 50 × 200ms = 10 seconds

HTTP/2 approach:
- Single TCP+TLS connection (1 RTT handshake, amortized)
- 50 multiplexed streams in parallel
- Total time: ~200ms (1 RTT + minimal framing)
- 50× faster for gateway scenarios

41.6.9 HTTP/3: QUIC Transport for Unreliable Networks

HTTP/3 replaces TCP with QUIC (UDP-based), offering advantages for IoT deployments with unstable connectivity:

Key features for IoT:

Feature HTTP/2 (TCP) HTTP/3 (QUIC) IoT Use Case
Connection establishment 3-way handshake + TLS 0-RTT or 1-RTT Mobile/cellular IoT
Head-of-line blocking Stream blocked by packet loss Independent streams Lossy wireless links
Connection migration Breaks on IP change Survives network switch Vehicle telematics
Congestion control Per-connection Per-stream Mixed priority data

0-RTT connection resumption:

Scenario: Cellular IoT device waking from sleep

TCP + TLS 1.3:
1. TCP SYN                           (1 RTT)
2. TCP SYN-ACK + TLS ClientHello
3. TLS ServerHello + Application data (1 RTT)
Total: 2 RTT before sending data (~200-400ms on cellular)

QUIC (HTTP/3) with 0-RTT:
1. ClientHello + Encrypted data       (0 RTT!)
2. Server response
Total: 0 RTT for resumed connections
Power savings: 30-50% reduction in radio active time

Handling packet loss gracefully:

HTTP/2 (TCP): Sensor telemetry + firmware chunk in same connection
- Firmware packet lost → ALL streams blocked waiting for retransmit
- Telemetry delayed by firmware recovery (100-500ms)

HTTP/3 (QUIC): Independent stream handling
- Firmware packet lost → only firmware stream waits
- Telemetry continues unaffected
- Critical for real-time + bulk data mixing

41.6.10 When to Use HTTP/2 or HTTP/3 for IoT

Choose HTTP/2 when:

  • IoT gateway aggregating many device messages to cloud
  • Existing HTTP/REST infrastructure must be preserved
  • Devices have sufficient memory (32KB+ RAM for TLS+HTTP/2 stack)
  • Connection reuse is possible (persistent connection to backend)
  • Example: Smart building gateway sending HVAC data to cloud API

Choose HTTP/3 when:

  • Cellular IoT with frequent sleep/wake cycles
  • Mobile assets changing networks (vehicles, drones, wearables)
  • Lossy wireless links (satellite, rural cellular)
  • Mixing real-time telemetry with bulk transfers
  • Example: Fleet tracking devices on LTE/5G

Avoid HTTP/2/3 when:

  • Extremely constrained devices (<16KB RAM)
  • Simple request-response suffices (use CoAP instead)
  • UDP is blocked by network (enterprise firewalls)
  • Battery sensors with infrequent transmissions (MQTT over TCP simpler)

41.6.11 Implementation Considerations

Library support (as of 2025):

Platform HTTP/2 HTTP/3 Notes
ESP32 (ESP-IDF) Partial (nghttp2) Limited Memory-constrained
Linux/Raspberry Pi Full (libcurl, nghttp2) Full (quiche, ngtcp2) Recommended platform
Azure IoT SDK Yes Preview Cloud-native support
AWS IoT Yes (MQTT over WebSocket/HTTP/2) Roadmap Prefer MQTT
Nordic nRF9160 Limited No Focus on LwM2M/CoAP

Memory requirements:

HTTP/1.1 minimal: ~8KB RAM (no TLS)
HTTP/2 minimal: ~32KB RAM (HPACK tables + stream state)
HTTP/3 minimal: ~64KB RAM (QUIC connection + crypto state)

Recommendation:
- <32KB RAM: Use CoAP or MQTT over TCP
- 32-128KB: HTTP/2 feasible for gateway scenarios
- >128KB: HTTP/3 viable for advanced applications

Configuration best practices:

# nginx HTTP/2 config for IoT backend
http2_max_concurrent_streams 128;  # Many IoT clients
http2_recv_buffer_size 256k;       # Handle burst uploads
keepalive_timeout 3600s;           # Long-lived IoT connections
ssl_protocols TLSv1.3;             # Require TLS 1.3 for IoT security

# QUIC/HTTP/3 (nginx 1.25+)
listen 443 quic reuseport;
http3 on;
quic_retry on;                     # Mitigate amplification attacks

Performance tuning for IoT:

# Python example: HTTP/2 with connection reuse for gateway
import httpx

# Create reusable HTTP/2 client
client = httpx.Client(
    http2=True,
    timeout=30.0,
    limits=httpx.Limits(
        max_keepalive_connections=5,
        max_connections=10,
        keepalive_expiry=3600  # 1 hour for IoT
    )
)

# Batch sensor readings efficiently
async def upload_sensor_batch(readings: list[dict]):
    """Upload multiple sensor readings in parallel over single HTTP/2 connection"""
    tasks = [
        client.post(f"/api/v1/sensors/{r['device_id']}/data", json=r)
        for r in readings
    ]
    responses = await asyncio.gather(*tasks)
    # All 50 requests share single connection, multiplexed
    return responses

41.6.12 Summary: HTTP Protocol Selection for IoT

Protocol Best For Overhead Latency Reliability
CoAP Constrained devices, 6LoWPAN Very Low Lowest Application-managed
MQTT Event streams, pub-sub Low Low QoS 0/1/2
HTTP/1.1 Simple prototyping, debugging High Medium TCP
HTTP/2 Gateways, cloud APIs, bulk uploads Medium Medium TCP + multiplexing
HTTP/3 Mobile IoT, unreliable networks Medium Lowest QUIC streams

Key Takeaway: HTTP/2 and HTTP/3 don’t replace MQTT or CoAP for constrained IoT devices, but they significantly improve HTTP performance for gateways, cloud integration, and mobile IoT scenarios where HTTP infrastructure already exists.

41.7 Knowledge Check

Test your understanding of these networking concepts.

:

41.8 Summary

This chapter provided a comprehensive technical comparison of IoT application protocols:

Key topics covered:

  • CoAP characteristics: RESTful over UDP, 4-byte headers, resource discovery, DTLS security
  • MQTT characteristics: Publish-subscribe over TCP, 2-byte headers, QoS levels 0/1/2, broker architecture
  • Protocol comparison matrices: Architecture, transport, reliability, security, and resource usage trade-offs
  • HTTP/2 and HTTP/3: Modern HTTP improvements including multiplexing, header compression, QUIC transport
  • Selection criteria: When to use HTTP vs MQTT vs CoAP based on device constraints and communication patterns

41.9 What’s Next?

Now that you can compare and evaluate IoT application protocols, continue with the chapters below to apply this knowledge to real design scenarios:

Chapter Focus Why Read It
REST API Design for IoT Designing RESTful APIs for constrained devices and cloud backends Apply CoAP and HTTP knowledge to build well-structured APIs with versioning, pagination, and authentication
Real-time Protocols VoIP, SIP, and RTP for audio and video IoT applications Extend protocol selection skills to media streaming scenarios where MQTT and CoAP are insufficient
Worked Examples End-to-end protocol selection walkthroughs for real deployments Construct complete protocol stacks for smart home, industrial, and mobile IoT use cases
MQTT Fundamentals Deep dive into MQTT broker configuration, topic design, and QoS Implement MQTT for production systems with retained messages, Last Will, and scalable topic hierarchies
CoAP Fundamentals and Architecture CoAP message types, resource discovery, and Observe extension Configure CoAP on constrained devices and diagnose message reliability issues
Protocol Integration Patterns Gateway translation, bridging CoAP to MQTT, and hybrid stacks Design multi-protocol IoT architectures and justify bridging decisions across network tiers