33  MQTT: Comprehensive Review

In 60 Seconds

This comprehensive MQTT review consolidates the entire protocol: the publish-subscribe architecture with central broker, hierarchical topic design with wildcards, three QoS levels trading reliability for overhead, persistent sessions for offline message queuing, retained messages and Last Will Testament for state management, and production concerns including clustering, TLS security, and performance tuning.

This review chapter consolidates everything you’ve learned about MQTT. Here’s a quick mental model to anchor your understanding:

MQTT in One Sentence: A lightweight “postal service” for IoT where devices send messages to a central broker that delivers them to all interested subscribers.

The Five Core Concepts:

Concept What It Does Analogy
Broker Central server routing all messages Post office
Publish Send a message to a topic Mailing a letter
Subscribe Register interest in topics Setting up mail forwarding
Topic Message category (hierarchical) Address/mailbox name
QoS Delivery guarantee level Registered mail vs regular

When to Use Each QoS Level:

  • QoS 0 (At most once): Temperature readings every 5 seconds - missing one is fine
  • QoS 1 (At least once): Door sensor alerts - must be delivered, duplicates OK
  • QoS 2 (Exactly once): Payment confirmations - no duplicates, no losses

Common Gotcha for Beginners: Wildcards (+ and #) only work in SUBSCRIBE, not PUBLISH. You can subscribe to home/+/temperature but must publish to home/living_room/temperature.

Quick Troubleshooting:

Problem Likely Cause Solution
Messages not arriving QoS 0 + network issues Use QoS 1
New subscriber misses last value No retained message Publish with retain=true
Commands lost while offline Clean session Use persistent session (clean_session=false)
Connection drops silently Keepalive timeout Reduce keepalive interval

33.1 Overview

This comprehensive MQTT review has been organized into focused chapters for easier learning:

Chapter Focus Time
MQTT Architecture Patterns Pub/sub architecture, topics, wildcards 15 min
MQTT QoS and Reliability QoS levels, sessions, retained messages, LWT 15 min
MQTT Production Deployment Clustering, security, performance, pitfalls 15 min
MQTT Knowledge Check 10 MCQs + 4 application scenarios 20 min

Total Estimated Time: 65 minutes

33.2 Learning Objectives

By completing all review chapters, you will be able to:

  • Implement MQTT Clients: Configure publishers and subscribers with appropriate QoS levels, clean/persistent sessions, and TLS authentication for a given IoT scenario
  • Design Topic Hierarchies: Construct hierarchical MQTT topic schemas using wildcard patterns (+, #) that minimize subscription count and support scalable multi-device deployments
  • Analyze QoS Trade-offs: Calculate the message overhead, battery penalty, and broker CPU cost for QoS 0, 1, and 2 in a concrete sensor deployment scenario
  • Evaluate Protocol Selection: Compare MQTT against CoAP, HTTP, and AMQP and justify the choice of protocol based on quantifiable criteria (latency, overhead, delivery semantics)
  • Diagnose Production Issues: Identify and resolve common MQTT failure modes including session state inconsistency, retained message staleness, and broker throughput bottlenecks
  • Assess Scalability Designs: Distinguish between single-broker, clustered, and bridged MQTT architectures and select the appropriate topology for a given concurrency and availability requirement

Key Concepts

  • MQTT: Message Queuing Telemetry Transport — pub/sub protocol optimized for constrained IoT devices over unreliable networks
  • Broker: Central server routing messages from publishers to all matching subscribers by topic pattern
  • Topic: Hierarchical string (e.g., home/bedroom/temperature) used to route messages to interested subscribers
  • QoS Level: Quality of Service 0/1/2 trading delivery guarantee for message overhead
  • Retained Message: Last message on a topic stored by broker for immediate delivery to new subscribers
  • Last Will and Testament: Pre-configured message published by broker when a client disconnects ungracefully
  • Persistent Session: Broker stores subscriptions and pending messages allowing clients to resume after disconnection

33.3 Prerequisites

Required Chapters:

Recommended Reading:

Technical Background:

  • TCP/IP networking basics
  • Client-server vs pub/sub patterns
  • Basic security concepts (TLS)
Cross-Hub Connections

This comprehensive review integrates knowledge across multiple learning resources:

Learning Hubs:

  • Knowledge Map - Visualize MQTT’s role in the IoT protocol ecosystem
  • Quizzes Hub - Test your MQTT mastery with interactive assessments
  • Simulations Hub - Experiment with MQTT broker behavior in a safe environment
  • Videos Hub - Watch MQTT protocol demonstrations and use cases

Practice Resources:

Related Concepts:

33.4 Chapter Navigation

33.4.1 1. Architecture Patterns

MQTT Architecture Patterns covers:

  • Publish/subscribe architecture with central broker
  • Topic hierarchies and naming best practices
  • Wildcard subscription rules (+ and #)
  • Worked example: Smart building topic design

33.4.2 2. QoS and Reliability

MQTT QoS and Reliability covers:

  • QoS level selection (0, 1, 2) with trade-offs
  • Session management (clean vs persistent)
  • Retained messages for last-known values
  • Last Will and Testament for failure notification
  • Keepalive configuration

33.4.3 3. Production Deployment

MQTT Production Deployment covers:

  • Broker clustering architecture
  • TLS security and authentication
  • Performance troubleshooting
  • Common pitfalls to avoid
  • CoAP-MQTT protocol bridging

33.4.4 4. Knowledge Check

MQTT Knowledge Check covers:

  • 10 scenario-based multiple choice questions
  • 4 real-world application scenarios
  • Protocol selection guidance

Before proceeding to the calculators, verify your grasp of the four key MQTT features reviewed above.

33.5 Knowledge Check

33.6 Summary

This MQTT comprehensive review synthesizes knowledge across fundamentals, QoS, security, and implementation:

  • Protocol Selection Criteria: MQTT excels for publish-subscribe event-driven systems with many-to-many communication, while CoAP suits request-response patterns and HTTP works for traditional client-server interactions
  • QoS Level Trade-offs: QoS 0 provides maximum efficiency (2x fewer packets than QoS 1, 4x fewer packets than QoS 2) for replaceable sensor data, QoS 1 balances reliability and performance for important events, and QoS 2 guarantees exactly-once delivery for critical commands despite highest overhead (4-way handshake)
  • Wildcard Subscriptions: Single-level (+) and multi-level (#) wildcards enable efficient topic patterns, reducing subscription count from 50 individual topics to a single pattern like home/+/temperature
  • Session Management: Clean sessions (clean_session=1) suit pure publishers that don’t need offline queuing, while persistent sessions (clean_session=0) with fixed client IDs enable message queuing for devices receiving commands during sleep
  • Retained Messages: Last published value delivered immediately to new subscribers, essential for device status and slow-changing state data
  • Production Security: MQTT over TLS (port 8883) with username/password authentication, client certificates for mutual authentication, and topic-level ACLs prevent unauthorized access and eavesdropping
  • Scalability Challenges: Broker clustering and load balancing address throughput bottlenecks, with modern brokers handling 100K-1M concurrent connections when properly optimized with QoS selection and message batching

33.7 Interactive Calculators

33.7.1 MQTT QoS Energy Impact Calculator

Estimate the battery impact of different QoS levels for your sensor deployment.

33.7.2 MQTT Broker Throughput Calculator

Estimate broker resource requirements for your MQTT deployment.

33.7.3 MQTT Message Overhead Comparison

Compare the per-message byte overhead across QoS levels and protocol options.

33.7.4 MQTT Topic Wildcard Savings Calculator

Estimate how wildcard subscriptions reduce the number of individual topic subscriptions needed.

Use this framework to select the appropriate QoS level for different message types in your MQTT deployment:

Message Type Recommended QoS Rationale Performance Impact
Sensor telemetry (frequent) QoS 0 Next reading supersedes previous; 1-5% loss acceptable Lowest latency, minimal broker load
Alert notifications QoS 1 Must deliver at least once; duplicates handled by application Moderate overhead, good reliability
Actuator commands QoS 1 or 2 Command execution critical; QoS 2 if duplicates cause harm Higher latency, guaranteed delivery
Configuration updates QoS 2 Exactly-once ensures consistent state; no duplicate config application Highest overhead, strong guarantee
Heartbeat/keepalive QoS 0 Periodic signal; single miss won’t trigger alarm Minimal overhead
Device status (retained) QoS 1 + retain Last status must reach new subscribers reliably Stored on broker, moderate cost
Financial transactions QoS 2 No duplicate payments; exactly-once essential High latency acceptable for correctness

Performance Comparison:

QoS Level Messages/Transaction Broker CPU Latency When Packet Loss = 5%
QoS 0 1 (fire-and-forget) 1× baseline 10ms 5% messages lost permanently
QoS 1 2 (PUBLISH + PUBACK) 2× baseline 25ms All delivered, ~5% duplicates
QoS 2 4 (PUBLISH + PUBREC + PUBREL + PUBCOMP) 4× baseline 60ms All delivered, zero duplicates

MQTT QoS levels have dramatic battery and broker scaling impacts. Let’s quantify for a 1,000-sensor deployment:

Scenario: 1,000 sensors report every 60 seconds, 50-byte payloads.

QoS 0 analysis:

  • Messages/day/sensor: 1,440
  • Total broker throughput: \(1{,}000 \times 1{,}440 = 1{,}440{,}000\) msg/day
  • Radio time/message: \(\frac{50 \times 8}{250{,}000} = 1.6\) ms @ 30 mA = 0.013 mAh
  • Daily energy/sensor: \(1{,}440 \times 0.013 = 18.7\) mAh
  • Broker load: 1.44M messages × 1 packet = 1.44M packets/day

QoS 1 analysis (2× overhead): - Broker load: 1.44M × 2 (PUBLISH + PUBACK) = 2.88M packets/day - ACK wait time: 5 ms @ 10 mA = 0.014 mAh - Daily energy: \(1{,}440 \times (0.013 + 0.014) = 38.9\) mAh - Battery penalty: \(\frac{38.9}{18.7} = 2.08\times\) vs QoS 0

QoS 2 analysis (4× overhead): - Broker load: 1.44M × 4 (4-way handshake) = 5.76M packets/day - 3 ACK phases: 15 ms @ 10 mA = 0.042 mAh - Daily energy: \(1{,}440 \times (0.013 + 0.042) = 79.2\) mAh - Battery penalty: \(\frac{79.2}{18.7} = 4.24\times\) vs QoS 0

Broker CPU scaling: At 1.44M msg/day baseline (QoS 0), Mosquitto on 2-core @ 2 GHz uses ~5% CPU. QoS 2 → 20% CPU.

Worked Example: Smart Factory with 1,000 Sensors

Scenario: 1,000 temperature sensors report every 10 seconds; 100 actuators receive commands hourly.

QoS 0 for sensors

  • Messages/hour: 1,000 x 360 = 360,000
  • Broker work: 360,000 messages, no ACK packets
  • 5% packet loss: about 18,000 readings lost
  • Impact: acceptable because the next reading arrives in 10 seconds
  • Bandwidth: about 18 MB/hour at 50 bytes per reading

QoS 1 for actuators

  • Commands/hour: 100
  • Broker work: 200 packets, including PUBACKs
  • 5% retry rate: about 5 duplicate commands
  • Impact: acceptable when commands include an idempotent command ID
  • Bandwidth: about 10 KB/hour

If sensors incorrectly used QoS 1

  • Messages/hour: doubles to 720,000 packets with ACKs
  • Broker CPU: roughly 2x increase
  • Bandwidth: roughly 36 MB/hour
  • Benefit: zero reading loss
  • Trade-off: not worth it because the next reading quickly replaces a lost one

If actuators incorrectly used QoS 0

  • Lost commands: about 5 per hour at 5% packet loss
  • Impact: valves may not actuate and production can stop
  • Cost: operational downtime dominates the tiny ACK overhead
  • Trade-off: unacceptable risk for minimal bandwidth savings

Key Insights:

  1. Don’t over-specify: QoS 1 for all messages wastes 50%+ bandwidth/CPU for replaceable sensor data
  2. Don’t under-specify: QoS 0 for commands risks operational failures worth far more than the ACK overhead
  3. Use retain strategically: Device status with retain=true + QoS 1 ensures new subscribers get last known state
  4. Hybrid architectures work best: QoS 0 for high-volume telemetry, QoS 1 for important events, QoS 2 for critical transactions

Production Recommendation:

  • Default: QoS 0 (assume sensor telemetry)
  • Exceptions: QoS 1 for alerts/commands/device status
  • Rare: QoS 2 only for financial/legal/safety-critical operations where duplicates cause actual harm

Concept Relationships

MQTT as a comprehensive protocol connects to many IoT system concepts:

Protocol Fundamentals:

Alternative Protocols:

System Integration:

  • Edge Gateway Patterns - MQTT broker placement
  • Message Broker Architecture - Scaling brokers
  • Cloud Integration - AWS IoT Core, Azure IoT Hub

Transport Layer:

  • TCP Fundamentals - Underlying reliable transport
  • TLS/SSL Security - Encryption for MQTT over port 8883
  • Network Reliability - How QoS works with TCP

Prerequisites You Should Know:

  • TCP provides ordered, reliable delivery - MQTT builds on this
  • Publish-subscribe pattern decouples producers from consumers
  • Quality of Service (QoS) levels trade reliability for overhead
  • Broker clustering enables horizontal scaling beyond single-node limits

What This Enables:

  • Design scalable IoT systems handling millions of devices
  • Select appropriate QoS levels based on application requirements
  • Implement secure MQTT with TLS and topic-level ACLs
  • Compare MQTT with CoAP/HTTP for protocol selection decisions

See Also

MQTT Deep Dives:

Protocol Comparisons:

  • CoAP vs MQTT - When to use each
  • AMQP vs MQTT - Enterprise vs IoT messaging
  • Protocol Selection Guide - Decision framework

Broker Technologies:

Specifications:

33.8 What’s Next

Now that you have reviewed the full MQTT protocol, explore related IoT messaging protocols and system-level topics:

Chapter

33.8.1 CoAP Fundamentals and Architecture

Focus: UDP-based request-response protocol for constrained devices.

Why read it: Understand when CoAP outperforms MQTT for battery-powered sensors, CoAP Observe patterns, and RESTful IoT design.

Chapter

33.8.2 CoAP vs MQTT Comparison

Focus: Side-by-side protocol selection framework.

Why read it: Apply a structured decision process for latency, power, topology, and reliability constraints.

Chapter

33.8.3 AMQP Fundamentals

Focus: Enterprise message queuing with exchanges, queues, and bindings.

Why read it: Evaluate when AMQP’s richer routing model is a better fit for back-end IoT pipelines than simple MQTT pub/sub.

Chapter

33.8.4 AMQP vs MQTT and Use Cases

Focus: When to use AMQP and MQTT in the same system.

Why read it: Design hybrid architectures that pair MQTT for device-to-cloud messaging with AMQP for cloud service routing.

Chapter

33.8.5 MQTT Advanced Topics

Focus: MQTT 5.0 features, packet structure, and topic design patterns.

Why read it: Implement shared subscriptions, topic aliases, user properties, and other production-grade MQTT 5.0 enhancements.

Chapter

33.8.6 MQTT Labs and Implementation

Focus: Hands-on ESP32 MQTT projects with Mosquitto and Node-RED.

Why read it: Build working prototypes that use MQTT QoS, retained messages, and TLS on real hardware.