13  MQTT Fundamentals

In 60 Seconds

MQTT is the de facto IoT messaging standard, using a publish-subscribe pattern where sensors publish data to hierarchical topics, applications subscribe to topics of interest, and a central broker handles all routing. The three QoS levels (0: fire-and-forget, 1: at-least-once, 2: exactly-once) let you trade reliability for overhead based on your application’s needs.

13.1 Learning Objectives

By the end of this chapter series, you will be able to:

  • Explain the MQTT publish-subscribe pattern and how brokers decouple publishers from subscribers
  • Design hierarchical topic structures with wildcards for scalable IoT messaging
  • Select appropriate QoS levels (0, 1, 2) based on reliability, battery, and bandwidth trade-offs
  • Configure MQTT security including TLS encryption, authentication, and topic-level ACLs
  • Compare MQTT with CoAP and HTTP to choose the right protocol for your IoT application
  • MQTT: Message Queuing Telemetry Transport — pub/sub protocol optimized for constrained IoT devices over unreliable networks
  • Broker: Central server routing messages from publishers to all matching subscribers by topic pattern
  • Topic: Hierarchical string (e.g., home/bedroom/temperature) used to route messages to interested subscribers
  • QoS Level: Quality of Service 0/1/2 trading delivery guarantee for message overhead
  • Retained Message: Last message on a topic stored by broker for immediate delivery to new subscribers
  • Last Will and Testament: Pre-configured message published by broker when a client disconnects ungracefully
  • Persistent Session: Broker stores subscriptions and pending messages allowing clients to resume after disconnection

13.2 For Beginners: MQTT Fundamentals

MQTT (Message Queuing Telemetry Transport) is one of the most popular protocols for IoT communication. It works like a messaging app for devices: sensors publish updates to topics (like chat channels), and any interested device can subscribe to receive those updates. It is simple, lightweight, and works even on slow or unreliable connections.

“MQTT is basically a group chat for devices!” announced Max the Microcontroller. “Sammy, you publish your temperature to the topic garden/temperature. Anyone who subscribes to that topic gets your updates automatically.”

“It’s so lightweight!” said Bella the Battery happily. “The connect message is just a few bytes, and each publish is even smaller. Compare that to HTTP where just the headers are hundreds of bytes. MQTT was literally designed for devices like us – low bandwidth, unreliable networks, tiny batteries.”

Sammy the Sensor demonstrated: “I publish to garden/temperature and the dashboard subscribes to garden/# – that wildcard means it gets ALL garden messages. Temperature, humidity, soil moisture – one subscription covers everything. Wildcards are like subscribing to an entire TV channel instead of individual shows.”

“And it works even on terrible connections!” added Lila the LED. “MQTT was invented for monitoring oil pipelines via satellite – where every byte costs money and the connection drops constantly. If it can handle that, it can definitely handle our garden sensors on WiFi!”

13.3 MQTT - Message Queue Telemetry Transport

MQTT is the de facto standard for IoT messaging, enabling millions of devices to communicate efficiently through a publish-subscribe pattern. This chapter series provides comprehensive coverage of MQTT concepts, from basic pub/sub mechanics to advanced security and optimization.

Key Takeaway

MQTT decouples message producers (publishers) from consumers (subscribers) through a central broker. Sensors publish data to topics, applications subscribe to topics they care about, and the broker handles all routing. The three QoS levels (0: fire-and-forget, 1: at-least-once, 2: exactly-once) let you trade off between reliability and overhead based on your application’s needs.

13.4 Chapter Overview

This comprehensive guide to MQTT is organized into focused chapters:

Chapter Focus Difficulty Time
Publish-Subscribe Basics Topics, wildcards, broker architecture Beginner 25 min
Quality of Service QoS levels, reliability vs overhead trade-offs Intermediate 20 min
Security TLS, authentication, access control Intermediate 15 min
Advanced Topics Packet structure, topic design, MQTT 5.0 Intermediate 20 min
Practice and Exercises Hands-on labs, pitfalls, resources Intermediate 30 min

13.5 Learning Path

Recommended MQTT learning sequence showing five progressive steps from publish-subscribe basics through QoS levels, security configuration, advanced topics like MQTT 5.0, to hands-on practice exercises with prerequisite relationships between chapters
Figure 13.1: Recommended learning sequence for MQTT Fundamentals

Recommended sequence:

  1. Start with Pub/Sub Basics - Understand the publish-subscribe pattern, topics, and wildcards
  2. Learn QoS Levels - Master reliability vs overhead trade-offs
  3. Study Security - Configure TLS, authentication, and access control
  4. Explore Advanced Topics - Packet structure, topic design patterns, MQTT 5.0
  5. Complete Practice Exercises - Build hands-on skills and avoid common pitfalls

13.6 Why MQTT?

The Challenge: Scalable Device Communication

The Problem: Traditional request-response patterns (like HTTP polling) don’t scale for IoT:

  • N devices x M polls/hour = billions of wasted requests
  • Servers must track every device connection
  • Devices waste energy checking for messages that aren’t there

The Solution: Publish-subscribe messaging with a central broker:

  • Decouple senders from receivers
  • Store messages for offline devices
  • Push instead of poll
  • Scale to millions of devices

13.7 MQTT Cost at a Glance

Calculate monthly cloud messaging costs for your IoT fleet:

Show code
viewof mqttDeviceCount = Inputs.range([100, 100000], {
  value: 10000,
  step: 100,
  label: "Number of devices:",
  width: 300
})

viewof mqttMsgRate = Inputs.range([0.1, 10], {
  value: 1,
  step: 0.1,
  label: "Messages per minute per device:",
  width: 300
})

viewof mqttCostPer1M = Inputs.range([0.5, 5], {
  value: 1,
  step: 0.1,
  label: "Cost per 1M messages ($):",
  width: 300
})

mqttMonthlyMessages = mqttDeviceCount * mqttMsgRate * 1440 * 30
mqttMonthlyCost = (mqttMonthlyMessages / 1000000) * mqttCostPer1M
mqttYearlyCost = mqttMonthlyCost * 12
Monthly Messages
\({Math.round(mqttMonthlyMessages / 1000000).toFixed(1)}M</div> <div style="font-size: 0.9em; opacity: 0.9;">Monthly Cost: <span style="font-size: 1.3em; font-weight: bold;">\)
Yearly Cost: $

Key Insight: At devices publishing msg/min, you’ll send \({(mqttMonthlyMessages / 1000000).toFixed(1)}M messages/month costing **\)\({mqttMonthlyCost.toFixed(2)}/month** or **\)/year**.

13.8 MQTT at a Glance

Feature Description
Pattern Publish-Subscribe (via broker)
Transport TCP (port 1883) or TLS (port 8883)
Message Size Up to 256 MB (typically KB)
QoS Levels 0 (fire-forget), 1 (at-least-once), 2 (exactly-once)
Key Features Retained messages, Last Will, Session persistence
Protocol Versions 3.1.1 (2014), 5.0 (2019)

Compare bandwidth usage across MQTT QoS levels:

Show code
viewof qosPayloadSize = Inputs.range([10, 1000], {
  value: 100,
  step: 10,
  label: "Payload size (bytes):",
  width: 300
})

viewof qosMessagesPerDay = Inputs.range([100, 10000], {
  value: 1440,
  step: 100,
  label: "Messages per day:",
  width: 300
})

// MQTT packet overhead (approximate)
qos0Overhead = 2  // Fixed header only
qos1Overhead = 4  // Fixed header + PUBACK
qos2Overhead = 10 // Fixed header + PUBREC + PUBREL + PUBCOMP

qos0TotalPerMsg = qosPayloadSize + qos0Overhead
qos1TotalPerMsg = qosPayloadSize + qos1Overhead
qos2TotalPerMsg = qosPayloadSize + qos2Overhead

qos0DailyKB = (qos0TotalPerMsg * qosMessagesPerDay) / 1024
qos1DailyKB = (qos1TotalPerMsg * qosMessagesPerDay) / 1024
qos2DailyKB = (qos2TotalPerMsg * qosMessagesPerDay) / 1024

qos1Increase = ((qos1DailyKB - qos0DailyKB) / qos0DailyKB * 100)
qos2Increase = ((qos2DailyKB - qos0DailyKB) / qos0DailyKB * 100)
html`<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 15px; margin: 15px 0;">
  <div style="background: #2C3E50; color: white; padding: 15px; border-radius: 8px;">
    <div style="font-size: 0.85em; opacity: 0.9;">QoS 0 (Fire-and-Forget)</div>
    <div style="font-size: 1.8em; font-weight: bold; margin: 8px 0; color: #16A085;">${qos0DailyKB.toFixed(1)} KB/day</div>
    <div style="font-size: 0.8em; opacity: 0.8;">Baseline</div>
  </div>
  <div style="background: #E67E22; color: white; padding: 15px; border-radius: 8px;">
    <div style="font-size: 0.85em; opacity: 0.9;">QoS 1 (At-Least-Once)</div>
    <div style="font-size: 1.8em; font-weight: bold; margin: 8px 0;">${qos1DailyKB.toFixed(1)} KB/day</div>
    <div style="font-size: 0.8em; opacity: 0.8;">+${qos1Increase.toFixed(1)}% overhead</div>
  </div>
  <div style="background: #E74C3C; color: white; padding: 15px; border-radius: 8px;">
    <div style="font-size: 0.85em; opacity: 0.9;">QoS 2 (Exactly-Once)</div>
    <div style="font-size: 1.8em; font-weight: bold; margin: 8px 0;">${qos2DailyKB.toFixed(1)} KB/day</div>
    <div style="font-size: 0.8em; opacity: 0.8;">+${qos2Increase.toFixed(1)}% overhead</div>
  </div>
</div>
<div style="margin: 12px 0 4px; padding: 12px 14px; border-radius: 8px; background: #eef5f7; color: #2C3E50; font-size: 0.95em;">
  <strong>Key Insight:</strong> For ${qosPayloadSize}-byte messages sent ${qosMessagesPerDay} times/day, QoS 1 adds ${qos1Increase.toFixed(1)}% overhead while QoS 2 adds ${qos2Increase.toFixed(1)}% overhead compared to QoS 0.
</div>`

Compare the true cost of MQTT persistent connections vs HTTP polling:

viewof compDevices = Inputs.range([1000, 50000], { value: 10000, step: 1000, label: “Number of devices:”, width: 300 })

viewof compPollRate = Inputs.range([1, 60], { value: 1, step: 1, label: “Polls per minute:”, width: 300 })

viewof compCloudCost = Inputs.range([0.5, 3], { value: 1, step: 0.1, label: “Cost per 1M messages ($):”, width: 300 })

viewof compAlbCost = Inputs.range([100, 500], { value: 200, step: 50, label: “Monthly ALB/Gateway cost ($):”, width: 300 })

// MQTT: 1 message per poll compMqttMonthlyMsgs = compDevices * compPollRate * 1440 * 30 compMqttMonthlyCost = (compMqttMonthlyMsgs / 1000000) * compCloudCost

// HTTP: request + response = 2 messages per poll, plus ALB compHttpMonthlyMsgs = compMqttMonthlyMsgs * 2 compHttpMonthlyCost = (compHttpMonthlyMsgs / 1000000) * compCloudCost + compAlbCost

compSavings = compHttpMonthlyCost - compMqttMonthlyCost compSavingsPercent = (compSavings / compHttpMonthlyCost) * 100 compYearlySavings = compSavings * 12 html<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 20px; margin: 15px 0;"> <div style="background: linear-gradient(135deg, #16A085 0%, #2C3E50 100%); color: white; padding: 20px; border-radius: 8px;"> <div style="font-size: 1.1em; font-weight: bold; margin-bottom: 10px;">MQTT (Persistent)</div> <div style="font-size: 0.85em; opacity: 0.9;">Messages: ${(compMqttMonthlyMsgs / 1000000).toFixed(1)}M/month</div> <div style="font-size: 2.2em; font-weight: bold; margin: 15px 0; color: #8ff3cc;">$${compMqttMonthlyCost.toFixed(2)}</div> <div style="font-size: 0.85em; opacity: 0.9;">Monthly cost</div> </div> <div style="background: linear-gradient(135deg, #E67E22 0%, #E74C3C 100%); color: white; padding: 20px; border-radius: 8px;"> <div style="font-size: 1.1em; font-weight: bold; margin-bottom: 10px;">HTTP (Polling)</div> <div style="font-size: 0.85em; opacity: 0.9;">Messages: ${(compHttpMonthlyMsgs / 1000000).toFixed(1)}M/month</div> <div style="font-size: 2.2em; font-weight: bold; margin: 15px 0;">$${compHttpMonthlyCost.toFixed(2)}</div> <div style="font-size: 0.85em; opacity: 0.9;">Monthly cost incl. ALB</div> </div> </div> <div style="background: #2C3E50; color: white; padding: 20px; border-radius: 8px; margin: 15px 0; text-align: center;"> <div style="font-size: 0.9em; opacity: 0.9;">Monthly savings with MQTT</div> <div style="font-size: 2.1em; font-weight: bold; margin: 8px 0; color: #8ff3cc;">$${compSavings.toFixed(2)}</div> <div style="font-size: 1em; opacity: 0.9;">${compSavingsPercent.toFixed(1)}% reduction • $${compYearlySavings.toFixed(2)}/year saved</div> </div> <div style="margin: 12px 0 4px; padding: 12px 14px; border-radius: 8px; background: #eef5f7; color: #2C3E50; font-size: 0.95em;"> <strong>Key Insight:</strong> MQTT saves $${compSavings.toFixed(2)}/month (${compSavingsPercent.toFixed(1)}%) by using persistent connections instead of HTTP polling, eliminating ALB costs and halving message traffic. </div>

13.9 Quick Reference: When to Use MQTT

Choose MQTT when:

  • You need publish-subscribe (one-to-many communication)
  • Devices send frequent updates (every few seconds to minutes)
  • Multiple subscribers need the same data
  • Network is unreliable (automatic reconnection, QoS guarantees)
  • Battery-powered devices need efficient persistent connections

Choose CoAP instead when:

  • You need request-response (client-server pattern)
  • RESTful APIs are required (GET/POST/PUT/DELETE)
  • UDP is acceptable (lower overhead than TCP)

Choose HTTP instead when:

  • Integration with web services is primary goal
  • Infrequent communication (hourly/daily updates)
  • Large payloads (firmware updates, images)

13.10 Prerequisites

Before starting this series:

13.12 Start Learning

Ready to begin? Start with MQTT Publish-Subscribe Basics to understand the core concepts that make MQTT the leading IoT messaging protocol.

13.13 Knowledge Check

Match each MQTT term or feature to its correct definition or use case.

Place the following steps in the correct sequence for a successful QoS 1 message delivery from publisher to subscriber.

13.14 What’s Next

After completing this MQTT Fundamentals series, continue with:

Chapter Focus Why Read It
MQTT Publish-Subscribe Basics Topics, wildcards, broker architecture Build the foundation — understand how messages flow between publishers, the broker, and subscribers
MQTT Quality of Service QoS 0/1/2 reliability vs overhead trade-offs Select the right delivery guarantee for your application’s battery, bandwidth, and reliability requirements
MQTT Security TLS encryption, authentication, topic ACLs Secure your broker against unauthorized access and eavesdropping in production deployments
MQTT QoS and Session Advanced session management and persistent sessions Diagnose message loss and implement reliable delivery for offline devices
MQTT Labs and Implementation Hands-on ESP32 and Python MQTT projects Apply concepts immediately with working code for real IoT hardware
CoAP Overview RESTful request-response protocol for constrained devices Compare MQTT’s pub-sub model with CoAP to select the right protocol for each scenario