11  MQTT Introduction and Fundamentals

In 60 Seconds

MQTT (Message Queuing Telemetry Transport) is a lightweight publish-subscribe protocol originally designed for oil pipeline monitoring over satellite links. Devices publish sensor data to hierarchical topics on a central broker, and any number of subscribers receive messages matching their topic filters, enabling efficient one-to-many IoT communication with as little as 2 bytes of overhead.

11.1 MQTT

Time: ~10 min | Intermediate | P09.C23.U01

This section provides a stable anchor for cross-references to the MQTT protocol across the module.

11.2 Learning Objectives

By the end of this chapter, you will be able to:

  • Explain the MQTT Protocol: Describe the publish-subscribe messaging pattern and distinguish it from request-response protocols like HTTP
  • Implement MQTT Communication: Construct working IoT applications using MQTT publishers, subscribers, and a broker
  • Calculate Protocol Overhead: Compare MQTT packet sizes against HTTP equivalents and justify QoS level selection based on bandwidth and battery constraints
  • Design IoT Systems: Apply MQTT to real-world IoT scenarios and select appropriate QoS levels for different data types

Meet the Sensor Squad!

Temperature Terry, Light Lucy, Motion Marley, Pressure Pete, and Signal Sam are a team of sensor friends who work together to make smart homes and buildings work! Today, they’ll help explain how MQTT works.

Simple Story:

Imagine your school has a giant bulletin board in the hallway. Temperature Terry has important news: “It’s 75 degrees in the classroom!” But Terry doesn’t want to run around telling everyone individually - that would take forever!

Instead, Terry walks to the bulletin board and pins a note under the “Classroom Temperature” section. Now, anyone who wants to know the temperature can just check the bulletin board! The air conditioner checks it and says “Too warm, I’ll turn on!” Your teacher’s phone checks it and shows the temperature. The school computer logs it for records. Terry only had to post ONE message, but everyone who cared got the information.

That’s exactly how MQTT works! The bulletin board is called a “broker” - it’s the central place where all messages go. Sensors “publish” their readings to the board, and devices that want to know “subscribe” to topics they care about. Light Lucy might subscribe to “Light Levels” while Motion Marley subscribes to “Movement Alerts.” Each sensor posts to its topic, and only the devices that subscribed get those messages!

Fun Facts:

  • MQTT was invented to help oil pipelines in the desert send tiny messages over slow satellite connections!
  • MQTT messages can be as small as 2 bytes - that’s like sending a message with just two letters!
  • Amazon Alexa uses MQTT to talk to over 100 million smart home devices!
  • Tesla cars use MQTT to report their health and receive updates!

MQTT’s minimal fixed header is one of its key efficiency advantages:

Fixed header structure: $ H_{} = 2 $

First byte: Message type (4 bits) + flags (4 bits) Second byte: Remaining length (7 bits per byte, variable)

Full packet for QoS 0 PUBLISH with short topic: $ S_{} = 2 + 2 + T + P $

Concrete example (topic temp, payload 23.5):

  • Fixed header: 2 bytes
  • Topic length field: 2 bytes
  • Topic string: 4 bytes (temp)
  • Payload: 4 bytes (23.5)
  • Total: \(2 + 2 + 4 + 4 = 12\text{ bytes}\)

Compare to HTTP POST:

POST /api/sensors/temp HTTP/1.1
Host: example.com
Content-Length: 4

23.5

Total: ~150 bytes minimum (headers + body)

Bandwidth savings: $ = = = 92% $

For 10,000 sensors sending hourly readings: - MQTT: \(10,000 \times 24 \times 12 = 2.88\text{ MB/day}\) - HTTP: \(10,000 \times 24 \times 150 = 36\text{ MB/day}\) - Savings: \(36 - 2.88 = 33.12\text{ MB/day}\) (92% reduction)

This 2-byte overhead is why MQTT dominates battery-powered IoT.

Try This at Home:

Create your own family bulletin board! Put up different sections like “Chores Done,” “Weather Report,” and “Pet Fed.” Family members can post sticky notes (publish) to sections, and anyone interested can check those sections (subscribe). Notice how the person posting doesn’t need to know who will read it - just like MQTT!

Think of MQTT like a post office for IoT devices.

Imagine you have a weather station at home that measures temperature. You want your phone, your smart thermostat, and a data logger all to receive those temperature readings. Without MQTT, you’d need to set up separate connections to each device-messy and inefficient!

MQTT solves this with a simple pattern:

  1. Publisher (your weather station) sends temperature data to a central Broker (the post office)
  2. Subscribers (your phone, thermostat, logger) tell the broker “I want temperature updates”
  3. The broker delivers messages to everyone who subscribed-automatically!

Real-world analogy: It’s like subscribing to a YouTube channel. The creator (publisher) uploads videos to YouTube (broker), and all subscribers automatically get notified-the creator doesn’t need to know who’s watching.

Why MQTT for IoT?

  • Tiny overhead: Messages can be as small as 2 bytes-perfect for battery-powered sensors
  • Works on bad networks: Built-in features handle disconnections gracefully
  • Scales easily: One sensor can feed thousands of apps without code changes

Key Terms Explained:

Term Plain English Definition Real-World Example
Broker The central message router that delivers messages to interested parties Like a post office that sorts and delivers mail to subscribers
Topic The “address” or category for messages home/livingroom/temperature - similar to a mailing address
Publish Sending a message to a topic Weather station sending “22.5C” to temperature topic
Subscribe Signing up to receive messages from a topic Your phone app saying “send me all temperature updates”
QoS Quality of Service-reliability level (0, 1, or 2) Like choosing regular mail (0), certified mail (1), or registered mail (2)
Retained Message Last message stored by broker for new subscribers Like leaving a note on the fridge-anyone who arrives sees the latest message
Last Will Message sent if device disconnects unexpectedly Like “if I don’t check in, notify everyone I’m offline”

Why Should I Care About MQTT?

MQTT powers most real-world IoT systems you interact with daily: - Amazon Alexa: Uses MQTT to communicate with 100+ million smart home devices - Facebook Messenger: Chose MQTT to reduce mobile app battery drain by 60% - Tesla vehicles: Report diagnostics and receive updates via MQTT (saving 90% bandwidth vs HTTP) - Smart cities: Traffic lights, parking sensors, and environmental monitors all use MQTT for efficient communication

By the numbers: A typical smart home temperature sensor using MQTT lasts 6-8 months on batteries, while the same sensor using HTTP polling lasts only 2-3 weeks. That’s the power of MQTT’s efficiency!

By the end of this chapter, you’ll be able to build your own MQTT-based IoT system!

Key Takeaway

In one sentence: MQTT is a publish-subscribe protocol optimized for unreliable networks, enabling efficient many-to-many communication through a central broker.

Remember this rule: Use QoS 1 by default; QoS 0 for high-frequency non-critical data, QoS 2 only when exactly-once delivery truly matters (and even then, implement application-level acknowledgment).

11.3 Prerequisites

Before diving into this chapter, you should be familiar with:

  • Basic Networking Concepts: Understanding TCP/IP, client-server architecture, and network protocols helps contextualize MQTT’s role in IoT communication
  • IoT Reference Architectures: Familiarity with IoT system layers and components provides context for where MQTT fits in the overall architecture
  • Basic programming knowledge: Experience with any programming language (Python, C++, JavaScript) is helpful for implementing MQTT clients and understanding code examples
Cross-Hub Connections

This chapter connects to multiple learning resources across the module:

Video Resources: Visit the Videos Hub for curated MQTT protocol demonstrations, broker setup tutorials, and real-world IoT system walkthroughs

Interactive Simulations: Explore the Simulations Hub for hands-on MQTT simulators including QoS level comparisons, topic wildcard explorers, and publish-subscribe flow visualizations

Self-Assessment: Test your MQTT knowledge with comprehensive quizzes in the Quizzes Hub covering pub-sub patterns, QoS levels, security, and topic design best practices

Common Challenges: Check the Knowledge Gaps Hub for frequently misunderstood MQTT concepts like QoS guarantees, retained messages, and Last Will Testament

Common Pitfalls

Unencrypted MQTT exposes device credentials and sensor data to network eavesdroppers — in a building IoT deployment on shared WiFi, this means any connected device can read all sensor data. Always enable TLS 1.2+ on the broker and generate unique client certificates for each device class.

Without LWT, there is no automatic notification when a device disconnects ungracefully — missed timeout alarms and false-healthy device status are common consequences. Configure LWT on every device connection to publish an offline status message, enabling real-time fleet health monitoring.

A single MQTT connection serializes all publishes through one TCP socket — at 100 messages/second with QoS 1, TCP backpressure creates queuing latency. Use multiple parallel MQTT connections or partition topics across connection pools for throughput above 1,000 messages/second.

11.4 What is MQTT?

In Plain English

MQTT is like a radio station broadcasting system. Publishers broadcast messages to topics (channels), and subscribers tune in to receive only the channels they care about. A central broker routes all messages - publishers and subscribers never talk directly to each other.

Everyday Analogy: Think of a newspaper delivery service. You subscribe to specific sections you want (sports, weather, business), and the delivery person (broker) brings only those sections to your door. The newspaper writers (publishers) don’t know who’s reading their articles-they just write and send them to the distribution center (broker).

Why this matters for IoT: In a smart home, your temperature sensor doesn’t need to know whether 1 device or 100 devices are listening. It just publishes temperature readings to a topic, and the broker handles delivery to all interested subscribers. This decoupling makes systems easier to build and scale.

Definition

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe messaging protocol designed for constrained devices and low-bandwidth, high-latency networks. It’s the de facto standard for IoT communication.

Key Characteristics:

  • Lightweight: Minimal packet overhead (2 bytes minimum)
  • Publish-Subscribe Pattern: Decouples message producers from consumers
  • Quality of Service: Three levels (QoS 0, 1, 2) for reliability
  • Retained Messages: Latest message stored for new subscribers
  • Last Will and Testament: Notifies when clients disconnect unexpectedly
When to Use MQTT vs CoAP vs HTTP

Quick guide: MQTT for publish-subscribe (one-to-many), CoAP for constrained REST devices, HTTP for web integration. See the detailed Decision Framework below for a comprehensive comparison table and decision tree.

11.5 MQTT Chapter Guide

This comprehensive MQTT guide is organized into the following chapters:

Chapter Focus Content
MQTT Introduction Getting Started This chapter - fundamentals, terminology, when to use
MQTT Architecture System Design Broker architecture, pub-sub pattern, components
MQTT Topics & Wildcards Topic Design Naming conventions, wildcards, best practices
MQTT QoS Levels Reliability QoS 0/1/2, battery considerations, delivery guarantees
MQTT Security Security TLS, authentication, ACLs, production hardening
MQTT Labs Hands-On ESP32 labs, Python examples, simulators
MQTT Advanced Topics Production Clustering, HA, troubleshooting, performance

11.6 Videos

MQTT Protocol Overview
MQTT Protocol Overview
Overview of MQTT publish/subscribe model, QoS, retained messages, and LWT.
Application Protocols Overview (from slides)
Application Protocols Overview
From Lesson 4 - positioning MQTT alongside CoAP, HTTP, and AMQP.

11.7 Working Code: Your First MQTT IoT System

This minimal Python example demonstrates MQTT publish-subscribe in action. A temperature sensor publishes readings, and a dashboard subscribes to receive them – all through a broker.

"""Minimal MQTT IoT Example -- publisher and subscriber."""
import time
import json
import random

# pip install paho-mqtt
import paho.mqtt.client as mqtt

BROKER = "test.mosquitto.org"  # Free public broker for testing
PORT = 1883
TOPIC = "iotclass/demo/temperature"


# --- Subscriber (Dashboard) ---
def on_message(client, userdata, msg):
    """Called when a message arrives on a subscribed topic."""
    data = json.loads(msg.payload)
    print(f"  Dashboard received: {data['temp']}C from {data['sensor_id']} "
          f"(QoS {msg.qos}, topic: {msg.topic})")

def run_subscriber():
    """Subscribe to temperature topic and print incoming readings."""
    client = mqtt.Client(client_id="dashboard-001")
    client.on_message = on_message
    client.connect(BROKER, PORT, keepalive=60)
    client.subscribe(TOPIC, qos=1)  # QoS 1 = at-least-once delivery
    print(f"Subscribed to {TOPIC} -- waiting for readings...")
    client.loop_forever()  # Blocks, processing messages


# --- Publisher (Sensor) ---
def run_publisher(sensor_id="sensor-garden-01", readings=5):
    """Publish simulated temperature readings to broker."""
    client = mqtt.Client(client_id=sensor_id)

    # Last Will: broker sends this if sensor disconnects unexpectedly
    client.will_set(
        topic=f"iotclass/demo/status/{sensor_id}",
        payload=json.dumps({"status": "offline", "sensor_id": sensor_id}),
        qos=1, retain=True
    )

    client.connect(BROKER, PORT, keepalive=60)
    client.loop_start()  # Background thread for network I/O

    for i in range(readings):
        temp = round(20 + random.uniform(-3, 5), 1)  # 17-25 C
        payload = json.dumps({
            "sensor_id": sensor_id,
            "temp": temp,
            "unit": "C",
            "reading": i + 1,
        })
        # QoS 1: broker acknowledges receipt (at-least-once)
        result = client.publish(TOPIC, payload, qos=1)
        result.wait_for_publish()
        print(f"  Published: {temp}C (msg {i+1}/{readings})")
        time.sleep(2)

    client.loop_stop()
    client.disconnect()
    print(f"  Sensor {sensor_id} finished {readings} readings")


# Run publisher (in a real system, subscriber runs on a different device)
if __name__ == "__main__":
    print("MQTT IoT Temperature Sensor Demo")
    print(f"Broker: {BROKER}:{PORT} | Topic: {TOPIC}\n")
    run_publisher(readings=5)

Key concepts demonstrated: topic hierarchy (iotclass/demo/temperature), QoS 1 delivery, Last Will message for disconnect detection, JSON payloads, and the decoupled pub-sub pattern. Run the subscriber in one terminal and the publisher in another to see real-time message flow.

11.8 Interactive Calculators

11.8.1 MQTT Packet Size Calculator

Explore how MQTT packet size varies with topic length and payload size, and compare to HTTP overhead.

11.8.2 MQTT vs HTTP Bandwidth Calculator

Calculate daily bandwidth for an IoT deployment using MQTT vs HTTP, based on number of sensors and reporting interval.

11.8.3 QoS Battery Life Estimator

Estimate how QoS level affects battery life for a battery-powered IoT sensor.

11.8.4 Broker Connection Scaling Calculator

See how MQTT’s broker pattern reduces connection complexity compared to direct device-to-device links.

11.9 Knowledge Check

Part A: Match each MQTT term to its correct definition or use case.

Part B: Place the steps of an MQTT QoS 1 PUBLISH exchange in the correct order.

Scenario: You’re deploying 500 temperature sensors in a commercial building. Each sensor publishes a JSON payload every 60 seconds. Calculate the total bandwidth requirements and broker load.

Given Data:

  • Number of sensors: 500
  • Publishing interval: 60 seconds
  • Payload: {"device":"temp042","value":23.5,"unit":"C","timestamp":1706140800} = 65 bytes (JSON)
  • QoS: 1 (at-least-once delivery with PUBACK)

Step-by-Step Calculation:

  1. Calculate per-message overhead:
    • MQTT fixed header: 2 bytes minimum
    • Topic length field: 2 bytes
    • Topic name (building/floor3/room42/temperature): ~35 bytes
    • Payload: 65 bytes
    • Total per message: 2 + 2 + 35 + 65 = 104 bytes
  2. Add QoS 1 acknowledgment:
    • PUBACK response: 4 bytes
    • Total per publish cycle: 104 + 4 = 108 bytes
  3. Calculate messages per second:
    • 500 sensors / 60 seconds = 8.33 messages/second
  4. Calculate bandwidth:
    • 8.33 msg/s × 108 bytes = 900 bytes/second = 7,200 bits/second ≈ 7.2 kbps
  5. Broker processing load:
    • Incoming: 8.33 PUBLISH messages/second
    • Outgoing ACKs: 8.33 PUBACK messages/second
    • Total broker operations: 16.67 messages/second

With subscribers (e.g., 5 dashboard applications subscribing to all sensors): - Broker must forward each message to 5 subscribers - Outgoing bandwidth: 8.33 msg/s × 104 bytes × 5 = 4,332 bytes/second ≈ 34.7 kbps - Total broker throughput: 7.2 kbps (in) + 34.7 kbps (out) = 41.9 kbps

Result: A 100 Mbps Ethernet connection has 2,387x more capacity than needed. Bottleneck will be broker CPU/RAM handling 16-33 messages/second, not network bandwidth.

Optimization: Switch to CBOR encoding (reduces payload from 65 to ~20 bytes) saves 43% bandwidth: 7.2 kbps → 4.1 kbps.