12  MQTT Architecture

In 60 Seconds

MQTT uses a broker-centric publish-subscribe architecture where publishers send messages to named topics without knowing who receives them, and subscribers register interest in topics without knowing who publishes. The central broker handles all message routing, persistence, and delivery guarantees, enabling fully decoupled one-to-many communication that scales far better than direct device-to-device connections.

12.1 MQTT Architecture

This chapter explores the broker-centric architecture that makes MQTT efficient for IoT applications, including the publish-subscribe pattern, broker components, and real-world system design.

12.2 Learning Objectives

By the end of this chapter, you will be able to:

  • Explain the Publish-Subscribe Pattern: Distinguish how decoupled messaging enables scalable IoT systems compared to request-response models
  • Analyze Broker Architecture: Identify and evaluate broker components and their functions in a real deployment
  • Design MQTT Systems: Construct and justify architectural decisions for real-world IoT scenarios using pub-sub principles
  • Compare Connection Topologies: Calculate the connection reduction ratio and assess when MQTT’s broker-centric approach outperforms peer-to-peer

MQTT pub-sub framework

MQTT Publish-Subscribe Architecture
Figure 12.1: MQTT publish-subscribe framework with broker architecture

MQTT broker ecosystem

MQTT System Architecture
Figure 12.2: MQTT’s publish-subscribe pattern decouples message producers from consumers. Publishers send messages to topics without knowing who will receive them, while subscribers register interest in topics without knowing who publishes to them. The broker handles all routing, persistence, and delivery guarantees.

MQTT broker internals

MQTT Broker Architecture
Figure 12.3: Inside the MQTT broker: topic tree organization enables efficient message routing, while session state management supports persistent connections and offline message queuing. The broker stores retained messages and manages QoS acknowledgments.
  • MQTT: Message Queuing Telemetry Transport — pub/sub protocol optimized for constrained IoT devices over unreliable networks
  • Broker: Central server routing messages from publishers to all matching subscribers by topic pattern
  • Topic: Hierarchical string (e.g., home/bedroom/temperature) used to route messages to interested subscribers
  • QoS Level: Quality of Service 0/1/2 trading delivery guarantee for message overhead
  • Retained Message: Last message on a topic stored by broker for immediate delivery to new subscribers
  • Last Will and Testament: Pre-configured message published by broker when a client disconnects ungracefully
  • Persistent Session: Broker stores subscriptions and pending messages allowing clients to resume after disconnection

12.3 For Beginners: MQTT Architecture

MQTT uses a hub-and-spoke architecture with a central broker that receives all messages and forwards them to interested subscribers. Think of the broker as a radio station tower – sensors publish data to the tower, and the tower broadcasts it to all listeners tuned to that channel (topic). No device needs to know about any other device directly.

“How does the dashboard know about my temperature readings if we’ve never met?” asked Sammy the Sensor.

Max the Microcontroller drew a picture. “The MQTT broker is like a radio tower in the middle. You, Sammy, publish your readings to a topic called home/kitchen/temperature. The dashboard subscribes to that same topic. The broker connects you two without either of you knowing the other exists!”

“That’s the beauty of publish-subscribe!” said Lila the LED. “I publish my status to home/living-room/lights, and anyone interested – the dashboard, the voice assistant, the energy monitor – subscribes to that topic. If the dashboard goes offline, I keep publishing. I don’t even notice it’s gone.”

Bella the Battery explained why this matters: “In a direct-connection system, Sammy would need to know the address of every device that wants his data. Add a new dashboard? Reprogram Sammy. With pub-sub, Sammy just publishes. New devices subscribe to his topic whenever they want. Zero changes to Sammy, zero extra energy for me!”

12.4 The Publish-Subscribe Pattern

The publish-subscribe (pub-sub) pattern is the foundation of MQTT’s architecture. Unlike request-response models (HTTP, CoAP), pub-sub decouples message producers from consumers.

Direct device-to-device connections (no broker): For \(n\) publishers and \(m\) subscribers, each publisher maintains connections to all subscribers: $ C_{} = n m $

Broker-based architecture: Each device connects only to the broker: $ C_{} = n + m $

Connection reduction ratio: $ R = = = 1 - - $

Concrete example (100 sensors, 10 dashboards):

  • Direct: \(C = 100 \times 10 = 1000\) connections
  • Broker: \(C = 100 + 10 = 110\) connections
  • Reduction: \(\frac{1000 - 110}{1000} = 89\%\)

Memory savings per device: Each TCP connection consumes ~4 KB RAM. For a sensor with 32 KB RAM: - Direct model: \(10 \times 4\text{ KB} = 40\text{ KB}\) (exceeds capacity!) - Broker model: \(1 \times 4\text{ KB} = 4\text{ KB}\) (12.5% of RAM)

This \(O(n+m)\) scaling is why MQTT works for millions of devices—each device’s resource usage is independent of fleet size.

12.5 Interactive Calculator: Connection Scaling

Real-World Example: Smart Thermostat System

Scenario: You have a Nest thermostat in your living room that reports temperature every 5 minutes to maintain comfort and optimize energy usage.

What happens behind the scenes:

  1. Sensor reads: Thermostat measures temperature (22.5C)
  2. MQTT publish: Sends tiny message to topic home/living-room/temperature -> Broker
  3. Broker distributes: Forwards to 3 subscribers:
    • Your phone app (shows current temperature)
    • Home automation system (triggers AC if >25C)
    • Data logger (stores for analysis)

Why MQTT is perfect for this:

  • Small messages: Temperature reading is <100 bytes (vs HTTP’s 200+ byte overhead)
  • Works over flaky Wi-Fi: QoS 1 ensures important readings aren’t lost during brief disconnections
  • Battery-friendly: ESP32-based sensor lasts 6+ months on batteries (vs 2-3 weeks with HTTP polling)
  • Easy to add subscribers: Want to add a Google Home display? Just subscribe to the topic-no changes to the thermostat needed!

Actual message payload: {"temp": 22.5, "humidity": 45, "timestamp": 1702314567}

Real deployment: This exact pattern is used by Nest, Ecobee, and most commercial smart thermostats. The Nest thermostat publishes to Google’s MQTT broker every 1-5 minutes depending on activity.

12.6 Architecture Components

Components:

  1. Broker: Central server that routes messages (e.g., Mosquitto, HiveMQ)
  2. Publisher: Client that sends messages to topics
  3. Subscriber: Client that receives messages from topics
  4. Topic: Hierarchical message routing path (e.g., home/bedroom/temperature)

MQTT protocol methods

MQTT Protocol Methods
Figure 12.4: MQTT methods: CONNECT, PUBLISH, SUBSCRIBE, UNSUBSCRIBE

MQTT topic hierarchy

MQTT Topic Hierarchy
Figure 12.5: MQTT topics use hierarchical naming with forward slash separators. Wildcards enable flexible subscriptions: ‘+’ matches one level (home/+/temperature matches home/kitchen/temperature), while ‘#’ matches all remaining levels (building/# matches all sensors in a building).

12.7 Connection Scaling Benefits

12.8 Common Misconception: QoS 2 and End-to-End Delivery

Common Misconception: “QoS 2 Guarantees End-to-End Delivery”

The Myth: Many developers believe that using QoS 2 ensures their message reaches the subscriber and is processed successfully.

The Reality: QoS 2 only guarantees exactly-once delivery to the broker and exactly-once delivery from the broker to online subscribers. It does NOT guarantee: - The subscriber is online when you publish - The subscriber successfully processes the message - The subscriber takes the expected action

Real-World Data: In a study of 500 industrial IoT deployments, 42% of critical system failures were traced to developers assuming QoS 2 provided end-to-end confirmation. They sent door unlock commands with QoS 2 and assumed the door unlocked, when in reality the lock controller was offline.

The Solution: Implement application-level acknowledgment:

QoS vs app-level ACK
Figure 12.6: MQTT QoS vs application-level acknowledgment for reliable commands

Key Insight: QoS levels control transport reliability (network delivery), not application reliability (business logic execution). Always implement state feedback for critical commands.

Topic structure wildcards

This diagram shows MQTT’s hierarchical topic structure with wildcards: + matches one level, # matches multiple levels, enabling flexible subscription patterns.

Reliability layers
Figure 12.7: Alternative view: Reliability layers in IoT communication showing that MQTT QoS (Layer 2) only guarantees broker delivery, while application-level acknowledgment (Layer 3) confirms actual device action execution. Most failures occur when developers stop at Layer 2.

12.9 Interactive Simulator

12.9.1 Try MQTT in Your Browser

Below is a live ESP32 simulator running MQTT code. You can modify the code and see it work!

{sim id="mqtt-esp32-dht22"}

Using the Simulator
  1. Click the green “Play” button to run the simulation
  2. The ESP32 will connect to a public MQTT broker
  3. Watch the Serial Monitor for messages
  4. Try modifying the topic name or message
  5. No hardware needed - runs entirely in your browser!

12.10 Example Code: MQTT Publisher

Here’s a complete example using ESP32 to publish temperature data:

# Python MQTT Publisher Example (paho-mqtt 2.0+)
import paho.mqtt.client as mqtt
import time
import random

# MQTT Broker settings
BROKER = "test.mosquitto.org"  # Public test broker
PORT = 1883
TOPIC = "iotclass/sensors/temperature"

# Create MQTT client (paho-mqtt 2.0 requires CallbackAPIVersion)
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="TempSensor_001")

# Connect to broker
print(f"Connecting to {BROKER}...")
client.connect(BROKER, PORT)

# Publish temperature readings
try:
    while True:
        # Simulate temperature sensor
        temp = round(20 + random.uniform(-5, 5), 2)

        # Publish message
        client.publish(TOPIC, f"{temp}")
        print(f"Published: {temp}C to {TOPIC}")

        time.sleep(5)  # Wait 5 seconds

except KeyboardInterrupt:
    print("Stopping publisher...")
    client.disconnect()

12.11 Example Code: MQTT Subscriber

And here’s how to receive those messages:

# Python MQTT Subscriber Example (paho-mqtt 2.0+)
import paho.mqtt.client as mqtt

BROKER = "test.mosquitto.org"
PORT = 1883
TOPIC = "iotclass/sensors/temperature"

# Callback when connection is established (VERSION2 signature)
def on_connect(client, userdata, flags, reason_code, properties):
    print(f"Connected with reason code {reason_code}")
    client.subscribe(TOPIC)
    print(f"Subscribed to {TOPIC}")

# Callback when message is received
def on_message(client, userdata, msg):
    temp = float(msg.payload.decode())
    print(f"Received: {temp}C from {msg.topic}")

    # Alert if temperature is too high
    if temp > 25:
        print("Warning: High temperature detected!")

# Create and configure client (paho-mqtt 2.0 requires CallbackAPIVersion)
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="Dashboard_001")
client.on_connect = on_connect
client.on_message = on_message

# Connect and start listening
client.connect(BROKER, PORT)
client.loop_forever()  # Block and listen for messages

12.12 Knowledge Check

Match the Concepts

12.12.1 Order the Steps

Knowledge Check: MQTT Architecture Test Your Understanding

12.13 Broker Selection Decision Framework

Choosing an MQTT broker is one of the most consequential architecture decisions in an IoT project. The right broker depends on deployment scale, reliability requirements, and operational constraints.

12.13.1 Decision Table: MQTT Broker Selection

Requirement Mosquitto (Open Source) HiveMQ (Commercial) AWS IoT Core (Managed) EMQX (Open Source)
Max connections ~100K (single node) Millions (clustered) Unlimited (auto-scale) Millions (clustered)
Clustering No (single node only) Yes (native) Built-in Yes (native)
Cost (1K devices) Free + server ($20/mo) ~$500/mo ~$30/mo Free + server ($40/mo)
Cost (100K devices) Infeasible (no cluster) ~$5,000/mo ~$2,500/mo Free + servers ($400/mo)
Setup complexity 5 minutes Hours (cluster config) Minutes (console) 30 minutes
MQTT 5.0 support Full Full Partial Full
Bridging Yes Yes Limited Yes
Best for Prototyping, small deploys Enterprise, mission-critical AWS-native, serverless Large-scale, self-hosted

12.13.2 When to Use Each Broker

Mosquitto is the right choice when: you have fewer than 10,000 devices, can tolerate minutes of downtime during restarts, and want zero licensing cost. A Raspberry Pi running Mosquitto handles a typical smart home with ease.

HiveMQ or EMQX is the right choice when: you need high availability (99.99%+ uptime), horizontal scaling across multiple nodes, or must handle connection spikes (100,000 devices reconnecting after a network outage). Both support MQTT bridging for multi-region deployments.

AWS IoT Core is the right choice when: your backend is already on AWS, you want zero operational overhead for the broker, and you need deep integration with Lambda, DynamoDB, or S3. The pay-per-message pricing ($1 per million messages) is cost-effective at moderate scale but becomes expensive above 100 million messages/month.

12.13.3 Real-World Sizing Example

A fleet management company tracks 25,000 vehicles, each publishing GPS coordinates every 10 seconds:

Message rate: 25,000 vehicles x 6 messages/min = 150,000 messages/min
Daily volume: 150,000 x 1,440 = 216 million messages/day
Payload: 85 bytes (lat, lon, speed, heading, timestamp in JSON)
Bandwidth: 150,000 x 85 / 60 = 212 KB/s sustained
Concurrent connections: 25,000 persistent + 500 dashboard subscribers

Broker options:
  Mosquitto: Cannot handle 25K concurrent (single-threaded bottleneck)
  HiveMQ (3-node cluster): $3,000/mo, 99.99% SLA
  AWS IoT Core: 216M msgs/day x 30 = 6.48B msgs/mo x $1/M = $6,480/mo
  EMQX (3-node cluster): $120/mo (self-hosted on 3x c5.xlarge)

Decision: EMQX self-hosted at $120/month if the team has DevOps expertise. AWS IoT Core at $6,480/month if the team wants zero operational burden. The 54x cost difference funds a full-time DevOps engineer, so the decision comes down to whether managing broker infrastructure is a core competency.

12.14 Interactive Calculator: MQTT Bandwidth Estimator

Scenario: A city installs 50,000 smart LED streetlights reporting status (on/off/brightness/fault) every 5 minutes. Three backend systems consume this data: city dashboard (all lights), maintenance system (faults only), and energy analytics (brightness levels).

Comparing architectures:

Peer-to-peer (hypothetical, each system polls each light):

  • Connections: 50,000 lights × 3 systems = 150,000 TCP connections
  • Poll messages: 50,000 × 12 polls/hour × 3 systems = 1.8M requests/hour
  • Server load: Each light must handle 3 simultaneous connections
  • Complexity: Lights must know IPs of all 3 backend systems

MQTT broker architecture:

  • Connections: 50,000 lights + 3 backends = 50,003 connections to broker
  • Publish messages: 50,000 × 12/hour = 600K publishes/hour
  • Broker distributes to subscribers: ~900K deliveries/hour (dashboard gets all, others filtered)
  • Complexity: Lights publish to single topic, backends subscribe

Scaling calculation:

  • Connection reduction: 150,000 → 50,003 (67% fewer)
  • Each light saves RAM: 3 TCP states → 1 TCP state (12 KB saved per device)
  • Adding 4th consumer: Peer-to-peer requires 50,000 new connections; MQTT requires 1 subscription

12.15 Interactive Calculator: Broker Cost Comparison