18  MQTT Practice and Exercises

In 60 Seconds

This hands-on chapter provides practical MQTT exercises covering basic pub/sub setup with Mosquitto, multi-sensor topic hierarchies, QoS-level experimentation, and common debugging techniques. You will build real MQTT systems, learn to avoid frequent implementation pitfalls like forgetting client.loop(), and develop systematic troubleshooting skills.

18.1 Learning Objectives

By the end of this chapter, you will be able to:

  • Implement MQTT pub/sub systems: Construct working publisher and subscriber clients using paho-mqtt and configure topic hierarchies for multi-sensor deployments
  • Diagnose common pitfalls: Identify and correct implementation mistakes including non-unique client IDs, missing client.loop() calls, and incorrect wildcard syntax
  • Select appropriate QoS levels: Justify QoS 0, 1, or 2 choices for specific IoT message types by analyzing loss impact, duplicate impact, and power constraints
  • Evaluate broker resource requirements: Calculate message throughput, bandwidth, and memory needs for a given IoT deployment and assess thundering herd risk
  • MQTT: Message Queuing Telemetry Transport — pub/sub protocol optimized for constrained IoT devices over unreliable networks
  • Broker: Central server routing messages from publishers to all matching subscribers by topic pattern
  • Topic: Hierarchical string (e.g., home/bedroom/temperature) used to route messages to interested subscribers
  • QoS Level: Quality of Service 0/1/2 trading delivery guarantee for message overhead
  • Retained Message: Last message on a topic stored by broker for immediate delivery to new subscribers
  • Last Will and Testament: Pre-configured message published by broker when a client disconnects ungracefully
  • Persistent Session: Broker stores subscriptions and pending messages allowing clients to resume after disconnection

18.2 For Beginners: MQTT Practice

These practice exercises help you apply MQTT concepts through hands-on scenarios. You will design topic hierarchies, configure quality-of-service levels, and troubleshoot common issues. Think of it as a driving lesson for MQTT – you learn best by actually doing, not just reading about it.

“Time to practice!” said Max the Microcontroller. “Exercise one: design a topic hierarchy for a three-story building with temperature and humidity sensors on each floor.”

Sammy the Sensor thought for a moment. “How about building/floor1/temperature, building/floor1/humidity, building/floor2/temperature… and so on? Then the building manager subscribes to building/# to get everything, and the HVAC on floor 2 subscribes to building/floor2/+ to get just its floor.”

“That’s clean and logical,” said Lila the LED. “Exercise two: what QoS should the fire alarm use? QoS 0 – fire-and-forget? No way! That alert must arrive. QoS 1 – at least once – is the minimum. Getting two fire alerts is better than getting zero!”

Bella the Battery tackled exercise three: “If I send a reading every 30 seconds at QoS 0, each publish is about 30 bytes. That’s 2,880 messages per day, or about 86 KB. At QoS 1, the broker replies with a PUBACK — just 4 bytes — so total traffic rises to about 98 KB per day, roughly 14% more. Not doubling! But the extra radio wakeup for each ACK still costs battery power. Now I see why choosing the right QoS matters for battery life. Practice these calculations before deploying!”

18.3 Prerequisites

Before diving into this chapter, you should be familiar with:

18.4 Practice Exercises

18.4.1 Exercise 1: Basic MQTT Pub/Sub

Objective: Set up a basic MQTT publisher and subscriber.

Tasks:

  1. Install Mosquitto broker or use test.mosquitto.org
  2. Create a publisher that sends temperature readings to home/bedroom/temperature every 5 seconds
  3. Create a subscriber that listens and prints received messages
  4. Run both simultaneously and observe the message flow

Expected Outcome:

  • Subscriber receives temperature readings in real-time
  • You understand the decoupling between publisher and subscriber

Sample Code:

# Requires paho-mqtt 2.0+
import paho.mqtt.client as mqtt
import time

# Publisher
def publish_temperature():
    client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
    client.connect("test.mosquitto.org", 1883)

    for temp in [22.0, 22.5, 23.0, 23.5]:
        client.publish("home/bedroom/temperature", str(temp))
        print(f"Published: {temp}")
        time.sleep(5)

    client.disconnect()

# Subscriber
def on_message(client, userdata, message):
    print(f"Received: {message.payload.decode()} from {message.topic}")

def subscribe_temperature():
    client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
    client.on_message = on_message
    client.connect("test.mosquitto.org", 1883)
    client.subscribe("home/bedroom/temperature")
    client.loop_forever()

Hints:

  • Use client.loop_forever() in subscriber to keep listening
  • Start subscriber before publisher to ensure it doesn’t miss messages

18.4.2 Exercise 2: Topic Wildcards

Objective: Master topic design and wildcard subscriptions.

Tasks:

  1. Design a topic hierarchy for a smart home with:
    • 3 rooms (bedroom, kitchen, living_room)
    • 3 sensor types per room (temperature, humidity, motion)
  2. Create publishers for all 9 sensors
  3. Create subscribers that:
    • Subscribe to all bedroom sensors (home/bedroom/#)
    • Subscribe to all temperatures (home/+/temperature)
    • Subscribe to everything (home/#)
  4. Verify wildcards match correctly

Expected Outcome:

  • home/bedroom/# receives bedroom temperature, humidity, motion
  • home/+/temperature receives temperature from all 3 rooms
  • home/# receives all 9 sensor streams

Topic Structure:

home/
  bedroom/
    temperature
    humidity
    motion
  kitchen/
    temperature
    humidity
    motion
  living_room/
    temperature
    humidity
    motion
Try It: MQTT Wildcard Pattern Matcher

Enter an MQTT subscription pattern using + (single-level) and # (multi-level) wildcards, and see which topics from a smart home hierarchy it matches.

18.4.3 Exercise 3: Retained Messages and LWT

Objective: Implement device status monitoring using retained messages and Last Will Testament.

Tasks:

  1. Create a simulated sensor that:
    • Publishes “online” (retained) on startup to device/sensor1/status
    • Sets LWT to publish “offline” (retained) to same topic
    • Publishes temperature readings every 10 seconds
  2. Create a monitoring dashboard subscriber
  3. Test LWT by forcefully killing the sensor process (Ctrl+C or kill -9)
  4. Verify dashboard receives “offline” status automatically

Expected Outcome:

  • Dashboard shows “online” immediately upon subscribing
  • When sensor crashes, dashboard receives “offline” automatically

Sample LWT Configuration:

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.will_set("device/sensor1/status", payload="offline", qos=1, retain=True)
client.connect("broker.local", 1883)
client.publish("device/sensor1/status", "online", qos=1, retain=True)

Testing Checklist:

Try It: Device Status Timeline Simulator

Simulate a fleet of IoT devices connecting, publishing, and crashing. See how retained messages and Last Will Testament (LWT) update the device status dashboard in real time.

18.4.4 Exercise 4: QoS Comparison

Objective: Compare QoS 0, 1, and 2 behavior under different conditions.

Tasks:

  1. Set up local Mosquitto broker
  2. Create a publisher that sends 100 messages with each QoS level
  3. Use Wireshark or tcpdump to capture MQTT traffic
  4. Analyze packet captures:
    • QoS 0: Single PUBLISH packet
    • QoS 1: PUBLISH + PUBACK
    • QoS 2: PUBLISH + PUBREC + PUBREL + PUBCOMP
  5. (Optional) Simulate 20% packet loss and observe retransmissions

Measurement Template:

QoS Level Packets Sent Packets Received ACKs Latency (ms)
0 100 ? 0 ?
1 100 ? ? ?
2 100 ? ? ?

Hints:

  • Use mosquitto -v to run broker in verbose mode
  • Filter Wireshark by mqtt protocol
  • Simulate packet loss: tc qdisc add dev lo root netem loss 20%

Let’s quantify the QoS performance differences you’ll observe in this exercise:

100-message test with 20% packet loss (simulated network):

QoS 0 delivery:

  • Expected losses: \(100 \times 0.20 = 20\) messages never arrive
  • Retransmissions: 0 (fire-and-forget)
  • Total packets on wire: 100
  • Delivery rate: 80%

QoS 1 delivery:

  • Lost PUBLISH: 20 → broker never sends PUBACK
  • Client retransmits after timeout (2 s): 20 additional PUBLISH
  • Lost PUBACK: \(80 \times 0.20 = 16\) → client retries again
  • Total packets: \(100 + 20 + 16 = 136\) PUBLISH + ~80 PUBACK = 216 packets
  • Delivery rate: 100% (eventually)
  • Duplicates received: ~16 (PUBACK lost, client resent, both copies arrive)

QoS 2 delivery:

  • 4-way handshake per message: PUBLISH → PUBREC → PUBREL → PUBCOMP
  • Packet loss hits 20% of all 4 phases
  • Expected retransmissions: \(100 \times 4 \times 0.20 = 80\) packets
  • Total packets: \((100 \times 4) + 80 = 480\) packets
  • Delivery rate: 100%, zero duplicates
  • Latency: up to \(4 \times 2\text{s timeout} = 8\text{s}\) worst case (if every leg is lost once)

Bandwidth comparison: \[\text{Efficiency ratio} = \frac{\text{QoS 0}}{\text{QoS 2}} = \frac{100}{480} = 0.21 \text{ or } 4.8\times \text{ overhead}\]

This exercise makes the QoS trade-offs tangible: reliability costs bandwidth and latency.

18.4.5 Interactive: QoS Overhead Calculator

Use this calculator to explore how QoS level and packet loss affect total network traffic for a batch of MQTT messages.

18.4.6 Interactive: MQTT Message Rate Calculator

Estimate daily and monthly message volume and bandwidth for your IoT deployment.

18.5 Common Pitfalls

Pitfall: Wrong MQTT Wildcard Usage

The Mistake: Using wildcards incorrectly leads to missing or unintended messages.

Wrong approach:

# INVALID: # wildcard not at end
client.subscribe("sensors/#/temperature")

# INVALID: Mixing wildcards incorrectly
client.subscribe("sensors/+/#/data")

Correct approach:

# Correct: Use + for single-level wildcard
client.subscribe("sensors/+/temperature")

# Correct: Use # at the end only
client.subscribe("sensors/room1/#")

# Correct: Multiple + wildcards
client.subscribe("sensors/+/+/temperature")

Pitfall: Non-Unique Client ID

The Mistake: Multiple clients using the same client_id causes disconnection loops.

Why It Happens: MQTT brokers only allow one connection per client_id. When a second client connects with the same ID, the first is disconnected.

Wrong approach:

# All devices using same client ID - CAUSES DISCONNECTIONS
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="sensor")
client.connect(broker, 1883)

Correct approach:

# Unique client ID based on device identity
device_id = get_device_serial()
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=f"sensor-{device_id}")
client.connect(broker, 1883)

# Or use MAC address
mac = get_mac_address()
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=f"device-{mac}")
client.connect(broker, 1883)
Pitfall: Using QoS 2 Everywhere

The Mistake: Setting QoS 2 for all messages thinking “higher is better.”

The Impact:

  • 4x message overhead (PUBLISH + PUBREC + PUBREL + PUBCOMP vs a single PUBLISH for QoS 0)
  • Up to 4x battery drain from the additional radio transmissions
  • 2 RTTs of additional latency per message (one round trip for PUBLISH→PUBREC, one for PUBREL→PUBCOMP)
  • Unnecessary broker load storing per-session message state

The Fix:

# Temperature reading every 30 seconds - QoS 0 is fine
client.publish("sensors/temp", "24.5", qos=0)

# Motion alert - QoS 1 ensures delivery
client.publish("alerts/motion", "detected", qos=1)

# Door unlock command - QoS 2 prevents duplicates
client.publish("actuators/door/unlock", "1", qos=2)
Pitfall: Retained Message Accumulation

The Mistake: Publishing large payloads with retain=true causes broker memory exhaustion.

The Problem: Retained messages persist indefinitely until cleared. A 1MB firmware binary retained on 100 topics = 100MB broker memory.

Wrong approach:

# WRONG: Large retained payload
firmware = open("firmware.bin", "rb").read()  # 2MB
client.publish("device/firmware", firmware, retain=True)  # Memory leak!

Correct approach:

# CORRECT: Retain only a reference
metadata = json.dumps({
    "version": "2.1.0",
    "url": "https://firmware.example.com/v2.1/sensor.bin",
    "sha256": "a1b2c3d4..."
})
client.publish("device/firmware/available", metadata, retain=True)  # ~200 bytes

18.5.1 Interactive: Retained Message Growth Calculator

Estimate how retained messages accumulate on your broker over time and when cleanup becomes critical.

Pitfall: Not Handling Disconnections

The Mistake: Assuming connections stay up forever without implementing reconnection logic.

The Fix: Implement exponential backoff reconnection:

def on_disconnect(client, userdata, rc):
    if rc != 0:  # Unexpected disconnect
        backoff = 1
        max_backoff = 300

        while not connected:
            try:
                client.reconnect()
                break
            except:
                time.sleep(backoff)
                backoff = min(backoff * 2, max_backoff)
Try It: Exponential Backoff Visualizer

Explore how exponential backoff works for MQTT reconnection. Adjust the initial delay, multiplier, max backoff, and jitter to see retry timing across multiple attempts.

Pitfall: Ignoring Clean Session

The Mistake: Using clean_session=True causes missed messages after reconnection.

The Impact: When device reconnects after network loss:

  • Must re-subscribe to all topics
  • Messages published during disconnection are lost

The Fix:

# BAD: Clean session loses state
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="device-ABC123")
client.connect(broker, 1883, clean_start=True)

# GOOD: Persistent session retains subscriptions
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="device-ABC123")
client.connect(broker, 1883, clean_start=False)

Concept Matching: MQTT Terms and Definitions

Match each MQTT concept to its correct definition or use case.

18.6 Summary

Practice exercises covered:

  1. Basic pub/sub - Understanding message flow
  2. Wildcards - Efficient topic subscriptions
  3. Retained messages and LWT - Device status monitoring
  4. QoS comparison - Reliability vs overhead trade-offs

Common pitfalls to avoid:

  • Wrong wildcard syntax
  • Non-unique client IDs
  • Overusing QoS 2
  • Retained message accumulation
  • Missing reconnection logic
  • Ignoring clean session implications

18.7 What’s Next

You’ve completed the MQTT Fundamentals series. The table below shows the natural progression from here:

Chapter Focus Why Read It
MQTT Labs and Implementation ESP32 and Python hands-on projects Apply the pub/sub patterns from this chapter to real hardware and cloud brokers
MQTT Comprehensive Review Broker internals, session management, and scaling Deepen your understanding of how brokers store, route, and deliver messages at scale
CoAP Fundamentals REST-style IoT messaging over UDP Compare MQTT’s broker-centric model with CoAP’s direct request-response approach
MQTT Fundamentals MQTT chapter index and topic overview Revisit earlier chapters or jump to any MQTT concept you want to review
Application Protocols Overview Survey of IoT application-layer protocols Understand where MQTT fits among AMQP, WebSockets, DDS, and other options
Transport Protocols TCP, UDP, and DTLS as protocol carriers Understand the transport layer that MQTT and CoAP rely on