4 MQTT Packet and Broker Features

Concept Check

This section: Concept Check

Concept Relationships

This section: Concept Relationships — Check current vendor documentation and benchmark all candidates against the same connection, fan-out, persistence and failover workload.

Advanced MQTT features connect to both protocol internals and system architecture:

MQTT Protocol Layers:

• MQTT Publish-Subscribe Basics - Basic pub-sub concepts.
• MQTT QoS Levels - Understanding packet acknowledgments.
• MQTT Security - TLS encryption overhead.
• MQTT Architecture - Broker design patterns.

Advanced Features:

• MQTT 5.0 Specification - Official standard.
• Shared Subscriptions - Load balancing pattern.
• Message Expiry - Time-to-live for stale data.
• Topic Aliases - Bandwidth optimization.

Broker Technologies:

• Mosquitto - Single-node, self-managed broker often used for learning and smaller deployments.
• EMQX - Distributed clustering; its published 10M+ connection result is a cluster benchmark, not a per-node guarantee.
• HiveMQ - Enterprise broker with managed clustering and vendor-published large-fleet scale.
• VerneMQ - Distributed Erlang-based broker.

Check current vendor documentation and benchmark all candidates against the same connection, fan-out, persistence and failover workload.

System Integration:

• Message broker clustering addresses high availability.
• Load balancing distributes connections.
• Edge gateway design includes topic bridging.
• Capacity planning sizes brokers.

Prerequisites You Should Know:

• MQTT control packets have a fixed header of at least two bytes; a variable header and payload appear where the packet type calls for them.
• Variable-length encoding saves bytes for small messages.
• Broker memory depends on implementation, connection settings, subscriptions, queued messages and retained state; measure it under the target workload.
• Topic hierarchy and wildcard breadth affect message matching and delivery; benchmark the actual filters.

What This Enables:

• For a 25-byte topic, an established MQTT 5 alias saves 22 bytes on a later PUBLISH under the example assumptions.
• Design topic hierarchies that support precise wildcard filters.
• Size broker capacity for connections, throughput, memory and recovery behavior.
• Select a broker for the measured deployment scale, whether thousands or millions of devices.

See Also

This section: See Also

MQTT Protocol Internals:

• MQTT 3.1.1 Specification - Packet format reference
• MQTT 5.0 Specification - New features
• Wireshark MQTT Dissector - Packet analysis tool

Broker Comparison:

• Mosquitto Documentation - Lightweight broker
• EMQX Documentation - Distributed clustering
• HiveMQ Documentation - Enterprise features
• Broker Performance Benchmarks - Throughput comparison

Topic Design Patterns:

• MQTT Essentials: Topics - Best practices
• IoT Topic Naming Conventions - Hierarchy design
• Wildcard Subscription Patterns - Performance optimization

MQTT 5.0 Features:

• MQTT 5.0 Migration Guide - Upgrade considerations
• Shared Subscriptions Tutorial - Load balancing
• Topic Aliases Deep Dive - Bandwidth optimization

Implementation Guides:

• MQTT Hands-On Labs - ESP32 hands-on projects
• Broker Clustering Guide - High availability
• EMQX Cost Estimator - Workload-based planning tool

Try It Yourself

This section: Try It Yourself — Compare the time to submit 1,000 exact subscriptions with one wildcard filter for the same set of topic names.

Experiment 1: MQTT Packet Structure Analysis

Capture and analyze MQTT packets with Wireshark:

# Install mosquitto broker and clients
sudo apt install mosquitto mosquitto-clients

# Start Wireshark with MQTT filter
wireshark -f "tcp port 1883" -k

# In another terminal, publish a message
mosquitto_pub -h localhost -t "test/topic" -m "Hello MQTT" -q 1

What to Observe:

• Fixed header: 2 bytes (0x32 for PUBLISH QoS 1)
• Variable header: topic length (2 bytes) + topic (10 bytes) + packet ID (2 bytes)
• Payload: “Hello MQTT” (10 bytes)
• Total: 26 bytes (minimal overhead!)

Experiment 2: Topic Hierarchy Performance

Compare the time to submit 1,000 exact subscriptions with one wildcard filter for the same set of topic names. This local timing does not include SUBACK completion or message-matching throughput.

import paho.mqtt.client as mqtt
import time

# The same 1,000 topic names are used in both cases.
hierarchical_topics = [f"building/floor{i//100}/room{i%100}/temp" for i in range(1000)]

# Measure local API submission time; benchmark SUBACKs and routing separately.
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.connect("localhost", 1883)
client.loop_start()

start = time.time()
for topic in hierarchical_topics:
    client.subscribe(topic, qos=0)
flat_time = time.time() - start

# Clear subscriptions
client.disconnect()
client.loop_stop()
client.connect("localhost", 1883)
client.loop_start()

start = time.time()
client.subscribe("building/+/+/temp", qos=0)  # Single wildcard subscription!
hierarchical_time = time.time() - start
client.disconnect()
client.loop_stop()

print(f"Exact (1000 subscriptions, local API time): {flat_time:.3f}s")
print(f"Wildcard (1 subscription, local API time): {hierarchical_time:.3f}s")
print(f"Local API submission-time ratio: {flat_time/hierarchical_time:.0f}x")

What to Observe: The measured API submission time depends on the local client and host. Compare SUBACK completion and broker routing under load before drawing a broker-performance conclusion. One wildcard reduces the number of subscriptions, but can also broaden the messages delivered to a client.

Experiment 3: MQTT 5.0 Topic Aliases

Measure bandwidth savings with topic aliases (requires MQTT 5.0 broker):

from paho.mqtt.client import CallbackAPIVersion, Client as MQTTClient, MQTTv5
from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes
from threading import Event

ready = Event()
def on_connect(client, userdata, flags, reason_code, properties):
    if reason_code == 0 and getattr(properties, "TopicAliasMaximum", 0) >= 1:
        ready.set()

client = MQTTClient(callback_api_version=CallbackAPIVersion.VERSION2, protocol=MQTTv5)
client.on_connect = on_connect
client.connect("localhost", 1883)
client.loop_start()
if not ready.wait(5):
    raise RuntimeError("Broker did not accept a connection with Topic Alias Maximum >= 1")

# First message: establish alias
long_topic = "farm/northfield/zone1/row12/plant45/soil/moisture"
props = Properties(PacketTypes.PUBLISH)
props.TopicAlias = 1

client.publish(long_topic, "25.5", properties=props).wait_for_publish()
print(f"First message: {len(long_topic)} byte topic")

# Subsequent messages: use alias (empty topic)
for i in range(100):
    props = Properties(PacketTypes.PUBLISH)
    props.TopicAlias = 1
    client.publish("", f"2{i}.{i}", properties=props).wait_for_publish()  # Empty topic, uses alias!

client.disconnect()
client.loop_stop()

savings = (len(long_topic.encode("utf-8")) - 3) * 100  # Later packets vs plain PUBLISH packets
print(f"Saved {savings} bytes with topic alias")

What to Observe:

• Topic name: 49 bytes
• 100 later messages: saves (49 - 3) × 100 = 4,600 bytes
• Relevant to MQTT 5 over byte-metered TCP/IP links; the savings depend on the actual link and negotiated alias support

Challenge: Broker Capacity Planning

Calculate broker requirements for a smart city deployment:

Given:
- 100,000 streetlights
- Publish every 5 minutes (12 msg/hour each)
- 3 subscribers per message (dashboard, analytics, alerts)
- Average message: 120 bytes

Calculate:
1. Messages per hour
2. Broker fan-out factor
3. Required bandwidth
4. RAM for connections (assume 20 KB per connection)
5. Select appropriate broker (Mosquitto, EMQX, or HiveMQ)

Bonus: Build your own capacity planning calculator!