1199  MQTT Topic Design and Wildcards

1199.1 MQTT Topic Design Best Practices

Topics in MQTT use a hierarchical structure with / separators. This chapter covers best practices for designing topic hierarchies that scale and support efficient wildcard subscriptions.

1199.2 Learning Objectives

By the end of this chapter, you will be able to:

  • Design Topic Hierarchies: Create scalable, maintainable topic structures
  • Use Wildcards Effectively: Apply + and # wildcards for flexible subscriptions
  • Avoid Common Mistakes: Prevent topic naming pitfalls that cause issues
  • Optimize for Performance: Balance topic depth with broker efficiency

1199.3 Topic Hierarchy Fundamentals

Graph diagram

Graph diagram
Figure 1199.1: MQTT topic hierarchy design showing location-based organization with home/living_room/temperature structure, wildcard subscription patterns using plus sign for single-level and hash for multi-level matching, and best practice rules including lowercase naming, underscore separators, hierarchical structure, avoiding leading slashes and spaces, with maximum 5-7 hierarchy levels.

Best Practices:

  • Use descriptive, hierarchical names
  • Keep topics lowercase
  • Avoid spaces (use underscores)
  • Include location/device/measurement
  • Don’t start with /
  • Avoid deep nesting (max 5-7 levels)

1199.4 Wildcard Patterns

NoteUnderstanding Wildcards
  • + (Single-level wildcard): Matches any value at a single topic level
    • home/+/temperature matches home/bedroom/temperature and home/kitchen/temperature
    • Does NOT match home/bedroom/sensor/temperature (too many levels)
  • # (Multi-level wildcard): Matches zero or more levels
    • home/# matches everything under home/
    • Must be the last character in subscription
    • home/bedroom/# matches home/bedroom/temperature and home/bedroom/sensor/data

Performance Tip: Specific subscriptions are faster than wildcards. Use home/bedroom/temperature instead of home/+/+ when possible.

1199.5 Deep Dive: Topic Design Best Practices

Designing your MQTT topic hierarchy is like designing a database schema-do it right at the start or suffer later.

Hierarchical Topic Structure:

{domain}/{location}/{device_type}/{device_id}/{metric}

Good Examples:

factory/building_a/temperature/sensor_001/celsius
factory/building_a/temperature/sensor_001/humidity
factory/building_a/conveyor/motor_012/rpm
factory/building_a/conveyor/motor_012/current
home/living_room/thermostat/nest_01/temperature
home/living_room/thermostat/nest_01/target_temp

Why This Structure Works:

  1. Wildcard Efficiency:
    • All building_a data: factory/building_a/#
    • All temperature sensors: factory/+/temperature/#
    • Specific sensor: factory/building_a/temperature/sensor_001/#
  2. ACL Mapping:
    • Sensor can only publish to its own topics: factory/building_a/temperature/sensor_001/#
    • Dashboard reads all: factory/#
    • Mobile app reads one building: factory/building_a/#
  3. Scalability:
    • Add new building: factory/building_b/... (no code changes)
    • Add new metric: factory/building_a/temperature/sensor_001/battery (subscribers see it)

Common Mistakes:

  1. Starting with /:

    BAD: /home/bedroom/temp
    GOOD: home/bedroom/temp

    Leading / creates empty root level (wastes bandwidth)

  2. Spaces and Special Characters:

    BAD: home/Living Room/temp
    BAD: home/bedroom/temp(celsius)
    GOOD: home/living_room/temp_celsius

    Use underscores, lowercase, avoid parentheses/spaces

  3. Too Flat:

    BAD: sensor_001_temp
    BAD: sensor_002_temp
    GOOD: sensors/001/temp
    GOOD: sensors/002/temp

    Can’t use wildcards efficiently

  4. Too Deep:

    BAD: company/headquarters/building/floor/room/zone/device/sensor/metric
    GOOD: hq/floor3/room_301/sensor_12/temp

    Max 5-7 levels recommended (broker performance)

  5. Mixing Commands and State:

    BAD: lights/living_room (used for both commands and state)
    GOOD: lights/living_room/command (set desired state)
    GOOD: lights/living_room/state (actual current state)

    Separate concerns for clarity

Advanced Patterns:

Command-State Pattern:

devices/{device_id}/command/set_temp  -> Controller publishes
devices/{device_id}/state/current_temp -> Device publishes
devices/{device_id}/state/target_temp  -> Device confirms

Metadata Topics:

devices/{device_id}/status     -> "online"/"offline" (LWT)
devices/{device_id}/config     -> Device configuration (retained)
devices/{device_id}/telemetry  -> Battery, signal strength, uptime

Reserved Topics:

$SYS/broker/clients/connected  -> Broker publishes system info
$SYS/broker/messages/received  -> Message statistics

Topics starting with $ are reserved for broker internal use.

Performance Considerations:

Topic Depth Wildcard Matching Speed Memory Usage
3 levels Fast (microseconds) Low
5 levels Moderate Moderate
10 levels Slow (milliseconds) High

Real-World Example: AWS IoT Core Topics:

$aws/things/{thing-name}/shadow/update         -> Device updates
$aws/things/{thing-name}/shadow/update/accepted -> Cloud confirms
$aws/things/{thing-name}/shadow/get            -> Request current state

Migration Strategy:

If you need to change topic structure in production: 1. Publish to both old and new topics (6 months) 2. Update subscribers gradually 3. Monitor old topic usage (zero activity?) 4. Deprecate old topics (send final migration notice) 5. Remove old topic publishing

Topic Naming Conventions Document:

# MQTT Topic Standards v2.1

## Format
{domain}/{location}/{device_type}/{device_id}/{metric}

## Examples
- factory/building_a/temperature/sensor_001/celsius
- home/bedroom/light/bulb_12/state

## Wildcards
- `+` matches one level: `home/+/temperature`
- `#` matches all levels: `home/bedroom/#`

## Reserved Prefixes
- `$SYS/` - Broker system topics (read-only)
- `$aws/` - AWS IoT Core (if using AWS)

## ACL Template
- Devices: write to `{domain}/{location}/{device_type}/{device_id}/#`
- Dashboards: read from `{domain}/#`

Invest time in topic design-changing it later is painful!

1199.6 Worked Example: Smart Factory Topic Design

NoteWorked Example: MQTT Topic Hierarchy Design for Smart Factory

Scenario: A manufacturing plant needs to deploy 500 sensors across 4 production lines, each with 25 machines containing 5 sensors (temperature, vibration, current, pressure, and status).

Given:

  • 4 production lines (line_a through line_d)
  • 25 machines per line (machine_001 through machine_025)
  • 5 sensors per machine (temp, vibration, current, pressure, status)
  • Dashboard needs all data; maintenance teams need per-line views; safety system needs all pressure readings
  • Expected message rate: 1 message/sensor/second = 500 messages/second

Steps:

  1. Design hierarchical topic structure:

    factory/{line}/{machine}/{sensor_type}

    Example: factory/line_a/machine_012/temperature

  2. Calculate topic string overhead:

    • Average topic length: 35 characters = 35 bytes per message
    • At 500 msg/sec: 35 x 500 = 17,500 bytes/sec = 17.5 KB/sec topic overhead
  3. Define wildcard subscription patterns:

    • Dashboard (all data): factory/# - single subscription, receives 500 msg/sec
    • Line maintenance: factory/line_a/# - receives 125 msg/sec (25 machines x 5 sensors)
    • Safety system: factory/+/+/pressure - receives 100 msg/sec (4 lines x 25 machines)
    • Single machine: factory/line_a/machine_012/# - receives 5 msg/sec
  4. Calculate broker subscription table size:

    • 500 unique topics + 4 wildcard patterns
    • Memory per subscription: ~200 bytes (topic string + metadata)
    • Total: 504 x 200 bytes = 100.8 KB subscription table

Result: Topic hierarchy enables flexible subscriptions with 17.5 KB/sec topic overhead. Dashboard uses 1 subscription instead of 500 individual subscriptions. Maintenance team subscribes to 1 wildcard pattern instead of 125 topics. Safety system monitors 100 pressure sensors with single +/+/pressure pattern.

Key Insight: Hierarchical topic design with 3-5 levels balances flexibility against broker memory. Each wildcard level adds ~50 microseconds matching time - at 500 msg/sec, the factory’s 4 wildcard subscriptions add 100 microseconds total routing delay (negligible). Flatter hierarchies (line_a_machine_012_temp) prevent wildcard use and force explicit subscription lists.

1199.7 MQTT Broker Game

Master MQTT concepts through an interactive game where you act as an MQTT broker, routing messages from publishers to the correct subscribers. See the full MQTT Labs chapter for the complete interactive game with three difficulty levels.

1199.8 Knowledge Check

Question: Your smart home has 50 sensors. Dashboard subscribes to all using individual topics: home/sensor1/temp, home/sensor2/temp, …, home/sensor50/temp. This requires 50 SUBSCRIBE messages. How can you optimize?

Explanation: MQTT wildcards enable efficient multi-topic subscriptions. Wildcards: (1) + (single-level): Matches one topic level. home/+/temp matches home/sensor1/temp, home/living_room/temp, NOT home/bedroom/closet/temp, (2) # (multi-level): Matches zero or more levels. home/# matches ALL topics under home: home/temp, home/bedroom/light, home/bedroom/closet/humidity. Your solution: client.subscribe("home/+/temp") receives ALL sensor temperatures with 1 subscription vs 50. Benefits: (1) Reduced broker load (1 subscription entry vs 50), (2) Dynamic sensors (new sensor42 automatically included), (3) Lower memory (client tracks 1 subscription vs 50), (4) Faster connection (1 SUBSCRIBE message vs 50).

1199.9 What’s Next

Continue to MQTT QoS Levels to understand the three Quality of Service levels and their impact on reliability, battery life, and performance.

1199.10 See Also