1210  MQTT Knowledge Check

1210.1 Learning Objectives

By the end of this chapter, you will be able to:

  • Apply MQTT Knowledge: Solve real-world scenarios involving QoS selection, topic design, and session management
  • Debug Complex Systems: Analyze multi-client MQTT systems with mixed configurations
  • Compare Protocols: Determine when MQTT is optimal versus CoAP, HTTP, or AMQP
  • Design Production Systems: Make architectural decisions for scalable MQTT deployments

1210.2 Prerequisites

Required Chapters:

Estimated Time: 20 minutes

1210.3 Key Concepts Reference

  • MQTT: Message Queuing Telemetry Transport - lightweight publish-subscribe protocol
  • Broker: Central message server managing topic subscriptions and message routing
  • Publish: Send message to topic (publisher doesn’t need subscribers to exist)
  • Subscribe: Register interest in topic pattern (receives all matching messages)
  • Topic: Hierarchical message category (e.g., home/kitchen/temperature)
  • Wildcard: Single-level + or multi-level # subscriptions
  • QoS (Quality of Service): 0 (fire-and-forget), 1 (at-least-once), 2 (exactly-once)
  • Payload: Arbitrary binary data (protocol-agnostic)
  • Last Will Testament: Message sent if client disconnects unexpectedly
  • Retain: Server keeps last message to deliver to new subscribers
  • TLS/SSL: Encryption for security (port 8883)
  • Authentication: Username/password or certificate-based

1210.4 Knowledge Check Questions

Test your understanding of MQTT protocol with these scenario-based questions:

Question 1: Your IoT sensor publishes temperature to topic home/living_room/temperature with QoS 0. Occasionally, readings don’t reach the dashboard even though network is stable. What’s the cause?

Explanation: QoS 0 (“at most once”) offers no reliability - messages can be lost. Network momentary congestion, Wi-Fi interference, or broker load spikes cause packet drops without retransmission. Solution: Use QoS 1 (“at least once”) for important data. Publisher stores message until PUBACK received from broker. If PUBACK timeout (5s typical): retransmit. Broker acknowledges receipt before forwarding to subscribers. Tradeoff: QoS 1 uses 2x messages (PUBLISH + PUBACK), increases latency (+10-50ms), higher power consumption. When to use each QoS: QoS 0: High-frequency non-critical data (temp every 5s - missing one reading OK), QoS 1: Infrequent critical events (door opened, alarm triggered), QoS 2 (“exactly once”): Financial transactions, medical commands (expensive: 4-message handshake, rarely used in IoT). Battery impact example: Sensor publishes 1x/min for 1 year. QoS 0: 525,600 messages. QoS 1: 1,051,200 messages (2x). At 20mA x 50ms per message: QoS 0 = 291mAh, QoS 1 = 583mAh. Use QoS 1 selectively for critical messages only.

Question 2: Your smart home has 50 sensors. Dashboard subscribes to all using individual topics: home/sensor1/temp, home/sensor2/temp, …, home/sensor50/temp. This requires 50 SUBSCRIBE messages. How can you optimize?

Explanation: MQTT wildcards enable efficient multi-topic subscriptions. Wildcards: (1) + (single-level): Matches one topic level. home/+/temp matches home/sensor1/temp, home/living_room/temp, NOT home/bedroom/closet/temp, (2) # (multi-level): Matches zero or more levels. home/# matches ALL topics under home: home/temp, home/bedroom/light, home/bedroom/closet/humidity. Your solution: client.subscribe("home/+/temp") receives ALL sensor temperatures with 1 subscription vs 50. Benefits: (1) Reduced broker load (1 subscription entry vs 50), (2) Dynamic sensors (new sensor42 automatically included), (3) Lower memory (client tracks 1 subscription vs 50), (4) Faster connection (1 SUBSCRIBE message vs 50). Pattern examples: sensors/+/temperature - all sensor temps, home/+/+ - all devices in all rooms (2 levels), alerts/# - all alert types/severities. Best practices: Avoid overly broad subscriptions (# receives EVERYTHING - wastes bandwidth), use specific wildcards (home/bedroom/+/temp better than home/#), validate topics on reception (wildcard may receive unexpected messages). Production: Design topic hierarchy for wildcard efficiency: {location}/{device_type}/{metric} enables selective subscriptions.

Question 3: Your battery-powered sensor connects to MQTT broker, publishes data, then disconnects to save power. After reconnection, you discover missed commands sent while offline. What MQTT feature addresses this?

Explanation: Persistent session (Clean Session=0) enables offline message queuing. How it works: (1) Client connects with client_id="sensor_123" and clean_session=false, (2) Client subscribes to commands/sensor_123 (broker remembers subscription), (3) Client disconnects (sleep mode), (4) Server publishes command to commands/sensor_123 with QoS 1, (5) Broker queues message for offline sensor_123, (6) Client reconnects with same client_id, clean_session=false, (7) Broker delivers queued message immediately. Requirements: (1) Fixed client_id: Must use same ID across sessions (broker links queue to ID), (2) QoS >= 1: Only QoS 1/2 messages queued (QoS 0 discarded if client offline), (3) Subscription persistence: Subscriptions survive disconnection (no need to re-SUBSCRIBE). Tradeoffs: Broker stores queued messages (uses memory/disk - typically 100KB-10MB per client), potential message flood on reconnection (100s of queued messages delivered rapidly). Clean Session=1: Discards session state on disconnect - suitable for high-frequency sensors that don’t need commands (temperature logger). Best practices: Use persistent sessions for devices receiving commands (actuators, configuration), use clean sessions for pure publishers (sensors), set message expiry (MQTT 5.0) to prevent infinite queuing, monitor broker memory usage.

Question 4: Your MQTT sensor device reboots randomly. After reboot, subscribers must wait 30-60 seconds before receiving first sensor reading. How do you ensure subscribers get latest value immediately?

Explanation: Retained messages provide “last known value” to new subscribers instantly. Normal operation: Subscriber connects after sensor publishes -> must wait until next publish (30s) to receive data. With retained message: client.publish("home/temp", "24.5", qos=1, retain=True). Broker stores message permanently. When subscriber connects -> immediately receives retained “24.5” (even if sensor published hours ago). Use cases: (1) Device status: device/status retained “online” - dashboard always knows current state, (2) Configuration: device/config retained settings - new dashboards get current config without querying device, (3) Slow-changing data: Room temperature, door state - subscribers need current value immediately, not historical. Caution: Only ONE message retained per topic (new publish replaces old). Clearing retained message: client.publish("topic", "", retain=True) (empty payload). Why alternatives fail: (A) Faster sampling wastes power/bandwidth, (B) LWT signals disconnect, not current value, (C) QoS 2 doesn’t help subscribers that missed message. Best practice: Use retained for “state” topics (current temp, light status), don’t use for “events” (button pressed, motion detected - these are temporal, not states).

Question 5: Your smart lock uses MQTT over public internet. Security audit reveals credentials transmitted in plaintext. What’s the proper security configuration?

Explanation: MQTT over TLS (MQTTS) provides transport security. Default MQTT port 1883: Unencrypted - username, password, payload visible to network sniffers. Secure MQTT port 8883: TLS-encrypted TCP tunnel. Configuration: client.tls_set(ca_certs="ca.crt", certfile="client.crt", keyfile="client.key") enables TLS with mutual authentication. Security layers: (1) Transport encryption (TLS): Protects data in transit (prevents eavesdropping), (2) Authentication (username/password): Proves client identity to broker, (3) Client certificates: Mutual TLS (mTLS) - broker verifies client certificate, preventing impersonation, (4) Authorization (ACLs): Broker enforces which clients can publish/subscribe to which topics. Production example: broker.acl: user sensor_device topic readwrite sensors/# topic read commands/device_123. Sensor can publish to sensors/*, read commands addressed to it, cannot access other devices’ data. Why alternatives insufficient: (C) Application-layer encryption misses metadata (topic names visible), doesn’t protect credentials, (D) VPN adds latency/complexity, not always available on constrained devices. Cloud providers: AWS IoT Core, Azure IoT Hub, HiveMQ Cloud enforce TLS + certificate authentication by default. Never deploy production IoT with unencrypted MQTT.

Question 6: Your MQTT broker serves 10,000 sensors. Broker CPU constantly at 100%, message delivery delays more than 5 seconds. What’s the likely bottleneck and solution?

Explanation: MQTT broker scalability depends on architecture, QoS levels, and message patterns. 10,000 sensors: Modern brokers (Mosquitto, HiveMQ, EMQX) handle 100K-1M concurrent connections. Your issue: message throughput. Bottleneck analysis: CPU 100% suggests: (1) QoS overhead: QoS 1/2 require acknowledgment processing (CPU-intensive). 10K sensors x 1 msg/sec x QoS 1 = 20K msgs/sec (publish + puback), (2) Large messages: 10KB payloads x 10K/sec = 100MB/sec processing, (3) Complex ACLs: Authorization checks on every publish/subscribe. Solutions: (1) Broker clustering: Distribute load across multiple brokers (EMQX, VerneMQ support clustering), (2) Optimize QoS: Use QoS 0 for high-frequency non-critical data (10K->5K effectively), (3) Reduce message size: Send deltas, not full payloads (1KB vs 10KB), (4) Batch messages: Combine multiple sensor readings into single message, (5) Edge brokers: Local brokers per floor/building aggregate data before cloud broker. Capacity planning: Benchmark broker: HiveMQ Enterprise ~1M msgs/sec, Mosquitto ~200K msgs/sec (single instance). Your 10K sensors @ 1 msg/sec = 10K msgs/sec (well within limits). Investigate actual message frequency/size. Production: Use managed MQTT services (AWS IoT Core auto-scales to millions of devices), monitor broker metrics (Prometheus + Grafana), implement backpressure/rate limiting on publishers.

Question 7: Your device publishes sensor data every 10 seconds. Network is unstable with frequent 1-2 second disconnections. Keepalive is set to 60 seconds. What problem occurs?

Explanation: Keepalive interval determines disconnection detection latency. How keepalive works: Client sends PINGREQ if no messages sent in keepalive period. Broker responds PINGRESP. If broker doesn’t receive ANY message (publish or ping) within 1.5x keepalive: disconnects client. Your scenario: Keepalive=60s. Network drops for 2s during active publishing. Client doesn’t realize disconnection until keepalive timeout (60s) - continues buffering messages locally. When timeout occurs: 6 messages queued (10s x 6 = 60s), sudden burst on reconnection. Optimal keepalive: Set to 2-3x message interval. Publishing every 10s -> keepalive=30s detects disconnections within 30s maximum. AWS IoT recommendation: 30-1200 seconds depending on battery constraints. Mobile networks: 60-300s (cellular keep-alive). Tradeoffs: Short keepalive (10s): Quick disconnection detection, higher power (periodic pings), more broker load. Long keepalive (300s): Lower power, delayed disconnection detection, zombie connections. Production: Implement exponential backoff reconnection: First reconnect immediately, then 1s, 2s, 4s, 8s, max 60s. Prevents reconnection storms when broker restarts. Use MQTT 5.0 session expiry interval: Broker automatically cleans up stale sessions after timeout.

Question 8: Your temperature sensor needs to signal broker/subscribers when it unexpectedly goes offline (e.g., power failure). Which MQTT feature provides this?

Explanation: Last Will and Testament (LWT) enables graceful failure notification. Setup: During CONNECT, client specifies will message: client.will_set("devices/sensor1/status", "offline", qos=1, retain=True). Normal operation: Client publishes data, sends "online" to status topic, DISCONNECT gracefully -> LWT not sent. Unexpected disconnect: Power failure, network timeout -> broker doesn’t receive DISCONNECT -> broker publishes LWT "offline" to status topic -> subscribers notified of failure. Use cases: (1) Device availability monitoring: Dashboard shows which devices offline, (2) Automated failover: Backup sensor activates when primary goes offline, (3) Alert generation: Email/SMS when critical device loses connection. Implementation: while True: client.publish("device/status", "online", retain=True) # Heartbeat time.sleep(60) # LWT takes effect if this loop stops. LWT + retained: Combine for persistent status: LWT sets “offline” as retained message, ensuring new subscribers see current state. Clear retained on reconnection: client.publish("device/status", "online", retain=True). MQTT 5.0 enhancement: Will Delay Interval - delay LWT publication (e.g., 30s) to avoid false alarms during brief reconnections. Production: All production IoT devices should implement LWT for operational monitoring.

Question 9: You need to send urgent commands to 50 smart locks. You publish to locks/# expecting instant delivery, but locks respond slowly (5-30s latency). All locks subscribe to locks/{lock_id}/command. What’s wrong?

Explanation: MQTT wildcards are subscribe-only, not publish. You CANNOT publish to a wildcard topic. How MQTT topics work: Publisher specifies EXACT topic: locks/lock42/command. Broker matches against subscriber patterns: Subscriber subscribed to locks/lock42/command -> receives message. Subscriber subscribed to locks/+/command -> receives message. Subscriber subscribed to locks/# -> receives message. Your mistake: Publishing to locks/# (literal topic string with # characters) -> no subscribers listening to that exact string -> message goes nowhere. Correct approach: (1) Individual publishes: for lock_id in locks: client.publish(f"locks/{lock_id}/command", "unlock") - 50 messages, but guaranteed delivery. (2) Shared group topic: All locks subscribe to locks/all/commands, publish once to that topic - efficient but lacks per-lock targeting. (3) Hybrid: Group commands (locks/all/lights off) + individual commands (locks/lock42/unlock). Performance: MQTT broker handles 100K+ msgs/sec, 50 publishes complete in less than 10ms, latency from slow device response, not MQTT. Wildcard rules: + matches single level, # matches multiple levels, both SUBSCRIBE-only, $ prefix (e.g., $SYS/broker/clients) reserved for broker internal topics (non-subscribable by default).

Question 10: Your IoT application needs both CoAP (low-power sensors) and MQTT (cloud integration). How do you bridge these protocols?

Explanation: Protocol gateway bridges CoAP and MQTT by translating between request-response and publish-subscribe paradigms. Architecture: CoAP Sensors (battery-powered) <- CoAP -> Gateway <- MQTT -> Cloud Broker <- MQTT -> Applications. Gateway functions: (1) CoAP->MQTT: Sensor POST to coap://gateway/sensor/temp -> Gateway publishes to sensors/temp MQTT topic, (2) MQTT->CoAP: Application publishes command to commands/sensor1 -> Gateway converts to CoAP PUT coap://sensor1/config, (3) Observe->Subscribe: CoAP Observe on sensor -> Gateway maintains subscription, forwards updates to MQTT. Implementation: NodeJS/Python gateway, Californium (Java CoAP library) + Eclipse Paho (MQTT client). Benefits: (1) Sensors use power-efficient CoAP/UDP locally, (2) Cloud services use reliable MQTT/TCP, (3) Gateway caches sensor data (reduce sensor wake time), (4) Protocol translation invisible to both sides. Production examples: AWS IoT Greengrass (edge gateway with protocol translation), Eclipse IoT Gateway (open-source CoAP-MQTT bridge), Azure IoT Edge (custom modules). Topology mapping: RESTful CoAP resource /sensor/temp -> MQTT topic devices/{device_id}/sensor/temp, CoAP GET -> MQTT subscribe, CoAP POST -> MQTT publish, CoAP PUT -> MQTT publish with retained flag. Challenges: Mapping CoAP request-response semantics (synchronous) to MQTT pub-sub (asynchronous), handling connection state differences (stateless CoAP vs stateful MQTT).

1210.5 Understanding Check: Protocol Application

Apply your MQTT knowledge to these real-world scenarios:

Situation: You’re deploying 100 sensors to a factory network. The IT team asks which ports need to be opened in the firewall for MQTT communication.

Think about:

  1. Why does MQTT use port 1883 (not 80 like HTTP or 443 like HTTPS)?
  2. When would you use port 8883 instead of 1883?
  3. What’s the trade-off between these two ports in terms of security and performance?

Key Insight:

  • Port 1883 = unencrypted MQTT (fast, minimal overhead, ~50ms latency)
  • Port 8883 = TLS encrypted MQTT (secure, +10-20% overhead, ~65ms latency)

For factory sensors on an internal network (isolated from internet): Port 1883 may be acceptable - lower overhead, faster processing.

For cloud connectivity or sensitive data: Always use 8883 - TLS encryption prevents eavesdropping, man-in-the-middle attacks.

Verify Your Understanding:

  • Can you explain why IoT protocols often use non-standard ports rather than 80/443?
  • What security implications does each port have for industrial deployments?
  • How would firewall rules differ for internal vs cloud-connected sensors?

Situation: You’re configuring message delivery guarantees for different sensor types in a smart building.

Compare these requirements:

  • Temperature sensors: Reading every 30 seconds
  • Door lock commands: Critical security actions
  • Motion alerts: Important but occasional loss acceptable

Think about:

  1. What happens if a QoS 0 temperature reading is lost? (Next reading arrives in 30s)
  2. What happens if a QoS 0 door unlock command is lost? (Security breach - door stays locked)
  3. How does QoS 1 protect against the door lock scenario?

Key Insight:

QoS 0 (At most once): Fire-and-forget, no acknowledgment, 1x message overhead

  • Use for: Frequent replaceable data (temperature every 30s - missing one reading OK)
  • Energy: Lowest (~1.5 mJ per message)

QoS 1 (At least once): Publisher waits for PUBACK, retries if timeout

  • Use for: Critical commands (door locks, alarms) - must be delivered
  • Energy: Medium (~3 mJ per message, 2x overhead due to ACK)
  • Trade-off: Possible duplicates (handle with idempotent operations)

QoS 2 (Exactly once): 4-way handshake, no duplicates or losses

  • Use for: Financial transactions, medication dispensing
  • Energy: High (~6 mJ per message, 4x overhead)
  • Trade-off: Rarely needed in IoT due to complexity

Verify Your Understanding:

  • Can you map each QoS level to a specific use case in your project?
  • What percentage of battery life would you save using QoS 0 vs QoS 1 for a sensor sending 100 messages/day?

Situation: Your smart home has 50 devices across 5 rooms. You need to subscribe to all temperature readings without creating 50 individual subscriptions.

Topic structure:

home/living_room/temp
home/living_room/humidity
home/bedroom/temp
home/bedroom/humidity
home/kitchen/temp

Think about:

  1. How would you subscribe to ALL temperature readings from ALL rooms with a single subscription?
  2. What pattern matches only bedroom sensors (both temp and humidity)?
  3. Why can’t you publish to home/+/temp to update all temperature sensors?

Key Insight:

Wildcard subscriptions enable efficient pattern matching:

  • + (single-level): home/+/temp matches home/bedroom/temp, home/kitchen/temp
  • # (multi-level): home/bedroom/# matches ALL topics under bedroom

Important: Wildcards ONLY work for SUBSCRIBE, NOT for PUBLISH!

  • Subscribe: client.subscribe("home/+/temp") - receives all room temperatures
  • Publish: client.publish("home/+/temp", "25") - this doesn’t work! Must publish to specific topic.

Verify Your Understanding:

  • Can you design a topic hierarchy for a 3-floor office building with 20 rooms per floor?
  • How would you subscribe to all sensors on floor 2?
  • What’s the difference between home/+/temp and home/# subscriptions?

Situation: You’re choosing between MQTT and HTTP for a battery-powered environmental monitoring system with 200 sensors.

Requirements:

  • Sensors report temperature, humidity, air quality every 5 minutes
  • Central dashboard displays all sensor data in real-time
  • Battery life must exceed 2 years on coin cell (CR2032, 240mAh)

Compare approaches:

HTTP Polling:

  • Dashboard polls each sensor: GET /api/sensor/temp every 5 minutes
  • Requires TCP connection setup (3-way handshake) per request
  • ~165 bytes overhead per reading (headers + payload)

MQTT Pub/Sub:

  • Sensors publish to topics: sensors/room1/temp
  • Dashboard subscribes once to sensors/#, receives updates automatically
  • ~22 bytes overhead per reading (MQTT header + payload)
  • Persistent TCP connection (setup once, reuse for all messages)

Think about:

  1. Which protocol wastes more battery on connection setup overhead?
  2. How does the dashboard get data in each approach (pull vs push)?
  3. What happens when you add sensor #201? Does the dashboard code change?

Key Insight:

Use HTTP when:

  • Request-response pattern natural (user clicks “get temperature”)
  • Infrequent access (once per hour or less)
  • Direct browser access needed
  • Integration with existing REST APIs

Use MQTT when:

  • Many subscribers need same data (dashboard + mobile app + analytics)
  • Real-time push updates required
  • Battery-powered sensors (persistent connection amortizes setup cost)
  • Publish-subscribe decoupling valuable (add subscribers without publisher changes)

Verify Your Understanding:

  • Can you calculate the energy savings of MQTT vs HTTP for this scenario?
  • What happens to HTTP battery life if polling interval changes from 5 minutes to 30 seconds?
  • How would you architect a hybrid system using both MQTT (sensor data) and HTTP (user commands)?

1210.7 Summary

This chapter provided comprehensive practice for MQTT knowledge:

  • QoS Selection: Match QoS level to data criticality - QoS 0 for replaceable data, QoS 1 for important events, QoS 2 only for critical single-execution commands
  • Wildcard Mastery: Use + for single-level and # for multi-level matching, but only in SUBSCRIBE operations
  • Session Management: Persistent sessions (clean_session=false) enable offline message queuing; clean sessions suit pure publishers
  • Security Configuration: Production deployments require TLS (port 8883) with authentication and ACLs
  • Protocol Selection: MQTT excels for publish-subscribe patterns with many subscribers; HTTP suits request-response with infrequent access

1210.8 What’s Next

Continue your learning journey with these related chapters:

  • Compare: CoAP - Learn the alternative request-response protocol for constrained devices
  • Enterprise: AMQP Fundamentals - Understand advanced message queuing for enterprise IoT
  • Protocols: IoT Protocols Overview - Survey the full protocol landscape