Chapters

MQTT Production Operations: Reference

From a Test Message to a Running Service

A test message shows that one path works once. A building with 4,000 devices needs more evidence. Its team must know how much work the broker can carry, who may use each topic, and what happens when a node fails.

An MQTT broker receives messages and routes them to clients whose subscriptions match. Quality of service, or QoS, sets the delivery promise on each MQTT leg. A leg is one link, such as publisher to broker. That promise does not prove that a device took an action.

Start with the message promise. Then count the workload and test recovery. The building worksheet below starts at 440 publications and sensor deliveries per second. That number is an input to a load test. The second node still needs a failure test before the team can rely on it.

Start With Release Evidence

Choose the protocol version first. Then find the setting and the evidence needed to rely on it. A protocol rule comes from the standard. A broker policy depends on the implementation. A worksheet uses stated assumptions.

NeedMQTT 3.1.1MQTT 5.0
Resume a sessionClean Session 0; stable client IDClean Start 0; surviving session; positive Session Expiry for storage after disconnect
Start a new session nowClean Session 1Clean Start 1; choose Session Expiry separately
Discard session after disconnectClean Session 1Session Expiry 0, including when omitted
Expire an undelivered messageNo protocol expiry propertyMessage Expiry, in seconds; separate from Session Expiry
Limit in-flight publicationsBroker/client policyReceive Maximum for unacknowledged QoS 1 and 2; separate queue limits still needed
QoS, both versionsSuccessful exchange on one legUse when
0: at most oncePUBLISH; one packetA missed reading is replaceable
1: at least oncePUBLISH, PUBACK; two packetsDelivery matters and repeats are safe
2: exactly oncePUBLISH, PUBREC, PUBREL, PUBCOMP; four packetsProtocol duplicate control justifies the extra state

Application actions still need their own result evidence. Packet counts exclude retries.

Protocol sources: MQTT 3.1.1, sections 3.1.2.4, 4.1.1 and 4.3; MQTT 5.0, sections 3.1.2.4, 3.1.2.11.2, 3.3.2.3.3, 4.3 and 4.9.

DecisionLookupApproval evidence
Which QoS, filter or connection setting?Quick-reference cardTrace each MQTT leg; confirm application results separately
How much load and spare capacity?Building worksheetInclude fan-out, queues, acknowledgments and surviving-node load
Is this device allowed to act?Security configurationTest transport, identity and topic permissions separately
Will an outage be visible and recoverable?Observability boundariesControlled faults, alerts, recovered state and rollback record
Chapter Roadmap

Use the linked tables for a quick decision. Open the operational detail when collecting its evidence:

  1. Size the cluster: connections, publication rate, fan-out and recoverable state.
  2. Secure the path: TLS, client identity and topic access rules.
  3. Diagnose the load: delivery exchanges, client ID collisions and gateway mappings.
  4. Approve the release: calculator assumptions, the building worksheet, checks and a failure record.

The merged card keeps the connection and routing lookup together. The deeper worksheets retain their assumptions beside the numbers.

Learning Objectives

Use these goals to guide your release review. Each goal should lead to a setting you can check or a test you can repeat.

Use this reference to:

  • Size a cluster from connection, routing and stored-state measurements.
  • Configure security with TLS, authenticated identities and topic permissions.
  • Diagnose saturation by separating connection churn, payload traffic and acknowledgment work.
  • Choose QoS from the consequences of losing or repeating each message family.
  • Calculate a capacity plan with explicit memory, throughput and availability assumptions.
  • Prevent session failures with unique, stable client IDs and a version-specific recovery policy.

Key Concepts

Keep these terms distinct as you read. Each one names a different part of the message path or its saved state.

  • Broker: receives publications and routes them to permitted, matching subscriptions.
  • Topic and filter: the publication names a route; the subscription selects routes, with optional wildcards.
  • QoS: the protocol delivery level on one MQTT leg; it does not confirm a physical action.
  • Retained message: a topic’s stored snapshot for new matching subscriptions; it is not a history or freshness check.
  • Will: a preconfigured publication sent under the protocol’s Will conditions; it may follow unexpected connection loss or, in MQTT 5.0, a deliberate DISCONNECT requesting Will publication (reason code 0x04). See MQTT 5.0 sections 3.1.2.5 and 3.14.2.1.
  • Session: client-related state such as subscriptions and eligible queued messages. Persistence depends on the protocol settings and storage policy.

Prerequisites

Review the linked chapters if topics or delivery levels are new to you. The examples also assume some knowledge of secure links and load balancing.

Required Chapters:

Technical Background:

  • TLS/SSL concepts
  • Load balancing basics
  • Database fundamentals (Redis, PostgreSQL)

Estimated Time: 27 minutes for the full chapter; use the opening tables for a targeted lookup.

MQTT Broker Clustering Architecture

Separate connection distribution from state recovery. A load balancer can send a reconnecting client to a healthy node. That node still needs the subscriptions, queued publications and retained state required by the service contract.

In the broker-cluster figure, trace a device connection, a publication delivered through another node, and the stores needed after a failure.

Read the broker-cluster figure from the device to the receiver. The load balancer spreads connections across nodes. The brokers must also route messages and recover the saved state that the service needs. How they copy and store that state depends on the chosen broker. Stop one broker in a planned test, then check both an existing subscriber and a late subscriber. Check subscriptions, queued messages and retained messages. A publisher that reconnects has proved only one part of this path.

Diagram scope: rates round at the end: 2,778 outbound deliveries and 3,333 publication/delivery operations per second per node. The 100,000 operations/s limit is assumed; state recovery needs broker support and tests. The 53 millisecond sum does not prove the 100 millisecond service target.

Clustering Architecture Layers

Layer 1: IoT Devices (10,000+)

Device TypeRoleConnection Pattern
SensorsPublishersPeriodic data upload
ActuatorsSubscribersCommand reception
GatewaysPub/SubBidirectional

Layer 2: Load Balancer

FunctionMethod
DistributionRound Robin / Sticky Sessions
MonitoringHealth Checks
Ports1883 (TCP), 8883 (TLS)

Layer 3: MQTT Broker Cluster

NodeConnectionsInter-Node Communication
Broker Node 1~3K–4K in the 10K+ device architecture shownBroker-specific routing and state replication, if supported
Broker Node 2~3K–4K in the 10K+ device architecture shownBroker-specific routing and state replication, if supported
Broker Node 3~3K–4K in the 10K+ device architecture shownBroker-specific routing and state replication, if supported

The separate 50,000-device sizing example below assigns about 16,667 connections to each of three nodes.

Layer 4: Shared Storage

StoreTechnologyPurpose
Session StoreBroker-specific durable or replicated storePersistent sessions and subscriptions, if supported and configured
Message PersistenceBroker-specific persistenceRetained and queued messages, if supported and configured

Put the Cluster Numbers on One Node

Scenario: 3-node cluster serving 50,000 IoT devices, each publishing every 30 seconds.

Load distribution:

Devices per node=50,0003≈16,667Messages/sec per device=130≈0.0333Messages/sec per node=50,0003×30≈556 msgs/sec\begin{align} \text{Devices per node} &= \frac{50{,}000}{3} \approx 16{,}667 \\ \text{Messages/sec per device} &= \frac{1}{30} \approx 0.0333 \\ \text{Messages/sec per node} &= \frac{50{,}000}{3 \times 30} \approx 556 \text{ msgs/sec} \end{align}

With 5 subscriber deliveries per publication, rounding only the final rates:

Inbound msgs/node=50,00090≈556Outbound msgs/node=50,00090×5≈2,778 msgs/secPublication and delivery subtotal/node=50,00090×6≈3,333 msgs/sec\begin{align} \text{Inbound msgs/node} &= \frac{50{,}000}{90} \approx 556 \\ \text{Outbound msgs/node} &= \frac{50{,}000}{90} \times 5 \approx 2{,}778 \text{ msgs/sec} \\ \text{Publication and delivery subtotal/node} &= \frac{50{,}000}{90} \times 6 \approx 3{,}333 \text{ msgs/sec} \end{align}

Memory requirements (4 kB decimal per connection + queues):

Connection memory=16,667×4=66,668 kB=66.668 MBQoS queues (10 msgs avg)=16,667×10×100=16,667,000 bytes=16.667 MBSubtotal per node≈83.335 MB before broker, TLS, replication and indexes\begin{align} \text{Connection memory} &= 16{,}667 \times 4 = 66{,}668 \text{ kB} = 66.668 \text{ MB} \\ \text{QoS queues (10 msgs avg)} &= 16{,}667 \times 10 \times 100 = 16{,}667{,}000 \text{ bytes} = 16.667 \text{ MB} \\ \text{Subtotal per node} &\approx 83.335 \text{ MB before broker, TLS, replication and indexes} \end{align}

Latency budget:

Network RTT (internet)=50 msBroker processing=2 msQueue lookup (assumption)=1 msWorksheet timing sum=53 ms (compare with a 100 ms target)\begin{align} \text{Network RTT (internet)} &= 50 \text{ ms} \\ \text{Broker processing} &= 2 \text{ ms} \\ \text{Queue lookup (assumption)} &= 1 \text{ ms} \\ \text{Worksheet timing sum} &= 53 \text{ ms (compare with a 100 ms target)} \end{align}

Modeled utilization:

50,000×690×100,000≈3.3% of an assumed 100,000-operation/s node limit\frac{50{,}000 \times 6}{90 \times 100{,}000} \approx 3.3\% \text{ of an assumed 100,000-operation/s node limit}

The timing sum includes a network round trip. It is not a measured one-way delivery time or latency percentile. The rate subtotal excludes acknowledgment, retry and replication traffic. Benchmark those costs and the surviving nodes after failover.

Capacity Planning Metrics

Record limits for the actual broker, edition and configuration. A vendor’s large-scale result cannot supply an untested release limit.

MetricUnit and scopeRelease measurement
ConnectionsSimultaneous connections per nodeSteady load and a reconnect burst
Routing rateIncoming publications and outgoing deliveries per secondFan-out, QoS exchanges and inter-node traffic separately
LatencyMilliseconds, with endpoints and percentile namedPeak workload and failure recovery
MemoryBytes for connections, queues, retained state and runtimePeak storage and slow-consumer behavior
Surviving-node capacitySame workload with a node unavailableSaturation threshold and recovery time

Broker Bex, Messaging and broker domain expertCheckpoint: Cluster Sizing

Start with 50,000 devices across three nodes. Each device publishes every 30 seconds. Each publication has five deliveries. Per node, expect about 556 incoming publications and 2,778 subscriber deliveries per second. The unrounded subtotal is about 3,333 operations per second. The memory subtotal is 83.335 megabytes. It uses rounded connections and assumed queue sizes. Measure the timing and node limits before release. The 53 millisecond sum and the 100,000-operation node limit are model inputs.

Security Configuration

Treat encryption, identity and permission as separate acceptance tests. Transport Layer Security, or TLS, protects the connection. Authentication checks identity. An access-control list, or ACL, restricts the topics that identity may use.

Ports 1883 and 8883 are registered for non-TLS and TLS MQTT respectively. A port number does not configure security. Verify the listener and the connection. Plaintext MQTT exposes unencrypted credentials, topics and payloads to someone who can observe that path. MQTT 3.1.1, sections 4.2 and 5.

TLS Configuration

client.tls_set(
    ca_certs="ca.crt",
    certfile="client.crt",
    keyfile="client.key"
)

Knowledge check · self-test

Knowledge check

Test your understanding.

The snippet enables Paho TLS support and supplies a client certificate and key. Mutual TLS also requires the broker to validate the client certificate. Call tls_set before connecting and test rejection of an untrusted certificate. Paho client API.

The reconnect quiz is a configuration sketch, not a complete program. Its disconnect signature uses Paho’s version 2 callback API. The ellipsis in mqtt.Client(...) stands for the client configuration. Its reconnect bounds are one and 60 seconds; its connection uses a 60-second Keep Alive chosen by the example.

Security Layers

ControlWhat it establishesWhat to test
TLS with server verificationProtected transport to the expected serverReject an untrusted server certificate
Client authenticationAn accepted client identityReject invalid credentials
Mutual TLS, when requiredBroker validation of a client certificateReject an untrusted client certificate
Topic authorizationAllowed reads and writes for that identityPermit the assigned branch; deny another device’s branch

Access Control Lists (ACLs)

Production example:

# Mosquitto acl_file entries (the broker config points acl_file to this file)
user sensor_device
topic write sensors/device_123/#
topic read commands/device_123

These Mosquitto entries grant sensor_device writes under sensors/device_123/# and reads of commands/device_123. With no other grants, another device’s branch remains denied. Check the complete listener and authorization policy, then test both an allowed topic and a denied sibling topic. Mosquitto ACL syntax.

Why Alternatives Are Insufficient

  • Payload encryption alone: protects content, while MQTT topics and connection credentials remain outside that payload.
  • VPN alone: can protect a network tunnel, but does not define the broker’s identities or topic permissions.

For a managed service, check its supported identity methods and topic policy. For a self-managed listener, inspect the configured authentication and ACLs. In either case, keep separate evidence for encrypted transport, rejected identities and denied topics.

Performance Troubleshooting

Start with the workload and the symptoms you can measure. Use those results to choose a change, then test the same load again.

Symptom: Broker CPU at 100%, Message Delays

A saturated CPU identifies a resource limit, not its cause. Correlate CPU with connection churn, TLS handshakes, publication rate, fan-out and queued work.

  1. Count the exchanges. For 10,000 sensors publishing once per second at QoS 1, the publisher legs carry 10,000 PUBLISH and 10,000 PUBACK packets per second before retries or subscriber fan-out.
  2. Count payload bytes. A 10 kilobyte payload at 10,000 publications per second is 100 megabytes per second of payload alone, using decimal units.
  3. Inspect authorization and routing. Profile the actual topic filters and ACL checks before changing the broker topology.

Solutions

Observed costCandidate changeEvidence before adopting it
Unnecessary delivery exchangesLower QoS for replaceable dataLoss tolerance and packet/CPU comparison
Repeated large payloadsSend deltas or batch readingsRequired information survives; delay remains acceptable
Connection or routing saturationDistribute work across broker nodesMeasured limits, inter-node cost and failover behavior
Repeated upstream trafficAggregate through edge brokersLocal freshness, outage policy and upstream load

Keep the workload and configuration with every benchmark. A broker name and a throughput figure without QoS, TLS, payload, fan-out and persistence settings cannot determine capacity.

  • Monitor broker and consumer signals with an owned dashboard and alerts.
  • Bound publisher rates and stored work; test the response to a slow consumer.
  • For a managed service, verify its quotas and recovery contract for this workload.

Common Pitfalls

Two common choices can break a sound design: using the same delivery level for all data, and giving two clients the same ID. Check both before a release.

Pitfall 1: Using QoS 2 for All Messages

Choose QoS by the cost of loss and repeats. The strip compares three message needs and the packets used for each successful exchange.

Read the strip from the temperature sample to the alert and then the four-packet exchange. A later temperature sample can replace one that is lost. An alert needs a delivery policy that allows the consumer to handle repeats safely. QoS 2 adds steps to control duplicate delivery on one MQTT leg. It still needs a separate check of the action result. Compare these cases before setting one delivery level for the whole fleet.

The Mistake: Developers set QoS 2 (exactly-once delivery) for all messages, assuming higher QoS always means better reliability without considering the costs.

Why It Happens: QoS 2 sounds like the safest option, and developers don’t realize the significant overhead. The 4-way handshake (PUBLISH, PUBREC, PUBREL, PUBCOMP) seems like “extra safety” rather than a trade-off.

The Fix: Match QoS to actual requirements:

  • QoS 0 for high-frequency sensor data (temperature every 5 seconds) - missing one reading is acceptable
  • QoS 1 for important alerts and commands (door open, motion detected) - duplicates are acceptable, loss is not
  • QoS 2 when exactly-once delivery on an MQTT leg is required; use application IDs, acknowledgments and state checks for single-execution actions

Packet-count impact: A completed QoS 2 exchange uses four MQTT control packets versus one for QoS 0 and two for QoS 1. For 10,000 sensors publishing once per second, that is 40,000 versus 10,000 control packets per second on the publisher-to-broker legs, before subscriber fan-out. Actual latency and battery impact need measurement on the target hardware and network.

Use the QoS decision-tree figure to test that fix against message semantics before changing a fleet-wide default. The decision starts with what happens if one publication is lost or repeated, not with the assumption that a larger QoS number is always safer.

Read the QoS decision-tree figure from the loss question to the duplicate question. If a later periodic sample replaces a missed one, QoS 0 avoids acknowledgement traffic. If delivery matters but the consumer can make duplicate processing safe, QoS 1 supplies retransmission with a simpler exchange. Reach QoS 2 only when both loss and duplicate delivery are unacceptable and the extra handshake is justified. Actions still need IDs, result records and state checks. Its QoS 2 branch controls protocol delivery on one leg; it cannot establish single execution of an action.

Pitfall 2: Client ID Collisions

The Mistake: Provisioning two active clients with the same ID. When the second connection is accepted, the broker disconnects the existing client. A predictable name is acceptable if allocation keeps it unique; randomness alone is not a uniqueness check.

Why It Happens: In development, a single device works fine. In production with auto-scaling, containerized deployments, or device replacements, multiple instances may attempt to use the same client ID simultaneously.

The Fix: Allocate a unique ID for each simultaneously connected client, then preserve it for reconnects. These are provisioning sketches:

# Good: UUID-based client ID
import uuid
client_id = f"sensor_{uuid.uuid4().hex[:12]}"  # Use once at provisioning, then persist for reconnects

# Good: Device-specific identifier
client_id = f"sensor_{device_mac_address}_{deployment_id}"

# Bad: Sequential or predictable IDs
client_id = "sensor_1"  # Will collide with other "sensor_1" devices

The truncated UUID example has 12 hexadecimal digits, or 48 bits. Check for provisioning collisions. The device-specific example depends on the uniqueness of its inputs. The code’s “predictable” warning applies to reusing sensor_1, not to every sequential naming scheme.

Recovery risk: If both clients keep reconnecting, they can repeatedly take over the same identity. Look for:

  1. Interrupted delivery when each connection is displaced
  2. Broker log flooding with connect/disconnect events
  3. Session takeover or loss of the expected persistent-session continuity

Before releasing firmware, test that separate devices retain distinct IDs across restart and reconnect. Reuse the same ID for the same intended session; do not regenerate it at every connection.

Broker Bex, Messaging and broker domain expertCheckpoint: Secure and Efficient Operations

Test TLS, identity and topic ACLs separately. At 10,000 publications per second, a successful QoS 2 exchange uses 40,000 control packets per second on publisher legs, compared with 10,000 at QoS 0 before retries. Choose by loss and duplicate consequences. Keep each active client’s ID unique and stable across reconnects.

Protocol Bridging

A bridge joins two message systems. Define what each request means on the far side, then test that meaning when a link fails.

CoAP-MQTT Gateway

A gateway can translate a local CoAP request into an MQTT publication. The mapping is an application contract: CoAP resource operations and MQTT subscriptions have different meanings.

Example path:

  • Sensors use CoAP over UDP on the local link.
  • The gateway publishes normalized events through MQTT over TCP.
  • Applications subscribe to device updates and publish commands through the MQTT broker.

Mapping decisions to preserve:

  1. Sensor event: a POST to coap://gateway/sensor/temp becomes a publication on sensors/temp.
  2. Device configuration: a publication on commands/sensor1 becomes a PUT to coap://sensor1/config.
  3. Observed resource: the gateway observes the sensor’s CoAP resource and turns notifications into MQTT publications.

Review the boundary:

  • Define payload conversion and error handling.
  • Keep the identity and allowed topics explicit on both sides.
  • If the gateway caches readings, preserve their age.
  • Test repeated commands and loss of either network connection.
CoAP operationMQTT-side design choiceAvoid this assumption
Resource /sensor/tempA named topic such as devices/{device_id}/sensor/tempResource paths do not automatically define broker permissions
GETFetch a representation, then publish a result if requiredGET is not a standing subscription
POSTPublish the mapped eventProtocol delivery does not establish the event’s application result
PUTApply the mapped configuration and report its resultPUT does not require the MQTT retained flag
ObservePublish resource notificationsThe gateway is the CoAP observer, not an MQTT subscription on the sensor

The operation meanings come from RFC 7252, section 5.8 and RFC 7641, sections 2 and 3. The example translation is a design choice to test, not a mapping required by either standard.

Production MQTT in Plain Language

Use three questions to review the service: can it carry the work, can it recover, and can each device act only where allowed? Keep a test result for each answer.

Release questionPlain-language check
Can it carry the workload?Count connections, copies, stored work and bursts
Can it recover?Stop a node and verify the promised state and deliveries
Can this identity act?Test the allowed topic and a forbidden topic
  1. Too much work: measure the bottleneck before adding nodes.
  2. Repeated disconnects: check for two clients using one ID.
  3. Protected but overprivileged clients: test topic permissions after TLS succeeds.

Interactive Calculators

Use these models to explore a plan before a load test. Their results depend on the stated inputs and costs. Keep those assumptions with any result you share.

MQTT Broker Cluster Sizing Calculator

Use measured connection and routing limits as inputs. The result is a worksheet, not a release recommendation. It forces at least two nodes but does not prove that the remaining nodes can carry the workload after a failure.

Memory assumes four decimal kilobytes per connection, ten queued payloads per connection and 200 megabytes of runtime overhead. Bandwidth counts payload copies only. Protocol, TLS, retry and replication costs need separate measurements.

Cluster Availability Calculator

This model treats node failures as independent. It assumes instant failover. It also assumes enough capacity when any node is available. Shared power, networks and storage can break those assumptions. So can a delay in failover.

The calculator uses a 365.25-day year. A separate estimate uses a 30-minute incident to infer how many events occur. That duration is a model choice. It is not evidence of the failure rate. The main uptime result excludes the shown failover downtime. It does not promise a service level.

QoS Overhead Comparator

Compare a completed publisher-to-broker exchange using MQTT 3.1.1. This model includes the topic name and packet identifier where required. It excludes retries, subscriber fan-out, TCP and TLS. Acknowledgments carry no copy of the payload.

The default topic length is 12 bytes, the length of sensors/temp. Bandwidth uses kibibytes per second and gibibytes per day. The displayed 300 percent overhead compares packet counts, not bytes. MQTT 5.0 properties and reason codes need a different byte model. MQTT 3.1.1 packet layouts, sections 2.2.3 and 3.3 through 3.7.

MQTT Infrastructure Cost Estimator

Enter cost assumptions for nodes, compute, memory, a load balancer and durable state. These are worksheet prices, not provider quotes. The model uses 730 hours per month and twelve months per year. It excludes support, traffic, storage, backup and labor. Its defaults are separate from the building example’s cost assumptions.

Worked Example: Sizing an MQTT Broker Cluster for a Smart Building

The worksheet has 40 floors, each with 80 sensors and 20 actuators. Sensors report every 30 seconds. Assume one command per actuator every 60 seconds and three subscriber deliveries per sensor publication. The target is 99.9 percent uptime and delivery in under 200 milliseconds. These are scenario inputs, not measured results.

Step 1: Calculate Connection and Message Load

MetricCalculationResult
Device connections40 floors times 100 devices4,000 devices: 3,200 sensors and 800 actuators
Incoming sensor publications3,200 divided by 30 secondsAbout 106.67 per second
Incoming commands800 divided by 60 secondsAbout 13.33 per second
Dashboard and analytics subscribers40 floor dashboards plus one building view plus five analytics46 subscribers
Outgoing sensor deliveriesThree times 3,200 divided by 30 seconds320 per second
Publication and sensor-delivery subtotalAdd the unrounded rates440 per second

The connection subtotal excludes the dashboard, analytics and command-publisher connections. The traffic subtotal excludes command deliveries to actuators, acknowledgments, retries and inter-node traffic. Add those before using a broker limit.

Step 2: Determine Node Count

ConstraintRequired evidence
ConnectionsDevice connections plus application connections fit the measured limit
RoutingFull traffic, including the omitted copies and exchanges, fits the measured rate
Failure recoverySurviving capacity and required state meet the recovery target

A single node might carry the workload. A second node is an availability candidate only if the service can recover through it. Choose from measured limits for the selected broker, edition and settings.

Step 3: Design for 99.9% Uptime

99.9% uptime allows at most 8.76 hours of downtime in a 365-day year. If two nodes each had 99.5% availability, their failures were independent, and failover were instantaneous, the idealized parallel-availability calculation would be:

  • Cluster availability: 1 - (1 - 0.995)^2 = 1 - 0.000025 = 99.9975%
  • Downtime: 0.000025 x 365 x 24 x 60 = 13.14 minutes/year under those idealized assumptions; shared dependencies and failover time lower real availability

Architecture candidate: A redundant broker deployment with a load balancer; verify the chosen broker’s session, queue and retained-state recovery under a node-failure drill.

Step 4: Memory Sizing per Node

  • Connections per node: 4,000 / 2 = 2,000
  • Memory per connection: 4 kB decimal, an assumed session and subscription cost
  • Connection memory: 2,000 x 4 kB = 8 MB, using decimal units
  • Message queue: 2,000 x 100 x 200 bytes = 40 MB for QoS 1 with a 100-message buffer
  • Routing table: 4,000 topics x 64 bytes = 256 kB, or 0.256 MB
  • Broker overhead: ~200 MB illustrative placeholder; measure the selected broker runtime
  • Subtotal per node: 8 + 40 + 0.256 + 200 = 248.256 MB, or about 250 MB before omitted costs

Sizing starting point: Test two 1 GB nodes under TLS, queue and failover load. This device-only memory worksheet omits the application connections and broker-specific costs. It cannot prove a fourfold operational margin.

Step 5: QoS Selection by Data Type

Data TypeQoSRationale
Temperature/humidity (periodic)QoS 0 candidateA fresh reading follows in 30 seconds if the next transfer succeeds; expose staleness
Carbon dioxide threshold eventQoS 1 candidateTest duplicates, freshness and application confirmation of the alert
Occupancy countQoS 0Frequent updates, loss tolerable
HVAC desired-state commandsQoS 1 candidateRepeating “set temperature to 22 degrees Celsius” preserves the requested state; confirm the result
Fire alarm dashboard integrationQoS 1 plus retained snapshot candidateTest late-subscriber display and freshness; MQTT delivery and retention do not establish life-safety adequacy

Cost Summary:

ComponentSpecificationEstimated Cost
Two broker VMs2 vCPU, 1 GB RAM each$120/month (illustrative assumption)
Load balancer1 vCPU, 512 MB RAM$30/month (illustrative assumption)
Durable state or replicationBroker-specific design$25/month (illustrative placeholder)
Illustrative subtotalExcludes support, traffic, storage, backup and labor$175/month
Illustrative per-device subtotal$175 / 4,000 devices$0.04375/device/month

Key Insight: The assumed $175/month subtotal divided by 4,000 devices is $0.04375 per device per month before omitted costs. The 99.9975% figure is an idealized two-node calculation, not a service guarantee. QoS 0 removes the QoS 1 acknowledgment packet on each publish leg; actual CPU savings require measurement.

Broker Bex, Messaging and broker domain expertCheckpoint: Release Evidence

The building worksheet starts with 4,000 device connections. Its subtotal is 440 publications and sensor deliveries per second. Add the app connections and traffic that it leaves out. The uptime model gives 99.9975 percent. It assumes independent failures and instant failover. The cost example is 175 dollars per month, or 0.04375 dollars per device. It leaves out some costs. Check the measured load, recovered state and app results in separate tests.

The calculation companion uses a 365.25-day year and assumes one broker can carry the load. The building worksheet uses a 365-day year and still requires a capacity test. Compare the assumptions before using either result.

Maths · guided foundations

The Second Node Buys Uptime, Not Capacity

Check the quantities and assumptions.

Knowledge Check

Try a timed lookup: find the session-resume settings for MQTT 3.1.1 and MQTT 5.0, then state what happens when MQTT 5.0 omits Session Expiry. Record how long the lookup took. Explain why a surviving session still needs a freshness rule before answering the application questions.

Test Your Understanding

Use the QoS exchanges to choose a policy. A CPU percentage alone does not identify its bottleneck.

Match each MQTT production concept to its correct definition or use case:

Arrange the following steps in the correct order for onboarding a new secure MQTT device to a production cluster:

Knowledge check · self-test

Knowledge check

Test your understanding.

Label the Production Deployment

Trace the path from the device fleet through the broker to the backend. Then label who owns identity, message routing, saved state and alerts in the exercise below.

Knowledge check · self-test

Knowledge check

Test your understanding.

Scale-Out Migration Evidence

Keep a migration record that connects each design change to an observed result. Include a rollback route before moving production clients.

Migration questionEvidence
Why does one broker no longer fit?Connection, publish, retained-state, queue, CPU, memory and reconnect traces
How do consumers share work?Shared-subscription support and groups where available; consumer lag and processing limits
How does state recover?Session policy, broker-specific durable or replicated storage, retained-state recovery and a node-failure result
How do sites exchange traffic?Bridge topic allowlist, loop-prevention rule, latency budget and outage behavior
How is the change reversed?Canary observations, rollback route, alarm owner and saturation threshold

Adding nodes changes routing and recovery responsibilities. Verify the selected broker’s state contract rather than assuming that a shared database or load balancer supplies it.

Deep-Dive Note: Production Observability Boundaries

MQTT 5.0 reason codes can distinguish rejected operations such as authorization failures, quota limits and a busy server. Keep the reason code with the affected identity and operation. It explains a protocol response, not whether a device acted.

Shared subscriptions such as $share/workers/site/+/telemetry distribute matching traffic to one eligible session per shared subscription. They are standardized in MQTT 5.0; check broker support when using MQTT 3.1.1 extensions. Monitor worker lag as well as broker acceptance. MQTT 5.0, sections 4.8.2 and 4.13.

Message Expiry limits a message’s useful delivery window. Session Expiry controls stored session lifetime. Receive Maximum limits in-flight QoS 1 and QoS 2 publications per connection and direction. It does not bound offline queues, retained storage or QoS 0 traffic. Set separate storage limits and rate controls; test a slow consumer and an expired command before relying on either. MQTT 5.0, sections 3.3.2.3.3 and 4.9.

Monitor client churn, message rates, dropped messages, retained topic count, queue depth and age, authorization failures, certificate expiry and bridge reconnects. Give each alarm an owner and prove it with a controlled fault. A healthy connection count can coexist with stale data and growing queues.

Give each bridge its own client ID, TLS credentials, ACL, Keep Alive and topic prefix. Allow only the branches that belong in the destination. Test for loops, stale forwards and accidental command exposure.

If the broker exposes telemetry under $SYS, use an authorized $SYS/# subscription. A leading wildcard does not match a topic beginning with a dollar sign. The metrics tree and its permissions are implementation-specific; MQTT does not make every $SYS topic universally read-only.

Keep the reason-code trace, worker lag, expiry result, slow-consumer test, system-topic subscription, bridge policy and controlled-alert result in the acceptance record. Rehearse rollback with the same workload and state requirements.

Reference: MQTT Quick Reference Card

Use this card to look up a setting after you have chosen the message promise. Check the protocol version and the limits of the chosen broker before using a value.

MQTT Cheat Sheet

Use the card for connection and routing settings. Use the opening version table for sessions and expiry. The QoS counts describe one successful MQTT leg without retries; they do not prove application execution.

Limit or settingProtocol ruleDeployment boundary
Keep AliveClient-selected seconds; zero disables its timer. MQTT 5.0 can supply Server Keep AliveNo universal 60-second default; test reconnect separately
Remaining LengthAt most 268,435,455 bytes, excluding the fixed header, in both versionsNot a payload allowance; broker limits may be smaller
Maximum Packet SizeMQTT 5.0 receiver property; total control-packet bytesCheck each peer’s limit; MQTT 3.1.1 has no equivalent property

Sources: MQTT 3.1.1, sections 2.2.3 and 3.1.2.10; MQTT 5.0, sections 2.1.4, 3.1.2.11.4 and 3.2.2.3.14.

MQTT system-topic guardrail

A filter starting with # or + does not match a topic starting with $. If the broker exposes metrics under $SYS, subscribe explicitly to $SYS/# with an authorized identity. The dollar sign is part of that topic name, not another wildcard.

Summary

Keep a release record with these decisions and tests. Each test should show what passed and what still needs work.

  • Capacity: measure connection, routing and storage limits. Include the load after a node fails.
  • Security: test TLS and client identities. Test allowed and denied topic operations too.
  • Delivery: choose QoS for each message family. Handle repeats and check the app result.
  • Recovery: keep client IDs unique and stable. Set the session policy for the chosen version. Give messages a separate rule for freshness.
  • Gateway behavior: test how resources map to topics. Check permissions, stale data and repeated commands.

Key Takeaway

Find the versioned setting, state exactly what it promises, then test the missing boundary. A protocol receipt cannot prove device action. A capacity worksheet cannot prove recovery.

Legacy illustrations moved from the chapter

These illustrations contain lettering and were moved here whole from the chapter page.

QoS myth strip (CP-0086)

  1. For successful publisher-to-broker exchanges without retries, three device-to-broker comparisons: QoS 0 sends a temperature reading every 5 seconds with 1 control packet; a missed reading is acceptable. QoS 1 sends a door-open alert with 2 packets; repeats are acceptable, loss is not. QoS 2 uses 4 alternating packets for exactly-once delivery on this MQTT leg. A final strip says to use application IDs, acknowledgments and state checks for single-execution actions.

    Match each message's delivery level to what happens if it is lost or repeated.

Correct the belief that the highest message-delivery level is best for every message.

Delivery work strip (CP-0183)

  1. Two illustrated sensor-to-broker exchanges compare delivery choices for 10,000 sensors publishing once per second. Assume successful exchanges without retransmissions; these rates count publisher-to-broker MQTT control packets only, before subscriber fan-out. QoS 0 allows a missed reading and sends PUBLISH: one control packet per reading, or 10,000 per second. QoS 2 delivers exactly once to the broker using PUBLISH, PUBREC, PUBREL, and PUBCOMP: four control packets per reading, or 40,000 per second. The closing instructions say to use QoS 0 when missing a reading is acceptable and measure packet traffic and CPU before scaling.

    Use simpler delivery for readings that can be missed, then measure the broker's workload before adding capacity.

Four-step delivery adds protocol exchanges; profile the busy broker to test whether that work is its bottleneck.

Hi, I’m Broker Bex. What do you need?

Find help

Let’s find what you need

Find a lesson or return to your module guide.

Give feedback

Help improve this chapter

Tell us what helped or what needs fixing.

Tips

Try a small next step

A few ways to get more from this chapter.

Ask AI

Continue the conversation

Take the conversation further with your preferred assistant.

AI help is available in supported modules when enabled in Settings.

No live person answers on independent study. Feedback does not promise a personal reply.

For welfare or safety concerns, follow your school’s procedures outside IoTClass.