6 Cloud Platforms: Message Paths and Capacity

6.1 Start With the Decision

A sensor message crosses more than one cloud service before it creates value. Each hop adds capacity, delay, cost, and failure work.

6.2 Route Overview

This is part 1 of 2. Continue with Cloud Platforms: Provider Checks and Selection.

6.3 Part Objectives

• Trace cloud ingestion, processing, storage, and action paths.
• Size platform capacity without hidden traffic assumptions.

6.4 Chapter Roadmap

• Start With the Message Path
• In 60 Seconds
• Minimum Viable Understanding
• Platform Responsibility Map
• Broker, Queue, Or Stream?
• Message Lifecycle
• Platform Selection Route
• Managed or Self-Managed Platforms
• QoS And Reliability
• Capacity Records Without Assumptions

6.5 Start With the Message Path

Separate Urgent Events From Work That Can Wait

Picture a campus sending fire-door alarms, room temperatures, and monthly energy totals into one online platform. A burst of routine readings must not delay an alarm, and a lost worker must not erase the only copy of work still due.

Choose one message class at a time. Record its source, time, unit, quality, size, allowed delay, order need, repeat rule, saved period, receiver, and final action. Give urgent local safety its own path before adding any distant service.

A broker is a service that directs messages from senders to interested receivers. A queue is a waiting line for work. A stream is an ordered history that can be read again. Send a burst, stop a worker, repeat an item, change order, fill the waiting space, and restore service. Count items at every step and reconcile the final results.

An available platform does not prove a correct application. Old data, wrong identity, weak access rules, or a harmful receiver action can still produce failure.

Practitioner builds the service comparison and recovery proof. Under the Hood explains partitioning, order, duplicate work, saved state, pressure limits, identity, and how managed services divide responsibility with the customer.

Use this message-class review:

• Name the source and final receiver.
• Set the allowed wait time.
• Set the saved-work limit.
• Decide whether order matters.
• Decide whether repeats are safe.
• Count input, work, and results.
• Stop one worker during a burst.
• Fill the waiting space on purpose.
• Restore service and recount work.
• Reject a message with wrong identity.
• Show old state as old.
• Keep urgent action close to site.
• Mark work that cannot repeat.
• Give each worker one owner.
• Test a full store alarm.
• End with matched final counts.

A platform choice often starts with a messy stream of device messages: some are alarms, some are routine samples, some drive dashboards, and some feed batch analysis. Treating them all as one pipe hides the real design problem.

Start simple: write the path for one message class and name whether it needs a broker, queue, stream, storage service, or worker. The platform is a fit only when it protects that path and leaves proof that it did.

6.6 Learning Objectives

By the end of this chapter, you will be able to:

• Describe the core capabilities of an IoT cloud platform.
• Distinguish message brokers, work queues, and event streams in IoT architectures.
• Map message lifecycle stages from device publish to application action.
• Choose MQTT QoS and downstream reliability controls based on consequence, timing, and duplication tolerance.
• Build a capacity and operations record for platform selection without relying on brittle pricing or named-service assumptions.

6.7 Platform Responsibility Map

A cloud platform for IoT usually wraps several responsibilities into one operating surface. Some teams consume those capabilities as managed services. Others assemble them from self-managed components. The architecture check should focus on the responsibilities, not on brand names.

The overview depth layer shows the responsibility map that connects field-side devices and gateways to identity, ingestion, routing, observability, state, storage, and controlled return paths.

Evaluate these platform responsibilities separately:

• Device identity: how devices prove who they are, rotate credentials, and get retired.
• Ingestion: how telemetry, events, files, commands, and device-health signals enter the system.
• Routing: how messages move to alerts, storage, analytics, and operations workflows.
• State: how desired state, reported state, last-seen time, and configuration changes are tracked.
• Buffering: how bursts, retry storms, and downstream outages are absorbed.
• Retention: which events are kept, for how long, at what level of detail, and for which purpose.
• Observability: how operators see connection health, rejected messages, lag, errors, and data loss.
• Return path: how commands, configuration, model artifacts, and updates move back toward devices.

The result may be one managed platform, a self-managed broker plus shared cloud services, a hybrid site-cloud platform, or a federated design. The review record should explain why that operating shape fits the workload.

6.8 Broker, Queue, Or Stream?

IoT systems often use several message technologies at once. A device-facing broker is not the same as a processing queue, and neither is the same as a replayable event stream.

Before approving broker, queue, or stream?, test the depicted proposition in Figure 6.1: Message brokers, work queues, and event streams serve different roles in an IoT platform. Read from Broker toward dead-letter review.

The Broker label opens the diagram Figure 6.1. dead-letter review marks a different decision point, while stream preserves history prevents an early stop in broker, queue, or stream?. Together Broker and stream preserves history connect to the claim: Message brokers, work queues, and event streams serve different roles in an IoT platform.

Use a broker for connection-facing pub/sub, a queue for work distribution, and a stream for durable event history. Combining them is common: a gateway publishes to a broker, the platform routes accepted events into a stream, and processors read from queues or stream consumers.

6.9 Message Lifecycle

A message lifecycle diagram should show more than “device sends data to cloud.” It should expose the control points where errors, duplicates, routing mistakes, and visibility gaps occur.

Make the message lifecycle premise visible in the diagram Figure 6.2: A cold-storage gateway message moves through identity and topic checks, broker acknowledgment, routing rules, retry queues, retained streams, processing, storage, application action, inspection, and an approved command return path. Begin by distinguishing Cold-Storage Platform Message Lifecycle from stream lag.

Figure 6.2 becomes useful when Cold-Storage Platform Message Lifecycle is read alongside stream lag. command id, authorization, gateway ack, timeout, rollback owner adds the remaining acceptance cue. This supports message lifecycle. The visual summarizes: A cold-storage gateway message moves through identity and topic checks, broker acknowledgment, routing rules, retry queues, retained streams, processing, storage, application action, inspection, and an approved command return path.

For each message family, review:

1. Publish source: device, gateway, application, batch job, or operator action.
2. Identity check: device credential, gateway identity, application token, or service role.
3. Protocol and topic: transport, topic naming, schema, version, and payload size.
4. Acknowledgment: QoS, retry rules, duplicate detection, and timeout behavior.
5. Routing: which rules send the event to alerts, queues, streams, storage, or applications.
6. Processing: transformation, validation, enrichment, aggregation, or inference.
7. Storage: hot state, time-series history, object archive, audit record, or replay stream.
8. Response path: command, configuration, notification, dashboard update, or no return action.

6.10 Platform Selection Route

Choose the platform shape after the workload is understood. Starting with a named platform often hides the real constraints.

One relationship governs platform selection route. The diagram Figure 6.3 states it as: A platform selection route starts with cold-storage workload intent, then checks connection shape, message roles, technology roles, state retention, stress proof, ownership, portability, and final managed, self-managed, or hybrid platform fit. Study Platform Selection Route For IoT Messaging and 5. Retention.

Interrogate Platform Selection Route For IoT Messaging first, then find 5. Retention in Figure 6.3. Apply the 5. Retention review question before Record topic rules, identity assumptions, and state formats. Those answers support platform selection route; the figure states: A platform selection route starts with cold-storage workload intent, then checks connection shape, message roles, technology roles, state retention, stress proof, ownership, portability, and final managed, self-managed, or hybrid platform fit.

Use this route:

1. Name the workload. Examples: device telemetry, command delivery, firmware rollout status, gateway health, alarm workflow, replayable analytics feed.
2. Describe the connection shape. Count always-on connections separately from occasional uploads and batch transfers.
3. Classify message roles. Telemetry, command, alert, state update, audit event, configuration, model artifact, or file reference.
4. Choose broker, queue, stream, or combination. Match the technology role to the message role.
5. Define state and retention. Decide what is last-known state, what is retained history, and what is replayable record.
6. Check timing and burst behavior. Identify peak periods, reconnect storms, rollout waves, and downstream limits.
7. Define operations ownership. Who patches, scales, monitors, rotates credentials, handles incidents, and reviews cost or capacity?
8. Record portability boundaries. Identify service-specific routing rules, state formats, identity assumptions, and migration risks.

6.11 Managed or Self-Managed Platforms

The durable question is not “managed or self-hosted?” The question is “which operating responsibilities should this team own?”

6.12 QoS And Reliability

MQTT QoS is useful, but it is not the whole reliability design.

One relationship governs qos 2. The diagram Figure 6.4 states it as: MQTT QoS 0, 1, and 2 compared by their actual publish-acknowledgment handshake: QoS 0 is one-way with no acknowledgment, QoS 1 is a publish plus PUBACK round trip with a possible duplicate on resend, and QoS 2 is the four-step PUBLISH, PUBREC, PUBREL, PUBCOMP exchange. Study QoS 0 · at most once and One confirmed round trip.

The route through Figure 6.4 is figure-specific: QoS 0 · at most once states one concern, One confirmed round trip names another, and Reserve for narrow, critical cases closes the scope. That structure supports qos 2: MQTT QoS 0, 1, and 2 compared by their actual publish-acknowledgment handshake: QoS 0 is one-way with no acknowledgment, QoS 1 is a publish plus PUBACK round trip with a possible duplicate on resend, and QoS 2 is the four-step PUBLISH, PUBREC, PUBREL, PUBCOMP exchange.

Do not use QoS 2 as a blanket safety setting. Many systems are safer with QoS 1 plus idempotent processing, explicit command state, and a clear confirmation path. Reliability should be designed across the full path: device, broker, queue, processor, storage, application, and return command.

6.13 Capacity Records Without Assumptions

Capacity planning should avoid fixed product ratings and changing prices. Use workload records that can be recomputed during architecture checks.

For a message family, record:

• Connection count: always-on devices, gateway sessions, applications, and operators.
• Message rate: average rate, peak rate, burst duration, and reconnect behavior.
• Payload size: raw payload, envelope overhead, compression, and attachments.
• State size: device state documents, retained topics, session buffers, and queued commands.
• Retention: replay window, hot storage, cold archive, and deletion rules.
• Processing limits: consumer concurrency, downstream write rate, retry behavior, and dead-letter thresholds.
• Recovery: time to catch up after outage, maximum acceptable lag, and manual intervention triggers.

Use simple formulas during platform checks:

• Average messages per second = devices x messages per device per hour / 3600.
• Peak messages per second = average messages per second x burst multiplier.
• Daily payload volume = devices x messages per day x average payload bytes.
• Session memory estimate = active sessions x measured memory per session from a load test.

The formula is not the proof. The proof is a load test, replay test, outage drill, or production telemetry sample that validates the assumptions.

6.14 Continue to the Next Part

Carry this evidence into Cloud Platforms: Provider Checks and Selection, which begins with Simulation and Provider Checks.