13 Pub/Sub and Topic Routing

13.1 Publishers Speak to Topics, Not People

Prove Who Receives Each Copy

Picture a school sensor posting an air-quality notice. A broker is the service that matches a named message route to interested readers. A gateway is a device or service that joins different networks or data forms. The sensor may know neither the dashboard nor the gateway, so the route and access rules must carry the promise.

Write one message with its source, topic, unit, source time, age limit, access group, saved-copy rule, and expected readers. Name who owns the route when a new reader or bridge is added.

Test an exact route, a broad wildcard, no reader, two readers, an old saved copy, a duplicate, and a reader that returns after an outage. Check that approved readers receive the right copy and that unapproved readers receive none. A match proves routing, not permission or useful action.

Keep urgent room action local if the broker is late. The shared route can inform many users, but it should not become the only path to an immediate safety step.

This opening does not choose a topic tree or delivery level. Practitioner designs names, filters, and records. Under the Hood examines match rules, saved state, queue behavior, replay, access limits, and bridge failures.

Start with a notice board. A sensor pins a note under a named topic, and anyone with the right subscription sees a copy. The sensor does not know whether the subscriber is a dashboard, an automation rule, or a gateway bridge. That makes pub/sub flexible, but it also means the topic name has to carry the contract clearly enough that the broker can route the right message to the right readers, bound fan-out, review each wildcard filter, and make replay behavior explicit.

In publish-subscribe messaging, a publisher does not name the consumer it is talking to. It publishes a message to a named topic and moves on. A broker compares that topic against the active subscription filters and delivers a copy to each subscriber whose filter matches. The publisher does not know who, if anyone, is listening, and the subscriber does not know who produced the value. That decoupling is the whole point of pub/sub — and it is also where the risk lives.

Contrast this with request-response, where the sender names the receiver and waits for a reply. Pub/sub trades that direct relationship for flexibility: any number of consumers can subscribe to the same data without the publisher changing. The cost of that flexibility is that the topic name becomes the contract. Once dashboards, automations, and bridges depend on a topic, renaming a level or changing its meaning silently breaks them, because nothing on the publisher side knows they exist.

A simple mental model has five parts: a publisher contract (topic format, payload schema, timestamp, retain and delivery intent), a topic matcher (exact topics and wildcard filters), broker state (active subscriptions, retained messages, queued messages, dead-letter routes), a subscriber contract (filter scope, maximum delivery quality, duplicate handling), and a bridge output that forwards into another protocol, queue, database, or automation.

Before deciding the “event stream”–“broker” decision, inspect “event stream” in Figure 13.1 and compare it with “Broker”. That contrast matters because Pub/sub routing is a broker contract: publishers send once, then the broker matches subscriptions and fans out bounded copies to approved consumers.

Rather than scanning Figure 13.1, use “event stream” as the start. Relate it to “Broker”, then carry “topic matcher” toward “subscriptions”. The labelled route demonstrates Pub/sub routing is a broker contract: publishers send once, then the broker matches subscriptions and fans out bounded copies to approved consumers and returns the result to the “event stream”–“broker” decision.

Start With the Broker's Job

Everyday Topic Routes

Read the Everyday Topic Routes material as a decision path rather than as isolated entries. First identify the operating condition in each entry and keep its units, timing, source, and assumed system state attached to it. Next compare the entries at the point where responsibility changes between device, gateway, network, analytic service, and operator; that hand-off is where apparently similar choices often produce different outcomes. Then follow the failure case: ask what becomes stale, delayed, unavailable, or unsafe, who detects it, and what evidence permits recovery. Finally connect the result to the chapter's running design record by naming the selected behavior, the rejected alternative, the measurement that justifies the choice, and the condition that forces a recheck. That order turns the examples or comparison into an auditable engineering argument.

• A meter publishes to plant-a/line-2/meter/mtr-17/power; an energy service subscribes to a filter and receives a copy without the meter knowing it exists.
• A new dashboard can subscribe to existing telemetry with zero changes on the device side, which is the convenience pub/sub is built for.
• "The broker delivered the message" is not the same as "only the authorized consumers received it and the topic name will not change underneath them."

Overview Knowledge Check

If you can explain why topics, not addresses, connect the two sides, you have the core idea. Continue to Practitioner for topic trees, wildcards, and delivery choices.

13.2 Design Topic Names Like an API

Read the Design Topic Names Like an API material as a decision path rather than as isolated entries. First identify the operating condition in each entry and keep its units, timing, source, and assumed system state attached to it. Next compare the entries at the point where responsibility changes between device, gateway, network, analytic service, and operator; that hand-off is where apparently similar choices often produce different outcomes. Then follow the failure case: ask what becomes stale, delayed, unavailable, or unsafe, who detects it, and what evidence permits recovery. Finally connect the result to the chapter’s running design record by naming the selected behavior, the rejected alternative, the measurement that justifies the choice, and the condition that forces a recheck. That order turns the examples or comparison into an auditable engineering argument.

Because topic names are a routing API, design them deliberately. A stable hierarchy puts the most stable routing dimensions first, so that consumers can subscribe to a meaningful slice with a simple filter. A common shape is site / area / device-role / device-id / metric-or-event, giving topics such as plant-a/line-2/meter/mtr-17/power and plant-a/line-2/valve/vlv-04/state.

Five topic-design rules keep the contract healthy:

• Routing dimensions first. Put site, area, tenant, or device role early, where consumers filter on them.
• Keep volatile values out of early levels. A value that changes often, high in the tree, breaks stable subscriptions.
• No secrets, tokens, or personal data in topic names. Topic names are routinely logged, so anything in them can leak.
• Version schemas deliberately, in the payload or a chosen topic level, never through accidental naming drift.
• Separate telemetry, command, event, and status paths, because they carry different authorization and retention rules.

Wildcard Filters and the Authorization Question

MQTT-style filters use two wildcards: + matches exactly one topic level, and # matches the remaining levels and must be the final segment of the filter. A filter that matches more topics is not automatically better; the decisive review question is whether the subscriber is allowed to receive every topic the filter can match.

Delivery Quality and Retained State

Delivery quality (QoS) and retained messages are tools, chosen per data flow rather than as a blanket setting. QoS 0 is fire-and-forget, good for high-rate telemetry where the next value soon replaces the last and the consumer tolerates loss. QoS 1 retries until acknowledged, so it can deliver duplicates and the consumer must handle them. QoS 2 adds a handshake to avoid duplication on that exchange, justified only when the extra cost pays for itself. A retained message stores the last value on a topic so a new subscriber immediately receives the current state — ideal for status or a configuration pointer, but a hazard for commands.

Practitioner Knowledge Check

If you can design a stable topic tree, scope wildcard filters to the authorization boundary, and pick delivery quality per flow, you can stop here. Continue to Under the Hood for fan-out, end-to-end delivery semantics, and retained-command risk.

13.3 What the Broker Cannot Promise Alone

The deeper layer is about the things a broker does not solve for you: scaling fan-out, end-to-end exactly-once, and the replay behavior of retained state. Each is a place where a routing design that passed a happy-path test fails under load or after a restart.

Fan-Out and Backpressure

One publication can produce many deliveries, one per matching subscriber. Fan-out is useful, but a broad subscription feeding a slow consumer can build a backlog faster than the consumer drains it. For each topic family, estimate the expected number of matching subscribers, decide what happens when one consumer is slow or disconnected, and put a bounded behavior behind it — a queue limit, a drop policy, or backpressure — rather than letting an unbounded backlog form silently.

QoS Is a Hop Behavior, Not End-to-End Exactly-Once

The most common misconception is that a high QoS guarantees each message is processed exactly once across the whole pipeline. It does not. QoS governs the delivery exchange between the broker and a client; it says nothing about a bridge that retries, a consumer that restarts after partial processing, or a database write that repeats. End-to-end exactly-once effect is an application property, built from a stable message identifier, a source identifier, a timestamp, and a consumer-side idempotency rule that recognizes a repeat and does not act on it twice. Treat QoS as one input to reliability, not the whole answer.

Retained Messages and the Replay Trap

A retained message is delivered to every new subscriber the moment it subscribes. For state — "the valve is open," "the configuration pointer is X" — that is exactly what you want. For a command it is dangerous: a retained command can execute when a device reconnects long after the command was relevant, firing a stale action with no operator intent. Retain state deliberately, and treat any retained command as a special risk that needs an explicit replay and expiry rule.

Dead Letters and Observability

Messages that are malformed, unauthorized, expired, or undeliverable need a defined destination — a dead-letter route — rather than vanishing. And a routing design is only reviewable if it is observable: keep counters for matched messages, dropped messages, retries, dead letters, queue depth, and retained-state changes, so a reviewer can see fan-out and failure behavior instead of guessing.

The Routing Review Record

Under-the-Hood Knowledge Check

At this depth, pub/sub routing is a contract you must keep reviewable as the system grows. Design stable topic names, scope filters to the authorization boundary, bound fan-out, build idempotency for exactly-once effect instead of trusting QoS, treat retained commands as a replay risk, and keep counters so a reviewer can see what the broker is actually routing.