AMQP · Study deck

AMQP Reliability: Acknowledgements and Crash Recovery

AMQP routes messages through a broker, the service that sends them on.

Broker Bex is your guide for this deck.

implmisconceptions

Broker Bex, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

Compare auto-ack and manual-ack behavior after a consumer crash.
Route unroutable messages with alternate exchanges and wildcard bindings.
Explain why a publisher confirm does not prove that a message reached a queue.
Detect unroutable publishes with mandatory=true, basic.return, and alternate exchanges.

Major section

Interactive Calculator: Auto-ack vs Manual-ack Crash Simulator

This simplified model has one worker processing message k when its connection closes.

It assumes earlier messages were acknowledged; with auto-ack, a retained queue holds later messages.
With auto-ack RabbitMQ applies no prefetch limit, so a crash loses every message already pushed to the consumer; the simulator shows the minimum loss of one.
Manual ack without a prefetch limit can leave every remaining delivery unacknowledged; prefetch caps the number waiting with the worker.

Try it: Interactive Calculator: Auto-ack vs Manual-ack Crash Simulator in the chapter

Major section

Unroutable and Wildcard Contracts

Advanced Message Queuing Protocol, or AMQP, is a protocol: a shared set of rules for moving messages.

A broker is the service that accepts and routes them.
Telemetry means readings and status sent from equipment for remote use.
The wider path can report and coordinate, but silent loss must never look like success.
The confirm branch remains separate.

AMQP 0-9-1 evaluates every binding independently, so one publish may reach zero, one, or several distinct queues. Publisher confirms report broker acceptance; alternate-exchange and mandatory-return behaviour determine how a final zero-route outcome becomes visible.

Major section

Unroutable and Wildcard Contracts (continued)

Message Queuing Telemetry Transport, or MQTT, is another publish-and-subscribe protocol, but its wildcard marks do not mean the same thing as AMQP marks.

This opening does not choose every queue policy.
Practitioner configures returns and alternate routes.
Many AMQP surprises are not crashes — they are messages that vanish with no error at all.

Major section

Unroutable and Wildcard Contracts (continued)

In both cases the publish can appear successful while data never becomes available to a consumer.

Under the Hood examines binding words, wildcard limits, storage, confirms, replies, duplicates, and the evidence that catches a silent drop.
The broker may accept a message, find no matching queue, and drop it unless mandatory returns, alternate exchanges, and wildcard tests make that failure visible.
A clean confirm without routing evidence is not proof that a consumer can receive the message.

Major section

Unroutable and Wildcard Contracts (continued)

If the analytics queue is bound to sensor.*, every four-word alert misses the queue even though the producer receives a confirm.

This page tightens one operational contract inside those pitfalls: a publish can be accepted by the broker and still reach no queue unless unroutable handling and routing-pattern tests are deliberate.
At 2,400 alerts/hour, that is 19,200 missing records across one shift.
Confirms alone will happily ack data straight into the void.

Major section

Unroutable and Wildcard Contracts (continued)

Two common sources of "the broker is eating my messages" incidents are a message that reaches no queue and a topic binding pattern that does not match the routing key the developer assumed it would.

With a catch-all alternate exchange, the broker extends the routing chain and captures the records instead; a message successfully routed by that alternate counts as routed for the mandatory flag.
If the answer is only "publisher confirms are on", the design is still missing part of the failure story.
To detect unroutable messages you must combine confirms with the mandatory flag (or an alternate exchange).

Major section

Unroutable and Wildcard Contracts (continued)

When an exchange receives a message that matches no binding, its default behavior is to discard it silently.

The misconception that bites hardest: a publisher confirm does not mean "routed to a queue".
A practical production check uses two intentionally bad routes before launch.
If you want "everything under sensor" you need sensor.#.

Major section

Unroutable and Wildcard Contracts (continued)

The most common bug is expecting to span multiple levels the way a shell glob would.

If the alternate chain also reaches no queue, mandatory=true makes that final zero-route result visible as basic.return.
Publisher confirm remains a separate acceptance signal; for a finally unroutable mandatory publish, the return is delivered before the confirm acknowledgement.
A single publish can match zero queues, one queue, or several queues; it is not a first-match router.

Major section

Unroutable and Wildcard Contracts (continued)

If the plant publishes 40 messages/s and a bad release misroutes 10% for 15 minutes, the alternate queue needs room for 40 x 0.10 x 900 = 3,600 messages plus margin.

Topic routing keys are dot-separated words, and the two wildcards mean precise, different things: matches exactly one word; # matches zero or more words.
This also differs from MQTT, whose wildcards are + (one level) and # (rest of the tree, and only as the final token).
With bindings sensor.#, sensor.*.temp, and sensor.floor1.#, the key sensor.floor1.temp routes to all three queues.

Deck summary

Key takeaways

This simplified model has one worker processing message k when its connection closes.

Advanced Message Queuing Protocol, or AMQP, is a protocol: a shared set of rules for moving messages.
Message Queuing Telemetry Transport, or MQTT, is another publish-and-subscribe protocol, but its wildcard marks do not mean the same thing as AMQP marks.
In both cases the publish can appear successful while data never becomes available to a consumer.
If the analytics queue is bound to sensor.*, every four-word alert misses the queue even though the producer receives a confirm.

Retrieval practice

Recall check 1 of 4

Broker Bex says: answer from memory, then check your reasoning.

Q1What queue property lets its definition survive a broker restart and retain persistent messages?

APublisher delivery_mode=2

BDurable exchange

CConsumer manual ACK

DDurable queue

Show answer

Answer: D A durable queue definition survives restart and can retain persistent messages routed to it.

Q2Fix the common AMQP misconception: configure BOTH a durable queue and persistent delivery for restart recovery (assuming the route and publisher confirms are configured):

Achannel.queue_declare(queue='critical_alerts', durable=True)

Bchannel.queue_declare(queue='critical_alerts', durable=False)

Cchannel.queue_declare(queue='critical_alerts', persistent=True)

Dchannel.queue_declare(queue='critical_alerts', auto_delete=False)

Show answer

Answer: A For a routed message in a durable classic queue, durable=True on the queue and delivery_mode=2 on the message are both needed for restart recovery.

Retrieval practice

Recall check 2 of 4

Broker Bex says: answer from memory, then check your reasoning.

Q3Per this chapter's Overview section, in the packaging-line example, what happens to an alert with routing key sensor.floor1.temperature.critical when the analytics queue is bound with pattern sensor.*, even though the producer receives a publisher confirm?

AThe alert is queued twice, once for the confirm and once for the actual delivery

BThe publisher confirm fails, so the application immediately knows to retry

CThe broker automatically widens the binding pattern to sensor.# to prevent message loss

DAlert misses queue; confirm means broker acceptance, not routing.

Show answer

Answer: D see answers page

Retrieval practice

Recall check 3 of 4

Broker Bex says: answer from memory, then check your reasoning.

Q4Per this chapter's Practitioner section, if a plant publishes 40 messages/second and a bad release misroutes 10% of them for 15 minutes, roughly how many messages should the alternate-exchange catch-all queue be sized to hold (plus margin)?

AAbout 600 messages (40 x 15, treating the 15 minutes as 15 seconds)

BAbout 36,000 messages (40 x 900 seconds of publishing)

CAbout 400 messages (40 x 10, treating 10% as a flat count of 10)

DAbout 3,600 messages (40 x 0.10 x 900 seconds)

Show answer

Answer: D The chapter's incident-buffer sizing formula: rate x misroute-fraction x duration-in-seconds.

Retrieval practice

Recall check 4 of 4

Broker Bex says: answer from memory, then check your reasoning.

Q5Which binding matches both a base route and routes with extra dot-separated words?

Asensor.*

Bsensor.#

Csensor.?

Dsensor/+

Show answer

Answer: B The hash wildcard accepts zero or more words after sensor.

Q6A topic binding change is ready for release. Which test set best detects silent drops and accidental overmatching?

AUse one known match as the routing test, because that match establishes the route for the remaining message patterns.

BBroker startup alone as proof topic bindings route selectively

CPublisher confirm alone as proof every key found its intended queue

DA matching route, a required miss and a depth-boundary route test for the binding

Show answer

Answer: D Positive, negative, and boundary routes test both intended matches and unintended misses.

Print reference

Answers 1 of 3

Answer key.

D · A durable queue definition survives restart and can retain persistent messages routed to it.
A · For a routed message in a durable classic queue, durable=True on the queue and delivery_mode=2 on the message are both needed for restart recovery.

Print reference

Answers 2 of 3

Answer key.

D · The chapter's opening point: 'Most AMQP surprises are not crashes -- they are messages that vanish with no error at all.' The packaging-line example makes this concrete -- a four-word routing key doesn't match the sensor.* binding, so the alert is silently unroutable, even though the publisher confirm still comes back successful.
D · The chapter's incident-buffer sizing formula: rate x misroute-fraction x duration-in-seconds.
B · The hash wildcard accepts zero or more words after sensor.

AMQP Reliability: Acknowledgements and Crash Recovery

Learning objectives

Interactive Calculator: Auto-ack vs Manual-ack Crash Simulator

Unroutable and Wildcard Contracts

Unroutable and Wildcard Contracts (continued)

Unroutable and Wildcard Contracts (continued)

Unroutable and Wildcard Contracts (continued)

Unroutable and Wildcard Contracts (continued)

Unroutable and Wildcard Contracts (continued)

Unroutable and Wildcard Contracts (continued)

Unroutable and Wildcard Contracts (continued)

Key takeaways

Recall check 1 of 4

Recall check 2 of 4

Recall check 3 of 4

Recall check 4 of 4

Answers 1 of 3

Answers 2 of 3

Answers 3 of 3