AMQP · Study deck
AMQP Reliability: Testing and Pitfalls
Some messages are casual status updates, and some are smoke alarms.
Broker Bex is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: Raising prefetch to ceil((20+5)/5) = 5 lets five messages overlap the round trip, so the consumer stays busy and approaches 200/s — a 5x gain with no change to durability or acks.
- Explain: If the consumer's channel or connection drops, the broker requeues everything still unacked for that consumer and marks it redelivered=true so the next consumer knows it may be a repeat.
- Explain: A handler that is normally 5 ms but becomes 100 ms during database compaction should not be given a window that lets it reserve minutes of work.
Major section
Common Pitfall: Prefetch Starvation
The mistake: Using high prefetch counts (or unlimited prefetch) with multiple consumers of varying processing speeds, causing fast consumers to starve while slow consumers hoard messages they cannot process quickly.
- With prefetch=1000, a slow consumer receives 1000 messages immediately.
- While it processes them one by one, those messages are unavailable to faster consumers.
- The queue looks full, but messages are stuck in slow consumer buffers.
Major section
AMQP Prefetch Reliability Contracts
It sets how many unacknowledged messages a consumer may hold before the broker gives other workers a chance.
- A large waiting set may keep the link busy while old work fills memory and delays the alarm that matters now.
- A protocol is a shared set of rules for exchanging data.
- AMQP means Advanced Message Queuing Protocol.
Major section
AMQP Prefetch Reliability Contracts (continued)
A broker is the service that accepts and routes messages.
- Latency means the elapsed time from a stated start event to a stated finish event.
- This runway does not select one prefetch value for every workload.
- AMQP Reliability Patterns teaches acknowledgments, persistence, dead-lettering, queue limits, and competing consumers.
- Second, unacked messages are the recovery unit.
Major section
AMQP Prefetch Reliability Contracts (continued)
The trap is treating it as "bigger is faster".
- This page tightens one operational contract inside that reliability model: manual acknowledgments need a deliberate prefetch window, or one stalled worker can hoard unacked messages and cause large redelivery bursts.
- Durable queues and persistent messages protect the broker side, but the consumer side has its own decisive knob: prefetch, set with basic.qos(prefetch_count=N).
- A large prefetch therefore means a large redelivery burst on failure.
Major section
AMQP Prefetch Reliability Contracts (continued)
If a handler writes to an idempotent database table, a redelivery window of 20 may be harmless.
- Prefetch is the number of messages the broker will hand a consumer before it must acknowledge any of them.
- Reliability tuning is finding the prefetch that keeps a consumer busy without hoarding.
- The monitoring threshold should match the prefetch policy.
Major section
AMQP Prefetch Reliability Contracts (continued)
It quietly controls three things at once — throughput, fair distribution across workers, and how much work is at risk if a consumer crashes.
- Concrete example: a queue has 3,000 telemetry enrichment jobs and three consumers.
- With unlimited prefetch, the first fast TCP connection can receive hundreds or thousands of deliveries before the broker notices other consumers are available.
- It defines the largest batch that can be delayed or redelivered by one consumer failure.
Major section
AMQP Prefetch Reliability Contracts (continued)
With prefetch_count=20, the stalled worker can hold only 20 at a time, leaving the broker free to keep feeding the other two consumers.
- A good first setting is usually based on the worst acceptable redelivery burst.
- If each message triggers a physical device action, even five unacked commands may be too many to replay at once.
- The broker cannot infer that risk from the queue name; the application owner has to set the window deliberately.
Major section
AMQP Prefetch Reliability Contracts (continued)
With prefetch=1, a consumer processes a message, sends basic.ack, and only then receives the next one.
- During the network round trip it is idle, so throughput is capped at roughly one message per (processing time + round trip).
- Worked example.: Handlers take 5 ms each; the broker round trip is 20 ms.
- At prefetch 1, each consumer does about one message per 25 ms (40/s).
Major section
AMQP Prefetch Reliability Contracts (continued)
Raising prefetch to ceil((20+5)/5) = 5 lets five messages overlap the round trip, so the consumer stays busy and approaches 200/s — a 5x gain with no change to durability or acks.
- If image analysis takes 500 ms and the broker round trip is 20 ms, ceil((20+500)/500) = 2 is already enough to hide network latency.
- A handler that is normally 5 ms but becomes 100 ms during database compaction should not be given a window that lets it reserve minutes of work.
- The picture connects fair throughput to monitoring both ready and unacked counts, not simply increasing concurrency.
Major section
AMQP Prefetch Reliability Contracts (continued)
Its share of the queue is frozen even though the queue is full and other consumers may be idle.
- If the consumer's channel or connection drops, the broker requeues everything still unacked for that consumer and marks it redelivered=true so the next consumer knows it may be a repeat.
- Operationally, a prefetch problem shows up as a mismatch between queue depth and worker activity.
- The queue may appear to drain because messages were delivered, but useful throughput stalls because they are sitting unacked inside one process.
Major section
AMQP Prefetch Reliability Contracts (continued)
A healthy bounded setup has ready messages falling, unacked messages spread across consumers, and acknowledgements continuing at roughly the handler rate.
- With three consumers and prefetch 20, more than 60 unacked messages means the configuration is not what you think, or consumers are using multiple channels.
- With unlimited prefetch, the same count tells you very little until a stall has already trapped a large batch.
- If that consumer then blocks on a database call, those deliveries are invisible to the other workers because they are already assigned and unacknowledged.
Deck summary
Key takeaways
The mistake: Using high prefetch counts (or unlimited prefetch) with multiple consumers of varying processing speeds, causing fast consumers to starve while slow consumers hoard messages they cannot process quickly.
- It sets how many unacknowledged messages a consumer may hold before the broker gives other workers a chance.
- A broker is the service that accepts and routes messages.
- The trap is treating it as "bigger is faster".
- If a handler writes to an idempotent database table, a redelivery window of 20 may be harmless.
Retrieval practice
Recall check 1 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q1Place each AMQP reliability mechanism where it lives so you can tell broker receipt, durable storage, application completion, and failed-message handling apart.
Show answer
Answer: A Place each AMQP reliability mechanism where it lives so you can tell broker receipt, durable storage, application completion, and failed-message handling apart.
Q2Complete the AMQP publisher with persistent message delivery:
Show answer
Answer: A Message persistence requires both a durable queue and delivery_mode=2 in BasicProperties.
Retrieval practice
Recall check 2 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q3Per this chapter's Overview section, in the 3,000-job / three-consumer example, why does setting prefetch_count=20 keep the broker able to feed the other two consumers even if one worker stalls on a database call?
Show answer
Answer: A Bounding prefetch caps how much of the queue one consumer can hold unacknowledged at once.
Retrieval practice
Recall check 3 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q4Three competing consumers share a busy queue with prefetch set to unlimited. One consumer stalls on a slow external call while holding thousands of unacked messages. What is the observed effect and the fix?
Show answer
Answer: A Unacked messages are pinned to a consumer and counted against prefetch; unbounded prefetch causes starvation and large crash redeliveries, so use a modest prefetch with manual acks.
Retrieval practice
Recall check 4 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q5A warehouse management system uses AMQP with auto-acknowledgment (auto_ack=True) for order processing. During a peak sales event, a consumer crashes while processing an order worth $5,000. What happens to that order message?
Show answer
Answer: C With auto-acknowledgment, the broker considers the message delivered and removes it from the queue as soon as it is sent to the consumer -- before processing completes.
Retrieval practice
Recall check 5 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q6A developer configures a durable AMQP queue but sets delivery_mode=1 (transient) on messages. After a broker restart, what happens to the messages that were in the queue?
Show answer
Answer: A Durability in AMQP requires BOTH a durable queue AND persistent messages (delivery_mode=2).
Print reference
Answers 1 of 2
Answer key.
- A · Place each AMQP reliability mechanism where it lives so you can tell broker receipt, durable storage, application completion, and failed-message handling apart.
- A · Message persistence requires both a durable queue and delivery_mode=2 in BasicProperties.
- A · Bounding prefetch caps how much of the queue one consumer can hold unacknowledged at once.
- A · Unacked messages are pinned to a consumer and counted against prefetch; unbounded prefetch causes starvation and large crash redeliveries, so use a modest prefetch with manual acks.
Print reference
Answers 2 of 2
Answer key.
- C · With auto-acknowledgment, the broker considers the message delivered and removes it from the queue as soon as it is sent to the consumer -- before processing completes.
- A · Durability in AMQP requires BOTH a durable queue AND persistent messages (delivery_mode=2).