AMQP · Study deck
AMQP Reliability: Testing and Pitfalls
Some messages are casual status updates, and some are smoke alarms.
Broker Bex is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: If the consumer's channel or connection drops, the broker requeues everything still unacked for that consumer and marks it redelivered=true so the next consumer knows it may be a repeat.
- Explain: The mistake: Using high prefetch counts (or unlimited prefetch) with multiple consumers of varying processing speeds, causing fast consumers to starve while slow consumers hoard messages they cannot process quickly.
- Explain: This page tightens one operational contract inside that reliability model: manual acknowledgments need a deliberate prefetch window, or one stalled worker can hoard unacked messages and cause large redelivery bursts.
Major section
Common Pitfall: Prefetch Starvation
The mistake: Using high prefetch counts (or unlimited prefetch) with multiple consumers of varying processing speeds, causing fast consumers to starve while slow consumers hoard messages they cannot process quickly.
- With prefetch=1000, a slow consumer can reserve up to 1000 available messages.
- While it processes them one by one, those deliveries are unavailable to faster consumers.
- The queue may have little ready depth even though many messages are unacknowledged on slow consumers.
Major section
AMQP Prefetch Reliability Contracts
It sets how many unacknowledged messages a consumer may hold before the broker gives other workers a chance.
- A large waiting set may keep the link busy while old work fills memory and delays the alarm that matters now.
- AMQP means Advanced Message Queuing Protocol.
- A broker is the service that accepts and routes messages.
Major section
AMQP Prefetch Reliability Contracts (continued)
Latency means the elapsed time from a stated start event to a stated finish event.
- This runway does not select one prefetch value for every workload.
- AMQP Reliability Patterns teaches acknowledgments, persistence, dead-lettering, queue limits, and competing consumers.
- The trap is treating it as "bigger is faster".
- Second, unacked messages are the recovery unit.
Major section
AMQP Prefetch Reliability Contracts (continued)
Reliability tuning is finding the prefetch that keeps a consumer busy without hoarding.
- This page tightens one operational contract inside that reliability model: manual acknowledgments need a deliberate prefetch window, or one stalled worker can hoard unacked messages and cause large redelivery bursts.
- Durable queues and persistent messages protect the broker side, but the consumer side has its own decisive knob: prefetch, set with basic.qos(prefetch_count=N).
- The monitoring threshold should match the prefetch policy.
Major section
AMQP Prefetch Reliability Contracts (continued)
Prefetch is the number of messages the broker will hand a consumer before it must acknowledge any of them.
- It quietly controls three things at once — throughput, fair distribution across workers, and how much work is at risk if a consumer crashes.
- Concrete example: a queue has 3,000 telemetry enrichment jobs and three consumers.
- It defines the largest batch that can be delayed or redelivered by one consumer failure.
Major section
AMQP Prefetch Reliability Contracts (continued)
With unlimited prefetch, the first fast TCP connection can receive hundreds or thousands of deliveries before the broker notices other consumers are available.
- With prefetch_count=20, the stalled worker can hold only 20 at a time, leaving the broker free to keep feeding the other two consumers.
- A good first setting is usually based on the worst acceptable redelivery burst.
- If a handler writes to an idempotent database table, a redelivery window of 20 may be harmless.
Major section
AMQP Prefetch Reliability Contracts (continued)
If each message triggers a physical device action, even five unacked commands may be too many to replay at once.
- The broker cannot infer that risk from the queue name; the application owner has to set the window deliberately.
- With prefetch=1, a consumer processes a message, sends basic.ack, and only then receives the next one.
- Worked example: Handlers take 5 ms each; the broker round trip is 20 ms.
Major section
AMQP Prefetch Reliability Contracts (continued)
If image analysis takes 500 ms and the broker round trip is 20 ms, ceil((20+500)/500) = 2 is already enough to hide network latency.
- During the network round trip it is idle, so throughput is capped at roughly one message per (processing time + round trip).
- At prefetch 1, each consumer does about one message per 25 ms (40/s).
- The picture connects fair throughput to monitoring both ready and unacked counts, not simply increasing concurrency.
Major section
AMQP Prefetch Reliability Contracts (continued)
Raising prefetch to ceil((20+5)/5) = 5 lets five messages overlap the round trip, so the consumer stays busy and approaches 200/s — a 5x gain with no change to durability or acks.
- A handler that is normally 5 ms but becomes 100 ms during database compaction should not be given a window that lets it reserve minutes of work.
- Its share of the queue is frozen even though the queue is full and other consumers may be idle.
- A large prefetch therefore means a large redelivery burst on failure.
Major section
AMQP Prefetch Reliability Contracts (continued)
If the consumer's channel or connection drops, the broker requeues everything still unacked for that consumer and marks it redelivered=true so the next consumer knows it may be a repeat.
- Operationally, a prefetch problem shows up as a mismatch between queue depth and worker activity.
- The queue may appear to drain because messages were delivered, but useful throughput stalls because they are sitting unacked inside one process.
- A healthy bounded setup has ready messages falling, unacked messages spread across consumers, and acknowledgements continuing at roughly the handler rate.
Deck summary
Key takeaways
The mistake: Using high prefetch counts (or unlimited prefetch) with multiple consumers of varying processing speeds, causing fast consumers to starve while slow consumers hoard messages they cannot process quickly.
- It sets how many unacknowledged messages a consumer may hold before the broker gives other workers a chance.
- Latency means the elapsed time from a stated start event to a stated finish event.
- Reliability tuning is finding the prefetch that keeps a consumer busy without hoarding.
- Prefetch is the number of messages the broker will hand a consumer before it must acknowledge any of them.
Retrieval practice
Recall check 1 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q1What lets a broker recover message content after restart when topology and storage also support recovery?
Show answer
Answer: B Persistence lets the broker recover message content across a restart when the durable topology and storage policy also support it.
Q2Complete the AMQP publisher with persistent message delivery:
Show answer
Answer: A Message persistence requires both a durable queue and delivery_mode=2 in BasicProperties.
Retrieval practice
Recall check 2 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q3Per this chapter's Overview section, in the 3,000-job / three-consumer example, why does setting prefetch_count=20 keep the broker able to feed the other two consumers even if one worker stalls on a database call?
Show answer
Answer: C Bounding prefetch caps how much of the queue one consumer can hold unacknowledged at once.
Retrieval practice
Recall check 3 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q4Three competing consumers share a busy queue with prefetch set to unlimited. One consumer stalls on a slow external call while holding thousands of unacked messages. What is the observed effect and the fix?
Show answer
Answer: B Unacked messages are pinned to a consumer and counted against prefetch; unbounded prefetch causes starvation and large crash redeliveries, so use a modest prefetch with manual acks.
Retrieval practice
Recall check 4 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q5A warehouse order consumer uses auto_ack=True. It crashes after receiving a $5,000 order but before recording or completing the order. What happens to that broker delivery?
Show answer
Answer: D With auto-acknowledgement the broker considers the message handled on delivery.
Retrieval practice
Recall check 5 of 5
Broker Bex says: answer from memory, then check your reasoning.
Q6A durable queue contains delivery_mode=1 (transient) messages. After broker restart, which part has a restart-survival guarantee?
Show answer
Answer: C A durable queue definition survives broker restart.
Print reference
Answers 1 of 2
Answer key.
- B · Persistence lets the broker recover message content across a restart when the durable topology and storage policy also support it.
- A · Message persistence requires both a durable queue and delivery_mode=2 in BasicProperties.
- C · Bounding prefetch caps how much of the queue one consumer can hold unacknowledged at once.
- B · Unacked messages are pinned to a consumer and counted against prefetch; unbounded prefetch causes starvation and large crash redeliveries, so use a modest prefetch with manual acks.
Print reference
Answers 2 of 2
Answer key.
- D · With auto-acknowledgement the broker considers the message handled on delivery.
- C · A durable queue definition survives broker restart.