Integration & Gateways · Study deck

Practice: Message Queue Challenges

A freezer gateway loses its uplink while temperature messages keep arriving every ten seconds.

Gateway Gus is your guide for this deck.

commbridgequeue
iotclass.org

After studying this chapter

Learning objectives

You will be able to:

  • diagnose the five queue failure shapes (expiry, dead letters, duplication, ordering, backpressure) from evidence
  • distinguish at-most-once, at-least-once, and exactly-once delivery semantics
  • size a deduplication window against replay behavior, not message rate
  • choose safe circuit-breaker states and resume conditions for a stuck queue
iotclass.org

Major section

A Longer Line Does Not Fix a Stuck Message

The waiting line grows.

  • Making it longer hides the delay but does not fix the cause.
  • A larger store may help a short burst, but it cannot make unsafe replay safe.
  • In the “ingress”–“message id + ttl” decision, use the same stops to locate evidence and assign the recheck.
A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
iotclass.org

Major section

Read the Failure Route Before Replaying

The split protects later messages from a poison record, but the dead-letter queue is not a bin to forget.

  • It needs the original payload, failure reason, attempt count, and enough identity to repair or reject the message.
  • Replaying before the defect is fixed merely sends the same poison message around the loop again.
  • If the queue limit is 100 records, 20 records cannot fit.

Key terms

If the queue limit
If the queue limit is 100 records, 20 records cannot fit.

Why it matters

After the third failure, route position 37 to the dead-letter queue and allow position 38 to proceed.

A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
iotclass.org

Major section

Read the Failure Route Before Replaying (continued)

A sensor publishes one 240-byte record every 10 s, or six records each minute.

  • During a 20-minute uplink loss, the gateway receives (6 \times 20=120) records.
  • With ordered single-message processing, that record can block positions 38 through 120 if the consumer retries forever.
  • On restart, the queue may redeliver that record.
iotclass.org

Major section

Read the Failure Route Before Replaying (continued)

A screenshot of an empty queue proves none of those states by itself.

  • The lab must state whether the oldest 20 are dropped, the newest 20 are rejected, or acquisition is slowed; an unspecified overflow policy is already a failure.
  • If a counter differs, stop and explain the queue transition rather than adjusting the expected answer after the run.
  • Any unexplained remainder is a message whose fate the queue evidence cannot show.
iotclass.org

Major section

Read the Failure Route Before Replaying (continued)

The lab is complete only when normal traffic, overflow, poison-message isolation, and replay each have an observed result.

  • This challenge separates transport completion from application completion and reveals whether at-least-once behavior is safe for the chosen message.
  • End by reconciling counts: produced records equal acknowledged records plus queued, dead-lettered, and explicitly dropped records.
  • New live records and replayed old records now compete for the consumer.
iotclass.org

Major section

Summary

Queue challenges are reliability problems, not sizing problems; a bigger buffer postpones the symptom and hides whether the cause is a burst, a slow consumer, or a stuck message.

  • Expiry bounds stale work with a time-to-live, but only helps when expiry is recorded and the consumer checks freshness before applying side effects.

Why it matters

A dead-letter queue isolates messages that cannot be processed, which prevents head-of-line blocking and preserves the context needed to diagnose and safely replay them.

A concrete poison-message trace for record 37: in an ordered flow it fails with a decoder error three times and is isolated as one dead-letter entry with the original 240-byte payload unchanged. After the decoder is repaired, only record 37 is replayed with deduplication enabled; record 38 can proceed after isolation, and replay produces one accepted result with no duplicate side effect.
A concrete poison-message trace for record 37: in an ordered flow it fails with a decoder error three times and is isolated as one dead-letter entry with the original 240-byte payload unchanged. After the decoder is repaired, only record 37 is replayed with deduplication enabled; record 38 can proceed after isolation, and replay produces one accepted result with no duplicate side effect.
iotclass.org

Deck summary

Key takeaways

The waiting line grows.

  • The split protects later messages from a poison record, but the dead-letter queue is not a bin to forget.
  • A sensor publishes one 240-byte record every 10 s, or six records each minute.
  • A screenshot of an empty queue proves none of those states by itself.
  • The lab is complete only when normal traffic, overflow, poison-message isolation, and replay each have an observed result.
iotclass.org

Retrieval practice

Recall check 1 of 4

Gateway Gus says: answer from memory, then check your reasoning.

Q1A gateway queue keeps growing during an outage recovery. An engineer proposes simply increasing the maximum queue size. Why is that often the wrong first move?

AA larger queue always reorders and corrupts messages.
BExtra capacity only postpones backlog if drain rate is too low.
CQueues must remain fixed-size for messages to stay valid.
DA larger queue automatically makes consumer delivery exactly once.
Show answer

Answer: B Queue challenges are reliability problems.

iotclass.org

Retrieval practice

Recall check 2 of 4

Gateway Gus says: answer from memory, then check your reasoning.

Q2A queue's oldest-message age keeps rising while CPU and network use stay normal, and a subset of messages fail with the same validation error on every retry. What is the best next action?

ADead-letter poison messages and keep flowing.
BIncrease the queue size and keep retrying every message indefinitely
CDisable acknowledgments so the consumer can move faster
DReplay the entire queue immediately to clear the backlog
Show answer

Answer: A Repeated validation failures are poison messages causing head-of-line blocking.

iotclass.org

Retrieval practice

Recall check 3 of 4

Gateway Gus says: answer from memory, then check your reasoning.

Q3A queue consumer keeps failing. When should its circuit breaker resume normal delivery?

AImmediately when breaker opens, by replaying the full queue.
BAs soon as a timer expires, under full load.
COnly after manual restart, even if trial calls pass.
DAfter a small half-open trial succeeds.
Show answer

Answer: D Successful trial work shows recovery before the full load returns.

iotclass.org

Retrieval practice

Recall check 4 of 4

Gateway Gus says: answer from memory, then check your reasoning.

Q4An at-least-once path sizes its deduplication cache from the average message rate. Operators replay several hours of old messages after an outage and observe duplicated side effects. What is the root cause?

AAt-least-once delivery guarantees no duplicates, so the queue itself must be corrupted
BReplaying old messages is always safe, so the duplicates must come from the network
CThe fix is to make the queue larger so the originals are not evicted
DThe deduplication window is shorter than the replay horizon
Show answer

Answer: D A dedup window must cover the worst-case retry-and-replay horizon, not the average rate; otherwise late duplicates fall outside the window and are reprocessed.

iotclass.org

Print reference

Answers

Answer key.

  1. B · Queue challenges are reliability problems.
  2. A · Repeated validation failures are poison messages causing head-of-line blocking.
  3. D · Successful trial work shows recovery before the full load returns.
  4. D · A dedup window must cover the worst-case retry-and-replay horizon, not the average rate; otherwise late duplicates fall outside the window and are reprocessed.
iotclass.org