Integration & Gateways · Study deck

Practice: Message Queue Challenges

A freezer gateway loses its uplink while temperature messages keep arriving every ten seconds.

Gateway Gus is your guide for this deck.

commbridgequeue
iotclass.org

After studying this chapter

Learning objectives

You will be able to:

  • diagnose the five queue failure shapes (expiry, dead letters, duplication, ordering, backpressure) from evidence
  • distinguish at-most-once, at-least-once, and exactly-once delivery semantics
  • size a deduplication window against replay behavior, not message rate
  • choose safe circuit-breaker states and resume conditions for a stuck queue
iotclass.org

Major section

A Longer Line Does Not Fix a Stuck Message

The waiting line grows.

  • Making it longer hides the delay but does not fix the cause.
  • A larger store may help a short burst, but it cannot make unsafe replay safe.
  • In the “ingress”–“message id + ttl” decision, use the same stops to locate evidence and assign the recheck.
A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
iotclass.org

Major section

Read the Failure Route Before Replaying

The split protects later messages from a poison record, but the dead-letter queue is not a bin to forget.

  • It needs the original payload, failure reason, attempt count, and enough identity to repair or reject the message.
  • Replaying before the defect is fixed merely sends the same poison message around the loop again.

Key terms

If the queue limit
If the queue limit is 100 records, 20 records cannot fit.

Why it matters

After the third failure, route position 37 to the dead-letter queue and allow position 38 to proceed.

A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
A growing queue is a symptom, not a diagnosis: route each failure explicitly — expire stale work on its TTL, dead-letter poison messages, deduplicate repeats, scale while keeping order, and apply backpressure — instead of hiding the problem behind more depth.
iotclass.org

Major section

Read the Failure Route Before Replaying (continued)

A sensor publishes one 240-byte record every 10 s, or six records each minute.

  • During a 20-minute uplink loss, the gateway receives (6 \times 20=120) records.
  • If the queue limit is 100 records, 20 records cannot fit.
  • A screenshot of an empty queue proves none of those states by itself.
iotclass.org

Major section

Read the Failure Route Before Replaying (continued)

The lab must state whether the oldest 20 are dropped, the newest 20 are rejected, or acquisition is slowed; an unspecified overflow policy is already a failure.

  • With ordered single-message processing, that record can block positions 38 through 120 if the consumer retries forever.
  • If a counter differs, stop and explain the queue transition rather than adjusting the expected answer after the run.
  • On restart, the queue may redeliver that record.
iotclass.org

Major section

Read the Failure Route Before Replaying (continued)

The lab is complete only when normal traffic, overflow, poison-message isolation, and replay each have an observed result.

  • This challenge separates transport completion from application completion and reveals whether at-least-once behavior is safe for the chosen message.
  • End by reconciling counts: produced records equal acknowledged records plus queued, dead-lettered, and explicitly dropped records.
  • Any unexplained remainder is a message whose fate the queue evidence cannot show.
iotclass.org

Major section

Summary

Consumer groups raise throughput but require per-key routing to preserve ordering, and retries must not let later messages overtake a failed earlier one for the same key.

  • Queue challenges are reliability problems, not sizing problems; a bigger buffer postpones the symptom and hides whether the cause is a burst, a slow consumer, or a stuck message.
  • Expiry bounds stale work with a time-to-live, but only helps when expiry is recorded and the consumer checks freshness before applying side effects.
  • A dead-letter queue isolates messages that cannot be processed, which prevents head-of-line blocking and preserves the context needed to diagnose and safely replay them.
iotclass.org

Deck summary

Key takeaways

The waiting line grows.

  • The split protects later messages from a poison record, but the dead-letter queue is not a bin to forget.
  • A sensor publishes one 240-byte record every 10 s, or six records each minute.
  • The lab must state whether the oldest 20 are dropped, the newest 20 are rejected, or acquisition is slowed; an unspecified overflow policy is already a failure.
  • The lab is complete only when normal traffic, overflow, poison-message isolation, and replay each have an observed result.
iotclass.org

Retrieval practice

Recall check 1 of 3

Gateway Gus says: answer from memory, then check your reasoning.

Q1A gateway queue keeps growing during an outage recovery. An engineer proposes simply increasing the maximum queue size. Why is that often the wrong first move?

ACapacity only postpones the symptom.
BA larger queue is guaranteed to reorder messages and corrupt data
CQueues must always stay the same size to remain valid
DIncreasing the size will switch the consumer to exactly-once delivery automatically
Show answer

Answer: A Queue challenges are reliability problems.

iotclass.org

Retrieval practice

Recall check 2 of 3

Gateway Gus says: answer from memory, then check your reasoning.

Q2A queue's oldest-message age keeps rising while CPU and network use stay normal, and a subset of messages fail with the same validation error on every retry. What is the best next action?

AIncrease the queue size and keep retrying every message indefinitely
BDead-letter poison messages and keep flowing.
CDisable acknowledgments so the consumer can move faster
DReplay the entire queue immediately to clear the backlog
Show answer

Answer: B Repeated validation failures are poison messages causing head-of-line blocking.

iotclass.org

Retrieval practice

Recall check 3 of 3

Gateway Gus says: answer from memory, then check your reasoning.

Q3An at-least-once path sizes its deduplication cache from the average message rate. Operators replay several hours of old messages after an outage and observe duplicated side effects. What is the root cause?

AThe deduplication window is shorter than the replay horizon
BAt-least-once delivery guarantees no duplicates, so the queue itself must be corrupted
CReplaying old messages is always safe, so the duplicates must come from the network
DThe fix is to make the queue larger so the originals are not evicted
Show answer

Answer: A A dedup window must cover the worst-case retry-and-replay horizon, not the average rate; otherwise late duplicates fall outside the window and are reprocessed.

iotclass.org

Print reference

Answers

Answer key.

  1. A · Queue challenges are reliability problems.
  2. B · Repeated validation failures are poison messages causing head-of-line blocking.
  3. A · A dedup window must cover the worst-case retry-and-replay horizon, not the average rate; otherwise late duplicates fall outside the window and are reprocessed.
iotclass.org