13  Retries and Sequence Numbers

transport-protocols
reliability
retry-sequencing
Keywords

IoT retry review, sequence number evidence, duplicate message handling, acknowledgement timeout review, transport reliability retest

13.1 Start With the Duplicate Message

A retry design is tested by the second copy of a message, not the first. Ask how the receiver distinguishes a valid retry from a duplicate command, how sequence state wraps, and when acknowledgements stop the loop. That story turns retry counts and sequence numbers into application safety evidence.

13.2 Overview: Retry Only Means Something With Boundaries

Retries and sequence numbers help an IoT system reason about loss, duplication, delay, and reordering. They do not prove that every message arrived or that every command was safe. They support a reliability decision only when the sender limit, receiver state, duplicate rule, and side-effect boundary are explicit.

A retry policy says when a sender tries again, when it stops, and what state remains after the limit. A sequence policy says how the receiver recognizes a new, duplicate, stale, missing, or reordered message.

For example, gateway-east sends command C-512 with sequence 181 to a valve controller and expects acknowledgement A-181 before the response window closes. If the acknowledgement is missing, a reviewable retry policy might wait a short interval, add jitter so a fleet does not retry in the same millisecond, try twice, then mark the command unresolved and leave the valve state at last-confirmed. That record is stronger than “three retries” because it names the command identity, the acknowledgement identity, the stop condition, and the state after the sender gives up.

The receiver side needs the same discipline. If sequence 181 arrives twice, the receiver may acknowledge the duplicate so the sender can stop, but it must not apply the valve command twice unless the command is explicitly idempotent. If sequence 183 arrives after 181, the receiver needs a rule for the missing 182: wait inside a reordering window, reject the gap, or flag the state as incomplete. Without those sequence rules, retries can turn a recoverable timeout into a duplicate write, stale dashboard value, or unsafe actuator action. The important evidence is the tested state transition, not the fact that a retry occurred.

Retry and sequence review loop from observed timeout or sequence gap through retry decision, acknowledgement or sequence state, final state, and retest trigger.
Retry and sequence evidence should close the loop from observation to final state.

Message Boundary

Name the telemetry report, command, queue item, gateway handoff, or state update that retry and sequencing protect.

Retry Boundary

Record timeout observation, delay behavior, retry limit, acknowledgement result, and the final state after stop or success.

Sequence Boundary

Record accepted sequence state, duplicate rule, missing-message rule, reordering window, reset behavior, and wraparound rule.

Side-Effect Boundary

Show whether repeated or late messages can change stored state, actuator state, command status, or operator decisions.

Review rule:

Do not approve retry behavior from repeated attempts alone. The evidence must show a bounded decision and the state left behind.

13.3 Practitioner: Build the Evidence Record

A practitioner should review retry and sequence behavior as a record, not as a slogan. The record starts with the message boundary and observed condition, then ties retry policy, acknowledgement handling, sequence state, side effects, final decision, and retest trigger together.

This works for TCP-based reconnect decisions, UDP paths with application acknowledgements, CoAP or MQTT message IDs, gateway queues, and application command logs. The exact protocol changes the signals available, but the record still needs bounded evidence.

Retry and sequence decision record fields for message boundary, observed condition, retry policy, acknowledgement result, sequence state, side-effect boundary, final decision, and retest trigger.
A compact record keeps retry and sequencing claims tied to the tested transport path.

Evidence Flow

1. State the boundary Identify the message, receiver expectation, state update, and observation point.
2. Record the condition Name the timeout, missing acknowledgement, duplicate, stale value, sequence gap, reorder, or reset.
3. Review the retry Capture delay behavior, jitter if used, retry limit, stop condition, and final sender state.
4. Review receiver state Capture acknowledgement identity, late response rule, accepted sequence value, duplicate rule, and gap rule.
5. Close the record Record side-effect guard, final decision, owner, and retest trigger.
Review Signal
Weak Evidence
Stronger Evidence
Decision Pressure
Timeout
A log says "timeout" without the expected response or observation point.
Expected acknowledgement, message identity, timer start, observation point, and state at timeout.
Retry, hold, fail safe, reconnect, or mark unknown.
Retry
A counter increments with no delay rule, limit, or final state.
Delay behavior, limit, stop condition, final state, and congestion or battery concern when relevant.
Accept for scope, revise policy, or add operational alert.
Sequence value
A sequence number appears in a trace without an accept or reject rule.
Prior accepted state, duplicate rule, gap rule, reset boundary, and wraparound rule when finite.
Accept, ignore, reject, wait for repair, or retest after reset.
Side effect
The command is retried without saying whether repeating it is safe.
Idempotency rule, command identifier, receiver state, duplicate action, and rollback or hold behavior.
Guard side effects before accepting the retry path.

13.4 Under the Hood: State Rules Prevent False Confidence

Under the hood, retry and sequencing are state machines. A sender tracks attempts, delays, acknowledgements, and stop conditions. A receiver tracks accepted values, stale values, duplicate decisions, gaps, resets, and sometimes a finite sequence space.

The hard part is not naming a retry or a sequence number. The hard part is proving what happens when evidence arrives late, twice, out of order, after reconnect, after a reset, or after the retry limit has already changed the final state.

State Boundaries to Inspect

Boundary
Failure Mode
Evidence Needed
Retest Trigger
Acknowledgement identity
A late or unrelated acknowledgement is accepted for the wrong message.
Message identifier, expected response, late-response rule, and association with the retry attempt.
Acknowledgement format, gateway mapping, batching, or session behavior changes.
Retry stop state
The sender keeps retrying, hides a persistent fault, or leaves command state ambiguous.
Retry limit, stop condition, state after limit, alert or hold rule, and owner of next action.
Timeout policy, retry limit, firmware, power mode, or network path changes.
Duplicate and stale values
A repeated or old message changes state again or overwrites newer data.
Accepted sequence state, duplicate action, stale-data rule, and side-effect guard.
Receiver storage, reconnect behavior, queue behavior, or reset logic changes.
Gap and wraparound
A missing value is assumed lost too early, or a wrapped value is mistaken for stale traffic.
Gap handling, reordering window, reset boundary, wraparound rule, and tested final state.
Sequence space, window size, session boundary, or message rate changes.
Common pitfall:

A retry trace is not a delivery guarantee. Treat it as evidence only after the receiver state and side-effect rules are visible.

Acceptance Checklist

  • Name the protected message boundary and expected response.
  • Record timeout, missing acknowledgement, duplicate, stale value, gap, or reorder evidence.
  • Show retry delay behavior, limit, stop condition, and final state.
  • Show accepted sequence state, duplicate rule, stale rule, gap rule, reset rule, and wraparound rule when relevant.
  • Guard side effects before accepting repeated commands, repeated writes, or repeated actuator actions.
  • List retest triggers for policy, firmware, gateway, receiver state, message shape, and path changes.

13.5 Summary

  • A retry or sequence claim needs a named message boundary, acknowledgement identity, retry timer, sequence window, and receiver side-effect rule.
  • Retry evidence needs timeout observation, acknowledgement identity, retry delay behavior, retry limit, stop condition, and final state.
  • Sequence evidence needs accepted state, duplicate handling, stale-message rules, gap handling, reset behavior, and wraparound boundaries when relevant.
  • Receiver state and side-effect guards decide whether repeated or delayed messages are safe.
  • Retest when timeout policy, retry policy, acknowledgement format, sequence-window behavior, firmware, gateway behavior, message shape, or transport path changes.

13.6 Key Takeaway

Approve retry and sequence behavior only when the record proves bounded retry, receiver state, duplicate handling, side-effect safety, and retest ownership for the tested path.

13.7 See Also