3 Latency Budgets for Tier Choice

3.1 Start Simple

Imagine a vibration sensor notices a motor fault and must stop a machine before damage spreads. The first question is not where the code is fashionable; it is how many milliseconds the whole loop can spend from sensing to actuation. Everyday IoT latency work starts by timing that loop, then moving only the responsibilities that miss the budget closer to the device. Start with one event, one deadline, one fallback action, and one owner for the measurement.

Edge Eddie

“Send the decision, not the raw feed — the edge earns its keep in milliseconds and megabytes saved.”

Here Eddie starts the stopwatch at the physical event, not the dashboard, and follows the budget through every segment that can eat it.

3.2 Learning Objectives

After this chapter, you should be able to:

• define a latency budget from physical event to useful response;
• separate immediate response ownership from later review evidence;
• identify the path segments that belong in a latency decision record;
• explain why averages can hide placement failures;
• state when edge, fog, cloud, or a hybrid split is defensible.

3.3 Start the Clock at the Physical Event

A latency budget starts when the physical condition changes, not when the dashboard receives a message. A cold-room door sensor may need a local alarm within a site-defined window, while the supervisor dashboard and monthly energy review can wait. If local firmware can alarm reliably during backhaul loss, the edge can own the immediate response. The fog gateway can still coordinate nearby devices and buffer records, while the cloud receives the event later for reporting.

Inspect Figure 3.1 before continuing. Latency starts with the physical condition that demands a response, not with a convenient software timestamp. Figure 3.1 identifies every segment that must fit inside the cold-room alarm’s site-defined window.

In the diagram Figure 3.1, start the timer at Physical Event, then account for Sense + Validate sampling, debounce, and timestamp work. Transfer + Queue adds hops, retries, buffering, and burst load before Process + Decide applies a rule, inference, or site policy. The path ends only after Command + Act sends and confirms the response or invokes local fallback; Evidence Record then separates device, site, and review proof. Record p50 and p95 duration, retry count, queue age, decision time, command acknowledgement, and the owner of each segment so a missed alarm window can be traced to a boundary rather than hidden inside one end-to-end average. This budget supports edge ownership when the local confirmed action must survive backhaul loss.

3.4 Tier Ownership From the Budget

3.5 Measure the Path, Not the Label

A latency decision record should show the start event, stop event, path segments, measurement method, stressed condition, accepted percentile, owner, and retest trigger. The record should explain why the chosen tier is acceptable and what evidence must remain visible after deployment.

Prefer percentile evidence over averages when the workload is safety-related, control-related, or user-facing. The relevant percentile depends on the risk and operating agreement; the decision record should state which percentile is accepted and what load condition it represents.

3.6 Tail Latency Is Where Placement Fails

Many placement failures hide in the slow cases. A path may look acceptable when a device is idle, a gateway is nearby, a queue is empty, and the cloud service is healthy. The same path may miss its response budget during radio retries, gateway overload, cold starts, model loading, authentication checks, congestion, actuator delay, or an upstream outage.

For a tail-latency review, trace one event through every boundary. A vibration threshold might wait for the next sensor sample, retry on a noisy radio link, sit behind a gateway upload, run a model, request command authorization, and then wait for the actuator to respond. Each segment can be small alone, yet the combined slow path can exceed the budget. Placement is credible only when the measured trace includes those waits and the fallback path.

Cloud is not wrong for latency-sensitive systems; it is wrong as the sole owner of an immediate physical response unless the measured path proves it. A common hybrid pattern is edge for the minimum safe action, fog for site coordination and buffering, and cloud for history, comparison, governance, and model improvement.

The under-the-hood rule is to review the worst useful path, not the happiest path. If the design depends on an edge model, measure preprocessing, inference, postprocessing, command output, and fallback. If the design depends on a gateway, measure local queueing and outage behavior. If the design depends on cloud, measure the full upstream and downstream path under the relevant operating condition.

3.7 Response Budget Flow

Trace your own path before you commit to a tier. The workbench below breaks end-to-end latency into request transfer, queueing, processing, and response segments across device, edge, fog, and cloud placements, so you can see which segment a given payload size, congestion level, or cloud distance actually costs before you defend a placement decision.

3.8 Summary

• Edge-fog latency is an end-to-end response budget from physical event to useful system response.
• The path includes sensing, queueing, network transfer, gateway behavior, processing, command delivery, actuation, and retained evidence.
• Edge ownership fits immediate local action; fog ownership fits site coordination; cloud ownership fits history, governance, training, and fleet review when delay is acceptable.
• A latency decision record should state start and stop points, segment budgets, measurement conditions, accepted percentile, owner, fallback, and retest trigger.
• Averages can hide the slow cases that matter; tail latency, jitter, congestion, retries, and outages need explicit review.
• Retest latency after sensor, firmware, model, gateway, topology, network, actuator, command policy, workload, or ownership changes.

3.9 See Also