15 Compute Offloading and Placement

15.1 Start With a Choice: Compute Here or Transmit

Time One Decision on the Device and Across the Link

Picture a battery camera that must spot a person before opening a gate. It can inspect an image nearby or send data elsewhere, but either path may waste energy or arrive too late. The first useful comparison measures the same decision both ways.

DSP means digital signal processing, or math that changes or extracts meaning from sampled signals. A gateway means a device or service that joins two system paths. Latency means the time from an input event to the result that matters.

Run one known input locally, then through the remote path. Measure energy, elapsed time, bytes, queue delay, and returned result. Weaken the link, repeat the request, restart the gateway, and keep a safe local response when the remote answer is missing or stale.

This runway does not prove that local or remote work is always cheaper. The deeper sections compare processor fit, radio states, transfer size, privacy, waiting cost, and the break-even evidence for each placement.

A device can process data locally, send features to an edge node, or push raw samples to the cloud. None of those choices is automatically cheaper; the winner depends on radio energy, data size, latency, privacy, and accelerator fit.

The start-simple calculation is the break-even point: how many bytes can you afford to move before local compute becomes the lower-energy path?

Phoebe’s Field Notes: From a Millijoule Ledger to Days Between Charges

The mathematical gist. The chapter’s local path costs $180\ \mathrm{mW}\times5.0\ \mathrm{s}=900$ mJ, while its Wi-Fi path costs 261 mJ. A 150 mAh, 3.7 V cell with 2% monthly self-discharge and 15% reserve supplies about $1.66\times10^6$ mJ, so 1,440 events per day yield 1.28 local days or 4.43 offloaded days.

Math Bridge · guided foundationsHow does per-event energy become days of service?Let Battery Bruno carry local and offloaded ledgers through a derated wearable battery. Learn the maths

15.2 Compute Offloading and Placement

Offloading is not a synonym for “use the cloud.” In an IoT design review, it is a placement decision for one workload at one moment. The candidate locations are usually local MCU or application processor, local accelerator, nearby gateway or edge node, and remote cloud service.

The decision should be made with an energy ledger and service constraints. If the ledger uses guessed radio costs, ignores retry and tail states, or assumes cloud processing is always free to the device, the resulting policy can shorten battery life instead of extending it.

15.3 Learning Objectives

By the end of this chapter, you will be able to:

• Build a local-versus-remote energy ledger for an IoT workload.
• Explain why upload size, result size, radio state, retries, and latency affect offloading decisions.
• Apply MAUI-style runtime profiling without treating cloud execution as automatically cheaper.
• Decide when edge offload, cloud offload, local acceleration, partial offload, or deferral is the best placement.
• Match CPU, DSP, GPU, and NPU execution to workload characteristics.
• Record the evidence needed to promote an offloading policy into deployment.

15.4 The Offloading Ledger

An offloading decision starts by comparing two ledgers at the same workload boundary.

The diagram at Figure 15.1 earns attention because the offloading ledger spans @Remoteable annotated method and E_local vs. Reading both prevents a local success from becoming a system claim.

E_remote, Offload to">

The visual logic of Figure 15.1 can be checked at three named points. @Remoteable annotated method adds a distinct review condition; after that, E_local vs adds a distinct review condition; at the boundary, E_remote? adds a distinct review condition. Together they mean that mAUI Code Offloading Decision: Local vs. Remote: Mobile Device, @Remoteable annotated method, E_local vs., E_remote?, E_local E_remote, Offload to. Keep that logic attached to the offloading ledger.

Use the following model as a review scaffold:

E_local  = P_compute_local * T_compute_local

E_remote = E_upload
         + E_download
         + E_wait
         + E_radio_transition
         + E_retry
         + E_local_preprocess

Offload only when the remote path saves enough energy while still meeting the service requirement:

choose_remote when:
  E_remote < E_local
  latency_remote <= deadline
  privacy_policy_allows_transfer
  remote_service_available

The ledger should use measured values from the device, firmware, network, and deployment environment. Vendor datasheet currents are useful for early sizing, but promotion decisions need current traces or power-monitor logs.

15.5 Worked Example: Feature Extraction

Scenario: A wearable collects a 150 kB motion window and runs a feature-extraction model. Local CPU execution takes 5 seconds at 180 mW.

Local CPU:
  E_local = 180 mW * 5.0 s = 900 mJ

Wi-Fi offload path: upload the 150 kB window, receive a small result, and wait for remote processing.

Upload:   400 mW * 0.60 s = 240 mJ
Download: 180 mW * 0.05 s =   9 mJ
Wait:      30 mW * 0.40 s =  12 mJ
Total Wi-Fi remote path     = 261 mJ

For this workload, Wi-Fi offload is attractive because the compute is heavy and the transfer is moderate.

Cellular offload path: the same transfer uses a higher-power radio and includes state-transition overhead.

Radio ramp:                    250 mJ
Upload:  1000 mW * 1.20 s =  1200 mJ
Download: 400 mW * 0.08 s =    32 mJ
Wait:     100 mW * 0.40 s =    40 mJ
Tail/connected state:          500 mJ
Total cellular remote path  = 2022 mJ

For the same workload, cellular offload is worse than local CPU execution. The policy should process locally, use a local accelerator, defer until Wi-Fi is available, or transmit a smaller feature vector instead of raw data.

15.6 Placement Gates

A robust offloading policy applies gates before making a final placement decision.

Use the placement calculator after applying the gates: change the workload, timing, privacy, and link assumptions, then record which execution site changes and why.

15.7 Runtime Decision Frameworks

MAUI-style systems profile methods, estimate energy, and decide at runtime whether a component should run locally or remotely. The important idea is the discipline, not one fixed formula:

1. Split the application into offloadable components.
2. Measure or estimate local execution time and energy for each component.
3. Measure or estimate network upload, download, wait, and radio-state costs.
4. Reject candidates that violate latency, privacy, availability, or dependency constraints.
5. Choose the placement with the lowest measured total energy.
6. Re-profile when firmware, model size, input size, radio conditions, or battery policy changes.

15.14 Heterogeneous Local Computing

Heterogeneous computing changes the offloading tradeoff because the local option is no longer only “general CPU.” Many IoT and mobile-class devices include lower-power accelerators for specific work.

For heterogeneous local computing, a result at checks workload type, deadline, is incomplete without CPU baseline. The figure in Figure 15.2 shows why those labels belong in one review.

Read Best fit: control flow, last in Figure 15.2. checks workload type, deadline, has already sets the timing constraint, and CPU baseline has establishes the starting condition; the final label adds a distinct review condition. This order makes the caption operational: A heterogeneous mobile system-on-chip dispatches work through a context-aware scheduler to a CPU baseline at about 1 watt per core, a DSP at about 50 milliwatts for always-on sensing, a GPU at 2 to 4 watts for parallel work, and an NPU at about 0.5 watts for machine-learning inference. The figure shows relative efficiency claims of 5 to 20 times for GPU, 10 to 50 times for DSP, and 50 to 100 times for NPU workloads, then sends unsuitable workloads to an edge gateway fallback. It supplies the evidence hand-off for heterogeneous local computing.

15.15 Breakeven Thinking

The breakeven point is the local compute duration where remote placement becomes cheaper:

T_breakeven = E_remote / P_local_compute

If a remote path costs 300 mJ and local compute power is 150 mW, remote placement starts to save energy only when local compute would take more than 2 seconds.

T_breakeven = 300 mJ / 150 mW = 2.0 s

Breakeven is not stable. It moves when:

• Input size changes.
• Signal strength changes.
• Retransmissions increase.
• The radio is already awake for another transfer.
• A local accelerator becomes available.
• The model grows or shrinks.
• The service deadline tightens.
• Privacy rules require local handling.

15.16 Measurement Record

An offloading review should leave a compact evidence record:

1. Workload boundary: input size, output size, model version, and deadline.
2. Local path: processor used, execution time, power trace, setup overhead, and thermal throttling notes.
3. Remote path: upload bytes, download bytes, radio state, retries, signal quality, wait time, and failure handling.
4. Placement decision: local CPU, local accelerator, edge, cloud, partial, or deferred.
5. Policy gate: privacy, availability, safety, data residency, or model-fit constraint that affected the choice.
6. Recheck trigger: new firmware, new model, changed radio profile, changed battery target, or changed deployment site.

15.17 Common Pitfalls

15.24 Related Chapters

15.29 Knowledge Check

15.30 Knowledge Check: Offloading Ledger

15.31 Knowledge Check: Heterogeneous Execution

15.32 Matching Quiz: Placement Options

15.33 Ordering Quiz: Offloading Review

15.34 Label the Diagram: Offloading Decision Ledger

15.35 What’s Next

15.40 Compute Or Communicate

Offloading moves a computation off the device to an edge server or the cloud. It never removes energy cost; it trades one kind for another. Instead of paying the processor to run the work, the node pays the radio to ship the input out and pull the answer back. Whether that trade wins depends on how much computation you avoid versus how much data you must move.

The rule of thumb is simple: offload heavy computation on small data, and keep light computation on large data local. A one-second inference on a two-kilobyte feature vector is a good offload. A one-line threshold check on a hundred-kilobyte image is a terrible one, because sending the image costs far more than the check ever would.

Worked example: a vibration node can either classify a ten-second window locally or send extracted features to a gateway. If local inference draws 120 mW for 2.5 s, the local path costs 300 mJ. Sending a 2 KB feature vector over a short-range link at 80 mJ, then receiving a 2 mJ result, leaves plenty of margin for offload. Sending the raw 120 KB waveform at 0.04 mJ per byte would cost 4,800 mJ before any wait or retry energy, so the same algorithm should stay local unless the node first compresses or summarizes the data.

Before approving an offload path, inspect Figure 15.3. It puts the complete local and remote ledgers beside the service gates that can reject an apparent energy win.

Read Figure 15.3 from local processor power and run time into the local-energy total, then compare it with upload, wait, download, transition, retry, and preprocessing costs on the remote path. Apply the latency, privacy, connectivity, and reliability gates last: only a remote path that wins the full ledger and passes those gates is a defensible offload decision.

15.41 The Two Paths

15.42 Overview Knowledge Check

15.43 Find The Break-Even Data Size

Give the radio an energy-per-byte figure from its active current and effective throughput, then compare against the local compute charge. Offload wins when E_local > e_byte x D + E_result + E_setup, where D is the payload size and e_byte is radio energy per byte.

15.44 Worked Example: Offload An Inference Over BLE

The local inference holds the MCU active at 15 mA for 2 s, so E_local = 30 mA-s. The BLE radio draws 6 mA while active at an effective 50 kbps, giving e_byte = 6 mA x (8 bit / 50000 bit/s) = 0.00096 mA-s per byte. Ignoring setup for the moment, the break-even payload is D* = 30 / 0.00096 = 31250 bytes, about 30 KB.

• Small input (2 KB feature vector): offload costs 2048 x 0.00096 = 2.0 mA-s. Offloading beats the 30 mA-s local run by about 15x.
• Large input (100 KB raw capture): offload costs 102400 x 0.00096 = 98 mA-s. Now local computing at 30 mA-s wins by more than 3x.
• Break-even: at roughly 30 KB the two paths cost the same; the decision flips around that size.

The same inference is a good offload for a small feature vector and a bad offload for a raw capture. Placement is a property of the data-to-compute ratio, not of the algorithm alone.

Now add setup overhead. If the link costs 8 mA-s to wake, negotiate, and settle before payload transfer, the break-even becomes D* = (30 - 8) / 0.00096 = 22917 bytes, about 22 KB. The 2 KB feature vector still wins, but the 30 KB case no longer breaks even: it costs about 8 + 30720 x 0.00096 = 37.5 mA-s. Record that fixed overhead separately so a later firmware change or already-awake radio state can be reviewed without rebuilding the whole ledger.

15.45 Placement Ledger

15.46 Practitioner Knowledge Check

15.47 The Radio Does Not Stop When The Packet Does

A naive offload model counts only the active transmit time. Real radios add two costs that can flip a break-even. First, connection setup: bringing up a BLE connection or attaching a cellular modem burns energy before any payload moves, which punishes small transfers most. Second, the tail: after the last byte, many radios hold a high-power state for a while before releasing. A cellular modem in particular can stay connected for several seconds after transmission, drawing tens of milliamps, before it drops to a low-power idle or power-save mode. That tail is pure overhead a payload-only calculation misses.

The energy-per-bit also varies enormously by technology, so the break-even data size is radio-specific. A short-range link like BLE moves bits cheaply. A long-range low-rate link like LoRa has such low throughput that a modest payload occupies the radio for a long airtime, making its energy per bit high. A cellular link adds the setup and tail costs above. The same 30 KB break-even computed for BLE could be a few kilobytes on a slow or tail-heavy radio.

Worked example: suppose a small cellular offload transmits for 0.4 s at 120 mA, so a payload-only model records only 48 mA-s. The modem also spends 25 mA-s attaching and then holds a 40 mA connected tail for 8 s, adding 320 mA-s. The real radio charge is 25 + 48 + 320 = 393 mA-s. If the local compute path is 90 mA-s, the payload-only model says offload wins, while the measured device says local compute is more than four times cheaper. That is why tail timers and network state must be captured with the same power trace as payload transfer.

15.48 Costs A Payload-Only Model Misses

15.49 Under-the-Hood Knowledge Check

15.50 LEO-Style Heterogeneous Scheduling

A heterogeneous scheduler is more than a table saying “DSP is efficient” or “GPU is fast.” It needs a live path from arriving sensor work to a compatible execution unit. A LEO-style architecture makes that path explicit:

1. Sensor applications describe jobs with data size, deadline, accuracy mode, and permitted execution targets.
2. A workload monitor observes the queued jobs and their recent runtimes.
3. A resource monitor records processor availability, frequency, memory pressure, thermal state, radio state, and the measured energy model for CPU, DSP, GPU, or cloud execution.
4. Jobs wait in a sensor-job buffer so compatible work can be batched without losing deadline order.
5. A scheduler running on a low-power unit scores feasible placements and dispatches each job to a CPU, DSP, GPU, or off-device target.
6. Completion measurements update the models; a timeout or unavailable accelerator takes a declared fallback path.

For job $j$ on processing element $k$, a useful score is not runtime alone:

where $E_{j,k}$ includes migration and data movement, $d_j$ is the deadline, and $R_{j,k}$ represents a quantified service risk such as unsupported precision or cloud unavailability. Infeasible placements receive infinite score. The scheduler then chooses the lowest-score feasible target, rechecking shared memory and thermal constraints before dispatch.

Historical keyword-spotting experiments illustrate why the control path must stay cheap. A heuristic schedule for a ten-application workload completed in roughly 100 ms on a low-power unit and consumed under 0.5% of the workload energy, whereas making the scheduling decision through a cloud path cost several percent in the reported comparison. These figures are workload- and platform-specific; the durable requirement is $E_{schedule}\ll E_{work}$ and $t_{schedule}$ small relative to the tightest deadline.

15.51 Reading the Keyword-Spotting Benchmarks

The same experiments reported two different speedups because they used two different baselines. An optimized local GPU path was about 6.5 times faster than the cloud comparison in one measurement and about 21.3 times faster than a sequential CPU implementation in another. Neither ratio means “GPU is always best.” Read a speedup as

and always name $A$, $B$, the batch size, model, precision, data-transfer boundary, and whether setup time is included.

Cloud energy also changes with bandwidth and batching. A complete device-side model is

while a local batched accelerator can be approximated as

Increasing batch size $b$ amortizes the fixed local wake and setup costs, but it also delays the oldest sample by roughly the batch-formation time. Higher link throughput reduces the payload-time terms in $E_{cloud}$, yet it does not erase attach, wait, retry, or tail energy. That is why the reported optimized-GPU batching curve could beat cloud execution across its tested bandwidths and batch sizes without proving a universal result.

To reproduce the decision on a new device, drive the comparison step by step: benchmark the same keyword model and inputs on every eligible processor; include buffer copies and accelerator setup; measure cloud attach and tail states; sweep batch size without violating the detection deadline; then choose the lowest-energy placement that still meets accuracy, privacy, and availability requirements. A bar chart supplies a hypothesis. Only this end-to-end ledger supplies a scheduler policy.

Trace every scheduling input and placement output in Figure 15.4.

In the diagram Figure 15.4, queue a described job records deadline, tensor shape, and memory; Join two monitor views combines workload urgency with resource and battery state. Dispatch to the best engine then chooses CPU, DSP, or GPU while retaining placement and fallback evidence.

Challenge any universal offloading claim with the two axes in the diagram Figure 15.5.

In the diagram Figure 15.5, lOCAL shows an Optimized GPU runtime near 0.05 s, while CLOUD is bandwidth-led and pays batching, radio, and queue costs. The There is no universal winner panel directs the review toward the crossover among batch size, link rate, wake cost, and transfer energy.

15.52 Summary

This chapter evaluates when IoT devices should compute locally, offload to edge or cloud, or use heterogeneous hardware. It weighs energy, latency, data-transfer cost, privacy, reliability, and fallback behavior.

15.53 Key Takeaway

Offloading saves energy only when communication, waiting, retry, privacy, and failure costs are lower than local execution. Compare total task energy and service risk before moving work away from the device.