← Back to the chapter · Previous detail

Technical detail · Compute Offloading and Placement

Read keyword-spotting benchmarks with their baselines

15.13 Reading the Keyword-Spotting Benchmarks

The keyword-spotting experiments discussed in the LEO study reported two different speedups because they used two different baselines. An optimized local GPU path was about 6.5 times faster than the cloud comparison in one measurement and about 21.3 times faster than a sequential CPU implementation in another. Petkogeorgiev thesis Neither ratio means “GPU is always best.” Read a speedup as

speedupA→B=tAtB,\text{speedup}_{A\rightarrow B}=\frac{t_A}{t_B},

and always name AA, BB, the batch size, model, precision, data-transfer boundary, and whether setup time is included.

Cloud energy also changes with bandwidth and batching. A complete device-side model is

Ecloud=Eattach+PtxDupRup+Pwaittremote+PrxDdownRdown+Etail,E_{cloud}=E_{attach}+P_{tx}\frac{D_{up}}{R_{up}}+P_{wait}t_{remote} +P_{rx}\frac{D_{down}}{R_{down}}+E_{tail},

while a local batched accelerator can be approximated as

Elocal(b)=Ewake+Etransfer(b)+bEkernel+Eidle−gap.E_{local}(b)=E_{wake}+E_{transfer}(b)+bE_{kernel}+E_{idle-gap}.

Increasing batch size bb amortizes the fixed local wake and setup costs, but it also delays the oldest sample by roughly the batch-formation time. Higher link throughput reduces the payload-time terms in EcloudE_{cloud}, yet it does not erase attach, wait, retry, or tail energy. That is why the reported optimized-GPU batching curve could beat cloud execution across its tested bandwidths and batch sizes without proving a universal result.

To reproduce the decision on a new device, drive the comparison step by step: benchmark the same keyword model and inputs on every eligible processor; include buffer copies and accelerator setup; measure cloud attach and tail states; sweep batch size without violating the detection deadline; then choose the lowest-energy placement that still meets accuracy, privacy, and availability requirements. A bar chart supplies a hypothesis. Only this end-to-end ledger supplies a scheduler policy.

Keyword-spotting placement varies with local accelerator runtime, cloud link bandwidth, and batch size.
Keyword-spotting results depend on batch size and bandwidth because local setup and cloud communication contribute different costs.

← Back to the chapter · Previous detail