← Back to the chapter · Previous detail
Technical detail · Compute Offloading and Placement
Read keyword-spotting benchmarks with their baselines
15.13 Reading the Keyword-Spotting Benchmarks
The keyword-spotting experiments discussed in the LEO study reported two different speedups because they used two different baselines. An optimized local GPU path was about 6.5 times faster than the cloud comparison in one measurement and about 21.3 times faster than a sequential CPU implementation in another. Petkogeorgiev thesis Neither ratio means “GPU is always best.” Read a speedup as
and always name , , the batch size, model, precision, data-transfer boundary, and whether setup time is included.
Cloud energy also changes with bandwidth and batching. A complete device-side model is
while a local batched accelerator can be approximated as
Increasing batch size amortizes the fixed local wake and setup costs, but it also delays the oldest sample by roughly the batch-formation time. Higher link throughput reduces the payload-time terms in , yet it does not erase attach, wait, retry, or tail energy. That is why the reported optimized-GPU batching curve could beat cloud execution across its tested bandwidths and batch sizes without proving a universal result.
To reproduce the decision on a new device, drive the comparison step by step: benchmark the same keyword model and inputs on every eligible processor; include buffer copies and accelerator setup; measure cloud attach and tail states; sweep batch size without violating the detection deadline; then choose the lowest-energy placement that still meets accuracy, privacy, and availability requirements. A bar chart supplies a hypothesis. Only this end-to-end ledger supplies a scheduler policy.