MACs Per Second Versus the Deadline

Ada re-derives this chapter’s own numbers step by step, at full precision

foundations
math-foundations
calculation-audit
edge-fog
Ada ADA · CALCULATION AUDIT

MACs Per Second Versus the Deadline

The chapter sizes a model needing 8 million MACs per inference at 10 inferences per second, demanding 80 million MACs/s — and a 20-million-MAC/s board delivers only 2.5 of the required 10. A good TOPS number can hide a missed deadline. This audit makes the arithmetic explicit, weighing MACs per second against the deadline to expose a 4x shortfall no sleep-current figure can close.

Companion to the chapter Hardware Accelerators for Edge AI — every number here comes from that chapter.

Ada: The sizing section states that a model needing 8 million MACs per inference at 10 inferences per second demands 80 million MACs per second, and that a 20-million-MAC/s board misses the deadline “even if its sleep current looks attractive.” Compute budgets are where a good TOPS number hides a missed deadline, so let me make the arithmetic explicit.

  • Sustained compute demand: 8,000,000 MACs/inference x 10 inferences/s = 80,000,000 MACs/s.
  • In raw operations (one MAC = one multiply + one add): 80,000,000 x 2 = 160,000,000 operations/s.
  • What the weak board can actually deliver: at 20,000,000 sustained MACs/s it manages 20,000,000 / 8,000,000 = 2.5 inferences/s.
  • The gap: 80,000,000 / 20,000,000 = 4x short – it meets only 2.5 of the required 10 inferences per second.

The audit conclusion is that the deadline, not the datasheet, sets the bar: 2.5 inferences/s against a 10 inferences/s requirement is a 4x shortfall no sleep-current figure can close. The design meaning is that the compute budget must be MACs/inference x inferences/s measured against sustained throughput on the real converted graph – because a headline TOPS rating that assumes every operator stays on the accelerator says nothing about the board that drops one layer to the CPU and quietly delivers a quarter of the needed rate.

Every number above is taken from the chapter’s own material and re-derived step by step.