MACs Per Second Versus the Deadline

MACs Per Second Versus the Deadline

Ada re-derives this chapter’s own numbers step by step, at full precision

foundations
math-foundations
calculation-audit
edge-fog
Ada ADA · CALCULATION AUDIT

MACs Per Second Versus the Deadline

The chapter sizes a model needing 8 million MACs per inference at 10 inferences per second, demanding 80 million MACs/s — and a 20-million-MAC/s board delivers only 2.5 of the required 10. A good TOPS number can hide a missed deadline. This audit makes the arithmetic explicit, weighing MACs per second against the deadline to expose a 4x shortfall no sleep-current figure can close.

Companion to the chapter Hardware Accelerators for Edge AI — every number here comes from that chapter.

See the relationship before changing it

The figure reads from left to right. The blue card is inference rate. The middle card applies the page rule. The green card is compute demand. Walk the arrows once: set the input, apply the rule, then read the result with its unit.

Inference rate changes compute demand An input card leads through the rule demand = 8 million MAC/inference x inferences/s to the compute demand result. INPUT PAGE INPUT APPLY THE RULE predict calculate check units OUTPUT RESULT
Walk the arrows. The deadline sets the sustained compute demand, not the peak number on a data sheet.

Derive the baseline in four named moves

  1. 1

    Name the input. The chapter baseline is 10 inferences/s.

  2. 2

    Name the relationship. demand = 8 million MAC/inference x inferences/s

  3. 3

    Substitute with units. 8 x 10 = 80.00 million MAC/s

  4. 4

    Read the result. Keep the unit beside the value. Use it only inside the technical boundary on this page.

Predict, then change inference rate

Try Predict the direction of demand = 8 million MAC/inference x inferences/s. Test another inference rate, then compare compute demand.

10 inferences/s
Chapter baseline
Compute demand

Observe The deadline sets the sustained compute demand, not the peak number on a data sheet. Reset inference rate to 10 and compare compute demand.

Explain The deadline sets the sustained compute demand, not the peak number on a data sheet.

Check yourself

What should you do before trusting a moved-control result?
Answer: Predict its direction, apply the shown relationship, keep the units, and reset to the worked baseline.
What does this small model leave out?
Answer: Only inference rate moves here. Field effects named in the technical boundary stay fixed.

Honesty note. This teaching model changes only inference rate. It is not a field prediction.

TryThe chapter sizes a model needing 8 million MACs per inference at 10 inferences per second , demanding 80 million MACs/s — and a 20-million-MAC/s board delivers only 2.5 of the required 10. Use Check derivation.
ObserveThe displayed ledger resolves 8 million MACs, 10 inferences per second, 80 million MACs/s, 20-million-MAC/s, 2.5 at full precision. This audit makes the arithmetic explicit, weighing MACs per second against the deadline to expose a 4x shortfall no sleep-current figure can close. Check derivation shows this.
ExplainThe design meaning is that the compute budget must be MACs/inference x inferences/s measured against *sustained* throughput on the real converted graph -- because a headline TOPS rating that assumes every operator stays on the accelerator says nothing about the board that drops one layer to the CPU and quietly delivers a quarter of the needed rate. Check derivation confirms it.

Ada: The sizing section states that a model needing 8 million MACs per inference at 10 inferences per second demands 80 million MACs per second, and that a 20-million-MAC/s board misses the deadline “even if its sleep current looks attractive.” Compute budgets are where a good TOPS number hides a missed deadline, so let me make the arithmetic explicit.

  • Sustained compute demand: 8,000,000 MACs/inference x 10 inferences/s = 80,000,000 MACs/s.
  • In raw operations (one MAC = one multiply + one add): 80,000,000 x 2 = 160,000,000 operations/s.
  • What the weak board can actually deliver: at 20,000,000 sustained MACs/s it manages 20,000,000 / 8,000,000 = 2.5 inferences/s.
  • The gap: 80,000,000 / 20,000,000 = 4x short – it meets only 2.5 of the required 10 inferences per second.

The audit conclusion is that the deadline, not the datasheet, sets the bar: 2.5 inferences/s against a 10 inferences/s requirement is a 4x shortfall no sleep-current figure can close. The design meaning is that the compute budget must be MACs/inference x inferences/s measured against sustained throughput on the real converted graph – because a headline TOPS rating that assumes every operator stays on the accelerator says nothing about the board that drops one layer to the CPU and quietly delivers a quarter of the needed rate.

Technical boundaries
The throughput model deliberately does not simulate memory stalls, unsupported operators, accelerator utilisation, scheduling jitter, or thermal throttling; it treats sustained MAC/s as a constant divisible by MACs per inference.

Work the audit first, then check the displayed derivation.

Every number above is taken from the chapter’s own material and re-derived step by step.