← Back to the chapter · Previous detail · Next detail →
Technical detail · Hardware Accelerators for Edge AI
Size compute and memory from the deployed workload
13.8 Deep Dive: Sizing Hardware From Workload Evidence
There is no single edge AI chip. Size compute from the converted model’s MACs per inference multiplied by required inferences per second. At 8 million MACs per inference and 10 inferences per second, sustained demand is 80 million MACs per second. Counting a MAC as a multiply plus an add gives roughly 160 million operations per second. A board sustaining only 20 million MACs per second on the real graph misses the required rate despite attractive sleep current. Ample TOPS helps only when operators and tensors stay accelerated.
Memory can be harder. Weights fitting in flash do not prove that activations, input frames, tensor arena and runtime fit in RAM. Record memory headroom for the converted artifact. Camera frames still travel through resize, normalization, inference, postprocessing, action and logging. Allocation failure, swapping, or needing a smaller batch or window than required signals a failed memory assumption.