← Back to the chapter

Technical detail · Practical Edge Computing: Bandwidth and Inference Budgets

Derive the shared memory allocation

Establish the unoptimized baseline first. Always-resident weights total 91 MB, simultaneous peak activations total 250 MB, the example reserves about 50 MB for the TensorFlow Lite runtime and 20 MB for input and output buffers, and the combined baseline is therefore 411 MB. That fits inside 3.2 GB, but the breakdown reveals where sharing and quantization could create reserve for other workloads.

Next inspect scheduling before allocating three independent activation arenas. The example schedules occupancy every second, HVAC every five seconds, and air quality every ten seconds. Those periods do not prove statistical independence, so multiplying nominal duty cycles into a two-percent overlap estimate is only a planning approximation. The defensible optimization is to serialize access explicitly: give each model the same 120 MB arena, release it after inference, and verify that the scheduler prevents overlap. With 91 MB of weights, the shared arena, 50 MB of runtime overhead, and 20 MB of buffers, the intermediate allocation becomes 281 MB, 130 MB below the baseline.

Then evaluate INT8 quantization, representing model values with eight-bit integers, model by model. The exercise’s nominal fourfold model-size reduction does not establish accuracy parity. The worked arithmetic reduces occupancy weights to 11.25 MB and activations to 30 MB, HVAC weights to 7 MB and activations to 21 MB, and air-quality weights to 4.5 MB and activations to 11 MB. That gives 22.75 MB of weights and a 30 MB maximum shared arena. Adding the stated runtime and buffer allowances yields 122.75 MB before contingency. These size reductions must be verified from the converted artifacts, and each quantized model still needs accuracy and feature-parity tests against its accepted baseline.

Finally test the worst scheduling case rather than assuming overlap cannot happen. Sequential execution of the stated 50 ms, 35 ms, and 25 ms measurements totals 110 ms, inside the one-second occupancy period. The final worked allocation rounds weights to 23 MB, keeps the 30 MB shared arena, reserves 50 MB for the runtime and 20 MB for buffers, and adds a 25 MB margin, producing 148 MB in total. The release record should confirm those peaks on the target device, include operating-system pressure and allocator behavior, and define what happens if a model misses its slot or exceeds the shared arena.

← Back to the chapter