Math Bridge: NPU Precision and Decision Cadence

← Back to Edge AI Hardware
Math BridgeEdge & FogStruggle-friendly runway

What does INT8 spend to make an NPU efficient?

Price numeric precision and the inference clock without pretending they are the same limit.

Edge Eddie, the guideEdge Eddie guides
The one targetKeep precision and cadence as two hardware budgets.
The chapter caseRange [−1,1], INT8, 10 inferences/s.
What it buys youAn accelerator claim with named numeric and timing costs.

A field team faces an unresolved physical question: What does INT8 spend to make an NPU efficient? They must answer it before changing npu arithmetic bit depth on the real device. Predict the direction first.

See the relationship before changing it

The figure reads from left to right. The blue card is npu arithmetic bit depth. The middle card applies this page's relationship. The green card is quantisation step. Walk the arrows once: set the input, apply the rule, then read the result with its unit.

The retained audit below checks several chapter fixtures. This added model holds every other chapter fixture fixed, so the numeric fixture does not switch without explanation.

NPU arithmetic bit depth changes quantisation step An input card leads through the page relationship to the quantisation step result. SET INPUT ONE CONTROL APPLY RULE predict calculate check units READ RESULT
Walk the arrows. The control changes how values are represented inside the accelerator. Only a scheduler or latency change can move the physical decision cadence.

Derive the baseline in four named moves

  1. 1

    Name the input. The chapter baseline for npu arithmetic bit depth is 8.

  2. 2

    Name the relationship. q=range/2^N; qrms=q/√12; SNR=6.02N+1.76; fevent≤finfer/2

  3. 3

    Substitute the chapter fixture. Set npu arithmetic bit depth to 8. The page ledger gives quantisation step as 0.008.

  4. 4

    Read the result. Keep the stated output unit beside the value. Use it only inside the technical boundary on this page.

Predict, then change npu arithmetic bit depth

Try Predict the direction of quantisation step. Move one control, calculate, then check your prediction.

8
Chapter baseline
Quantisation step

Observe The control changes how values are represented inside the accelerator. Only a scheduler or latency change can move the physical decision cadence. Reset the control to 8 and compare quantisation step.

Explain Only npu arithmetic bit depth moves here. The other chapter fixtures remain fixed.

Check yourself

What should you do before trusting a moved-control result?
Answer: Predict its direction, apply the shown relationship, keep the units, and reset to the worked baseline.
What does this small model leave out?
Answer: Only npu arithmetic bit depth moves. Field effects named in the page's technical boundary stay fixed.

Honesty note. This teaching model changes only npu arithmetic bit depth. It replays the chapter fixture; it is not a field prediction.

1. One array, two limits

Fewer bits let hardware pack more multiply-accumulate units into an area and move less data. That efficiency widens each numeric step. A separate inference cadence decides how often the physical system gets a new decision.

Edge Eddie: Faster arithmetic inside a call does not guarantee enough calls per second.

2. Name each move

1

Count levels2^N levels span the normalized range.

2

Divide the rangeq = 2/2^N.

3

Convert step to RMSqrms = q/√12.

4

Estimate the ideal ceilingSNR = 6.02N + 1.76 dB.

5

Halve the decision rateA 10 Hz cadence has a 5 Hz Nyquist boundary.

3. Work INT8 and INT16

INT8: q = 2/256 = 0.0078125; qrms = 0.002255; SNR = 49.9 dB

INT16 gives 65,536 levels and an ideal 98.1 dB ceiling, 48.2 dB above INT8. Neither result changes the chapter's 10 inferences/s cadence or its 5 Hz event boundary.

4. Try one controlled change

q=range/2^N; qrms=q/√12; SNR=6.02N+1.76; fevent≤finfer/2

TryMove only the arithmetic bit depth. The normalized range and 10-inference/s physical decision schedule stay fixed.

Levels
Quantisation step
RMS floor
Ideal ceiling
Inference cadence
Event boundary
Decision period

ObserveAt 8 bits the ideal ceiling is 49.9 dB and the RMS floor is 0.002255. Raising precision improves those numbers, while the 10 Hz cadence, 100 ms period, and 5 Hz event boundary remain fixed.

ExplainThe control changes how values are represented inside the accelerator. Only a scheduler or latency change can move the physical decision cadence.

Technical boundaries.

The precision calculation is an ideal full-range uniform quantiser.

Distribution
Real weights and activations may not use the full range uniformly
Network
Clipping and rounding accumulate across operations and layers
Timing
Event bandwidth is not always a sinusoid and latency includes the whole pipeline

Benchmark the final artifact, operators, preprocessing, memory traffic, scheduler, and event set on the target.

5. Interpret the 48.2 dB gap

It is an ideal representation-margin difference, not a predicted accuracy drop. A well-calibrated INT8 model can preserve the task while a poorly ranged INT16 model can still clip or fail elsewhere.

6. Record the complete accelerator claim

Keep bit depth, scale policy, calibration set, operator support, accuracy delta, tensor memory, inference latency, cadence, energy per decision, thermal state, and fallback together.

7. Check yourself

Why does INT8 have 256 levels over [−1,1] here?
Answer: Eight bits provide 2^8 code positions across the modeled range.
Does 49.9 dB predict the model's accuracy?
Answer: No. It is an ideal quantiser ceiling, not a task metric.
Can a faster MAC array alone move the 5 Hz event boundary?
Answer: No. The end-to-end scheduler must deliver more than 10 decisions/s.
Honesty boundary.

This bridge explains the chapter's first-order hardware trade without claiming NPU or model completeness.

2.0 range
Normalized teaching interval
49.9 dB
Ideal INT8 ceiling
5 Hz
Ideal boundary at 10 decisions/s

Go deeper in the chapter and validate the complete runtime on representative events.

Return to Edge AI Hardware with the maths unpacked →