Edge ML Latency Calculator
Try Change Model inference (ms) from 65 to 100.
Observe Watch Pipeline total and Total with margin against the 200 ms budget.
Explain Explain why a 10% safety margin reduces Available for inference.
Calculate latency
Reset
Technical boundaries. Pipeline total adds the five stage times. Safety margin uses a percentage of the total latency budget, and fps uses pipeline total plus safety margin.
Edge ML Latency Calculator
Use this tool to break down your edge ML inference pipeline and check if it fits within your latency budget.
viewof totalBudget = Inputs. range ([10 , 1000 ], {value : 200 , step : 10 , label : "Total latency budget (ms)" })
viewof captureTime = Inputs. range ([1 , 100 ], {value : 33 , step : 1 , label : "Camera capture (ms)" })
viewof preprocessTime = Inputs. range ([1 , 50 ], {value : 8 , step : 1 , label : "Preprocessing (ms)" })
viewof inferenceTime = Inputs. range ([5 , 500 ], {value : 65 , step : 5 , label : "Model inference (ms)" })
viewof postprocessTime = Inputs. range ([1 , 50 ], {value : 5 , step : 1 , label : "Post-processing (ms)" })
viewof decisionTime = Inputs. range ([1 , 50 ], {value : 10 , step : 1 , label : "Decision + action (ms)" })
viewof safetyMarginPct = Inputs. range ([0 , 30 ], {value : 10 , step : 1 , label : "Safety margin (%)" })
viewof minFps = Inputs. range ([1 , 60 ], {value : 5 , step : 1 , label : "Minimum required fps" })
pipelineTotal = captureTime + preprocessTime + inferenceTime + postprocessTime + decisionTime
safetyMs = totalBudget * (safetyMarginPct / 100 )
availableForInference = totalBudget - captureTime - preprocessTime - postprocessTime - decisionTime - safetyMs
fitsInBudget = pipelineTotal <= (totalBudget - safetyMs)
achievedFps = 1000 / (pipelineTotal + safetyMs)
meetsFps = achievedFps >= minFps
inferencePct = (inferenceTime / pipelineTotal) * 100
html `<div style="background: var(--bs-light, #f8f9fa); padding: 1.5rem; border-radius: 8px; border-left: 4px solid #E67E22; margin-top: 0.5rem;">
<h4 style="color: var(--bs-body-color); margin-top: 0;">Latency Budget Analysis</h4>
<table style="width: 100%; border-collapse: collapse; color: var(--bs-body-color);">
<tr style="border-bottom: 1px solid #ddd;"><th scope="row" style="font-weight: normal; text-align: left;">Camera capture</th><td style="text-align:right;"> ${ captureTime} ms</td></tr>
<tr style="border-bottom: 1px solid #ddd;"><th scope="row" style="font-weight: normal; text-align: left;">Preprocessing</th><td style="text-align:right;"> ${ preprocessTime} ms</td></tr>
<tr style="border-bottom: 1px solid #ddd;"><th scope="row" style="font-weight: normal; text-align: left;"><strong>Model inference</strong></th><td style="text-align:right;"><strong> ${ inferenceTime} ms</strong> ( ${ inferencePct. toFixed (0 )} % of pipeline)</td></tr>
<tr style="border-bottom: 1px solid #ddd;"><th scope="row" style="font-weight: normal; text-align: left;">Post-processing</th><td style="text-align:right;"> ${ postprocessTime} ms</td></tr>
<tr style="border-bottom: 1px solid #ddd;"><th scope="row" style="font-weight: normal; text-align: left;">Decision + action</th><td style="text-align:right;"> ${ decisionTime} ms</td></tr>
<tr style="border-bottom: 2px solid #333;"><th scope="row" style="font-weight: normal; text-align: left;"><strong>Pipeline total</strong></th><td style="text-align:right;"><strong> ${ pipelineTotal} ms</strong></td></tr>
<tr style="border-bottom: 1px solid #ddd;"><th scope="row" style="font-weight: normal; text-align: left;">Safety margin ( ${ safetyMarginPct} %)</th><td style="text-align:right;"> ${ safetyMs. toFixed (0 )} ms</td></tr>
<tr><th scope="row" style="font-weight: normal; text-align: left;"><strong>Total with margin</strong></th><td style="text-align:right;"><strong> ${ (pipelineTotal + safetyMs). toFixed (0 )} ms</strong></td></tr>
</table>
<div style="margin-top: 1rem; display: grid; grid-template-columns: 1fr 1fr; gap: 0.5rem;">
<div style="padding: 0.5rem; background: ${ fitsInBudget ? '#d4edda' : '#f8d7da' } ; border-radius: 4px; text-align: center;">
<strong style="color: ${ fitsInBudget ? '#155724' : '#721c24' } ;"> ${ fitsInBudget ? 'PASS' : 'FAIL' } : Budget ${ totalBudget} ms</strong><br/>
<small style="color: var(--bs-body-color);">Available for inference: ${ availableForInference. toFixed (0 )} ms</small>
</div>
<div style="padding: 0.5rem; background: ${ meetsFps ? '#d4edda' : '#f8d7da' } ; border-radius: 4px; text-align: center;">
<strong style="color: ${ meetsFps ? '#155724' : '#721c24' } ;"> ${ meetsFps ? 'PASS' : 'FAIL' } : ${ achievedFps. toFixed (1 )} fps (need ${ minFps} )</strong><br/>
<small style="color: var(--bs-body-color);">Frame interval: ${ (pipelineTotal + safetyMs). toFixed (0 )} ms</small>
</div>
</div>
</div>`
Back to the chapter