Edge & Fog Computing · Study deck
Edge AI Fundamentals: Why and When
Picture a pump that makes a new sound before a bearing fails.
Edge Eddie is your guide for this deck.
After studying this chapter
Learning objectives
Local inference is useful when it improves a defined decision within the target's operating limits.
- Training and edge inference are separate jobs.The pump’s field computer can apply the frozen artifact to live input, while development or cloud systems retain training and model-review responsibilities.
- A local decision can improve timing, bandwidth, and offline operation.The pump can report a compact alert instead of every waveform and keep its safe local response available during a WAN outage.
- Optimization must preserve useful behavior within measured target limits.A smaller file still shares memory with input buffers and the runtime, and its scores, latency, power, and thermal behavior need testing.
- Uncertainty handling, monitoring, and rollback remain necessary after deployment.The pump needs a defined weak-score response, retained model context, reviewed field evidence, and a controlled path back from a failed update.
Major section
In 60 Seconds
A local prediction needs a safe, clearly bounded job.
- The pump’s sound window can support a local maintenance warning.A computer beside the machine can label the input promptly while a remote service compares the longer operating history for later review.
- The owner must define known classes, a deadline, and permitted responses.A useful warning depends on when the result arrives and what the application may do with that particular classification of the sound.
- Weak or unknown results need a safe fallback.The local result may raise an alarm or enter a safe state while a person or separate proven rule approves a costly change.
- Input time, device identity, model version, score, and action belong together.That record lets maintenance staff connect the pump’s prediction with the exact input and deployed artifact instead of reviewing an isolated score.
Major section
Field conditions and failed connections
Field cases test whether the prediction remains useful beyond clean training examples.
- Noise, sensor loss, heat, and restarts can change device behavior.The pump test includes normal, worn, noisy, missing-sensor, hot, restarted, and disconnected cases rather than relying on clean development examples.
- False alarms and missed faults are part of target validation.A fast classification can still be unsuitable when its false-positive cost, missed-event cost, or recovery behavior breaks the intended maintenance workflow.
- Held-back examples and operator findings can help evaluate deployed results.Reviewed input evidence can tie the warning to the later field outcome, distinguishing an actual fault from a changed sensor or environment.
- The safe local fallback must remain available during network loss.Remote review can improve a later model version, but it cannot be the pump’s only response when the current remote connection is unavailable.
Major section
Training, local inference, and cloud review
An edge design changes the division of work across the whole system.
- Training usually remains outside the device performing local inference.The pump runs a frozen model artifact, while training datasets, model development, and repeatable evaluation usually remain on development or cloud systems.
- Sensors, processor, memory, power, and update policy constrain the artifact.The model must run alongside the real input pipeline and operating software, with measured headroom instead of a file-size-only approval.
- Compact events and selected samples can replace continuous raw uploads.The remote pump can report a short evidence window when vibration changes, leaving the cloud to track reviewed alerts, versions, and site drift.
- Cloud inference remains reasonable when local placement adds little value.A planner’s weekly forecast can use richer centralized history when the decision does not require an immediate response beside the sensor.
Major section
Placement Drivers
This placement map compares local, hybrid, and cloud choices against the workload's needs.
- The edge column supports decisions requiring local timing or autonomy.Its response, safety, offline, and privacy conditions need evidence for the actual local action rather than a general preference for device processing.
- The hybrid column keeps fast filtering near the event.Cloud services can review selected evidence, retrain models, and coordinate rollout while the local system retains responsibility for its immediate response.
- The cloud column supports rich history and less urgent planning.Batch planning and long-horizon analysis can wait for centralized processing when a local prediction would not improve the required workflow.
- The lower questions connect placement with the workload’s constraints.Latency, safety, network dependence, and data volume explain why the chosen edge, hybrid, or cloud arrangement fits this particular decision and fallback.
Major section
Timing, uplink savings, and local resilience
A placement driver must change the practical decision rather than merely sound attractive.
- Local inference can meet a deadline before the physical situation changes.A camera that must stop a conveyor needs the full sensing-to-action path measured against the required response, including the safe fallback.
- A compact event can use far less uplink than raw input.The chapter’s 2 Mbps stream becomes roughly 160 bits per second when local inference sends a 200-byte event every 10 seconds.
- Local processing can preserve useful behavior when the WAN fails.The remote pump can keep monitoring and retain an alert window for later review while its wider network connection is unavailable.
- Data minimization still needs sample and retention rules.Raw debug clips, timestamps, labels, and logs can expose sensitive behavior even when the prediction itself runs locally beside the sensor.
Major section
Edge AI Lifecycle
This lifecycle connects representative data with deployment evidence and later model review.
- The early stages define the decision and prove a useful signal.The figure begins with action, cost, and fallback, then real sensor data and a baseline before conversion or accelerator choices enter the review.
- Optimization can adapt the baseline to the intended runtime.Quantization, pruning, and architecture choices need behavioral checks because converting the pump model does not establish that its field decision remains useful.
- Hardware validation measures latency, memory, and power before release.The final artifact and sensor pipeline must also demonstrate thermal behavior, startup, accuracy, and failure handling on the intended target device.
- Monitoring returns field evidence to the next lifecycle decision.Drift, model versions, reviewed samples, and rollback results help the team reopen the right data or modeling choice after deployment.
Activity 1 · Draw it
✎ Rebuild the model lifecycle
I want you to follow the evidence all the way back when a field prediction changes.
On paper, sketch representative data, baseline modeling, optimization, hardware validation, deployment, and monitoring. Add an arrow from field evidence back to an earlier decision. Mark where latency, memory, and power must be measured.
4 minutes · Pen and paper · Answer: Activity 1
Major section
Optimization Basics
Optimization must preserve the useful decision while reducing the target's resource demands.
- Quantization changes numeric representation and can shift model scores.Lower-precision weights and activations can reduce memory or improve runtime efficiency, but the resulting classifications still need testing on the target.
- Pruning and distillation produce smaller models through different changes.Pruning removes selected connections or channels; distillation trains a smaller model to imitate a larger model or a reviewed decision process.
- Smaller architectures and feature choices can reduce target resource demands.Windows, spectra, counts, or summary statistics change the input workload as well as model size, so preprocessing remains part of the deployment decision.
- The optimized artifact needs fresh target validation.A small model file can still fail the pump’s memory, latency, confidence, or thermal requirements when the complete sensor pipeline is running.
Major section
Input processing and runtime headroom
The deployed input pipeline and runtime can invalidate a good notebook result.
- Preprocessing must match the pipeline used to validate the model.Different scaling, filtering, windowing, or feature extraction can change the pump’s input before the frozen model produces its confidence score.
- Thresholds and confidence behavior determine which results reach the application.Low confidence and sensor faults need explicit policies so model uncertainty cannot silently become permission to take an unsafe machine action.
- Memory must hold the runtime and buffers alongside the model.Input windows, tensor and activation buffers, preprocessing state, networking, and application logic need simultaneous RAM headroom on the target device.
- Latency, power, and heat need measurements from the intended device class.Sensor acquisition, preprocessing, inference, action, and logging all contribute to the deadline, while duty cycle and enclosure conditions affect power and thermal behavior.
Major section
When Cloud or Hybrid Is Better
Cloud and hybrid placement remain useful when local inference adds little value.
- Batch planning can benefit from richer centralized history.A weekly demand forecast or long-horizon analysis can use a larger cloud model when the workflow tolerates the remote processing delay.
- A target without measured resource margin may be unsuitable.Memory, power, thermal, or latency limits can justify a smaller architecture, a gateway serving several devices, or a different inference placement.
- Hybrid systems can handle common cases locally and review uncertainty remotely.The edge performs fast filtering, while selected hard samples help the cloud review behavior and prepare the next training set or model version.
- Every chosen split still needs a safe uncertain-result response.A disconnected pump cannot rely solely on sending its low-confidence input to a remote service before deciding how to keep the machine safe.
Major section
Release pitfalls and recovery control
A converted artifact is only one part of a deployable inference system.
- Hardware selection must follow the local decision and operating requirements.Choosing a board or accelerator first can leave the pump with mismatched memory, power, update, and cost assumptions before its useful action is defined.
- Target validation needs realistic sensor conditions beyond clean lab data.Noise, vibration, placement, sensor aging, and environmental changes can alter the field input even when the development model’s accuracy looks good.
- Reviewed evidence and operator feedback help explain changing predictions.Selected input samples and later maintenance findings let the team investigate a warning and decide whether data, preprocessing, thresholds, or models need revision.
- Staged updates need version inventory, health checks, and rollback.A small device cohort provides field observations before wider rollout, while a safe reversion route limits the effect of an unsuitable model update.
Activity 2 · Predict
✎ Lose the cloud connection
I want you to keep the pump’s safe response available when the remote link disappears.
The pump model produces a low-confidence result during a network outage. Write the local response and the evidence to retain. Explain why sending the uncertain result to the cloud cannot be the only fallback.
3 minutes · Pen and paper · Answer: Activity 2
Major section
Sampling Before Inference
Sampling and artifact sizing constrain the evidence available to an edge model.
- A vibration model needs samples that preserve the fault frequencies.The chapter’s 1200 Hz fault band must survive acquisition before a local classifier can learn or recognize useful features from its input window.
- A 1200 Hz fault band has a 2400 samples/s Nyquist boundary.The chapter contrasts that frequency requirement with aliasing: at 1600 samples/s, a 2000 Hz component folds to 400 Hz.
- Too-low sampling can disguise a signal before inference begins.The aliased 400 Hz result shows why a strong classifier cannot recover the original frequency identity from an already misleading input window.
- ADC resolution determines amplitude detail separately from sampling rate.The pump example’s ±4g, 12-bit ADC has a 1.95×10⁻³g step, while the sample rate determines which input frequencies remain distinguishable.
Major section
Size the Deployed Runtime
The deployed artifact shares device resources with input processing and operational evidence.
- The frozen artifact is only part of deployed memory use.The chapter’s 5 million weights need about 20 MB as FP32 or about 5 MB as INT8, before runtime overhead.
- Input windows and the inference runtime need their own memory.The illustrative gateway has 16 MB available, with a 4 MB tensor arena and a 2 MB input buffer alongside the optimized model.
- Target validation measures resources with the sensor pipeline running.Preprocessing, allocation, inference, confidence policy, alert timing, and logging can invalidate a file-size-only approval even when a desktop test succeeds.
- Cloud records support model review and controlled later rollouts.The remote pump’s reviewed alert windows, model versions, and site-drift evidence help the team investigate changes and coordinate a replacement with rollback support.
Major section
The action and evidence in the release record
A release record connects model behavior with the action that the application may take.
- The release record must state false-alarm and missed-event costs.The local warning needs an agreed permitted action and safe fallback, keeping the model’s uncertain prediction separate from authority for a dangerous change.
- Exact preprocessing and model versions identify the measured artifact.A target result belongs to the input scaling, filtering, windowing, and frozen model used in that test rather than every future exported file.
- Target tests must cover resource use, uncertainty, and network loss.Memory headroom, latency, power, low-confidence behavior, and the local alert path need evidence from the actual sensor and intended device class.
- Monitoring and rollback keep acceptance useful after the first deployment.Reviewed alerts, version tracking, drift signals, staged updates, and a tested reversion route connect later field behavior with the original release decision.
Deck summary
Key takeaways
A justified edge placement joins one local decision with target and operational evidence.
- A local model needs a timely, useful job.The pump’s short alert window can avoid continuous waveform uploads and support local monitoring when a remote inference call is unavailable.
- Optimization must preserve the decision within sensor and runtime limits.The final graph shares resources with acquisition, preprocessing, buffers, and action code, so target validation must extend beyond notebook accuracy or file size.
- Uncertain input and sensor faults need explicit fallback behavior.The application retains evidence and follows its safe policy instead of turning a weak model score into an unsupported automatic machine action.
- Evidence review and controlled updates connect operation with future improvements.The cloud retains reviewed alert windows and model versions, while staged rollout and rollback keep later pump-model changes observable and reversible.
Retrieval practice
Recall check 1 of 6
Edge Eddie says: answer from memory, then check your reasoning.
Q1A vibration monitor can summarize a machine state locally and only upload alerts and short evidence windows. Which placement driver is strongest?
Show answer
Answer: A A strong edge AI candidate has a measurable reason to make the decision near the data source, such as bandwidth reduction, resilience, latency, privacy, or local control.
Retrieval practice
Recall check 2 of 6
Edge Eddie says: answer from memory, then check your reasoning.
Q2Why should a team train and evaluate a baseline model before optimizing for a microcontroller or accelerator?
Show answer
Answer: C The lifecycle starts by proving the decision and data are viable.
Retrieval practice
Recall check 3 of 6
Edge Eddie says: answer from memory, then check your reasoning.
Q3A camera counts room occupancy locally and uploads only counts. What privacy or governance issue still needs review?
Show answer
Answer: B Data minimization must cover raw data, logs, labels, evidence, timestamps, telemetry, retention, and access control.
Retrieval practice
Recall check 4 of 6
Edge Eddie says: answer from memory, then check your reasoning.
Q4Place each edge AI element where it lives so you can explain what stays local and what remains connected.
Show answer
Answer: A Separate observation, nearby decisions, and learning governance so you can explain why inference and action stay local while monitoring and updates remain connected.
Retrieval practice
Recall check 5 of 6
Edge Eddie says: answer from memory, then check your reasoning.
Q5Which statement best describes edge AI?
Show answer
Answer: C Edge AI places optimized inference near the source when that placement improves the application.
Retrieval practice
Recall check 6 of 6
Edge Eddie says: answer from memory, then check your reasoning.
Q6What is the safest way to decide whether a model is ready for edge deployment?
Show answer
Answer: B Edge deployment readiness combines target validation, representative data, defined fallback behavior, monitoring, and update control.
Print reference
Answers 1 of 2
Answer key.
- A · A strong edge AI candidate has a measurable reason to make the decision near the data source, such as bandwidth reduction, resilience, latency, privacy, or local control.
- C · The lifecycle starts by proving the decision and data are viable.
- B · Data minimization must cover raw data, logs, labels, evidence, timestamps, telemetry, retention, and access control.
- A · Separate observation, nearby decisions, and learning governance so you can explain why inference and action stay local while monitoring and updates remain connected.
Print reference
Answers 2 of 2
Answer key.
- C · Edge AI places optimized inference near the source when that placement improves the application.
- B · Edge deployment readiness combines target validation, representative data, defined fallback behavior, monitoring, and update control.
Print reference
Activity 1 answer
Model answer.
Draw it: The model passes from representative data to baseline, optimization, hardware validation, deployment, and monitoring. Hardware validation measures latency, memory, and power on the target. Reviewed field evidence returns to data or modeling decisions, while update control preserves a rollback route.
Print reference
Activity 2 answer
Model answer.
Predict: The application follows its safe fallback policy, retains the input evidence and model/device context, and requests review when possible. A cloud-only fallback is unavailable during the outage. Model uncertainty must remain separate from permission to take a dangerous action.