Analytics & ML · Study deck
Sensor Fusion Applications
Imagine a shuttle enters a busy depot and several sensors disagree about an object near its path.
Data Dora is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: A low-rate Bluetooth path may be enough for a 200 Hz quaternion stream, while a faster 2.4 GHz path may be needed for roughly 2500 Hz quaternions or higher-rate gyroscope and accelerometer evidence.
- Explain: The record should therefore name the body placement, sample rates, radio mode, latency budget, wireless charging state, calibration pose, dropped-frame policy, and whether the display is a demonstration view or validated motion evidence.
- Explain: Optical rigs, marker suits, depth cameras, and body-worn IMUs can all produce a plausible moving body, but they carry different infrastructure and failure boundaries.
Major section
A Clear First Route
The team must decide whether the shuttle may move, slow down, or stop.
- Latency means the wait from a real event to the system's response.
- This page starts with one job.
- Last, choose the safe next step and the fallback if one sensor fails.
- This first route is a guide to the main choice.
Major section
A Clear First Route (continued)
A neat result can still be wrong when clocks drift or two readings refer to different objects.
- Under the Hood adds clock drift, data links, gradual sensor faults, and error growth.
- They do not reverse its main claim.
- If two sources differ, keep that fact in the record.
- A person may inspect the site.
Major section
A Clear First Route (continued)
A late result may be true about the past and still be unsafe now.
- A missing result is also useful news when the system shows it at once.
- A local rule may hold a safe state.
- A remote team may ask for more proof.
- The right step depends on the claim that was tested.
Major section
Applications Define Fusion Contracts
A phone needs a stable orientation for display and interaction.
- A wearable needs activity or fall evidence.
- A building system needs a trustworthy air-quality or occupancy state.
- A mobile robot needs an obstacle, pose, or track estimate with known uncertainty.
- The same raw sensor can play different roles in different applications.
Major section
Applications Define Fusion Contracts (continued)
Optical rigs, marker suits, depth cameras, and body-worn IMUs can all produce a plausible moving body, but they carry different infrastructure and failure boundaries.
- An accelerometer can supply gravity for roll and pitch, step features for activity recognition, vibration evidence for equipment monitoring, or a fault signal when it saturates.
- Daily-life context recognition often uses indirect traces rather than one obvious sensor.
- Motion capture is a useful stress test for that contract.
Major section
Applications Define Fusion Contracts (continued)
A good application design records the state being estimated, the sensors that can observe it, the failure modes each sensor introduces, and the evidence that downstream users need before acting.
- A faucet, toilet, or tub event may appear as a pressure drop and recovery shape; a gaze trace may show attention only after it is tied to scene context.
- An Orient-style wearable IMU design is attractive because it can be fully wireless and interactive, but the application still has to choose what travels over the radio.
- Camera-based 3-D motion capture is a complementary fusion case, not just a bigger camera array.
Major section
Applications Define Fusion Contracts (continued)
Performance and game cases make the same contract stricter because the fused pose may drive an actuator immediately.
- A low-rate Bluetooth path may be enough for a 200 Hz quaternion stream, while a faster 2.4 GHz path may be needed for roughly 2500 Hz quaternions or higher-rate gyroscope and accelerometer evidence.
- The record should therefore name the body placement, sample rates, radio mode, latency budget, wireless charging state, calibration pose, dropped-frame policy, and whether the display is a demonstration view or validated motion evidence.
- Decision Boundary The threshold for display, alerting, control, or actuation, including confidence and freshness limits.
Major section
Applications Define Fusion Contracts (continued)
Each camera or marker view sees only part of the body and can be blocked by occlusion, lighting, reflective clothing, or calibration drift.
- A defensible capture record names the camera layout, calibration volume, marker set, synchronization method, reconstruction error, missing-marker policy, and the conditions where a depth-camera or wearable-IMU fallback is allowed to contribute.
- For avatar, game, or sports demos, the screenshot is only a demonstration artifact; the validation record still has to prove latency, synchronization, dropped frames, and realistic movement.
- Without those limits, a fused value can look precise while being stale, unobservable, or driven by a failed input.
Major section
Applications Define Fusion Contracts (continued)
A wearable demo may show a MotionViewer axis trace on a phone, a human arm driving a humanoid robot, or Speckled Tango Dancers with chest, waist, and foot landmarks.
- Three or more well-placed sensors are usually enough to multilaterate a single source position from those time differences alone, without needing every sensor to agree on amplitude or timbre.
- The application should say what "good enough" means: maximum age, minimum sensor set, uncertainty limit, degraded-mode behavior, and validation dataset.
- The first stage of Figure: A fusion application should name the decision pipeline separates Temperature,: Humidity,: Motion,: Light, and: Door readings.
Major section
Match Sensors by State and Rate
Application fusion is usually more than averaging values.
- Measurements must describe the same state at compatible times, use the same coordinate frame and units, and include uncertainty or quality labels.
- A simple inverse-variance fusion is a useful design check when several sensors report the same scalar state with independent errors.
Major section
Match Sensors by State and Rate (continued)
For tracking applications, a data-association step decides whether camera, lidar, radar, radio, or inertial evidence belongs to the same object or person before any update is applied.
- The application should still keep camera classification evidence and radar velocity evidence if those are needed for the decision.
- Inverse-variance weights therefore change with range: the camera's geometric error grows as $r^2$ even when its pixel matching stays constant.
- Interpretation: The output is pulled mostly by lidar because its stated uncertainty is smallest.
Major section
Timing Failures in Fusion
application rule: If stale error budget is 0.10 m, the 120 ms camera measurement needs prediction, downweighting, or rejection before range fusion.
- Many fusion failures are not caused by the filter equation.
- A measurement that was accurate when captured can be unsafe when used too late.
- Event Time Use the physical capture time for fusion.
Major section
Timing Failures in Fusion (continued)
Tracking systems therefore need event timestamps, prediction to the fusion time, stale thresholds, and a rule for publishing lower confidence when evidence is old or out of sequence.
- The 40 ms lidar measurement is inside the budget if its other quality gates pass.
- Arrival time can hide network, buffering, or processing delay.
- Ground Truth Validate against labeled walks, surveyed anchors, reference instruments, or reviewed incidents that match the use case.
Major section
Timing Failures in Fusion (continued)
Operating Envelope Define the environment, speed, load, lighting, weather, placement, and sensor health where the fused output is valid.
- The system reports a precise state after losing a key sensor.
- Fallback did not update uncertainty or mode labels.
- Activity, occupancy, or condition labels degrade after deployment changes.
- Training labels, sensor placement, or environment no longer match production.
Deck summary
Key takeaways
The team must decide whether the shuttle may move, slow down, or stop.
- A neat result can still be wrong when clocks drift or two readings refer to different objects.
- A late result may be true about the past and still be unsafe now.
- A phone needs a stable orientation for display and interaction.
- Optical rigs, marker suits, depth cameras, and body-worn IMUs can all produce a plausible moving body, but they carry different infrastructure and failure boundaries.
Retrieval practice
Recall check 1 of 3
Data Dora says: answer from memory, then check your reasoning.
Q1What should a sensor-fusion application define before choosing the algorithm?
Show answer
Answer: B Application requirements define the fusion output, evidence, and fallback rules.
Retrieval practice
Recall check 2 of 3
Data Dora says: answer from memory, then check your reasoning.
Q2In the obstacle-range example, why does lidar receive the largest normalized weight?
Show answer
Answer: A Inverse-variance fusion assigns larger weights to measurements with lower stated variance.
Retrieval practice
Recall check 3 of 3
Data Dora says: answer from memory, then check your reasoning.
Q3Why can a sensor measurement become unsafe even if it was accurate when captured?
Show answer
Answer: A Application-level fusion must account for event time, state motion, and stale evidence.
Print reference
Answers
Answer key.
- B · Application requirements define the fusion output, evidence, and fallback rules.
- A · Inverse-variance fusion assigns larger weights to measurements with lower stated variance.
- A · Application-level fusion must account for event time, state motion, and stale evidence.