44 Sensor Fusion Foundations

44.2 Fusion Combines Evidence

Sensor fusion combines measurements, features, or decisions from more than one source so an IoT system can estimate state with a clearer evidence boundary than any single sensor can provide. A fusion design should name what each sensor contributes, how time is aligned, how uncertainty is represented, and what the system does when sensors disagree or disappear.

The most common beginner mistake is treating fusion as simple averaging. Averaging can reduce random noise when sensors measure the same quantity with independent errors, but many IoT systems need complementary evidence. A GPS receiver provides slow absolute position; an IMU provides fast motion changes but drifts. A camera gives rich scene evidence but is sensitive to lighting; radar can keep range evidence under different visual conditions. Fusion is the discipline of combining those strengths while keeping their failure modes visible.

The contract starts with a state sentence that is narrow enough to test. "Estimate the temperature field in a greenhouse bay every 60 seconds" is reviewable; "understand the greenhouse" is not. Once the state is named, each sensor gets a role: one may provide absolute correction, another may provide fast relative change, and another may act as a fault witness when the first two disagree. This prevents a fusion design from mixing every available channel simply because data exists.

Fusion level is the next choice. Signal-level fusion combines readings close to the sensor. Feature-level fusion waits until useful features have been extracted. Decision-level fusion combines detector outputs when raw evidence cannot be shared. State fusion tracks a quantity over time, carrying uncertainty and health state forward from one update to the next.

A practical input-output view makes the same choice more concrete. Data-in/data-out fusion keeps the output as data, such as filtering raw image or sensor samples with a Gaussian, median, or total-variation-style filter. Data-in/feature-out fusion turns raw data into features, such as extracting labelled connected components from image pixels. Feature-in/feature-out fusion combines already-derived features, such as video and audio features for an activity window. Feature-in/decision-out or data-in/decision-out fusion drives pattern or object recognition. Decision-in/decision-out fusion combines separate recognizer outputs, votes, or incident labels. The review question is always what evidence was lost, summarized, or made harder to replay at that level.

To see why fusion level is an output contract, compare the two handoffs in Figure 44.1 before classifying the source relationships. The diagram holds the raw input and segmentation step constant, isolating what the next stage receives.

In Figure 44.1, the Raw pixels enter Segmentation and become the same binary mask. The Data-in / data-out branch returns a filtered image that remains pixel data, whereas Data-in / feature-out returns a feature list, including Component 1 at 42 px and smaller labelled components. This difference carries the narrative from representation to evidence: the feature branch is easier to classify but cannot replay every pixel-level choice made before the handoff.

Classical multisensor fusion also includes related information, not only live sensor streams. A calibration table, floor plan, map, equipment database, or known user context can make a fused estimate more accurate or more specific than one sensor could make alone. The review record should therefore name every source of evidence and whether it is measured now, looked up from an associated database, or assumed from the deployment model.

Source relationships give that record a useful first classification. Complementary fusion combines sources that see different parts of the problem, such as cameras with different fields of view. Redundant fusion combines sources that observe the same target so confidence or fault tolerance can improve. Cooperative fusion combines sources whose joint interpretation creates new information, such as using an accelerometer, gyroscope, and magnetometer together to estimate rotational angles.

Before choosing a fusion method, identify the relationship among its sources. The three-panel diagram in Figure 44.2 contrasts three relationships because each supports a different claim about confidence, coverage, or newly inferred information.

The diagram’s COMPLEMENTARY panel in Figure 44.2 combines Accel, Gyro, and Magno because different fields of view cover more of an IMU orientation problem. REDUNDANT shows five co-located temperature sensors observing the same target, where agreement can raise confidence only if errors are sufficiently independent. COOPERATIVE combines Cam A and Cam B to create a 3D motion result that neither view supplies alone. Those distinctions set the chapter’s running evidence boundary: coverage, confidence, and emergent information need different validation tests and must not be collapsed into the generic claim that more sensors are better.

The cooperative case rewards a closer look, because it is the only relationship of the three that manufactures a measurement no input owns. Follow Figure 44.3 as two sensor teams build heading and height for one asset tracker.

Trace the diagram Figure 44.3 from the top: the Attitude team hands gyroscope turn rate, accelerometer gravity, and magnetometer north to an orientation filter whose output is heading 118°, while the Height team below pairs the barometer’s pressure trend with the same accelerometer working a second job to produce height +2.4 m. The published track is genuinely new information; the closing caution earns its place in the fusion contract because that track inherits every input’s failure modes, so a nearby magnet or a passing weather front must surface as per-input health flags rather than silently bending the fused number.

The benefit of multiple sensors should be stated in that same evidence language. Redundancy can let a system survive one failed sensor; independent uncertainty can increase confidence; different sensor dimensions can describe more of the environment or process; and denser or differently placed sensors can improve spatial or temporal resolution. None of those benefits is automatic, so the design still has to prove independence, placement, calibration, and degraded-mode behavior.

The first practical challenge is often not the algorithm but making the inputs comparable. A fusion record should say whether evidence arrived as analogue voltage, digital counts, text, audio, image, or video; which coordinate frame, unit, sample frequency, and timestamp each source uses; and how clock skew, buffering, or propagation delay is handled before samples are combined. Without that record, a fused estimate can quietly mix different dimensions, different coordinate systems, or measurements that describe different physical moments.

Source relationship says why evidence is combined; fusion level says what representation is combined. The pump-bay scenario in Figure 44.4 makes that placement choice concrete: one maintenance question, four different places the merge could happen, and a different survivor set of evidence after each.

Reading Figure 44.4 from the top, Signal level merges the two probes sample-by-sample at the sensor node, where the shared 1 kHz clock still exists, and keeps full waveforms replayable. Pixel level averages eight thermal frames inside the camera; Feature level joins vibration RMS 4.2 mm/s with a 6 px hot-spot area at the gateway, after which only features can be replayed; Decision level lets the WORN and HOT verdicts vote in the service backend. The replay chips carry the provenance story: each step upward ships fewer bytes and discards detail no later stage can reconstruct, which is why the fusion record must be captured at the level where fusion actually ran.

Overview Knowledge Check

44.3 Roles and Complementary Filter

A good introductory fusion design begins with the state variable. For a small IMU example, the state may be device tilt angle. The gyroscope updates quickly and captures short-term motion, but integrated gyro rate drifts over time. The accelerometer can estimate gravity direction over longer intervals, but it is noisy during vibration or acceleration. A complementary filter combines the short-term gyro estimate with the long-term accelerometer correction.

Worked example: one complementary-filter tilt update
previous fused angle: 10.0 deg
gyro rate: 20.0 deg/second
time step: 0.020 second
accelerometer tilt estimate: 8.0 deg
alpha: 0.98

gyro prediction:
10.0 + (20.0 x 0.020) = 10.4 deg

fused angle:
(0.98 x 10.4) + (0.02 x 8.0) = 10.352 deg

interpretation:
The gyro carries most of the fast motion for this update.
The accelerometer gently pulls the estimate back toward gravity.
The alpha value is a design choice that must be validated for the device,
motion pattern, vibration, and latency target.

Practitioner Knowledge Check

44.4 Fusion Quality Constraints

Fusion improves decisions only when the inputs carry useful, timely, and sufficiently independent evidence. If two sensors share the same bias, combining them can make the estimate look more confident without making it more correct. If timestamps are misaligned, a fast sensor and a slow sensor may describe different physical moments. If a sensor health state is hidden, the fused result can keep using stale or invalid evidence.

Redundancy reduces random noise only when the sensors do not share the same systematic error. The time-series chart in Figure 44.5 tests that assumption over time before the quality constraints turn agreement into a fusion rule.

Across Day 0 to Day 28 in the chart Figure 44.5, S1, S2, S4, and S5 remain clustered while S3 (biased) separates and reaches the annotated S3 drifts +3 deg C condition. Averaging the clustered independent noise can stabilise the estimate, but folding S3 into the result without diagnosis would preserve its calibration bias rather than cancel it. This is why the quality record below needs covariance or shared-cause checks, calibration history, and an exclusion decision instead of treating sensor count as automatic confidence.

Diagnosing S3 is only half the job; the fusion rule must also say what happens at the instant of publication. The pipeline in Figure 44.6 freezes a single timestamp from that same five-sensor bay and walks the readings through an explicit disagreement rule.

At the top of Figure 44.6, the Median gate measures every probe against the median 22.5 °C and quarantines S3 at +3.1 °C, while the Keep four branch below averages the survivors to 22.45 °C. Two details make that output defensible. The published record carries 4-of-5 and S3 out beside the value, so a reviewer sees the disagreement decision instead of reconstructing it; and the amber comparison prices the alternative — a blind five-way average would publish 23.08 °C with the calibration fault hidden inside the fused number.

The independence assumption is easy to overstate. Suppose sensor A reports 10.0 units with random uncertainty of about 0.5 units and sensor B reports 10.4 units with the same uncertainty. If their errors are independent, a simple average gives 10.2 units and the random uncertainty can shrink because the noise sources partly cancel. But if both sensors were calibrated with the same offset reference that is +0.4 units high, the average is still biased. The fused output may display a smaller random-error band while the real value remains shifted in the same direction.

Shared-bias example
sensor A: 10.0 units, random uncertainty about 0.5
sensor B: 10.4 units, random uncertainty about 0.5
simple average: (10.0 + 10.4) / 2 = 10.2

if errors are independent:
random noise is reduced by averaging

if both sensors share a +0.4 calibration bias:
the average still carries the same hidden offset
publish the common calibration source as a risk,
not just the smaller-looking random uncertainty

Timing can create a different kind of false confidence. An IMU may update at 100 Hz while GPS updates at 1 Hz. A valid fusion design states whether it buffers, interpolates, predicts the slow measurement forward, or rejects stale evidence. Without that rule, the filter may combine a current acceleration sample with a position fix from a different maneuver and then blame the disagreement on sensor noise.

Under the hood, every fused estimate should therefore carry three kinds of evidence: the measurement values, the timing relationship among them, and the error relationship among them. That record is what allows a later reviewer to separate random noise, shared calibration bias, stale input, saturation, and true state change.

Under-the-Hood Knowledge Check