22 Sensor Linearization Methods

Choose Calibration Methods Without Hiding Measurement Error

fundamentals

signal

processing

linearization

22.1 In 60 Seconds

Linearization turns a curved sensor response into physical units, but it must not hide uncertainty. Pick the simplest method whose held-out residuals meet the target, keep the calibrated range explicit, and flag or clamp values outside the evidence instead of extrapolating confident numbers.

22.2 Start With the Story

Start with a physical signal that is noisy, delayed, sampled, quantized, calibrated, filtered, packed, and finally sent as a number someone will trust. The core idea in Sensor Linearization Methods is simple: signal processing is the bridge between the physical world and digital evidence, so every sampling, ADC, filter, and calibration choice changes the value that leaves the device. This page focuses that idea on Practice for selecting Taylor, piecewise-linear, lookup-table, and polynomial sensor linearization methods from calibration evidence, residual checks. In everyday IoT, temperature drift, vibration spikes, audio snippets, and lab traces all become decisions only after their limits and uncertainty are made visible. Start simple: trace one measurement through the chain, keep the raw-to-processed evidence, and move advanced math into the deeper review only when the simple chain no longer explains the result.

Phoebe’s Field Notes: Sampling and Quantizing Before You Linearize

Phoebe the physics guide

Phoebe’s Why

Before a calibration curve can be trusted, two earlier stages have to be trusted first. The sample times have to occur often enough to catch the real signal, and the ADC codes have to be fine enough that the linearization residual is not just quantization noise wearing a decimal point. Sampling too slowly does not simply lose detail – a fast event folds back and impersonates a slow one, so a smooth-looking trace can be hiding an aliased fake. Quantizing too coarsely adds a fixed staircase error to every reading, baked in before any fit ever runs. The thermistor’s own physics adds a third bound: its resistance-temperature law is exponential, so the sensor’s sensitivity in volts per degree changes across the range, and that changing sensitivity is exactly what turns one ADC code into a temperature error.

The Derivation

Sampling copies the signal spectrum every \(f_s\):

\[X_s(f) = \sum_{k=-\infty}^{\infty} X(f - k f_s)\]

The baseband copy and its neighbor stay separate only while \(f_{max} \leq f_s - f_{max}\), so

\[f_s \geq 2f_{max}\]

A component sampled too slowly folds back to

\[f_{alias} = |f_{signal} - n f_s|\]

Quantizing the ADC code:

\[q = \frac{V_{ref}}{2^N}\]

The rounding error is uniform on \([-q/2, q/2]\), so its RMS value and the ideal signal-to-noise ratio are

\[q_{rms} = \frac{q}{\sqrt{12}} \qquad \mathrm{SNR} = 6.02N + 1.76 \text{ dB}\]

The thermistor’s governing equation is exponential, not linear:

\[R(T) = R_0\, e^{B(1/T - 1/T_0)}\]

so its slope \(dR/dT = -BR(T)/T^2\) changes across the range – the entire reason a single straight-line fit fails.

Worked Numbers: The Cold-Chain Thermistor

The chapter does not fix a part number, so take a standard 10 k\(\Omega\) NTC (\(B = 3950\) K) divided against a 10 k\(\Omega\) fixed resistor and read by a 12-bit, \(V_{ref} = 3.3\) V ADC.

At \(0\,^{\circ}\mathrm{C}\) (\(T = 273.15\) K): \(R(T) = 10{,}000\,e^{3950(1/273.15\, -\, 1/298.15)} = 33{,}600\ \Omega\) (33.6 k\(\Omega\)), and divider sensitivity there is \(dV/dT \approx -30.9\) mV/K
Quantization: \(q = 3300/4096 = 0.806\) mV; RMS noise \(q/\sqrt{12} = 0.233\) mV; \(\mathrm{SNR} = 6.02(12) + 1.76 = 74.0\) dB
One ADC code near \(0\,^{\circ}\mathrm{C}\) is worth \(0.806/30.9 = 0.0261\,^{\circ}\mathrm{C}\) of thermistor temperature – finer than most field calibrations can hold, so quantization is not the error the residual check should chase
Nyquist side: a 90 s door-opening excursion has \(f_{max} = 1/90 = 0.0111\) Hz, so \(f_s \geq 2(0.0111) = 0.0222\) Hz – sample at least every \(1/0.0222 = 45\) s. A logger sampling only every 200 s (\(f_s = 0.005\) Hz, Nyquist frequency \(0.0025\) Hz) violates that bound and folds the excursion to \(f_{alias} = |0.0111 - 2(0.005)| = 0.00111\) Hz – a fake 15-minute drift where a 90 s door-opening actually occurred

22.3 Turn a Curved Response Into Correct Units

Many sensors do not produce an output that is proportional to what they measure. A thermistor’s resistance, for example, changes steeply and nonlinearly as temperature changes. Linearization is the step that converts that curved raw response into the correct physical units.

Picture a bathroom scale whose spring stiffens as it compresses. The pointer still moves, but not evenly, so you need a conversion that knows the spring’s curve to read true weight. Linearization is that conversion for a sensor.

If you only need the intuition, this layer is enough: linearization maps a sensor’s curved output to real units, and a smooth, precise-looking converted number is only as accurate as the calibration it rests on, never more.

The honesty rule matters most here. Making the numbers come out smooth is not the same as making them accurate. A conversion can present a clean value to many decimals while the real uncertainty lives in the calibration behind it. No method can be more accurate than the calibration data it was built from.

Consider an NTC thermistor on a cold-chain logger. The ADC code may change quickly near freezing and slowly at warmer temperatures, so a single slope can make the freezer end look acceptable while the room-temperature end drifts. A better workflow is to collect reference points across the shipping range, fit a Steinhart-Hart equation or a lookup table, then compare the converted temperatures against held-out reference readings. If the held-out residuals are still larger than the product’s tolerance, the display should not imply more accuracy than the calibration proves. The firmware also needs to know the calibrated range: an ADC code outside the tested span should be flagged or clamped, not silently converted into a confident Celsius value.

Linearization is valid only where calibration evidence supports the correction.

flowchart TD
  A["Raw sensor code"] --> B["Apply calibration method"]
  C["Reference calibration points"] --> B
  B --> D["Converted physical units"]
  D --> E["Compare held-out residuals"]
  E --> F{"Residuals within tolerance?"}
  F -- "yes" --> G["Ship with valid range and uncertainty"]
  F -- "no" --> H["Change method, add points, or relax claim"]
  A --> I{"Inside calibrated range?"}
  I -- "no" --> J["Flag or clamp instead of extrapolating"]

The One-Minute Linearization Decision

Know the curve

Get the sensor’s characteristic from the manufacturer or your own calibration before choosing a method.

Match method to constraints

Balance accuracy, operating range, and the device’s available compute and memory.

Keep error visible

Track residuals and the valid range so the conversion does not hide measurement uncertainty.

Beginner Examples

A nearly linear sensor over a small range may need only a straight-line fit with an offset and a slope.
A steeply curved sensor over a wide range needs a method that follows the curve, such as a lookup table or a fitted model.
A value shown to six decimals is not accurate to six decimals if the calibration was only good to a fraction of a unit.

Linearization Honesty Knowledge Check

If this gives you the core principle, you can stop here. Continue to Practitioner when you need to choose a method for a real sensor.

22.4 Apply It: Choose a Method From the Evidence

The practical job is to choose the simplest linearization method whose residuals meet the accuracy target within the device’s limits, then prove it on held-out data and protect against out-of-range inputs.

The Method Options

Method

How It Works

Best When

Watch Out For

Local (Taylor)

A straight-line approximation around one operating point.

The sensor runs in a narrow band near a known point.

Error grows quickly away from that point.

Piecewise-linear

Straight segments between chosen breakpoints.

Moderate curvature with limited compute.

Kinks at breakpoints; error set by segment count.

Lookup table

Stored output values, interpolated between entries.

Constrained compute with a repeatable curve.

Memory use and table density set accuracy.

Polynomial fit

One smooth equation fit to calibration data.

A smooth curve with enough calibration points.

High degree overfits and oscillates.

Characteristic equation

Invert the manufacturer’s published model.

The model and its coefficients are trustworthy.

Only as good as your calibration of its parameters.

Method choice starts with range, residual behavior, and maintainability, not with the smoothest-looking formula.

Walkthrough: From Sensor Curve to Validated Conversion

Characterize the sensor. Gather calibration points across the real operating range, not only a few convenient values.
Quantify nonlinearity and range. A nearly straight, narrow-range sensor needs far less than a steeply curved, wide-range one.
List device constraints. Available compute, memory, and required conversion speed narrow the field; a small microcontroller may favor a lookup table over a high-degree polynomial.
Choose the simplest method that meets the target. Prefer the least complex method whose residuals stay within budget.
Check residuals on held-out points. Large or patterned residuals mean the method or its parameters are wrong.
Guard the range and validate at release. Clamp or flag inputs outside the calibrated range, and re-validate against references before shipping and after any sensor or firmware change.

A releaseable conversion keeps the raw code, reference value, fit choice, residuals, and gate decision together.

Choosing by Evidence

Situation

Evidence

Reasonable Method

Why

Narrow operating band

Residuals small across the band.

Local (Taylor) linearization.

Minimal compute for a small range.

Wide range, compute available

Low-degree polynomial residuals acceptable.

Low-degree polynomial.

One compact equation covers the curve.

Wide range, constrained device

Needs fast, repeatable conversion.

Lookup table with interpolation.

Trades memory for compute.

Sharp local features

Curvature concentrated in a region.

Piecewise-linear, denser where curvature is high.

Bounded per-segment error where it matters.

Incremental Practice

Beginner

For a near-linear sensor over a small range, justify using a simple slope-and-offset fit.

Intermediate

For a constrained device and a steeply curved sensor, argue for a lookup table over a high-degree polynomial.

Advanced

Given residuals that form a systematic curved pattern after a linear fit, state what that pattern reveals and your next step.

Method Selection Knowledge Check

If your job is to select and validate a conversion, you can stop here. Continue to Under the Hood for how each method fails.

22.5 Under the Hood: How Each Method Fails

The deeper layer explains the error behavior behind each method, because choosing well means knowing how each one breaks.

Local (Taylor) Linearization

A local linearization keeps the first-order term of a series expanded around an operating point, which is the local slope, and discards the rest. Those discarded higher-order terms are the error, and they grow roughly with the square of the distance from the expansion point. The result is excellent near the point and increasingly wrong far from it.

Polynomial Fit and Overfitting

A polynomial of degree n has n + 1 coefficients and can pass exactly through n + 1 points. Passing through the points is not accuracy. High-degree fits, especially with equally spaced calibration points, can swing far above and below the true curve between the points, a behavior known as the Runge phenomenon, and they diverge quickly outside the calibrated range. Prefer the lowest degree whose held-out residuals are acceptable.

Piecewise-Linear and Lookup Tables

Both approximate the curve with straight segments, and a lookup table is essentially a piecewise-linear model stored as data. The error inside a segment depends on the curve’s bend and the segment width. For a smooth curve the worst-case interpolation error scales with the square of the segment width, so halving the segment width roughly quarters the error. Accuracy is therefore bought with more breakpoints or table entries, which costs memory.

Calibration Uncertainty Is the Floor

No linearization can be more accurate than the calibration points it is built from. If the reference is uncertain, every converted value inherits that uncertainty. Keep two ideas separate: model error is how well the curve fits the points, and calibration error is how good those reference points were in the first place.

Residual Analysis

Residuals are the differences between the model and held-out calibration points. Small, randomly scattered residuals suggest a good fit, while a systematic pattern, such as a consistent smile or trend, means the model is missing real structure. Residuals measured on the same points used to fit understate the true error, so always check against points the model did not see.

Validation uses independent holdout points and range guards; failing residuals send the method back for better evidence or a simpler fit.

Extrapolation Is Unbounded

Every method is valid only inside the calibrated range. Outside it, polynomial terms explode and tables have no data, so out-of-range inputs must be clamped or flagged rather than silently converted into confident, wrong numbers.

Common Pitfalls

Equating fit with accuracy. Passing through every calibration point can be overfitting, not correctness.
Raising polynomial degree to chase residuals. It can oscillate and worsen accuracy between and beyond the points.
Reporting more precision than the calibration supports. The converted value cannot beat its reference.
Extrapolating beyond the calibrated range. Error there is unknown and can be large.
Checking residuals only on the fitting points. That hides the real generalization error.

Overfitting Knowledge Check

At this depth, linearization is a trade among model flexibility, compute, memory, and honesty about uncertainty. The best method is the simplest one whose held-out residuals meet the target, applied only inside the range where the calibration can vouch for it.

22.6 Summary

Linearization converts a sensor’s nonlinear raw output into correct physical units.
A smooth, precise-looking converted value is never more accurate than the calibration behind it.
Common methods are local (Taylor) linearization, piecewise-linear, lookup tables with interpolation, and polynomial fits, each with different error behavior.
Choose the simplest method whose held-out residuals meet the accuracy target within the device’s compute and memory limits.
High-degree polynomials can overfit and oscillate, so passing through every calibration point is not proof of accuracy.
Every method is valid only inside the calibrated range, so out-of-range inputs must be clamped or flagged and residuals checked on held-out points.

Key Takeaway

Pick the simplest linearization method whose held-out residuals meet the accuracy target, keep the calibration uncertainty visible rather than hidden by smooth numbers, and never trust the conversion outside its calibrated range.