← Back to the chapter · Next detail →
Technical detail · Validation and Outlier Detection
Agreement limits and significance
Do the extra arithmetic before accepting a method comparison. A regression line between method A and method B can look strong even when the differences are too wide for release. The Bland-Altman check should name the mean difference d, the standard deviation of the differences s, and the limits of agreement d +/- 1.96s after checking that the differences are roughly normal. If d = -27.2 and s = 34.8, the practical agreement band is about -95.4 to 41.1: method A may read nearly 95 units below method B or 41 units above it. That is a release decision, not just a plot annotation.
Normality and significance also need careful wording. A histogram with a normal curve or a Shapiro-Wilk, D'Agostino-Pearson, or Kolmogorov-Smirnov test can support the limits-of-agreement assumption, but it does not prove the devices are interchangeable. A p-value should be read against a stated null hypothesis such as "no difference between methods." Smaller values mean the observed difference would be surprising under that null; they do not by themselves measure engineering importance, safety margin, or field fitness.