1 Answers
In numerical analysis, catastrophic cancellation is the phenomenon that subtracting good approximations to two nearby numbers may yield a very bad approximation to the difference of the original numbers.
For example, if there are two studs, one L 1 = 254.5 cm {\displaystyle L_{1}=254.5\,{\text{cm}}} long and the other L 2 = 253.5 cm {\displaystyle L_{2}=253.5\,{\text{cm}}} long, and they are measured with a ruler that is good only to the centimeter, then the approximations will be L ~ 1 = 255 cm {\displaystyle {\tilde {L}}_{1}=255\,{\text{cm}}} and L ~ 2 = 253 cm {\displaystyle {\tilde {L}}_{2}=253\,{\text{cm}}}. These may be good approximations, in relative error, to the true lengths: the approximations are in error by less than 2% of the true lengths, | L 1 − L ~ 1 | / | L 1 | < 2 % {\displaystyle |L_{1}-{\tilde {L}}_{1}|/|L_{1}|<2\%}.
However, if the approximate lengths are subtracted, the difference will be L ~ 1 − L ~ 2 = 255 cm − 253 cm = 2 cm {\displaystyle {\tilde {L}}_{1}-{\tilde {L}}_{2}=255\,{\text{cm}}-253\,{\text{cm}}=2\,{\text{cm}}} , even though the true difference between the lengths is L 1 − L 2 = 254.5 cm − 253.5 cm = 1 cm {\displaystyle L_{1}-L_{2}=254.5\,{\text{cm}}-253.5\,{\text{cm}}=1\,{\text{cm}}}. The difference of the approximations, 2 cm {\displaystyle 2\,{\text{cm}}} , is in error by 100% of the magnitude of the difference of the true values, 1 cm {\displaystyle 1\,{\text{cm}}}.
Catastrophic cancellation may happen even if the difference is computed exactly, as in the example above—it is not a property of any particular kind of arithmetic like floating-point arithmetic; rather, it is inherent to subtraction, when the inputs are approximations themselves. Indeed, in floating-point arithmetic, when the inputs are close enough, the floating-point difference is computed exactly, by the Sterbenz lemma—there is no rounding error introduced by the floating-point subtraction operation.