| title | description |
|---|---|
Error Propagation |
Explanations about propagation of errors in floating-point math. |
While the errors in single floating-point numbers are very small, even simple calculations on them can contain pitfalls that increase the error in the result way beyond just having the individual errors "add up".
In general:
- Multiplication and division are "safe" operations
- Addition and subtraction are dangerous, because when numbers of different magnitudes are involved, digits of the smaller-magnitude number are lost.
- This loss of digits can be inevitable and benign (when the lost digits also insignificant for the final result) or catastrophic (when the loss is magnified and distorts the result strongly).
- The more calculations are done (especially when they form an iterative algorithm) the more important it is to consider this kind of problem.
- A method of calculation can be stable (meaning that it tends to reduce rounding errors) or unstable (meaning that rounding errors are magnified). Very often, there are both stable and unstable solutions for a problem.
There is an entire sub-field of mathematics (in numerical analysis) devoted to studying the numerical stability of algorithms. For doing complex calculations involving floating-point numbers, it is absolutely necessary to have some understanding of this discipline.
The article What Every Computer Scientist Should Know About Floating-Point Arithmetic gives a detailed introduction, and served as an inspiration for creating this website, mainly due to being a bit too detailed and intimidating to programmers without a scientific background.