Small errors introduced by finite-precision arithmetic when representing real numbers and performing arithmetic operations; while individually tiny, they can accumulate, cancel, or interact with algorithms to produce significant deviations from exact results.

Principle

Finite word length and binary (or base-b) representation impose quantization and truncation; algorithmic operations (subtraction of similar magnitudes, ill-conditioned transforms, repeated accumulation) amplify these tiny discrepancies into observable error.

Demonstration

Summing a long list of alternating-sign floating-point numbers in single precision yields a biased result compared to double precision; iterative linear solves with poorly conditioned matrices show stagnation or drift due to accumulation of rounding in updates and residual computations.

Misapplication

Blaming model failure solely on round-off without examining truncation, modeling error, or algorithmic instability; increasing precision blindly when algorithmic reformulation (e.g., Kahan summation, preconditioning) would be more effective.

Consequence

Recognizing round-off error leads to choices like using appropriate precision, stable algorithms, compensated summation, scaling, and conditioning; ignoring it can produce nonphysical oscillations, loss of significance, or incorrect convergence behavior.

Reversal

The reversal is exact arithmetic behavior in which finite precision effects are negligible compared to modeling or discretization errors; numerics then behave deterministically with respect to arithmetic representation.

Boundary

Applies to floating-point and fixed-point computations across simulations, optimizations, and linear algebra; excluded are modeling errors, discretization truncation error, and machine-level faults (bit flips) which have distinct causes and remedies.

Semantic Tension

Often conflated with truncation error: truncation arises from approximating operators or series, round-off arises from finite representation and arithmetic; both interact and must be balanced when choosing precision and algorithms.

Synthesis

Round-off error is the finite-precision arithmetic noise that can grow through unstable operations or long accumulations; mitigating it requires numerically stable algorithms, appropriate precision, and error-aware software practices.