Computer Arithmetic

Overview

This section reviews techniques to minimize roundoff error in numerical computations. Key strategies include:

Example: Avoiding Catastrophic Cancellation

For the function \( f(x) = \frac{x^2 - 1}{x - 1} \), direct evaluation near \( x = 1 \) leads to an indeterminate form. Instead, simplify:

\( f(x) = \frac{(x-1)(x+1)}{x-1} = x+1 \).

This method is crucial in scenarios such as pivoting in LU decomposition or handling trigonometric limits.

Improving Numerical Stability

When evaluating expressions prone to rounding errors or cancellation, try these techniques: