3.2 Experimental error investigation

Source: Error-tradeoff ideas from Computational Physics (Landau, Páez, Bordeianu), §3.2. Rewritten in original wording. Continues 3.1 Errors.

An algorithm is not “done” when it prints a number. You still have to decide:

1. Does it converge, and how fast?
2. Even if it converges, how many digits are real?
3. What did those digits cost (steps, and therefore round-off)?

“Just run longer” is not a method. Some expansions only work in a patch of parameter space and stall. Even a series that is uniformly convergent can lose digits if you keep adding terms: approximation error falls, round-off error climbs, and the sum of the two has a best $N$, not an infinite one. A cheaper algorithm is often more accurate for that reason — fewer steps, less garbage.

Two errors, one $N$

Let the method take $N$ steps. Approximation error falls as a power of $N$; round-off climbs like a random walk. Their sum has a best $N$, not an infinite one. Walk $\epsilon_{\mathrm{app}}$, $\epsilon_{\mathrm{tot}}$, the $A(N)-A(2N)$ check, and $d\epsilon/dN=0$ — at each step pick the matching equation (lookalikes included).

Theorem (best $N$). Total error is smallest near the $N$ where $\epsilon_{\mathrm{app}}\simeq\epsilon_{\mathrm{ro}}$. Past that trough, extra steps buy round-off, not truth.

On a realistic integrator you would see a steep drop (power-law convergence), then a noisy slow rise. The trough for a careful double-precision run often sits around $10^{-14}$ on that plot — roughly fourteen digits — before the walk upward begins. Quit before the uptick.

Plot $\log_{10}$ of the relative error versus $\log_{10} N$. The negative of that ordinate is the number of decimal places you can defend: if $(\mathcal{A}-A(N))/\mathcal{A}\simeq 10^{-9}$, then $\log_{10}|(\mathcal{A}-A(N))/\mathcal{A}|\simeq -9$.

Quiz

Approximation vs round-off

1 / 3

Why can adding more terms of a convergent series make the printed answer worse?

Pick an answer to see if you are right.

You do not know $\mathcal{A}$

If you already had the exact answer, you would not be computing. You might have a cousin problem with a known result, and you can tune the method there. Or you watch $A(N)$ itself — the guide above walks $A(N)-A(2N)$ and the log–log slope $-\beta$.

A straight steep drop on that plot means you are in the converging region. When the line flattens and then wanders upward, round-off has arrived. A good production $N$ sits just before that change.

In the mantissa you will see the same story without a plot: stable digits march rightward as $N$ grows, then the last places start to flicker and the flicker slowly gets worse. Stop before the flicker.

Stationary point of the cartoon

The same walk sets $d\epsilon_{\mathrm{tot}}/dN=0$. For a general power,

$$N^{*}=(2\alpha\beta/\epsilon_m)^{1/(\beta+1/2)}.$$

Slide $\beta$ and $\epsilon_m$ on the first chart; the dashed line is that $N^{*}$. For $\alpha=1$, $\beta=2$ this is $N^{5/2}=4/\epsilon_m$.

A method with $\epsilon_{\mathrm{app}}\simeq 2/N^{4}$ reaches its trough at smaller $N$ (higher power in the denominator of $N^{*}$). Fewer steps also means a shorter random walk, so the round-off piece is smaller too. The “better” algorithm is not only faster — it is often cleaner.

Quiz

The best N

1 / 2

For $\epsilon_{\mathrm{tot}}=\alpha N^{-\beta}+\sqrt{N}\,\epsilon_m$, the model minimum solves…

Pick an answer to see if you are right.

Lab: estimate the double-precision trough

Pick $\epsilon_m=10^{-15}$ (a working double-precision scale). Implement $N^{*}=(2\alpha\beta/\epsilon_m)^{1/(\\beta+1/2)}$ and $\epsilon_{\mathrm{tot}}(N)=\alpha/N^{\beta}+\sqrt{N}\,\epsilon_m$. Report $N^{*}$ in scientific notation with three digits after the decimal.

Activity: Best $N$

Input: alpha beta eps_m. Print $N^{*}$ as :.3e.

Those two cases are the lesson’s cartoons: $\alpha=1,\beta=2$ wants on the order of $10^{6}$ steps; $\alpha=2,\beta=4$ wants a few thousand — and a smaller $\epsilon_{\mathrm{tot}}$. Check the first chart’s dashed line against your function.

Next: 3.3 Errors with power series.