5.7 Monte Carlo variance reduction

Source: Ideas from Computational Physics (Landau, Páez, Bordeianu), §5.7. Rewritten in original wording. Continues 5.6 Mean-value theorem and high-D Monte Carlo. Labs: 5.7b Variance-reduction labs.

When plain sampling wastes effort

Crude Monte Carlo draws $x$ uniformly on $[a,b]$ and averages $f(x)$. That is fine when $f$ is smooth and comparable everywhere. It is wasteful when $f$ is rapidly varying — for example a narrow Gaussian peak on a wide interval. Most samples land where $f\approx 0$ and barely change the average, while the rare hits near the peak dominate both the estimate and its variance. Raising $N$ still helps like $1/\sqrt{N}$, but you are paying for samples that teach you almost nothing.

Control-variate idea

Pick a comparison function $g$ that tracks $f$ closely,

$$\bigl|f(x)-g(x)\bigr|\le\varepsilon,$$

and whose integral is known exactly:

$$J=\int_a^b g(x)\,dx.$$

Then rewrite the target as a small correction plus the known piece:

$$I=\int_a^b f=\int_a^b\bigl(f-g\bigr)+J.$$

Estimate only the residual by Monte Carlo:

$$I\simeq\frac{b-a}{N}\sum_{i=1}^{N}\bigl(f(x_i)-g(x_i)\bigr)+J$$

(with $x_i$ uniform on $[a,b]$; on $[0,1]$ the factor $b-a$ is $1$).

Walk the same four equations — each step offers three lookalike formulas:

Goal: shrink the variance

The method pays off when

$$\mathrm{Var}(f-g)<\mathrm{Var}(f).$$

If $g$ mirrors the shape of $f$, the residual $f-g$ is flatter and less noisy under uniform sampling. Exact $J$ absorbs the bulk of the integral; Monte Carlo only has to resolve a small leftover. A simple textbook calibration on $[0,1]$ is $f(x)=e^{-x}$ with $g(x)=1-x$ (so $J=1/2$): the control-variate estimate typically beats plain mean-value MC at the same $N$ and seed.

Quiz

Variance reduction

1 / 3

Order the control-variate derivation (top = first).

OrderDrag the handle to rearrange — top should be first.

1. ⋮⋮1Rewrite I=∫(f−g)+J
2. ⋮⋮2MC-average the residual and add known J
3. ⋮⋮3Require |f−g|≤ε so g tracks f
4. ⋮⋮4Introduce known J=∫g

Drag items into the correct order (or use ↑↓), then check.

Next: 5.7b Variance-reduction labs.