12. Forecast Value Added (FVA)
Teaching rewrite — Part 3, a data-driven forecasting process (the Review process step). Original wording; illustrative numbers.
Introduction
A typical process: software produces a baseline → demand planners adjust → sales adjust → a consensus meeting agrees the final number. The final forecast has 45% MAE and +4% bias. Is that good? Is the software set up well? Is sales over-forecasting to avoid shortages? Is the consensus meeting about forecasting — or about matching the budget?
The Forecast Value Added (FVA) framework answers these questions by measuring the accuracy of every step and comparing it with the step before. It needs little investment, often delivers the best return of any planning improvement, and should be the backbone of your improvement journey.
flowchart LR
B[Benchmark] -.compare.-> M[Baseline model]
M --> P[Demand planners] --> S[Sales team] --> C[Consensus meeting]
This lesson covers:
- Internal vs industry benchmarks
- Measuring efficacy (does each step improve accuracy?) and efficiency (is it worth the time?)
- Best practices from research on judgmental adjustments
- Setting objectives without inviting metric hacking
- How to get started
Track FVA for every relevant lag (lesson 6) and with value-weighted metrics (lesson 11).
12.1 Compare the process with a benchmark
Internal benchmarks — do this
Run a simple method (best moving average or seasonal moving average, lesson 10) through history and compare it with your process.
| Step | MAE | Bias |
|---|---|---|
| Benchmark | 52% | −1% |
| Overall process | 45% | +4% |
The process beats the benchmark on accuracy — good news. How much investment (software and people) it takes to gain those 7 points is the efficiency question.
Industry benchmarks — don’t
| Problem | Why |
|---|---|
| Unknown method | Often survey-based: you don’t know how others compute accuracy |
| Different levels | Country vs region, month vs week, different lags |
| Different strategies | A 20-SKU low-cost competitor will naturally beat a 500-SKU catalogue |
| Different footprints | Different markets and channels |
| Single lag | You need value across the whole horizon |
Expensive, unreliable, apples vs pears. Use internal benchmarks.
12.2 Process efficacy: does each step add value?
FVA = accuracy improvement versus the previous step. For MAE, FVA = previous MAE − current MAE. For bias, FVA = previous |bias| − current |bias|.
| Step | MAE | FVA (MAE) | Bias | FVA (bias) |
|---|---|---|---|---|
| Benchmark | 52% | −1% | ||
| Baseline model | 45% | +7 | −3% | −2 |
| Demand planners | 43% | +2 | +1% | +2 |
| Sales team | 45% | −2 | +5% | −4 |
| Consensus | 45% | 0 | +4% | +1 |
Reading it:
- The model adds most of the accuracy.
- Planners improve accuracy and remove the model’s negative bias (they handle new products and key-customer information).
- Sales makes it worse and pushes bias to +5% (“we want to secure stock”).
- Consensus adds nothing to accuracy (“we want to match the yearly budget”).
FVA creates ownership and accountability: each team’s contribution is visible instead of diluted in the final number. You can even track individuals and customer-supplied forecasts (often trusted, not always helpful).
12.3 Process efficiency: is it worth the time?
Add person-hours per cycle:
| Step | Person-hours | MAE | FVA (MAE) |
|---|---|---|---|
| Baseline model | — | 45% | +7 |
| Demand planners | 72 | 43% | +2 |
| Sales team | 20 | 45% | −2 |
| Consensus | 8 | 45% | 0 |
Marginal gains shrink at each step: fixing the model’s big blind spots is easy; improving an already-reviewed forecast is hard. Past a point, extra hours aren’t worth it — FVA shows where to stop.
Checkpoint: FVA
Checkpoint — sections 12.1 to 12.3
1 / 4Why avoid industry accuracy benchmarks?
Pick an answer to see if you are right.
12.4 Best practices
- Don’t overreact to a few negative FVA rounds by deleting process steps. Find root causes — bias, ownership, incentives — and fix them. Removing a step that should add value is the last resort.
- Report FVA by product group, channel, region where teams and information differ.
- Track it over the whole horizon and value-weighted.
Research on judgmental adjustments in companies (Fildes & Goodwin, 2007) suggests:
| Finding | Action |
|---|---|
| Many small adjustments add almost nothing and cost time | Don’t tweak within the error margin |
| Large adjustments more often help (they need justification and carry personal risk) | Focus effort on big, well-founded changes |
| Positive (upward) adjustments often reduce accuracy; negative ones more often help | Track FVA separately for upward and downward edits |
Present results tactfully. FVA can embarrass non-value-adders; share early results privately. The aim is to improve the process, not humiliate anyone. (For a consultant, a process that loses to a benchmark is easy to improve: switch to the benchmark.)
Setting objectives
| Do | Don’t |
|---|---|
| Base targets on what’s achievable (benchmarks) | Set arbitrary targets |
| Remember external shocks (e.g. a pandemic year) | Blame or reward teams for things they don’t control |
| Use metrics to discuss and improve | Use them to reward and punish — that invites metric hacking (choosing a convenient lag, excluding zero-demand periods, dropping “top offenders”) |
12.5 Getting started
FVA needs data: one forecast version per step per lag. With 5 steps (benchmark, model, planners, sales, consensus) and 6 lags, that is 30 versions per cycle — you need automation, not spreadsheets.
Two ways to start:
- Quick check: compare the consensus forecast with a moving-average benchmark. Promotion-heavy businesses can beat benchmarks by ~20%; if you beat it by less than ~5%, there is probably room for improvement.
- Full FVA: use software that stores versions, or build a data pipeline (e.g. in Python).
Worked example: is the planners’ time worth it?
Planners spend 72 hours per month and improve weighted MAE by 2 points on a portfolio with €10 M monthly value. If each MAE point is worth roughly €15,000 a month in lower safety stock and fewer lost sales, the gain is about €30,000/month for 72 hours ≈ €417/hour — clearly worthwhile. Sales’ 20 hours with −2 points destroys about €30,000 a month.
Checkpoint: practice and objectives
Checkpoint — sections 12.4 and 12.5
1 / 4Planners make hundreds of ±1% tweaks each month. What does research suggest?
Pick an answer to see if you are right.
Rank: rolling out FVA
FVA roll-out
1 / 2Rank the steps to roll out FVA.
OrderDrag the handle to rearrange — top should be first.
- 1Store every step’s forecast version for each lag.
- 2Compare the final forecast with the best simple benchmark.
- 3Compute value-weighted MAE and bias for each step.
- 4Share results tactfully and fix root causes of negative FVA.
- 5Add person-hours per step.
Drag items into the correct order (or use ↑↓), then check.
Check: forecast value added
Forecast value added
1 / 5What does FVA measure?
Pick an answer to see if you are right.
Takeaways
- Compare your process with an internal benchmark, never with industry surveys
- FVA = accuracy and bias change at each step → ownership and accountability
- Add time spent to judge efficiency; marginal gains shrink
- Avoid tiny tweaks; track upward vs downward edits; present results tactfully
- Use metrics to discuss and improve, not to reward and punish
- Start with a benchmark check; scale with automated version storage
Further Reading
- 11. Portfolio accuracy
- Next: 13. What do you review? ABC XYZ and beyond
- M. Gilliland, “Is forecasting a waste of time?” (2002) and The Business Forecasting Deal (2010); R. Fildes & P. Goodwin, “Good and Bad Judgment in Forecasting” (Foresight, 2007)
- Background: N. Vandeput, Demand Forecasting Best Practices (Manning) — used as a study outline