Is Your Forecast Biased, Or Just Slow?

In the last two posts, we talked about catching demand trend changes early and about measuring how late your own detection process typically is. This post is about what happens in the meantime: bias. In this post, I will explore why a missed trend break manufactures bias even in a well-behaved forecasting process, how the Tracking Signal (TS) can serve as an early-warning system, and where demand sensing fits into the picture. (Long-time readers may recognize the title; I wrote a post called “Is Your Forecast Biased Much?” back in 2015. You can find it here. The echo is intentional.)

A Trend Break Is a Bias Factory

Let us think about what happens in the periods after demand shifts and the forecast has yet to catch up. Say demand steps up from about 100 units a month to about 130, and the forecast keeps saying 100. In the first period after the break, the actual comes in about 30 units above forecast. Same story in the second period. And the third. The misses all point the same way, period after period, until the process catches up.

Now, sustained one-sided error is pretty much the definition of bias: a tendency, more often than not, to under-forecast (or over-forecast, if demand steps down). So a missed trend break is a bias factory. This holds even if your forecasters are disciplined, your process has no sandbagging, and nobody is inflating numbers to hit a bonus. The bias comes from the lag itself.

How long does this window stay open? How much one-sided error accumulates before one notices? And who notices first: the planner, the metric, or the customer calling about a stockout?

Let Us Put a Number on It

The classic instrument for this is the Tracking Signal (TS): the running sum of forecast errors divided by the Mean Absolute Deviation (MAD). The running sum keeps the sign of each error, so one-sided misses accumulate; the MAD is the average size of the errors regardless of sign. A forecast with no bias bounces around zero. A forecast that keeps missing in one direction climbs (or sinks) steadily.

Let us run our step-up example through it. The forecast stays at 100 while actuals come in at 130, 128, 132, and 129, giving errors of +30, +28, +32, and +29.

• Period 1: running sum = 30; MAD = 30 / 1 = 30; TS = 30 / 30 = 1
• Period 2: running sum = 30 + 28 = 58; MAD = (30 + 28) / 2 = 29; TS = 58 / 29 = 2
• Period 3: running sum = 58 + 32 = 90; MAD = 90 / 3 = 30; TS = 90 / 30 = 3
• Period 4: running sum = 90 + 29 = 119; MAD = 119 / 4 = 29.75; TS = 119 / 29.75 = 4

A common rule of thumb says a TS beyond +/-4 to +/-4.5 indicates a biased (out of control) forecast. So in this example, the alarm sounds in 4 periods. In practice, the MAD is typically computed over a longer history that includes the calm periods before the break, where errors were small. A smaller MAD makes the TS climb faster, so the crossing often happens in 3 periods, sometimes fewer. All of this is super easy to set up in MS Excel.

Again, the cutoff values are somewhat arbitrary. I have seen 4 to 6 used, and shorter windows argue for wider bands. Experiment with your data and use what works.

Biased, Or Just Slow?

An analogy I like: a car whose wheel alignment is off pulls gently to one side. On short city trips, with all the turning and stopping, one may never notice. On a long straight highway drive, the constant pull in one direction becomes impossible to ignore. But correcting your grip every second is its own problem; you end up over-steering on every gust of wind. The TS is the long straight drive, and the band is what keeps you from over-steering on noise.

This brings me to a distinction I find useful. Some bias is behavioral: optimism from the sales team, sandbagging ahead of bonus season, anecdotes outweighing data. (I listed several of these in the 2015 post.) And some bias is structural: the forecast is honest, just late. The remedy differs. Behavioral bias calls for a people conversation (incentives, triangulation, forecast value added analysis). Structural bias is a detection-speed problem, and no amount of coaching the forecasters will fix it.

This is where demand sensing earns its keep as a bias watchdog. By reacting to fresher signals, it shortens the window during which the forecast keeps missing in one direction. The TS tells you the window is open; demand sensing closes it sooner. One is the alarm, the other is the response. In my experience, you want both, because an alarm with a slow response still leaves you with weeks of one-sided error baked into your plans.

Wrapping Up the Series

So, there you have it: three posts on catching the turn early. First, detection lead time and why it matters. Second, how to backtest your own history and measure how late your process really is. And third, this post: while you are late, your forecast is biased, and the Tracking Signal will tell you so if you let it.

Do you compute a tracking signal on your forecasts today? What band do you use, and has it ever caught a trend break before your planners did? I am interested in hearing from you. Please share via comments.

And if you would like help setting up this kind of early-warning measurement on your own data, reach out to us at Arkieva.