Myths and Truths About Forecast Accuracy
By Tiffany LaReau, Human Numbers
Forecast accuracy is a workforce manager’s lifelong “frenemy.” When it’s correct, it’s something to be personally proud of, and when it’s wrong… well, hindsight is always 20/20.
Some forecasters’ performance review scores are based solely on accuracy percentages, and some outsourcer contracts have monetary compensation tied to forecast accuracy. The reality is that even the best forecast can produce a poor outcome. The accuracy is a direct result of how the calls are performing, and not always a result of how well the forecaster performs.
Still, this metric is valuable, and understanding more about it may help migrate the forecaster’s goal from a simple “lucky guess” routine to a more meaningful and scientific conclusion, based on solid facts and good judgment. To help you improve your outcomes, let’s take a moment to separate truth from fiction.
Myth: As long as the forecast accuracy is good, leave the forecast alone.
Truth: Even when the forecast is performing well, you should still be on the lookout for abnormal call drivers.
I’ve been in situations where my original forecast was really too high, but because there was an unplanned event that drove up calls, my forecast result appeared to be right on target. In these circumstances, the abnormal call volumes still require cleaning up prior to joining the general population of historical trends. The other thing that is affected when this happens is the immediate future forecast; if the over-forecast trend can be identified and rectified quickly enough (by making a manual adjustment) then those service levels may be saved, too.
Myth: Good forecast accuracy = a good forecaster.
Truth: A thorough and properly prepared forecast = a good forecaster.
The forecast results reflect both the forecaster’s ability and the actual nature of the current call volumes. A good forecaster will use a consistent, thorough method to collect data, normalize data into predictive trends and categories, and follow up by monitoring the performance with documented results that are measured and evaluated on a recurring schedule. It is through this repetitious labor that future forecasting accuracy will emerge, providing the center with long-term stability in the forecasts, rather than short-term hit-and-misses.
Myth: Academic averages yield proper accuracy results (e.g., A = 90%, B = 80%, etc.)
Truth: Accuracy goals need to be tailored to the forecast group.
The variations of forecast deviance that are considered “acceptable” should be directly tied to three circumstances:
- Length of history
- Volatility of calls
Like service level, there is no industry standard for forecast accuracy because the nature of each forecast group is so diverse. But just as service level can be properly determined with an easy customer patience level exercise, forecast accuracy can be properly determined by considering how heavy the calls are, the maturity of the forecast group, and the natural environment of the industry.
For example, a multiskilled, high-volume call center with more than one year of history and cyclical seasonality may perform well with a forecast accuracy goal within 5%. But a single-skilled, low-volume group that blends inbound/outbound calls with off-phone activity may never see accuracy that tight, and their goal may work better within 15% to 20%. An emergency dispatch center with calls driven by accidents or weather may need to see one goal assigned during normal times, and another for special factors. There is a right answer for measuring accuracy (and service level), just recognize that the answer is not going to be the same for all groups unless they all happen to share the same drivers.
Myth: Measuring forecast accuracy is as simple as dividing the forecast into the actual calls.
Truth: The proper calculation is a little more complicated, but will pay off in the end.
Assume 800 calls were forecasted, and 1,000 were received. Dividing 800 into 1,000 equals 80%, or stated backward, 20% lower than actual. The 80% result may feel like something that is easy to interpret because it relates to those same academic concepts we all grew up with (i.e., 90% is A-grade work, 80% to 89% is B-grade, etc.). But those 200 extra calls are truly 25% of the original 800 forecast, so when the forecast is coming in too low, you really need a more robust formula that won’t hide this fact.
A better method is to take the difference between the forecast and the actual, and divide that number into the original forecast. This formula results in a number that is going to be either positive or negative, and you can measure this result similar to the way net staffing works (positive = overstaffing, negative = understaffing, and zeros = perfect staffing.) In the case of forecast accuracy, a negative result means the forecast was too low, a positive result means the forecast was too high, and the closer you can get to zero, the more perfect the result
Myth: It’s OK to measure a forecast on normalized volumes.
Truth: Hypothetical, future forecasting may be calculated on normalized volumes.
Historical forecast accuracy should be calculated using raw, uncleaned volumes.
Workforce management formulas and calculations are no longer in the adolescent stage they were at the turn of the century. There was a time when unscrupulous managers tried to get away with tricking their systems by excluding abandons from the service level reports, or using their switch to force a busy rather than rack up calls in the queue. The same kind of exploitation can easily occur by calculating forecast accuracy using normalized volumes and, in extreme cases, using recalculated forecasts based on actuals. This is harmful behavior that creates a vicious circle of future bad forecasting.
Once forecasters can let go of the idea that their accuracy results are a reflection of how well they do their job, and accept that it is just an indicator that shows what’s going on with the calls, it is so much easier to embrace this metric and put it to good use (yes, it’s hard to do when it’s tied to your review).
What needs to be happening instead is for the forecaster to generate a baseline, and use that baseline as a historical record and the key to documenting when and why volumes are fluctuating. Reforecasting data that has already occurred, using current information for the purpose of making your own forecasts look better, also has the undesirable effect of hiding what’s really happening. The call deviations are the forecaster’s clue to research and drill down into specific reasons why it’s different. This is the key to knowing if these events and patterns are repeating in the next week, month or year.
Once the baseline forecast is set, it does make sense to reforecast the future data as new trends are pulled in; just leave the historical forecast alone. It’s unethical to retouch it.
Myth: Volume is the only metric that needs to be measured for accuracy.
Truth: Abandons, handle times, shrinkage, and off-phone activities also benefit from constant evaluation.
Volume is one of the easier metrics to measure because there is a phone report that produces the quantities to measure. But the other ingredients to your total required staff may benefit from regular assessments. Abandons will come from two sources — a trunk report as well as an ACD report. Abandoned calls play a special role in your forecast accuracy, especially when they are high, because those offered calls may include repeat, three-peat, or four-peat customers. When this happens, use the forecast accuracy to your advantage by taking the forecasted calls vs. the actual calls and backing into the difference to figure out how much to clean up those calls.
Handle times can also be forecasted, based on seasonality or just on recent trends. When they are evaluated, use the actual customer time of investment (ACTI) method to pull in the total customer experience.
Shrinkage and off-phone activities may require a more manual approach, but since we know what we measure will improve, it will pay off. When reporting tools are lacking, you can always resort to stop-watch measurements. Sometimes these exercises are the most enlightening to the forecaster and even to the people performing the work because it sheds light on time requirements that may have been too exaggerated or understated.
Tiffany LaReau, CWPP, may be reached at firstname.lastname@example.org.