Correcting Heterogeneous and Biased Forecast Error at Intel For Supply Chain Optimization

Download as pdf or txt
Download as pdf or txt
You are on page 1of 14

informs ®

Vol. 39, No. 5, September–October 2009, pp. 415–427 doi 10.1287/inte.1090.0452


issn 0092-2102  eissn 1526-551X  09  3905  0415 © 2009 INFORMS

Correcting Heterogeneous and Biased Forecast


Error at Intel for Supply Chain Optimization
Matthew P. Manary
Server Platform Group, Intel Corporation, Hillsboro, Oregon 97124,
matthew.manary@intel.com

Sean P. Willems
School of Management, Boston University, Boston, Massachusetts 02215,
willems@bu.edu
Alison F. Shihata
Planning and Logistics Group, Intel Corporation, Hillsboro, Oregon 97124,
alison.f.shihata@intel.com

In 2007, Intel’s Channel Supply Demand Operations launched an initiative to improve its supply chain per-
formance. To ensure success, the process had to fit within the existing planning processes. In practice, this
meant that setting service-level and inventory targets, which had previously been external inputs to the pro-
cess, had to become part of the structured decision-making process. Although other Intel business units had
achieved success implementing a multiechelon inventory optimization model, the boxed processor environment
posed some unique challenges. The primary technical challenge required correcting for the impact of forecast
bias, nonnormal forecast errors, and heterogeneous forecast errors. This paper documents the procedure and
algorithms that Intel developed and implemented in 2008 to counter the impact of forecast imperfections. The
process resulted in safety stock reductions of approximately 15 percent. At any given time, Intel applies this
process to its 20–30 highest-volume boxed processors, determining an on-hand inventory commitment between
$50 million and $75 million.
Key words: forecasting: applications; industries: computer/electronic; inventory/production: applications,
multiechelon safety stock optimization.
History: Published online in Articles in Advance August 27, 2009.

S ince 2005, Intel has used more rigor in its in-


ventory-planning processes, as manifested in two
ways. First, Intel has improved its processes by
Palmatier (2003), Franses (1998), and Montgomery
et al. (1990), recommend addressing these problems
prior to loading data into an optimization model.
developing an integrated sales, inventory, and oper- However, as Manary and Willems (2008) document,
ations process (SIOP). Second, it has improved its planners cannot always remove bias from the raw
decision-support technologies by integrating multi- forecast data, especially when groups that do not
echelon inventory optimization with its supply chain reside in the supply chain organization have loaded
planning solvers. the data.
The SIOP process must operate within the con- From a decision-support perspective, it was nec-
straints of Intel’s existing organizational responsibility essary to address the three forecasting problems.
structure. In particular, the sales and marketing orga- Intel began with forecast bias. In an unbiased fore-
nization is responsible for entering the forecast data cast, the forecast residuals, found by subtracting the
that are so critical to any SIOP process. At Intel, three period’s forecast from actual demand, have a zero
forecasting problems—forecast bias, heterogeneity of mean; in a biased forecast, the mean is nonzero. Based
errors, and nonparametric residuals—have nontrivial on the Intel experience that Manary and Willems
impacts on the output of its optimization models. Tra- (2008) cover, we expected bias to be the dominant
ditional forecasting textbooks, including Crum and error behavior to correct. Unfortunately, heterogeneity
415
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
416 Interfaces 39(5), pp. 415–427, © 2009 INFORMS

(i.e., the errors could not be characterized by a single multiple service levels. If we detect no significant dif-
probability distribution) and nonparametric behavior ference in variability estimates, we apply a weighted
(i.e., errors could not be characterized by a stan- proxy of variability through the root mean of the
dard probability distribution) error structures compli- collective adjustment procedure at the sampled ser-
cated solutions to Intel’s boxed central processing unit vice levels. We treat stock-keeping units (SKUs) with
(CPU) forecasts and required a more rigorous solu- significant differences in variability by using a kernel-
tion than bias alone would need. By addressing these smoothing technique that accounts for both hetero-
forecasting-process problems, Intel could quantify the geneity and nonparametric error distributions, and
impact of service and inventory changes as part of its generates a single estimate of variability.
SIOP process. Our approach, which Intel uses today, This paper presents the solution that Planning de-
addressed these issues. veloped to determine the joint SKU-location safety
stock and service-level targets in the presence of
forecast bias, nonnormal forecast errors, and fore-
Methodological Contribution
cast error heterogeneity by reparameterization of the
Manary and Willems (2008) proposed a way to cor- estimate of the standard deviation of forecast errors.
rect for forecast bias in inventory optimization models Since Intel implemented this process in early 2008, the
without altering the raw demand signal. Their adjust- process has maintained the high service levels affili-
ment procedure provided the necessary improvement ated with prior MEIO models; simultaneously, it has
required for Intel management to implement multi- reduced total corresponding boxed CPU inventory
echelon inventory optimization (MEIO) models for its levels by approximately 15 percent.
embedded processor division; the result was a signif-
icant reduction in aggregate inventory in comparison
to the prior policy, a weeks-of-inventory strategy Intel’s Channel Products Group
with management selecting the target number of Intel’s Channel Products Group (for which Plan-
weeks. The adjustment procedure was conditioned ning manages supply and demand coordination) sells
on assumptions that forecast bias was stationary and boxed CPUs in unique finished goods SKUs that are
service levels were predetermined. Intel’s Channel differentiated by CPU design, boxing material, and
Supply Demand Operations (hereafter referred to as warehousing location. The data that this paper dis-
Planning), a division with approximately $5 billion plays come from over a year of weekly forecasts
in annual boxed CPU sales to distributors, runs sim- of approximately 20 high-volume boxed CPUs that
ilar MEIO models; its business processes also do not represent Intel’s mobile, desktop, and server CPU
allow the supply chain organization to alter the raw businesses.
forecast data, which the Sales and Marketing Group The boxed CPU supply chain is similar to a stan-
controls. However, the boxed CPU procedure has dard reseller’s supply chain but includes no product
one fundamental process difference: service levels are manufacturing. Instead, the supply chain modeling
dynamic and optimized as a function of a single focuses on boxing processes and distribution. Each
demand variance estimate for all service levels. This SKU is modeled as a worldwide distribution network
negates direct application of the embedded processor of packaged and unpackaged finished goods CPUs
division’s adjustment procedure, which determines a that consist of three echelons with multiple physical
demand variance estimate for each service level. locations. CPU manufacturing echelons are beyond
Our solution to this problem builds upon the ad- the scope of Planning’s distribution network problem.
justment procedure in Manary and Willems (2008) by The stages of the supply chain include component
combining it with Bartlett and Kendall’s (1946) meth- warehousing facilities; assembly facilities that box the
ods for testing heterogeneity of independent sample CPUs and package them with other components, such
variances. This solution relies principally on applying as fans and heat sinks; and location-specific distribu-
the adjustment procedure to extract a random sam- tion centers that serve end customers. SKUs share a
ple of variance estimates for a single product across common supply chain in both echelon structure and
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
Interfaces 39(5), pp. 415–427, © 2009 INFORMS 417

Packaging Finished goods 250

Increase in sigma (%)


Warehoused CPUs
facility warehousing
200

150

100

50

0
0 10 20 30
Forecast bias (%)

Figure 2: A squared error standard deviation estimate grows exponentially


as we introduce bias into the forecast. At a 20 percent forecast bias, the
sigma generated from a sum-of-squared-errors approach more than dou-
bles the estimate generated when bias is absent.

accuracy and forecast error homogeneity differ signif-


icantly. In the past, Planning’s optimization process
relied on the commonly employed standard devia-
Figure 1: The graph depicts the three echelons in Intel’s boxed CPU dis- tion of forecast error (SDFE) calculation to provide the
tribution network model. demand variability input,
 n
i=1 Fi − Ai 
2
number of stages (Figure 1). By echelon, the boxed SDFE =
  (1)
n−1
CPU distribution network has (1) one virtual com- where F and A are the paired forecast and actual
ponent stage that represents nonboxed, finished CPU demand, respectively, for a period i. Equation (1) is
inventory from Intel’s factory network, (2) four stages influenced by any bias and (or) heterogeneity in the
in boxing and kitting assembly stages, and (3) five relationship between the forecast and actuals (i.e., the
stages in geography-specific distribution centers. The forecast errors). Figure 2 demonstrates how even small
MEIO model generates inventory targets for between amounts of bias result in a significant increase in the
200 and 300 SKU locations every month. calculated SDFE. The boxed CPU global forecast bias
The planning organization monitors inventory lev- exceeded 20 percent for products whose safety stock
els weekly; a major reset, which is driven by a new targets were set using the optimization process.
forecast from the sales and marketing organization, Intel’s optimization technique relies largely on the
occurs monthly. Based on the new demand informa- demand variance estimates to derive safety stock tar-
tion, the monthly planning process resets the forward- gets; therefore, the forecast bias naturally impacted
looking demand at the SKU-location level. This reset Planning’s modeling output. Figure 3 presents the
modifies the current optimized safety stock targets; bias problem that Planning faced using the sales and
the targets feed an Intel-developed Systems Appli- marketing forecasts. MEIO SKUs were collectively
cation and Products (SAP) advanced planning and overforecast more than 60 percent of the time; the
scheduling (APS) optimizer that minimizes produc- expected rate was 50 percent. Because Planning did
tion costs, lost-sales costs, and costs for deviating not initially address the forecast bias (Figure 3), we
from the inventory targets. The APS optimizer out- can look at Figure 2 to understand how targeted
put is the constrained production and inventory plan, inventory and service levels differed from the levels
which constitutes the finalized factory schedule. that Intel actually achieved.

Addressing the Breakdown with Addressing the Boxed Processor


Intel’s Approach Forecast Bias
Although operating parameters, such as transit In 2005, Intel’s Communications Infrastructure Group
times, are essentially identical across SKUs, forecast (CIG) encountered a similar problem with forecast
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
418 Interfaces 39(5), pp. 415–427, © 2009 INFORMS

MEIO product forecast error Forecast/(Forecast + Actuals) SDFE as a function of service level, Equation (3) fac-
3
tors out the MEIO model impact of bias in the fore-
0.99 cast without changing the mean demand signal from
2
0.95 sales and marketing. That is, Equations (3) and (6) later

Normal quantile plot


0.90
1 in the paper correlate negatively with bias. Because a
0.75
product is more likely to be overforecast, based on its
0.50 0 historical performance, the sigma estimates decrease,
0.25 thus telling the planning system to plan for less safety
–1
0.10 stock (because the biased forecast already has “built-
0.05
–2
in” safety stock). In extreme cases in which a prod-
0.01 uct has never been underforecast, the sigma estimate
–3 is zero, thus signaling the planning system to gener-
ate no inventory for demand variability. If the prod-
uct has been underforecast historically, then the sigma
estimate is larger than it would be had no bias been
present. Products that do not display a bias either way
converge to the sum-of-squared-errors estimate. It is
worth noting that the method for calculating relative
forecast error in Equation (2) is largely arbitrary; any
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
number of forecast error measures would work. How-
ever, Equation (2) is a natural selection because Intel
Figure 3: The graph shows the distribution of relative forecast errors, i.e.,
Equation (2), for boxed CPU MEIO products representing over 500 paired planners are familiar with it. If a different error mea-
observations. The distribution’s heavier right tail represents an overall surement is used (e.g., centering the error on zero),
propensity to overforecast. At the SKU level, products displayed overfore- Equation (3) would need to be updated to reflect any
casting bias, underforecasting bias, and no bias.
change in Equation (2).
Unfortunately, the modified sigma solution that
CIG employed was incompatible with the boxed
bias in its MEIO models. To solve its problem, CIG
CPU optimization process because forecast errors
implemented a modified estimate of the demand vari-
were heterogeneous, and service levels were selected
ation that Manary and Willems (2008) discuss. The
dynamically. CIG’s modified sigma solution requires a
premise was to calculate a measure of relative forecast
stationary bias and was not robust to the heterogene-
accuracy,
ity present in the boxed CPU forecast error. In addi-
Forecasti
i =  (2) tion, boxed CPU service levels were determined using
Forecasti + Demandi
a proprietary selection process that balanced inven-
and reconstruct from the pattern of error measure- tory costs, lost sales associated with different service
ments, whether biased positively, negatively, or unbi- levels, and strategic management guidance. Although
ased, a modified estimate of SDFE, the service-level selection process is beyond the scope
   of this paper, its key attribute was that it required a
1 −  /  − 1 single constant estimate of demand variability to gen-

 Modified
= Max  0  (3)
t df erate a single efficient inventory frontier prior to the
service-level selection.
where  = 1 − ,  denotes the quantile point corre-
sponding to  from the distribution of s calculated in
Equation (2), t df is the Student’s t-distribution with a Determining a Constant
cumulative density of  and degrees of freedom com- Sigma Estimate
ing from the number of historical points from which Similar to CIG, Planning needed a process to deter-
to draw, and  is the average demand. By treating mine an SDFE estimate that would net out the forecast
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
Interfaces 39(5), pp. 415–427, © 2009 INFORMS 419

bias and heterogeneity without changing the sales 350

and marketing-loaded forecast. Using Equation (3) 300


was a natural first step; however, the equation’s
random-sampling approach across service levels to 250

Sigma modified
approximate demand variability did not allow for
200
optimizing service-level selection under a stationary
sigma assumption. The goal then was to approximate 150
a single representation of demand variability from the
100
random sample generated by sigma modified, which
then allowed Intel’s third-party software to calculate a 50
single inventory efficient frontier; this aided Planning
in ultimately selecting the optimized service level. 0
50 60 70 80 90 100
Under a normal and unbiased forecast error as-
Service level
sumption, sigma modified will generate a random
sampling of the true sigma across each service level. Figure 4: The graph shows sigma-modified estimates at given service lev-
Therefore, it is a natural extension to treat sam- els for CPU product A. Although sigma modified is obviously different for
ples independently and test for consistency of sigma each service level, per the adapted Bartlett’s variance test, the estimates
do not differ significantly.
across the service levels. One complication with this
is that most variance equality tests are functions of
the underlying data populations; however, because We considered product A’s sigma-modified vari-
sigma modified is a calculation from one data point, ability sampling, and thus its underlying forecast
it lacks individual observations to determine compar- error, to be well behaved (i.e., have no significant
ative statistics. This eliminated adapting a standard bias and be generally homogeneous). Unfortunately,
F -ratio method or borrowing from approaches devel- the forecasting behavior of many boxed CPU prod-
oped by O’Brien (1979), Levene (1960), and Brown and ucts did not fit this description. Figure 5 demonstrates
Forsythe (1974), among others. However, Bartlett’s our challenge in working with boxed CPU forecast
F -test of homogeneity (Barlett and Kendall 1946) is variability, as interpreted through sigma-modified
not a function of observations; it is determined by and relative forecast-accuracy histograms. The graphs
the sigma estimates themselves. Therefore, Bartlett’s in Figure 5 illustrate both the lower 50 percent
approach was adapted to test homogeneity of the underforecast error distribution and modified sigma
sigma-modified sampling (Appendix B) across service sampling at each service level for three products
levels by simply replacing the variances from inde- that represent the forecast error behavior across all
pendent samples with an array of sigma-modified esti- boxed CPU products. For purposes of setting safety
mates drawn from different management-determined stock, our interest is the propensity to underfore-
service levels. cast; therefore, we focus our forecast error analysis
If Bartlett’s test (Bartlett and Kendall 1946) indi- on the lower 50 percent of forecast error occurrences
cated homogeneity across the service levels, then the that represent the underforecast activity. In theory,
root mean square error (RMSE) of the sigma-modified each product’s forecast error should follow a normal
estimates was established as the point estimate of distribution with measurements independent of one
demand variability for all service levels, thus enabling another. In such a case, the histogram for the lower
Planning’s SIOP process to determine feasible service- 50 percent forecast accuracy will also follow a one-
level and safety stock targets. In Figure 4, the variabil- sided (tail portion) normal distribution, in which all
ity in the sigma-modified sampling for product A was  values from Equation (2) are 0.5 or less. For boxed
not significant enough to reject the null hypothesis CPUs, such as product A, some product forecast error
of equal variances across all service levels; therefore, follows approximately a normal distribution for the
the pooled variance was used as the constant demand lower 50 percent. Evidence of that lies with the sigma-
variability parameter in the optimization process. modified estimates at each service level that appear
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
420 Interfaces 39(5), pp. 415–427, © 2009 INFORMS

Product A Product B Product C


4,000
300
Daily sigma modified

2,000

Daily sigma modified

Daily sigma modified


250 3,000
1,500
200
150 2,000
1,000
100
1,000 500
50
0 0 0
50 60 70 80 90 100 50 60 70 80 90 100 50 60 70 80 90 100
Service level Service level Service level

Equation (2) Product A Equation (2) Product B Equation (2) Product C

0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6

Figure 5: The graphs show sigma-modified sampling for errors and represent the worst 50 percent of underfore-
casts across service levels for CPU products A, B, and C. Intel describes the forecast error for product A as
homogeneous unbiased, product B as homogeneous biased, and product C as heterogeneous biased.

randomly distributed and pass a homogeneity test, variability). Because of Bartlett’s natural log approach
as we saw earlier. In contrast, product B displays a (Bartlett and Kendall 1946), testing for homogene-
bias to overforecast, as evidenced by the histogram of ity requires strictly nonzero estimates of sigma. Fig-
forecast errors, which show that about half of what ure 5 shows how products B and C would clearly
should have been underforecasts were overforecasts. fail the equal variance test on just the 0-sigma fre-
The calculated sigma-modified values also reflect this quency that bias causes. Figure 6 shows a product
because the approximated sigma is zero for service count breakdown of the full boxed CPU portfolio
levels up through 75 percent. However, when forecast behavior. Approximately 40 percent of the boxed CPU
errors begin to represent underforecasts, the modified SKUs displayed a significant degree of overforecast
sigma values appear randomly distributed and stable,
or underforecast bias. However, Planning manage-
and the product is considered to have a biased fore-
ment established a minimum allowable service level;
cast; otherwise, the error structure is normal beyond
thus, we tested only sigma from service levels in the
the bias. Product C also displays some level of over-
top quartile. This allowed some SKUs with moderate
forecast bias; however, because its extremely long tail
to underforecast breaks with normality, it drives an
increasing estimate of sigma as higher service levels
70
are desired. In this case, the product is considered
60
biased, heterogeneous, and nonparametric, as we will
50
see later.
Percent

40
30
Deriving a Normalized Point Estimate 20
of Sigma with Bias and Heterogeneity 10
0
Sigma modified is designed such that when bias to Bias present No bias present
overforecast is demonstrated, the estimate of sigma
is zeroed out (because a product that is habitually Figure 6: More than 40 percent of Intel’s boxed CPU SKUs demonstrated a
overforecast will not need safety stock for demand significant bias based on a binomial test at the  = 5 percent level.
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
Interfaces 39(5), pp. 415–427, © 2009 INFORMS 421

bias to pass the homogeneity test. Some products for most robust technique for calculating sigma under
which bias was not a factor failed because of hetero- heterogeneous and nonparametric error was to apply
geneity or nonparametric error. a nonparametric approximation of the product error-
To allow Planning to run the SIOP process, an alter- density function across the full range of prior fore-
native to the pooled variance method needed to be casts, allowing a kernel-smoothing technique to help
developed for the products that failed the homogene- determine a weighted, localized error pattern. We
ity test. Any estimate developed needed to consider selected the kernel standard deviation based on Bow-
four major factors: bias, heterogeneity, the potential for man and Foster’s (1993) recommendation and then
nonparametric error, and the requirement that it be a performed a bivariate normal kernel smoother. This
single variance estimate translated back to an assump- allowed us to weight the influences of prior forecasts
tion of normality for the optimization software. that were similar in volume to the current forward-
The initial approach, which Intel applied in produc- looking estimate and helped us to address the prob-
tion in early 2008, was a minimum function of the lem that the previous uniform weighting approach
modified sigma based on both a relative error and had caused. Figure 7 presents the completed nonpara-
an absolute volume error, as Manary and Willems metric error-probability density function across the
(2008) outline. We knew in advance that this approach history of prior forecasts for product C, a product
would likely overstate a sigma estimate; however, we whose error pattern was seen to be biased, heteroge-
were comfortable with a conservative estimate for neous, and nonparametric.
what we thought would be largely an exceptions pro- The next step was to identify the error density
cess. During the spring of 2008, the data showed that localized to the current forward-looking forecast. That
almost half the boxed CPU products were failing the portion of the probability density function (PDF) usu-
homogeneity test. This meant that the sigma estimate ally defined at the forward-looking forecast becomes
for failed tests would play a prominent role in estab- the curve under which the new estimate of demand
lishing overall inventory targets and thus increase variability is weighted. Figures 8(a) and 8(b) represent
the necessity for a more robust approach. Although
the initial approach applied in early 2008 typically
delivered a 30 percent reduction in a product’s vari-
0.0020

ance estimate (versus a SDFE calculation in approxi-


mately 80 percent of the cases), the technique was not
0.0015

an unbiased estimate of sigma and was susceptible


Density
0.0005 0.0010

to significant overstatement because of heterogeneity


and nonparametric error structures. Our biggest con-
cern with the initial technique used in early 2008 was
that the sigma estimate was often driven by errors
0

that occurred at forecast levels significantly higher or


1.0

lower than the current forecast level (i.e., the fore-


0.9
0.8

cast driving the MEIO model). To correct for this,


0.7

we wanted to minimize the heterogeneity and (or)


0.6
Erro
0.5

nonparametric error impact by isolating a sigma esti-


r

,500
0.4

mate close to the current forecast level. One chal- 00 3


0 3,0
0.3

2,50
0 2,000
lenge we faced in doing this was a relatively low
0.2

0
0 1,5 st
1,00 a
500 Forec
0.1

number of observations around the current forecast 0

level to draw on as a population. To compensate


for this, we decided to apply a weighting system Figure 7: The graph shows CPU product C’s nonparametric error-density
field across all prior forecast levels. If the relative forecast error was nor-
to forecast error observations with decreasing influ-
mally distributed with no bias, we would see a bell-shaped “tunnel” lying
ence as error observations were further away from on the 0.5 error line. However, the error density for product C indicates
the current forecast. Ultimately, we found that the heterogeneity and nonparametrics.
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
422 Interfaces 39(5), pp. 415–427, © 2009 INFORMS

(a) (b) 0.07


020

0.06
0.0015 0.0

0.05

Density
0.04
Density
0.0005 0.0010

0.03

0.02
0 1.0

0.01
0 .9

0
0.8

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0


0.7

Error
0.6
Err
0.5
r o

00
0.4

0 3,5
3,00
2,500
0.3

2,000
1,500
0.2

1,000 cast
500 Fore
0.1

Figure 8: CPU product C’s forecast error density field (panel a) is localized at the current forward-looking forecast
of 1,708 units (panel b). The distribution localized at the current forecast level determines the product-level
sigma estimate.

CPU product C’s error density localized to the new point estimate of demand variability back in the units
forward-looking forecast—1,708 units. of the forecast,
Having obtained an estimate of the forecast error  
Modified 1 − ¯ 
/ ¯ −1
likelihood conditioned on a new forward-looking 
 =  (5)
∼ 0
8
forecast, the next step is to estimate the expected
underforecast and translate it into a singular normal- where  is the forward-looking forecast. This approach
ized parameter for the optimization software. In our is similar to Bowman and Azzalini’s (1997) sugges-
nonparametric case, we considered the underforecast
tail of the conditioned PDF (error values <0
5) and 0.07

computed the weighted average error in Equation (4). 0.06


 0
5
PForecast  ∗  d 0.05
¯ = 0

(4)
CDFForecast 0
5
Density

0.04

The integrated approach allowed us to weight across 0.03


the potential heterogeneity of the forecast error.
0.02
Figure 9 illustrates product C’s tail zone for under-
forecast error conditioned on the forward-looking 0.01
forecast.
¯ now represents the weighted-average underfore-
0
0.20 0.25 0.30 0.35 0.40 0.45
cast error for product C conditioned on the current Error
forward-looking forecast. We can now reintroduce
Equation (3) to normalize the new weighted-demand Figure 9: The graph shows product C’s underforecast region PDF trun-
¯ and replacing the
underforecast. Replacing  with , cated only to the values representing underforecasts (<0.50 error scores).
Under unbiased conditions, the cumulative probability under this curve is
t-distribution with the average tail value from the 50 percent. For product C, it is ∼ 30 percent, thus indicating a bias to
standard Normal distribution, generates a normalized overforecast.
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
Interfaces 39(5), pp. 415–427, © 2009 INFORMS 423

tion of using a median absolute deviation as a more Forecast level Equation (1) Equation (6)
robust sigma estimate when dealing with nonpara- Product C SDFE ˆOptimized

metric data. However, given the nonparametric nature 250 1,141 111
of Intel’s data, and to be conservative, we chose to 750 1,141 713
weight what otherwise would be considered outliers 1,250 1,141 1560
1,750 1,141 310
under a median absolute deviation calculation. Equa- 2,250 1,141 72
tion (5), unlike (3), does not require a maximum cri- 2,750 1,141 381
terion because the continuous nature of the kernel 3,250 1,141 225
estimation will guarantee some positive density to
Table 1: The table shows product C’s kernel-smoothed, weighted sigma
underforecast. Equation (6) will provide an additional estimate conditioned on a forecast level versus a more conventional root-
weighting that can drive the sigma-modified estimate sum-of-squared-errors approach. Note that the weighted technique does
to zero because an extreme overforecast bias exists. not guarantee a lower sigma estimate for every forward-looking forecast
level, even for a product that has an historical overforecast bias.
We have now addressed the heterogeneity at the
forward-looking forecast level but still need to ac-
count for the bias. We do that by considering the which corresponds with the historical, increased error
propensity of the bias (either over or under) by density to underforecast at that forecast level. In sim-
weighting Equation (5) with the modeled underfore- pler terms, our modified sigma estimate is negatively
cast likelihood against the expected underforecast correlated with bias.
likelihood—that is, a ratio of the conditioned, under-
forecast cumulative distribution function (CDF) up to
a value of 0.5 and 50 percent, the CDF for the same Validating Kernel-Smoothed, Modified
region under an unbiased assumption. If the forecast Sigma Estimates
error was unbiased, then this scalar would simply fall Our approach to validating the kernel-smoothed,
out of the equation. If the bias is to overforecast, then sigma-modified approach was threefold: it needed to
the new demand variability is decreased by a rela- replicate a theoretical underlying variance estimates
tive amount; if the bias is to underforecast, then the for a constant relative variance, a constant unit vari-
new demand variability is increased proportionately. ance, and a heterogeneous variance.
Equation (6) represents the algorithm for determin- We first validated the kernel-smoothed, modified
ing the weighted point estimate of demand variability sigma technique for the sigma of a well-behaved
conditioned on the forward-looking forecast for use (i.e., homogeneous, unbiased) forecast. We generated
in the SIOP process. a series of forecasts and randomly determined the
 0
5 actuals around the forecast using a percent error rep-
Modified P  Forecast  d
 
Optimized = 
 ∗ 0
(6) resented by an N 0 5%2  distribution. Equation (7), a
0
5
coefficient of variation (COV) that is the ratio of sigma
It is worth highlighting here that if service levels are at a given forecast and the forecast, measures the rel-
deterministic, then we recommend skipping Equa- ative error for our case:
tions (4)–(6) and using the alternative that Appen- Forecast
dix A shows. COV =
(7)
Forecast
Table 1 demonstrates the sigma estimate differences
for product C under Equation (6) based on the fore- With this approach, we were looking to see if the
cast level. It also displays the more common SDFE kernel-smoothing technique could consistently repli-
approach, Equation (1), to show readers the estimate cate the same 5 percent standard deviation across any
differences. Figure 10(a) gives the graphical represen- forecast level. Figures 11(a) and 11(b) display the fore-
tation of the sigma estimate from Equation (6) at a cast error distribution that was randomly generated
given forecast level; Figure 10(b) shows product C’s for the purposes of validating and the COV at each
error-density field. Note how the Equation (6) sigma forecast level generated by Intel’s kernel-smoothing
estimate peaks at a forecast level of ∼1200 units, technique.
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
424 Interfaces 39(5), pp. 415–427, © 2009 INFORMS

(a) (b)
0.9
1500
0.8
Equation (7) sigma

0.7

Equation (2)
1000
0.6

0.5

500 0.4

0.3

0 0.2
0 500 1,000 1,500 2,000 2,500 3,000 0 500 1,000 1,500 2,000 2,500 3,000
Forecast level Forecast level

Figure 10: The graphs illustrate product C’s new kernel-smoothed, weighted sigma estimate conditioned on a
forecast level and the forecast-density error field. As expected, the new procedure estimates its largest value for
sigma where (global max in panel a) the product has historically been underforecast most severely (i.e., density
around 1,200 in panel b).

After relative homogeneity, we tested unit homo- technique. As desired, the kernel-smoothing approach
geneity. In a similar fashion, we considered a series replicated the underlying variance pattern.
of actuals that were normally distributed around a With constant relative error and constant unit error
forecast by a constant unit variance, regardless of fore- validated, the final validation check was to see if the
cast value. Using the same arbitrarily chosen forecast smoothing technique could replicate a heterogeneous
range from the test for relative homogeneity, we gener- error pattern, a particularly important aspect given
ated the actuals around the forecasts with an N 0 252  its intent to address SKUs that fail the homogeneity
error pattern. Figures 12(a) and 12(b) demonstrate test. Figure 13(a) represents a split-forecast error pat-
both the resulting randomly generated forecast error tern in which forecasts from 100 to 350 have a rela-
pattern and sigma estimate from the kernel-smoothing tive error structure of N 0 5%2  and the relative error

(a) (b)
0.54
0.09
0.53 0.08
Forecast/Forecast + Actuals

0.52 0.07

0.51 0.06
COV

0.05
0.50
0.04
0.49
0.03
0.48
0.02
0.47 0.01

0.46 0
100 200 300 400 500 600 100 200 300 400 500 600
Forecast Forecast

Figure 11: Panel a depicts the Equation (2) forecast error that was randomly generated from an N0 5%2 
distribution. Panel b demonstrates the kernel-smoothed sigma appropriately estimated the underlying 5 percent
sigma across all forecast levels.
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
Interfaces 39(5), pp. 415–427, © 2009 INFORMS 425

(a) 0.64 (b) 45


40
0.60
Forecast/Forecast + Actuals

35

Sigma mod weighted


0.56
30
0.52
25
0.48
20
0.44
15
0.40
10
0.36
5
100 200 300 400 500 600 100 200 300 400 500 600
Forecast Forecast value

Figure 12: Panel a depicts the Equation (2) forecast error that was randomly generated from an N0 252  distribu-
tion. Panel b demonstrates the kernel-smoothed sigma estimate, which appropriately estimated the underlying
sigma (25) across all forecast levels.

for forecasts above 350 are structured as N 0 15%2 . smaller kernel-smoothing standard deviation creates
Figure 13(b) demonstrates how the kernel-smoothing higher variability in the sigma-modified estimate at
technique tracks to the underlying error pattern. any given forecast level. Optimal standard deviation
In validating, we learned that a larger ker- estimate selection for the kernel-smoothing technique
nel standard deviation estimate than Bowman and provides an area of further research for Intel.
Foster’s (1993) recommendation can sometimes result
in faster convergence to the true underlying sigma
when bias is present, but is otherwise homogeneous.
Optimized Targets Based on
However, a smaller standard deviation estimate helps
minimize the transition lag of sigma-modified esti- New Modified SDFE
mates when variance is heterogeneous (e.g., in the Since it implemented the modified sigma approach
range of 300 to 400, as in Figure 13(b)), but the in early 2008, Intel has seen safety stock inventory-

(a) 0.60 (b)


0.58 0.200

0.56 0.175
Forecast/Forecast + Actuals

0.54 0.150
0.52
0.125
COV

0.50
0.100
0.48
0.075
0.46
0.050
0.44
0.42 0.025

0.40 0
100 200 300 400 500 600 0 100 200 300 400 500 600 700
Forecast Forecast

Figure 13: Panel a depicts the Equation (2) randomly generated heterogeneous forecast error. Panel b demon-
strates the kernel-smoothed COV estimate, which appropriately estimated the underlying theoretical sigma (step
function) across all forecast levels.
Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
426 Interfaces 39(5), pp. 415–427, © 2009 INFORMS

2.5

2.0
WOI

1.5

1.0

0
Modified MEIO-CSO Standard MEIO-CSO
0.5
Process

Figure 14: The graphs show representative snapshots of boxed CPU inventory targets comparing the modified
sigma approach with a squared error standard deviation estimate. Panel a shows the overall reduction in both
variance and average weeks of inventory (WOI) targets; Panel b displays the inventory target under sigma mod-
ified relative to the squared error standard deviation.

level reductions of approximately 15 percent without Intel’s boxed processor inventory targeting has
any reduction in customer service levels. Figure 14(a) employed MEIO optimization techniques for several
demonstrates a snapshot of its inventory targets in years. Intel’s experience has shown that appropriate
the first half of 2008. Both average targets and target- demand characterization is a critical driver of the
ing variability have been reduced. Figure 14(b) pro- model’s efficacy. Demand characterization also tends
vides insight into inventory targeting from a paired to be the variable that departs most frequently and
perspective in which values are measured as the severely from the conventional assumptions upon
inventory target under the modified sigma approach which most optimization models and software are
standardized to the previous method of a squared built. The adaptations that Intel developed to address
error standard error. Approximately 70 percent of the ill-behaving demand characterization have focused
SKUs had a significantly lower safety stock target on correcting bias and heterogeneity impacts to opti-
than they would had the sigma not been modified; mized inventory target levels. In cases in which
approximately 10 percent saw a significant increase imperfections, such as bias or heterogeneity, cannot be
in safety stock targets. Because of the propensity to removed from the raw forecast data, these approaches
overforecast, as we expected, the realized safety stock
have proven effective in allowing a formal SIOP pro-
reductions (∼15 percent) were less than the targets
cess to achieve desired inventory and service-level
called for (∼50 percent) because the forecast bias
targets.
inherently carries with it extra “safety stock” that is
realized in finished goods when the demand does not
materialize. Appendix A
Although initially developed only to address prod-
If the service level is predetermined, then Equa-
ucts failing the homogeneity test, we found the
tions (4)–(6) can be replaced with a simple cumulative
kernel-smoothing technique robust enough to replace
density problem and the original sigma-modified cal-
the need for the initial Bartlett test (Bartlett and
culation in Manary and Willems (2008) in which the
Kendall 1946) and RMSE estimate of sigma modified.
¯ solved for in Equation (A1),
Intel fully implemented the kernel-smoothing tech-
nique (with retirement of the homogeneity testing) in  1
fall 2008. P  Forecast  d = Service Level Selection (A1)

Manary et al.: Correcting Heterogeneous and Biased Forecast Error at Intel for Supply Chain Optimization
Interfaces 39(5), pp. 415–427, © 2009 INFORMS 427

is then used in Equation (3); the result is decision support, Paul Bloomquist for his technical expertise
in the programming environment, and Pat Mastrantonio for
 
Modified ¯ 
1 − / ¯ −1 his advocacy at the management level. The success of this


 = Max ∗  0  (A2) project would not have been possible without their contribu-
t df tions. M. P. Manary’s current affiliation is Duke University’s
Fuqua School of Business, Durham, NC 27708.
where the probability density function is defined by
the kernel-smoothed density field localized at the cur-
References
rent forecast.
Bartlett, M. S., D. G. Kendall. 1946. The statistical analysis of
variance-heterogeneity and the logarithmic transformation.
J. Royal Statist. Soc. Supplement 8(1) 128–138.
Appendix B Bowman, A., A. Azzalini. 1997. Applied Smoothing Techniques for
Data Analysis. Oxford University Press, Oxford, UK.
To test for homogeneity, we calculated an adapta- Bowman, A., P. Foster. 1993. Density based exploration of bivariate
tion of Bartlett and Kendall’s (1946) statistic using the data. Statist. Comput. 3(4) 171–177.
sigma-modified calculation. In the adapted Bartlett’s Brown, M. B., A. B. Forsythe. 1974. Robust tests for the equality of
variances. J. Amer. Statist. Assoc. 69(346) 364–367.
test, (n − 1) represents the historical observations to Crum, C., G. E. Palmatier. 2003. Demand Management Best Practices:
draw against, k reflects the number of service levels Process, Principles and Collaboration. J. Ross Publishing, Boca
(SL) to test against, and sigma modified is calculated Raton, FL.
Franses, P. H. 1998. Time Series Models for Business and Economic
from Equation (3) for k service levels, as Equation (B1) Forecasting. Cambridge University Press, Cambridge, UK.
shows: Levene, H. 1960. Robust tests for the equality of variances. I. Olkin,
  ed. Contributions to Probability and Statistics: Essays in Honor
n−1k∗ln SL ˆ Modified
2
/k− SL lnˆ Modified
2
 of Harold Hotelling. Stanford University Press, Palo Alto, CA,
T= 
278–292.
1+ SL 1/n−1−1/k∗n−1/3∗k−1 Manary, M. P., S. P. Willems. 2008. Setting safety-stock targets at
(B1) Intel in the presence of forecast bias. Interfaces 38(2) 112–122.
Montgomery, D. C., L. A. Johnson, J. S. Gardiner. 1990. Forecast-
ing and Time Series Analysis, 2nd ed. McGraw-Hill Companies,
New York.
Acknowledgments O’Brien, R. G. 1979. A general ANOVA method for robust tests of
We thank the additional team members at Intel: Brian additive models for variances. J. Amer. Statist. Assoc. 74(368)
Wieland and Major Govindaraju for their role in managing 877–880.
Copyright 2009, by INFORMS, all rights reserved. Copyright of Interfaces is the property of INFORMS:
Institute for Operations Research and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print, download, or
email articles for individual use.

You might also like