The Measurable News

26

Winter 20072008

Measuring the Accuracy of Earned Value/Earned

Schedule Forecasting Predictors
By Mario Vanhoucke and Stephan Vandevoorde

Abstract
Earned value systems have been setup to deal with the complex task of controlling and adjusting the
baseline project schedule during execution, taking into account project scope, timed delivery, and total
project budget. Although earned value systems have been proven to provide reliable estimates for the
follow-up of cost performance within certain project assumptions, it often fails to predict the total duration of the project. In this article, we give a brief overview and summary of a simulation study that
investigates the potential of three earned-value-based methods to forecast the final project duration. The
study assumes a project setting where project activities and precedence relations are known in advance
and does not consider fundamentally unforeseeable events and/or unknown interactions among various
actions that might cause entirely unexpected effects in different project parts. This paper is the first in a
sequence that summarizes the results of the large simulation study initiated by Vanhoucke and Vandevoorde (2007). Each paper will discuss and highlight an aspect of the simulation study.

he planned value method (Anbari, 2003) and

the earned duration method (Jacob, 2003) are
two well-known methods that rely on the traditional schedule performance indicator SPI
(= EV/PV) and can be used to predict a projects final
duration. Lipke (2003) has criticized these approaches
and has presented a straightforward extension, known
as the earned schedule technique, which claims to
overcome some drawbacks of the traditional earned
value methods when predicting a projects final duration. More precisely, the earned schedule method
has been developed as a criticism on the use of the
classic SV and SPI metrics since they give false and
unreliable time forecasts near the end of the project.
Instead, Lipke (2003) provided a time-based measure
to overcome the quirky behavior of the SV and SPI
indicators, and calculates two alternative schedule
performance measures (referred to as SV(t) and SPI(t))
that are directly expressed in time units. This novel
method relies on similar principles of the earned value
method, and can be calculated as follows:
Find t such that EV PVt and EV< PVt+1
ES = t + (EV PVt)/( PVt+1 PVt)
with
ES Earned Schedule
EV Earned Value at the actual time
PVt Planned Value at time instance t.
1

Consequently, the cumulative value for the ES is

found by using the EV to identify in which time increment t of PV the cost value for EV occurs. ES is
then equal to the cumulative time t to the beginning
of that time increment, plus a fraction (EV PVt) /
(PVt+1 PVt) of it. The fraction equals the portion of
EV extending into the incomplete time increment divided by the total PV planned for that same time period, which is simply calculated as a linear interpolation
between the time-span of time increment t and t + 1.
Since the introduction of the earned schedule
concept by Lipke (2003), other authors have investigated the potential of the new method in various
ways. Henderson (2003) has shown the validity of
the ES concepts on a portfolio of six projects. We
(Vandevoorde & Vanhoucke, 2006) were the first authors to compare the three methods and test them to
a simple one activity project and a real-life data set.
Moreover, we summarized the often confusing terminology used in the earned value literature, which
will be used throughout this paper.
In Vanhoucke and Vandevoorde (2007), we argue
that the promising results of these various research
efforts need be taken with a critical eye. Inspired
by the shortcomings of the limited set of real-life
projects, we started a research project in 2005 that
aims at the validation of various earned value based

Ghent University, Tweekerkenstraat 2, 9000 Gent (Belgium) and the Vlerick Leuven Gent Management School, Reep 1, 9000 Gent
(Belgium), [email protected]
2
Fabricom GTI Suez, Rue Gatti de Gammond 254, 1180 Brussel (Belgium), [email protected]

Winter 20072008

The Measurable News

forecasting methods on a very large and diverse

set of fictive projects. We performed this research
project and aimed at tightening the bridge between
academic knowledge and current practice by generalizing case-specific results to a large and diverse set
of project settings. Since simulation studies require
computer power, in 2007, the Belgian Chapter of the
Project Management Institute decided to fund part of
the research.
In this article, we test the performance of the three
project duration forecasting methods in an objective
way by simulating a large dataset containing projects
of moderate size and calculating the forecast accuracy
of each method. The outline of this manuscript is as
follows:
In section two, the design of the simulation study
is presented in detail. The set of fictive projects
is presented, and the simulation methodology is
briefly outlined.
In section 3, an overview of the basic results is
presented. These results clearly show that the
new earned schedule forecasting methods outperform, on average, the existing earned value based
predictors.
In section 4, we give a conclusion and sketch a
brief summary of the future articles.

Test Design
In this section, the test design of the simulation
study is outlined. The set of simulated projects is
described in detail, followed by the presentation of
three project duration forecasting methods. In a last
subsection, the mathematical approach to measure
the forecast accuracy is discussed.
Set of Simulated Projects

The set of projects has been generated under a controlled design by carefully controlling the structure
of each project network. The generator RanGen
(Vanhoucke et al., 2007; Demeulemeester et al.,
2003) generates activity-on-the-node3 networks with
a predefined number of activities between which

27

precedence relations exist. The set of projects in the

simulation study contains 4,100 project networks.
Each project network has 30 activities but contains
a different amount of precedence relations between
the activities leading to a wide variety of different
network structures.
Each project is simulated by changing the original
planned duration for each activity under a controlled
design, leading to a final project duration that might
differ from its planned duration. Both projects, that
finish ahead of schedule and finish with a delay, have
been simulated. Each project is the subject of 100 simulation runs to guarantee simulation convergence.
Forecasting Methods

To the best of our knowledge, only three project duration forecasting methods have been presented in
literature. In the remainder of this paper, we refer to
these methods as the planned value method (Anbari,
2003), the earned duration method (Jacob, 2003),
and the earned schedule method (Lipke, 2003). The
first two methods rely on the traditional EV metrics
(SV and SPI) while the latter makes use of the novel
ES metrics (and SV(t) and SPI(t)). We use the abbreviation EAC(t) for time forecasting (similar to the
abbreviation EAC without the t between brackets
for cost forecasting). The simulation study simulates and compares three forecasting methods, each
of which can be considered under three scenarios,
according to the future expected performance, as
given in Table 1.4 The expected future performance
determines the calculations of the project duration
forecast and is a choice the project manager has to
make depending on the current project situation
(EAC(t) with subscripts PV1,2,3, ED1,2,3 and ES1,2,3). If
past performance is not a good predictor of future
performance, the expected future performance can
be done according to the plan. Problems/opportunities of the past will not affect the future, and the
remaining work will be done according to plan.
If past performance is a good predictor of future
performance (realistic!), the expected future per-

Alternatively, one could model a project network as an activity-on-the-arc network where arcs represent the project activities and
nodes represent project events to implicitly model the precedence constraints. Since most commercial software use the activity-on-thenode representation, we do not rely on this alternative project network representation.

Details about the mathematical formulas of each method can be found in Vandevoorde and Vanhoucke (2006).

28

The Measurable News

formance is likely to follow the current SPI trend.

Problems/opportunities of the past will affect future
performance, and the remaining work will be corrected for the observed efficiencies or inefficiencies.
If past cost and schedule problems are good indicators for future performance (i.e., cost and schedule
management are inseparable), the expected future
performance is likely to follow the SCI trend. The
SCI = SPI * CPI (schedule cost ratio) is often called
the critical ratio index
Forecast Accuracy

Winter 20072008

(see Table 1). The second part measures the accuracy

along the stages of completion for each project.
Overall Accuracy of the Three Methods

Figure 1 shows the average overall forecast accuracy as the average percentage deviation between the
planned duration and the average predicted duration
over all review periods (see earlier) of the three methods for early and late projects (note that the EAC(t)
measures have been abbreviated to PV, ED, and ES).
The figure clearly reveals that the earned schedule
method outperforms, on average, the two other forecasting methods (planned value method and earned
duration method), both for projects that finish ahead
of schedule and projects with a delay. Moreover, the
figure also shows that the forecasting method that assumes that the expected future performance follows
the current SPI or SPI(t) trend (i.e., EAC(t)ES2) outperforms the other methods.

The accuracy of the forecast is measured for each

project under an early and late performance. During
the simulation, the final project duration is forecast
by means of the three EAC(t) methods (where each
method can be calculated according to three scenarios, see Table 1). After the simulation run, the final
real project duration is compared with the average
Accuracy Along the Stages of Completion
forecast value (i.e., the average of all EAC(t) values
measured./0
during
the life *
of the
project)./$-!!
More"+-!./%*#
preIn this.part,
forecast accuracy is reported along
$! .%)0(/%+*
4 .%)0(/!.
+),-!.
)!/$+
!$the
+" 2$%$
is measured
as the average
the/+ /$!the
stage!3,!/!
of the project
* !cisely,
+*.% the
!-! accuracy
0* !- /$-!!
%
!-!*/ .!*-%+.
+- of%*#
"0/0-!
,!- completion for projects ahead
  $! !3,!/! "0/0-! ,!-"+-)*! !/!-)%*!. /$! (0
"+-)!*!
. #%1!*
%* /(!
absolute
values
of alldeviations
between the various
of schedule or for projects with a delay. The early
(/%+*.forecasts
+" /$! ,-+&!/
0-/%+*
"+-!./
%.  project
$+%! /$!
)*#!$. /+ as
)'!
measured
along
the life*of the
and,-+&!/stage
is defined
the first 30% of the project com!,!* %*# +* /$! 0--!*/ ,-+&!/ .%/0/%+* / 2%/$ .0.-%,/.   
the real project duration. This accuracy is expressed
pletion, the middle stage is defined as the interval
*   " ,./ ,!-"+-)*! %. *+/  #++ ,-! %/+- +" "0/0-! ,!-"+-)*! /$!
as
a
percentage
deviation,
with
lower
values
denotbetween 30% and 70% completion, and the late stage
!3,!/! "0/0-! ,!-"+-)*! * ! +*! +- %*# /+ /$! ,(* -+(!). +,,+-/0*%/%!.
better
forecast
accuracy.
the %*#
last/+30%
+" ,./ing
2%((a *+/

!/
/$! "0/0-!
* /$! -!)%*%*# 2+-' 2%(( ! equals
+*! +,(*completion of the project. Table
2 displays/$!
the!3,!/!
accuracy for the early, middle, and late
" ,./ ,!-"+-)*! %.  #++ ,-! %/+- +" "0/0-! ,!-"+-)*! -!(%./%
"0/0-!Computational
,!-"+-)*! %. (%'!(4 /+ Experiments
"+((+2 /$! 0--!*/  /-!*  -+(!). +,,+-/0*%/%!.
stage of the project +"
for all forecasting methods.
/$! ,./
2%((

!/
"0/0-!
,!-"+-)*!
*
/$!
-!)%*%*#
2+-'
2%((
!
+--!/!
"+In this section, the main results of the simulation
The results from the table confirm the previously
/$! +.!-1!
!%!*%!.
+- The
%*!%!*%!.
" ,./ +./
,-+(!).
study are
presented.
first part compares
the* .$! 0(!
found
results-!
and#++
show that the earned schedule
%* %/+-. "+- "0/0-! ,!-"+-)*! %! +./ * .$! 0(! )*#!)!*/ -! %*.!,-(!
overall accuracy of early and late projects for the
method outperforms, on average, the other forecast/$! !3,!/! "0/0-! ,!-"+-)*! %. (%'!(4 /+ "+((+2 /$!  /-!*  $!     
forecasting
methods
the three
scenarios
ing methods. The results also illustrate the quirky
.$! three
0(! +./
-/%+ %. +"/!*
((!under
/$! -%/%(
-/%+
%* !3
behavior of the SPI indicator
TABLE 1: FORECASTING MEASURES (SOURCE: VANDEVOORDE AND
(used in the planned value and
VANHOUCKE (2006))
earned duration methods) at the
(!  +-!./%*# )!.0-!. +0-! * !1++- ! * *$+0'! 



 ! !

late stage of the project. Indeed,
#! "!" 
 
 

the late stage forecast accuracy

is much better for the ES method
/

/

/

+- %*# /+ ,(*
compared to the PV and ED
/
/

+((+2. 0--!*/ 
methods. The SPI(t) indicator of
/
+- / /-!*


the earned schedule method is de/
/

+((+2. 0--!*/ 
veloped to overcome this quirky


/
+- / /-!*
behavior, leading to an improved






forecast accuracy at the end of
"
"!
"
the project.




 
  

$! 0-4 +" /$! "+-!./ %. )!.0-! "+- !$ ,-+&!/ 0* !- * !-(4 * (/! ,!-

Winter 20072008

The Measurable News

29

FIGURE 1: OVERALL FORECAST ACCURACY OF THE THREE FORECASTING METHODS

Conclusions and Future Work

In this paper, a brief overview of our simulation
study (Vanhoucke and Vandevoorde, 2007) has
been presented that compares a recently introduced
earned schedule project duration forecasting technique with the more traditional earned value based
forecasting metrics. This article presents basic results of the simulation study, while other, more detailed results will be presented in later articles. The
main results are that the earned schedule metrics
outperform, on the average, both the planned value
method (Anbari, 2003) and earned duration (Jacob,
2003) methods. Moreover, the studies reveal that the
earned schedule method is more reliable in all stages
(early stage, middle stage, and late stage) of the project life cycle.
Our next article will elaborate on the forecast accuracy of earned value predictors and investigate
whether there is a difference in accuracy between
critical and non-critical activities. Meanwhile, the
academic research remains ongoing and focuses on
the simulation of the p-factor study of Lipke (2004).
Details can be expected in the beginning of 2008.
In the somewhat more distant future, a new book
on earned schedule management will be published
where algorithmic details and case studies, and many
more issues will be discussed in detail. At the same
time, a novel software package will be launched that
enables earned value/schedule project tracking and
the simulation study presented in the current manuscript. This project is in the middle of the development phase. More details will follow in a later article
or will soon be accessible on www.or-as.be.

TABLE 2: FORECAST ACCURACY ALONG THE COMPLETION

STAGE OF THE PROJECT

Acknowledgements
We acknowledge the support by the research collaboration fund of PMI Belgium received in Brussels
in 2007 at the Belgian Chapter meeting.

References
Anbari, F. (2003). Earned Value Project Management
Method and Extensions. Project Management
Journal, 34:1223.
Demeulemeester, E., Vanhoucke, M., and Herroelen, W.
(2003). A Random Network Generator for Activityon-the-Node Networks. Journal of Scheduling,
6:1334.
Jacob, D. (2003). Forecasting Project Schedule
Completion with Earned Value Metrics. The
Measurable News, March, 2003, 79.

30

The Measurable News

Lipke, W. (2003). Schedule is Different. The Measurable

News, Summer 2003.
Lipke, W. (2004). Connecting Earned Value to the
Schedule. Crosstalk, June 2005.
Vandevoorde, S. and Vanhoucke, M. (2006). A
Comparison of Different Project Duration
Forecasting Methods Using Earned Value Metrics.
International Journal of Project Management,
24:289302.
Vanhoucke, M., Coelho, J.S., Debels, D., Maenhout,
B. and Tavares, L.V., 2007, An evaluation of the
adequacy of project network generators with
systematically sampled networks, European Journal
of Operational Research, 187, 511524
Vanhoucke, M. and Vandevoorde, S. (2007). A
Simulation and Evaluation of Earned Value Metrics
to Forecast the Project Duration. Journal of the
Operational Research Society, 58:13611374.

Author Biographies
Dr. Mario Vanhoucke is an associate professor at the Ghent University and the Vlerick Leuven Gent
Management School (Belgium).
He teaches Project Management,
Business Statistics, and Applied
Operations Research. He is the
program director of the Commercial Engineers and
the advanced Master in Operations and Technology
Management. He is partner of the company OR-AS
(www.or-as.be), where he is involved in the development of a project scheduling software package
with earned schedule tracking capabilities. His main
research interest lies in simulation and optimization models in project scheduling and scheduling in

Winter 20072008

the health-care sector. He is advisor to various PhD

projects, in collaboration with different university
hospitals. He has articles published in international
journals, such as Annals of Operations Research,
Management Science, Operations Research, The
Accounting Review, International Journal of Production Research, Journal of the Operational Research Society, Journal of Scheduling, International
Journal of Project Management, Project Management Journal, European Journal of Operational Research, and Lecture Notes on Computer Science.
Stephan Vandevoorde is a division
manager in Fabricom GTI Suez, for
airport baggage handling systems.
He has an industrial engineer diploma and is a member of PMI, Project
Management Belgium, PMI College
of Performance Management, and
director of programs of the PMI Belgium Chapter.
He has been working on a number of large-scale
international projects (Europe, Asia) across many
industries, including construction, retail, automotive, and airline. Stephan has extensive experience
in using EVM techniques to assist in evaluating and
predicting project performance, including the newly
developed earned schedule concept. On several occasions, he has given presentations on different project
management topics for V.I.K., Vlerick Management
School, Ghent University, and Boston University,
Brussels. In collaboration with the Institute for Business Development, Stephan is docent for the courses
Earned Value Management and Project Management
for the Construction industry.

EVM World 2008

CREATING THE ENVIRONMENT FOR EARNED VALUE MANAGEMENT

PMI-CPM 24th Annual International Conference

Clearwater Beach Resort
May 1416, 2008 Clearwater Beach, FL
Call for papers and more information at www.pmi-cpm.org