St.

Mary’s University

School of Graduate Studies

Evaluation of Ethiopia- Djibouti Railway project using

Organization for Economic Co-operation Development Criteria

By: Eskendir Adinew

ID: SGS/0562/2011A

A Thesis Submitted to St. Mary’s University, School of Business in Partial Fulfilment for
the Requirements of Masters of Arts Degree in Project Management

Advisor: Maru Shete (Dr)

Jun, 2021

Addis Ababa, Ethiopia

 Evaluation of Ethiopia- Djibouti Railway project using

Organization for Economic Co-operation Development Criteria

By
ESKINDER ADINEW

ID: SGS/0562/2011A

A Thesis Submitted to St. Mary’s University, School of Business in Partial Fulfilment for
the Requirements of Masters of Arts Degree in Project Management

Advisor: Maru Shete (Dr)

Jun,2021
Addis Ababa, Ethiopia
 St. Mary’s University

School of Graduate Studies

Evaluation of Ethiopia- Djibouti Railway project using

Organization for Economic Co-operation Development Criteria

By

ESKINDER ADINEW

ID: SGS/0562/2011A

APPROVED BY BOARD OF EXAMINERS

__________________________________ _____________________
Dean, Graduate Studies Signature

__________________________________ ______________________
Advisor Signature

___________________________________ ____________________
External Examiner Signature

___________________________________ ___________________
Internal Examiner Signature
 Declaration

I, hereby, declare that this thesis entitled Evaluation of Ethiopia- Djibouti Railway project using
OECD Criteria is my original work, prepared under the guidance of Maru Shete (PhD) and has
not been presented for a degree in any other university. All source of materials used for the thesis
have been duly acknowledged. I further confirm that the thesis has not been submitted either in
part or in full to any other higher learning institution for the purpose of earning any degree.

__________________________________ _________________
Name Signature
St. Mary`s University,
Jun, 2021
Addis Ababa, Ethiopia
 Endorsement

This is to certify that Eskinder Adinew has completed his thesis entitled Evaluation of Ethiopia-
Djibouti Railway Project using OECD Criteria. As I have evaluated, his thesis, it is appropriate
to be submitted as a partial fulfillment required for the award of Master of Arts Degree in Project
Management.

___________________________________ _____________________________
Advisor Signature
St. Mary`s University
Jun, 2021
Addis Ababa, Ethiopia
 Table of Contents

LIST OF TABLES...................................................................................................................... I
LIST OF FIGURES…………………………………………………………………………… II
ACKNOWLEDGEMENTS ....................................................................................................... III
List of Acronyms ........................................................................................................................ IV
Abstract …………………………………………………………………………………………V
Chapter One ................................................................................................................................ 1
Introduction ................................................................................................................................. 1
1.1.Background of the study................................................................................................. 1
1.2.Statement of the problem ............................................................................................... 2
1.3.Research Question........................................................................................................... 4
1.4.Objectives of the study ................................................................................................... 4
1.5.Significance of the Study ................................................................................................ 4
1.6.Scope and Limitation of the Study................................................................................... 5
1.7.Organization of the Study................................................................................................. 5
CHAPTER TWO ........................................................................................................................... 6
LITERATURE REVIEW .............................................................................................................. 6
2.1 Introduction ............................................................................................................................. 6
2.2 Project Evaluation ……………............................................................................................. 7
2.3. Steps to conduct Project Evaluation …....………….…………………………………..…….8
2.4. Types of Evaluation ………………………………….……………………………………..9
2.5 OECD Project evaluation criteria .......................................................................................... 13
2.6 Importance of Project Evaluation .......................................................................................... 17
2.7 The Project Evaluation Procedure ......................................................................................... 18
2.8 Overview of Railway industry ............................................................................................... 19
2.8.1 Railway in Ethiopia …………………………..…………………………………………..20
2.9. Empirical Review ………………………………………………………………..………….21
2.10 Conceptual Framework…………………………..…………………………………………23
CHAPTER THREE ..................................................................................................................... 24
RESEARCH DESIGN AND APPROACH ……. ....................................................................... 24
3.1 Introduction ............................................................................................................................ 24
3.2 Research design and approach ............................................................................................... 24
3.3 Data Type and Source ........................................................................................................... 25
3.4 Sampling technique and Sampling ………………................................................................. 25
3.5 Data Collection Method ………………………………...…………………………………...26
3.6 Data analysis method ............................................................................................................. 27

Chapter Four ................................................................................................................................ 29

Results and Discussions ............................................................................................................... 29
4.1 Introduction ………………………………………………………………………………….29
4.2 Respondent Profile ……………………………………………………………………….….29
4.3 OECD Evaluation Results of Ethiopia- Djibouti railway Project .......................................... 30
4.3.1 Relevance of the Project ..................................................................................................... 30
4.3.2 Efficiency of Ethiopia- Djibouti railway Project ................................................................ 35
4.3.3 Effectiveness of Ethiopia- Djibouti railway Project............................................................ 38
4.3.4 Sustainability of Ethiopia- Djibouti railway Project ........................................................... 44
Chapter Five ................................................................................................................................. 47
Summary, Conclusion and Recommendation .............................................................................. 47
5.1 Summary of Findings ............................................................................................................. 47
5.2 Conclusion.............................................................................................................................. 48
5.3 Recommendation: .................................................................................................................. 49
REFERENCES............................................................................................................................. 50
APPENDIX ................................................................................................................................. 56
 LIST OF TABLES

Table 4.1: Interview respondent profile ……………………………………………………29

Table 4.2: Earned value of the Project ……………………………………………………...34

I
 LIST OF FIGURES

Figure 1: Conceptual Framework for OECD project Evaluation ………………………23

II
 ACKNOWLEDGEMENTS

First, I would like to thank Almighty God who keep me healthy and strong to finalize my master
program and the paper itself.

I really thank and appreciate all ERC stuffs who involved in the completion of this paper which
makes things easy for me and especial thanks to Mr. Kassahun a project coordinator at ERC for
your time, patience and support.

And, I would like to give a huge appreciation and much thanks to my advisor Maru Shete (PhD)
for his guidance during the whole process of my paper.

Lastly, I would like to be grateful to my family especially my Mom who supported and
encouraged me during my studies and to all friends ―you were there for me when I need it so
special thanks to you all‖.

III
 List of Acronyms
AC Actual cost
CREC China Railway Group
CCECC China Civil Engineering Construction Cooperation
CV Cost Variance
CPI The cost performance index
DAC Development Assistance Committee
ERC Ethiopian Railway Cooperation
EDR Ethio- Djibouti railway
EU European Union
EVA Earned value analysis
GHG Green House Gas
GTP Growth and Transformation Plan
LFA Logical Framework Analysis
MDGs Millennium Development Goals
METEC Metal and Engineering Corporation
OECD Organization for Economic Co-operation and Development
PMBOK Project Management Body of Knowledge
PMI Project Management Institute
PRSP Poverty Reduction Strategy Paper
SDGs Sustainable Development Goals
SV Schedule variance
SPI Schedule performance index
TAG Technical Advisory Group
TCS The Technical Cooperation Strategy

IV
 Abstract

The 756km Ethiopia- Djibouti railway modernization project is the first electrified railway line
in East Africa that has strategically benefit for the country economy. This project was jointly
owned by the government of Ethiopia and Djibouti and constructed by China Railway Group and
China Civil Engineering Construction Corporation. As per initial plan the project has been
completed and was formally inaugurated for passenger in October 2016 and official commercial
operation for both passenger and freight services commenced in January 2018, in this research
using OECD (Organization for Economic Co-operation and Development) evaluation criteria;
the relevance, effectiveness. efficiency and sustainability aspects of the project is analyzed in
detail. Descriptive research approach is selected for this study. Data is collected for the study
using document review and semi-structured interview. The collected Data was analyzed and
interpreted using descriptive analysis technique and earned value analysis. The study found that
the project is consistent with local community need, country requirement and global priorities,
and it’s achieved cost, quality and schedule requirement of the project with expected long-term
benefits. Using semi-structured interview and document review the study point out that the
project is providing multiple benefits for the nation in terms of time saving, reduction in road
maintenance cost, fuel saving, employment generation, reduction in pollution, foreign exchange
earnings and revenue generation. These benefits are accruing to government, passengers,
general public and the business community at large. Based on that for recommendation the
project lacks utilization of local material which could contribute something for local economy
through acquisition of local material for construction projects. This might help to meet the GTP
goal to transform the country’s economy into industrialization by encouraging local
manufacturers and service activities. Therefore, in order to close the trade balance gap of the
nation in the global trade the government should set a cap on which kind of material imported
and which are manufactured locally and utilized.

Key Words: OECD Project Evaluation, Relevance, Effectiveness, Efficiency, Sustainability

and Ethiopia- Djibouti railway

V
 Chapter one

Introduction
1.1. Background of study
Evaluation is the systematic and objective assessment of an on-going or completed projects or
programs. It involves collecting and analyzing information about the project activities,
characteristics, and outcomes. The main purpose is to make a decision about an ongoing project
which help to improve its effectiveness or to provide on time information for programing and
project decisions. Evaluation can be a better means for learning from past experience for
different stakeholders to improve service delivery, planning and allocation of resources
(International Finance Corporation, 2008). It brings institutional development by creating a
system for its role and functions to enable them to carry out responsibilities in a better way, this
can be reflected through by introducing change and new development way by organized the
institution for a better result to meet its mission (World Bank, 2005). Although one benefit of
conducting evaluation is to provide information for actions which is for decision makings,
strategic planning, reporting or project modifications this help to understand the progress,
success, and effectiveness of a project.

Evaluation systems support development by generating relevant, accurate and timely information
which is used to improve program design, enhance decision-making and thus increase impact.
While closely related to research, operations research and the social sciences, functional
evaluation may neither practice nor aspire to emulate, exactly, the stringent scientific standards
of more academic approaches (Elkins, 2006). Elkins (2006) further states that without doubt
evaluation practices and approaches overrides the academic social- science domains,
nevertheless evaluation purposes and techniques are usefully distinguished as a variety of
information collection, processing and use.

According to Berhanu et al. (2011), implementation of development project is important to

reduce poverty and achieve sustainable livelihood. The success and speed with which
development project is achieved depends in part on the performance of the institution working to
promote the development project. Thus, any institution working in implementing development

1
project is concerned with the need to assess and understand its performance and to improve
relevance, effectiveness and efficiency of project through evaluation. In addition, Berhanu et al.
(2011) stated that, currently, the focus of management changes from activities to result. As a
result, the focus of project evaluation also changes from focusing on assessing inputs and
progressive monitoring to the assessment of the contribution of intervention to development
project outcomes or changes.

OECD evaluation criteria is well familiar to evaluate the development interventions around the
globe. It is the systematic assessment of an on-going or completed projects that determine the
relevance and fulfillment of objective, development efficiency, effectiveness, impact and
sustainability (OECD, 2002). OECD criterion contains five criteria: relevance efficiency,
effectiveness, impact and sustainability. Relevance the extent to which the objective of a
development intervention is consistent with beneficiaries‘ requirements, country needs, global
priorities and partners, Effectiveness the extent to which the development intervention‘s
objective were achieved, or are expected to achieved, Efficiency a measure of how economically
resources/inputs are converted in to result, and Sustainability the continuation of benefits from a
development intervention after the development completed.

Properly functioning of evaluation system in a development intervention are expected to control

against any negative result of the desired plan of the project. Therefore, the study wishes to
examine the Ethiopia-Djibouti railway project using OECD criteria.

1.2. Statement of the problem

The importance of developing a constraint-free and reliable work plan has long been recognized
by development, private and civil society. Evaluation provides government officials,
development managers, the development and private sector and civil society with better means
for learning from past experience, improving service delivery, planning and allocation of
resources and demonstrating results as part of accountability to key stakeholders (International
Finance Corporation (IFC), 2008).

2
Governments in developing countries, where approximately 85.4% of the world‘s population
lives in, develop different projects to achieve their social and economic sustainable development
objectives (Human Development Report,2011; Zeybek and Kaynak, 2006; Cohen, 2006). This is
accomplished through construction of different projects. Because Ethiopia is the nation that has
low living standards, undeveloped industrial base, and low human development index (HDI) the
2018 world bank report classify Ethiopia as developing countries.

As developing country Ethiopia also initiates different projects, construction of Railway is one of
them which begin with the Addis Ababa - Djibouti corridor that is the major import and export
corridor of the country handling more than 90% of the foreign trade. In Ethiopia, freight growth is
expected to continue to be strong and this will continue to exert pressure on the accommodation of
freight. The demand for freight transportation in Ethiopia has been rising reflecting growth in the
economy. Such growth will place an increasing pressure on a network seeking to provide capability
for growth in both passenger and freight traffic (AICD, 2010).

The Addis Ababa-Djibouti Railway modernization project is the first cross-border electrified
railway in Africa. The railway line is 753 kilometers (km) electrified single-track standard gauge
line between Ethiopia´s capital Addis Ababa and the Port of Djibouti, with 45 stations in total.
The new standard gauge line runs parallel to and replaces the abandoned one-meter gauge
railway, which was built more than 100 years ago (Ethiopian railway Cooperation (ERC), 2018).

The extent to which a development intervention has contributed to the achievement of desired
targets should be examined in every phase after its completion, as per the Addis Ababa Bureau
of Finance and Economic Development report (2016) criticizes an inadequacy of monitoring and
evaluation practices on development projects implemented by sector bureaus. Besides, two basic
points that strengthen this argument are: The first point the past 5 years government report states,
there is almost no project that successfully completed as per the schedule and allocated cost. The
second point is the problem that states the management capacity of projects is at its lowest level;
the work of project administering and leading as well as completing according to their cost,
schedule and quality is acute problem of the Administration. Therefore, the central focus of this
project work is to examine the Ethiopia- Djibouti railway project using OECD criterion.

3
1.3. Research Question

1. The extent to which the objectives of the project development intervention are consistent
with beneficiaries‘ requirement, country needs and global priorities?
2. Are resources and inputs converted into outputs in timely and cost-effective manner?
3. Dose the project activities attain its objectives?
4. The extent to which the benefits of the project likely to grow beyond the project life?

1.4. Objective of the study

1.4.1. General Objective

The general objective of the research is to evaluate the overall performance of Ethiopia –
Djibouti Railway project using OECD criteria of project evaluation.

1.4.2. Specific Objective

The research has also some specific objectives:

1. To examine the achievement of the planed objectives of the project using schedule
performance index.
2. To examine the performance of the project in terms of its cost performance index.
3. To assess the relevance of the project from the perspective of different stakeholders.
4. To analyze the project benefits maintained over time.

1.5. Significance of the study

As we know, Ethiopia is a landlocked and poor country, this Ethio-Djibouti railway line
is the critical path for country's import and export. Over 90% of Ethiopian external trades
are conducted through port of Djibouti; to be economical and to attain sustainable
development this project should meet its intended plan. Therefore, this research is try to
answer if the intended objective of this mega project has been meet as expected by using
OECD evaluation criterion.

1.6. Scope and Limitation of the study

1.6.1. Scope of the study
4
The study is concerned to evaluate the relevance, efficiency, effectiveness and sustainability of
the Addis Ababa – Djibouti railway project and due to the early completeness of the project and
some other constraints it will be tough to measure the impact of the intervention at a time. This
research focuses only the investment cost and works of the project like; land development,
infrastructures (Electricity, Buildings), station and other fixed equipment‘s, and some relevant
operating costs of the project, to avoid unnecessary broadness of the research pre-feasibility and
feasibility cost, land and right cost, and training costs will be excluded from the research.

1.6.2. Limitation of the study

And also, this study has been subject to various limitations that may have hindered its accuracy.
Some of them are:

 Lack of sufficient literature conducted on construction project evaluation using OECD

criteria‘s, are the major limitation of the studies.
 Limited data collection from limited number of samples only from the Ethiopia side is
another major limitation of the study.
 Due to the early completeness of the project failed to include the beneficiaries as a source
of data.
 Accesses information related to the project like (Project plan Document) will be much
difficult due to the poor cooperation culture of the stakeholder.
 In some extent the researcher might face time and cost limitations in the process.

1.7. Organization of the study

The study has five chapters, chapter one is the introduction part which discuss about the
background of the study, problem statement, objectives, research question, significance and
scope and limitation of the study. In chapter two is literature related review which cover
theoretical and empirical review. The third chapter is discussing how the study has been
designed and what kind of methodology has been used and then the fourth chapter deals about
data presentation, analysis and interpretation. The last chapter is cover conclusion, and
recommendation of the study.

5
 Chapter Two
Literature related review

This chapter presents the related literatures on the study so as to have an insight in to the research
topic and briefly expose the readers to some of the major areas of the subject matter under
consideration.

2.1. Introduction of Project

Project is a unique set of co-ordinate activities, with defined starting and finishing points
undertaken by individuals or organizations to meet specific objectives with in defined schedule,
cost and performance parameter. The word unique points out that every project has its own
genuine nature in the sense that there may not be a pre-existing blue print for the project‘s
execution and there may not be a need to repeat the project once completed. Its goal
characteristics may be well perceive as achieving stated objectives or solve a particular problem,
while its temporary nature signifies a discrete, definable commencement and conclusion. Project
is series of activities and tasks that have Specific objectives, defined start and end dates, funding
limits, and it also has characteristics of multifunctional (Kerzner, 2009).

Projects are one of the several instruments to achieve particular objectives in a process of
development. Thus, projects has to be discussed as an integral part of the national development
strategy for they have to be evaluated in close reference to the overall development policy of a
nation. Projects have been described as ―the cutting edge‘‘ of development; they embody the
policy choices flowing from development objectives and acts as the vehicle or the medium of the
described social change (Rondinlli, 1976). Therefore, projects are the means through which
development target are achieved and considered to be a tangible benefit for the project
beneficiaries. Without visible projects on the ground, policies, strategies, and plans for the
development are simply administrative.

Project management is the process by which projects are defined, planned, monitored, controlled
and delivered such that the agreed benefits are realized. Projects bring about change and project
management is recognized as the most efficient way of managing such change. Project

6
management, on the other hand, involves project planning, monitoring and includes such items
as: Project definition of work requirements, definition of quantity and quality of work,
description of resources needed, project monitoring, tracking progress, comparing actual
outcome to predicted outcome, analyzing impact, making adjustments. Successful project
management can then be defined as having achieved the project Objectives (within time, within
cost, at the desired performance/technology level keeping the desired quality) while utilizing the
assigned resources successfully and efficiently, and accepted by the customer/owners (Kernzer,
2013).

According to Project Management Institute (PMI,2015), Project management is the application

of knowledge, skills, tools, and techniques to a program in order to meet the program
requirements and to obtain benefits and control not available by managing projects individually.
In the other case Project Management engages good practices to support coordination,
organization, and completion of projects from start to finish (Wills, 2012).

2.2.Project evaluation

Project Evaluation is a useful tool for managers to use to carry out deep assessments of the
design, efficiency, effectiveness, implementation or impact of programs, and for identifying
improvements of the projects. Project evaluation can be defined as a process that attempts to
determine, as systematically and objectively as possible, the achievement of result in light of
relevance, efficiency, effectiveness, impacts and sustainability of project activities. It is the
process of determining the worth or significance of a development activity, policy or program to
determine the relevance of objectives, the efficiency of design and implementation, the
efficiency of resource use, and the sustainability of results. An evaluation should incorporate
lessons learned into the decision-making process of both partner and donor (MoFED, 2008 and
Berhanu et al, 2010).

In order to compare the actual progress of the project with plan and to try to decide whether the
things and action taken were important and progressive evaluation is necessary. Evaluation is the

7
systematic collection of information about activities, characteristics and outcomes of the project
(Patton, 1986). The field of evaluation in the contemporary world are relatively new, meaning
the terms itself greatly depending on the industry, country, and general attitude towards business.

A well-functioning evaluation system is a critical part of good project management and

accountability. As Berhanu et al, (2010) note timely and reliable evaluation have the following
importance. First, useful to provide timely and useful information to decision-maker and
stakeholder feedback, especially beneficiaries, to provide input into and perceptions of work,
modeling openness to criticism, and willingness to learn from experiences and to adapt to
changing needs. Secondly, good evaluation system helps governments and organizations to
develop knowledge base of the types of projects, programs and policies that have worked and did
not work, and why. Thirdly, evaluation systems can be used to promote greater transparency and
accountability within organizations and governments.

2.3. Steps to conduct project evaluation

According to Hunter (2009) there are six major steps to be followed in project evaluation
process. These include identifying the purpose and scope of the evaluation system, plan for data
collection and management, plan for data analysis, plan for information reporting and utilization,
plan for evaluation of human resources and capacity building and prepare the evaluation budget.
The details of the six major steps are:

i. Identify the purpose and scope of the Evaluation system answers, ―Why do we
need evaluation and how comprehensive should it be?‖ It serves as a reference point for the
evaluation system, guiding key decisions such as informational needs, methodological
approaches, capacity building and allocation of resources.

ii. Plan for data collection and management: -Once you have defined the
project/programme‘s informational needs, the next step is to plan for the reliable collection and
management of the data so it can be efficiently analyses and used as information. Both data

8
collection and management are firmly linked as data management begins the moment it is
collected.

iii. Plan for data analysis: - Data analysis is the process of converting collected (raw) data
into usable information. This is a critical step of the evaluation planning process because it
shapes the
information that is reported and its potential use. It is really a continuous process throughout the
project/programme cycle to make sense of gathered data to inform ongoing and future
programming. Such analysis can occur when data is initially collected, and certainly when data is
explained in data reporting.

iv. Plan for information reporting and utilization: - Reporting is the most visible part
of the evaluation system, where collected and analyzed data is presented as information for key
stakeholders to use. Reporting is a critical part of evaluation because no matter how well data
may be collected and analyzed, if it is not well presented it cannot be well used – which can be a
considerable waste of valuable time, resources and personnel.

v. Plan for evaluation human resources and capacity building: -An effective
evaluation system requires capable people to support it. While the evaluation plan identifies
responsibilities for the data collection on each indicator, it is also important to plan for the people
responsible for evaluation processes, including data management, analysis, reporting and
evaluation training. This section summarizes key considerations in planning for the human
resources and capacity building or a project/programme‘s evaluation system.

vi. Prepare the evaluation budget: -It is best to begin systematically planning the
evaluation budget early in the project/programme design process so that adequate funds are
allocated and available for evaluation activities.

2.4. Types of evaluation

Evaluation activities should also follow the project developmental stages. In general, there is a
natural developmental sequence that the intervention project follows and the evaluation activities
should match the development level of the intervention appropriately. The project stage will
9
determine the level of effort and methods to be used. There are several types of evaluation that
can be used to conduct evaluation. Some of them including the following:

1. Formative Evaluation: is used before program design or implementation. It

generates data on the need for the program and develops the baseline for subsequent
monitoring. It also identifies areas of improvement and can give insights on what the
program‘s priorities should be. This helps project managers determine their areas of
concern and focus, and increases awareness of your program among the target population
prior to launch (Steven, 1993).
Formative Evaluation are mostly used for new projects or interventions and designed to
help identify needs or gaps in service which the new project should address or answer
other questions that need to be answered ( Wylle, 1992; Tessmer, 1994).

2. Process Evaluation: occurs when the program implementation began to determine

whether project activities has been implemented as intended and resulted a certain output.
You may conduct process evaluation periodically throughout the life of your project and
start by reviewing the activities and output components of the logic model.

When projects develop need to assess how well the implementation of the project is
going and if needed, to make corrections. In these stages, there are many evaluation
questions that could be ask, all having to do with project monitoring and evaluation
activities related to this problem. Answering this question involves process evaluation.
Process evaluation include documenting actual project functioning ( Dehar, 1993),
measuring exposure to and diffusion of the interventions ( Fortmann, 1982), and identify
barriers to implementation ( Demers, 1992). Process evaluation includes the identification
of the target population, a description of the service delivered, th use of resources, and the
qualification and experiences of personnel participating in them (NIDA, 1991). It
involves determining what service were actually delivered, to whom, and with what level
of resources.

10
 There are project monitoring tasks which must also be conducted before an outcome or
impact evaluation can take place. Project monitoring tasks are concerned with
documenting actual project functioning. Documenting project functioning is important
for two reasons, if the project is working well, there will be interest in replicating the
project in other locations that serve similar or other populations. In the other side if the
project isn‘t going well, it is of tremendous use to know exactly how the project failed, in
which component, and in what population (Chen, 2004). Some major questions posed in
this evaluation component:
 Which elements of the project actually have been implemented? Usually the
practical problem of here is that there are no data readily available to answer the
question. When that happen, the answer may be a guess rather than supported by
evidence.
 What are the types and volume of treatment or service actually provided to
clients? This question is important to answer both for accountability purpose and
to assist in the development of an outcome evaluation subsequent to project
implementation.
 What are the characteristics of project participants? It is important to determine if
the recipients of the project services resemble the intended target group as
identified in the project design and development stage. An effective intervention
administered to non-target group may be just as useless as an ineffective
intervention administered to a targeted group.

3. Outcome Evaluation: measures project effects in the target population by assessing

the progress in the outcomes that the project is to address. To design an outcome
evaluation, begin with a review of the outcome components of your logic model
(Drummond, 1987).
When process evaluation shows that the project was implemented properly, there is often
interest in measuring the effectiveness of the actual project (Mohr, 1995).
Outcome evaluations assess the effectiveness of a program in producing change.
Outcome evaluations focus on difficult questions that ask what happened to program
participants and how much of a difference the program made for them.

11
4. Economic Evaluation: considers both the outcomes of a project and the cost of
producing those outcomes. In some cases, the most effective project may also have the
lowest cost, but it‘s not necessarily true that the lowest-cost option is the most cost
effective. It is also possible that the project that produces the most units of a given
outcome maybe imperial to implement because it is so costly that it diverts too many
resources from other uses, or requires more resource than available. To conduct an
economic evaluation, it is necessary to know what resource are used in a project, and
what these resources cost. In some cases, the costs are not direct (they don‘t have to be
paid), but indirect (such as an opportunity cost, which is the cost of using a resource in a
given project that could be used elsewhere). This process involves measuring or
estimating the value of facilities, equipment, personnel, and other resources used.
Sometimes patient time commitment and travel cost are relevant, as well (Drummond,
1987). Adequately determining appropriate costs can be difficult, and should not be
undertaken without the help of someone familiar with economic analysis (Rossi, 1998).
Economic evaluation types:
 Cost analysis: the simplest form of economic evaluation is a cost analysis,
because it considers only the costs, however, it is a partial economic evaluation
(Drummond, 1987). To conduct a cost analysis the costs of a project must be
determined, making sure to collect all relevant costs for the perspective being
used (Haddix, 1996).
Once cost is determined, there are three common methods used for comparing
the costs and consequences of different interventions which are: cost
effectiveness, cost-utility and cost benefit analysis.
 Cost effectiveness analysis (CEA): it divides the net cost of a project by the
out-comes produced by the project. Since CEA involves in two metrics (cost
and effectiveness) one cannot obtain a single measure of social net benefits; one
can only compute the ratio of the two. This can be done in tow ways: cost per
unit of outcome effectiveness or outcome effectiveness per unit cost; for
instance;
CEi = Ci/Ei – average cost per unit of effectiveness – lower is better

12
 ECi = Ei/Ci – average effectiveness per unit cost – larger is better
As CEA computes the ratio of input to output (or vice versa), it is a measure of
technical efficiency and is not necessarily a good measure of allocate efficiency.
 Cost Utility Analysis (CUA): is form of economic analysis used to guide
procurement decisions. Its used to determine cost in terms of utilities, especially
quantity and quality of life. Its common specially in health center.
 Cost-Benefit Analysis (CBA): is also similar with CEA, except that it places a
monetary value on the outcomes of project.

2.5. OECD Project evaluation criteria

Currently most of projects are evaluated specially in developing countries are through the
five OECD/DAC evaluation criteria. Even though there are different criterion the five
evaluation criteria from the Development Assistance Committee of the Economic
Cooperation and Development (OECD/DAC) have been a strong foundation for project
evaluation since 1991 (Thomas, 2008). According to Australian Development Agency
(2009), OECD/DAC criterion is the most inclusive criterion because, it evaluates almost all
aspect of the project. The Organization for Economic Co-operation and Development‘s
(OECD) Development Assistance Committee (DAC) is a form to discuss issues surrounding,
development and poverty reduction in developing countries. Both OECD and Europe aid
project cycle management handbook list the following as the major criteria for project and
project evaluations:
Relevance: the extent to which the objective of a development intervention or the projects
are consistent with beneficiary requirements, country needs global priorities and partner and
donor policies. Questions and Dimensions of relevance:
 Assessing the relevance of a development intervention require to assess the extent to
which it responds to the needs and is also consistent with the priorities, livelihood
strategies and the culture of the main stakeholders and beneficiaries.
 It includes looking at the technical adequacy of the intervention, including in relation
to the institutional environment.
 Relevance also involves assessing consistency with other donors‘ interventions.

13
  To what extent does the project correspond with the most recent objective of the
partner country‘s development policy (Government: Poverty reduction Strategy Paper
(PRSP) or similar other relevant groups in case of conflict of interests, if applicable)?
 To what extent does the objective of the project in terms of development policy
correspond with the objective and directive of the Federal Ministry for Economic
Cooperation and Development (BMZ) (poverty reduction, Millennium Development
Goals MDGs)?
 To what extent does the basic orientation and conception regarding development
policy of the development intervention correspond with the most recent requirements,
standard of knowledge and framework conditions?

Effectiveness: the extent to which the development intervention‘s objectives were achieved
whether at output, outcome or impact levels, or are expected to be achieved, by taking into
account their relative importance.

Dimensions of Effectiveness:

 Effectiveness focuses on assessing whether the intended results of the development

intervention were achieved. Evaluating the effectiveness of an intervention involves
understanding how the project contributed to these results, whether it was appropriately
designed in relation to the expected results, what were the success factor, what was the
weight of the external factors, who were the beneficiaries of the intervention, etc. it
contributes to institutional learning by making you understand what worked and under
which conditions.
 Effectiveness aims at measuring the extent to which the objectives of the development
intervention are being achieved, whether at output, outcome or impact levels.
 Assessing the effectiveness of an intervention at output level requires to examine the
extent to which the project activities have taken place and produced the expected outputs.

Question to be addressed under effectiveness:

 To what extent were the objective achieved or likely to be achieved?

 What were the major factors influencing the achievement or non- achievement of the
objective?

14
Efficiency: a measure of how economically resources/ inputs (funds. Expertise, time etc) are
converted to results.

Dimensions of efficiency:

 Efficiency is a relation between resources allocated to the project and the results
achieved. The results are usually measured at output level, as outputs can easily be
observed and measured and are in the control of the development intervention.
 Efficiency involves assessing the extent to which the intervention produced the
intended with an economical use of resources. It also involves assessing whether the
same outputs could have been achieved with different and more economical use of
resource.
 Efficiency can also entail assessing whether outputs were efficient in achieving the
intended outcome (ultimate impact).

Questions raised while efficiency is evaluated includes:

 Were activities cost-efficient?

 Were objectives achieved at the least cost?
 Was the project or project implemented in the most efficient way compared to alternative
ways?

Consequently, Efficiency can be evaluated using Earned Value Analysis (EVA). Earned Value
Analysis is an industry standard method of measuring a project‘s progress at any given point in
time, forecasting its completion date and final cost, and analyzed variances in the schedule and
budget as the project proceeds. It compares the planned amount of work with what has actually
been completed to determine if the cost, schedule and work accomplished are progressing in
accordance with the plan. As work is completed its considered ―earned‖.

Key elements in EVA:

 Planned Value (PV): the approved cost baseline for the work package. It was earlier
known as Budgeted Cost of Work Schedule (BCWS).
 Earned Value (EV): the budgeted value of the completed work package. It used to be
known as Budgeted cost of work performance at a specific point (BCWP).

15
  Actual cost (AC): the actual cost incurred during the execution of the work package to a
specific point in time. It also previously called Actual cost of work performed (ACWP).

Sustainability: will the benefit last?

It is the continuation of benefits from the development intervention after major development
assistance has been completed. It looks at the longer-term effects of the development
intervention, it assesses the extent to which the effect of the development intervention will
continue after the withdrawal of the donor support.

Assessing sustainability allows evaluator to determine if an intervention benefits will last

financially, economically, socially and environmentally (Mansuri and Rao, 2013).

Sustainability focuses on the effect of the development intervention rather than on the
intervention itself. It may have different time frame apply when assessing sustainability which
depending on the intervention. There is different aspect of sustainability, including financial
sustainability, institutional sustainability, technological sustainability and many more which have
to be assessed when looking at the sustainability of an intervention.

Sustainability can be evaluated over different timeframes. Evaluator can assess for both the
actual sustainability which is the continuation of net benefits created by the intervention that are
already evident and prospective sustainability which is called the net benefits for key
stakeholders that are likely to continue into the future. Under such circumstance the evaluator
should carefully consider appropriate evaluation approaches to assess actual and prospective
sustainability, depending on the timing of the evaluation and the timescale of intended benefits.

Evaluating Actual Sustainability, the evaluator can examine the extent to which any positive
effect generated by the intervention demonstrably continued for key stakeholders including
intended beneficiaries, after the intervention has ended (Noltze and Verspohl, 2018).

Examining prospective sustainability, entails a slightly different approach which means that an
evaluation examining the future potential for sustainability would assess how likely it is that any
planned of current positive effects of the intervention will continue, usually assuming that the

16
current conditions hold. This evaluation will need to assess the stability and relative permanence
of any positive effect realized and conditions for their continuation.

2.6. Importance of Evaluation

Project evaluation plays an important role to the success of the project. It determines whether the
overall status of the project is acceptable in terms of the intended value to the client after the
project completed. Evaluation is a risk management tool which allows project manager to
reduced uncertainty when making decisions. (Chapman, 2005). It often generates a written report
that contribute for accountability and transparency and allow for to share lessons very easily.
Farbey (1992) summarized the importance of evaluation as:

 Evaluation will bring a lesson how to optimize for success and discover the story behind
the result.
 Evaluation could be used as part of justification for project.
 Evaluation enables organizations to compare between numbers of projects by providing a
set of measures which support monitoring and control system.

The purposes of project evaluation are to improve the quality of services, to ensure value for
money and to priorities proposed capital projects. This is achieved through a structured
process which makes it possible to:
• Clearly define project objectives, and consider a wide range of options to meet these
objectives;
• Link the project to the strategic objectives of the government, the State Capital Works
Program and an agency‘s physical asset strategic plan;
• Carry out economic, social, environmental and budgetary analyses of the project; and
• Identify the net benefit of the project to the community, and the effect on the State
Budget. Project evaluations assist departments to make decisions on proposed capital
projects. They provide the means to assess the viability of proposed capital projects, and
to rank competing projects in the department‘s annual capital works program. Project
evaluations also facilitate deliberations by the Cabinet Budget Committee during the

17
 Budget process. They assist in the selection of projects to be included in the State Capital
Works Program. (Queensland Treasury, February, 1997)

2.7. Project evaluation procedure

According to Steven (1993), project evaluation is a combination of a number of activities

ranging from setting indicators, developing model, defining measurable outcome, identifying key
stakeholders and their interests, selecting methodology for evaluation, collecting information,
analyzing data and disseminating evaluation result for further learning.

i. Source of evaluation information: a variety of information sources exist from which

to gather evaluative data. In a major project evaluation, it may be needed more than a
single source. The information source will select depend upon what is available and
what answers the evaluation questions most effectively. The most common source of
evaluative information falls into three categories: existing information, people and
pictorial record and observation.
Existing information: before start to collect data better to check to see what
information is already available
People: people are the most common source of information for an evaluation, they
provide information about the need for the project implementation and its outcome
(Ellen and Sara, 1996). Some of them are:
 Participant‘s beneficiaries: those who benefit directly or indirectly from the
project.
 Key informants – anyone who has particular knowledge about the project or
how it benefits participants.
 People with special expertise
 Project staff, admin, volunteers
 Collaborators; competitors
 Funders / donors
Pictorial record and observation: the third major source of evaluative information
is through visual accounts picture, photograph and video tape or a direct observation
of situation, behaviors, project activities and outcomes. Observation can provide
18
 information about real-life situation and circumstance that are useful in designing or
understanding what is happening in an extension projected why it is happening (Ellen
and Sara, 1996).
ii. Methods for collecting information about an evaluation: for some reason scientific
method have dominated in the field of evaluation for many years. These methods seek
to establish cause effect relationships, produce generalizable results and provide
quantitative data through structured data collection procedures.
according to Ellen and Sara (1996), there is no a single method to collect and analyze
project data which we have some list of categories; some of them are:
 Survey: collecting standardize information through structured questionnaire to
generate quantitative data. Survey maybe mailed completed on site or
administered through interview which conducted either face to face or
telephone electronically.
 Case study: an in-depth examination of a particular study, it can be a project, a
program, single individual, site or location. Case study rely on multiple
sources of information and methods to provide a complete picture as possible.
 Interview: collecting information by speaking with and listening to people.
 Observation: collecting information through seeing or listening. It can be
structured or unstructured.

2.8. An overview of Railway Industry

These days, railway transport becomes very popular across the globe. According to Chong
(2011), developing mass transport railway network as the backbone of transport systems has
been accepted as the right approach. This is because of the fact that, railway systems have the
advantage of being able to move masses of people efficiently, safely and in an environmentally-
friendly manner.

Railway becomes the most useful transport system in both developed and developing countries.
Railways have a low impact on the environment, particularly in comparison with other transport
modes and most notably, road. Overall, rail is one of the „greenest‟ ways there is to provide

19
mobility for goods and people. For instance, the total greenhouse gas (GHG) emissions in the EU
are described as: rail transport 0.9%, domestic navigation 2.2%, domestic aviation 2.5%, road
transport 93.4 % and other transport has only 1.0 % (www.railwaystrategies.co.uk/).

Furthermore, in Europe as European Commission (2013) described, the rail sector plays a
significant role in the economy of the European Union which made an economic benefit of the
annual turnover of EUR 73 billion and employing 800,000 people. To be a sustainable system,
Skilled and well-qualified train drivers are significant factor in the safety, interoperability and
competitiveness of railways. There are over 133,000 train drivers in the European Union at the
human interface with technical developments and cross-border operations.

United States of America has invested $148 billion in 2007 to keep the national network up to
acceptable service standards. Capacity improvements including railway construction are planned
at several levels of analysis because the demand of rail transport has been increasing from time
(Cambridge Systematic Inc, 2007).

Railway is not only widely spread in Europe and America; it has also been used widely in Asian
countries like: India, china, Japan and others for long times. Now a days, many African countries
including Ethiopia has been experiencing more.

2.8.1. Railway in Ethiopia

Ethiopia has experienced rail transport since the last ten decades. As the Ethio-Djibouti Railway
deteriorated from lack of maintenance, Ethiopia lost railroad access to the sea. The existing
meter gauge railway had been originally built by the French between 1894 and 1917.

Ethiopia is a mostly agricultural country with 84% of the population living in rural area. Since
road transport accounts for 90-95% of inter-urban freight and passenger movements, it is
considered essential to improve the road network in order to achieve the socio-economic
development and food security of the country. More than 95% of Ethiopia's trade passes through
Djibouti, accounting for 70% of the activity at the port of Djibouti. Because of these facts, the

20
country has demanded railway transport system so that to be able moving these goods and
commodities to and away the country.

In fact, both Ethiopia and Djibouti have agreed to construct the old gauge rail system into new
and modern railway system. Therefore, with common agreement, China was financing the
construction of a standard gauge railway network in East Africa, Ethiopia and Djibouti chose to
abandon the meter-gauge railway and build a new standard gauge link. In 2011, the Ethiopian
Railways Corporation awarded contracts to two Chinese state-owned companies for the
construction of a new standard gauge railway from Addis Ababa to the Djibouti border.

2.9. Empirical Literature review

Different project evaluation studies are conducted using OECD criteria‘s, for instance studies
conducted by Ninson (2018) on cosmetic and oil palm processor project on Ghana: all
respondent agreed that their project was Relevant. Approximately half, 21 out of 40, saw this
project as very beneficial, almost all 33 out of 40 respondents perceived the objective as fully
Effective while the remaining seven respondents believed they had been partially achieved. All
40 respondents agreed that much of the resource were contributed by the NGOs in the form of
building materials, expertise and finance.

According to Chianca (2008), the most widely used evaluation criteria after the 1990s has been
the OECD/DAC five project evaluation criteria (Relevance, Efficiency, Effectiveness, impact
and Sustainability). Several institution including African Development Bank, Asian
Development Bank, International Federation of Red cross and Red crescent society and many
more are either used this criterion or used as a criterion one of its element.

Welde (2016), also study the relevance, efficiency and effectiveness of Sandvika- Asker intercity
railway and in his study found that even though the project is relevant, like most of the projects
are it exposed to inefficient and ineffective.

The other study Conducted by Maxima Consulting (2018) on Leskovac Green Zone in Serbia,
The Green Zone is being developed as an Agroindustry. The Zone itself incorporates a 100

21
hectare site provided by the Government, as a warehousing and product distribution center. Their
evaluation also shows: ―The project shows overwhelmingly relevant to Serbia‘s needs and fit
within national priorities and donor policies‖, their effectiveness evaluation shows that, there are
missed outcomes and under-performance against expected results and it consume highest costly
resources (time/cost) possible in order to achieve the desired results.

These types of poor performance projects are common in developing countries like Africa
Lavagnon and Jan (2014). Even though the above study does not mention the reason behind poor
performance of projects, other studies list their findings in different areas.

The reasons behind project failures in different countries are various. For instance, a research had
done by Fidelis and Esther (2015) in Anambra State, South East Nigeria. Information collected
from sourced from a survey of one hundred (100) project professionals, with a minimum of 5
years of experience. Illustrate five most important causes of project ineffectiveness and
inefficiency: 1) Increase in the price of raw materials 2) Poor planning of Project Implementation
3) Variation of Project Scope 4) Award of Contract without reference to availability of funds
(corruption) 5) Political Pressure.

Other research done by Damoah (2015) on Ghana government project, data collected through
questionnaire surveys of 265 (contractors- 78, PMP- 81 and general public- 106) participants,
finding indicated that contractors, project management practitioners and general public agreed
that the top 10 causes of Ghanaian government project failure are poor monitoring, corruption,
political interference, change in government, bureaucracy, lack of continuity cooperation,
fluctuation of price, delays in payment and release of fund.

22
2.10. Conceptual Framework

As it describes above the comparison nature of project evaluation study the conceptual
framework also reflects how to illustrates the project base line which intended to achieve at the
end of the project and the actual performance of the project, and then the actual performance of
the project has been evaluated using the baseline as standard, therefore, the result of the
evaluation is used as a lesson learned practice for the ongoing and future projects.

Figure 1. Conceptual framework for OECD project evaluation in the case of Ethiopia- Djibouti
railway project.

Project Baseline Actual Performance

Cost
Schedule Cost
Quality Schedule
Beneficiaries‘ specification
Quality

Result planning Beneficiaries‘

expectations
Comparison
Relevance
Efficiency
Effectiveness
Sustainability

Evaluation Result

Lesson Learned

23
 CHAPTER THREE
Research Design and Methodology

3.1. Introduction

This chapter is discussing about the methodology and design of the undergoing research that has
been gathered, analyzed, and interpreted data to get relevant and reliable results. Under this
section the researcher is trying to explain the research design, target population, sample size
determination, research and data collection instrument, data analysis and ethical considerations.

3.2. Research Design and Approach

According to Yin (2009), any researcher can use diverse strategies in his/her research or more
than one design at a time that means different research designs may be employed both at a time,
one or two at a time for a single research program.

There are three types of research approaches Quantitative, qualitative and mixed method
approach (Leedy and Ormrod, 2005). Quantitative approach is used to answer questions about
relationships among measured variables with purpose of explaining, predicting and controlling
phenomenon. Whereas, Qualitative approach is used to answer questions about the complex
nature of phenomena and its purpose is describing and understanding the phenomena (Leedy and
Ormrod, 2005). To draw data from sources including individuals, groups and organizations, the
study was adopted mixed approach in which both qualitative and quantitative approach has been
employed.

Research Design is the general plan of how research question will be answered (Saunders, 2009).
There is no single way of conducting a research. Research design depends on many factors such
as research topics, audience of the research, time and resource availability and practical
considerations like access to people and information (Martelli & Greener, 2015). It is a master
plan to specify the methods and procedures for collecting and analyzing the needed information
(Adams et al, 2007). It describes the plan used in collection of information, so that in this
research the researcher was applied both descriptive and analytical method. Why descriptive
because to describe the state of affairs as it exists currently and try to attempt to describe

24
systematically a situation, problems, phenomenon, projects or provide information about the
current condition (Kothari, 2004). Therefore, by using the above stated methods and reasons try
to evaluate the Ethiopia- Djibouti railway project objective, budget, schedule and the actual
conditions with the intended plan. The result of the data has been analyzed to make a critical fact
or evaluation result.

3.3. Data type and source.

This study was employed both primary and secondary data. Primary data represents to data obtained
first hand by the researcher on the variable of interest for the specific purpose of study, while
secondary data is those collected from sources already existing in the concerned organizations or by
stakeholders of the project to be studied.

3.4. Sampling Technique and Sampling

3.4.1. Sampling Population

The population of the study are 64 employees of an Ethiopian railway corporation which engages in
management and supervision areas that works in the railway projects. The sampling frame of this
study was the management staffs who have worked directly in the Addis Ababa – Djibouti railway
project.

3.4.2. Sampling Techniques

The applied sampling technique for this population is purposive sampling where the respondents
were selected based on the criteria (Black, 2010; Saunders& Thornhill, 2012).

3.4.3. Sample Size

There are suggestions (Burns, 2000; May, 2001; Bryman, 2004) that sample size is not
necessarily the major consideration in designing the research method, as long as it fulfilled the
basic requirements. According to Bryman, (2004), the decision about sample size is not a straight
forward once as it depends on numbers of considerations: so, there is no definitive answer.
Voorhis and Morgan (2001) propose rules of thumb for sample size of multiple regressions is 50
to 300 samples which suggested that different statistical procedures require different numbers of
sample size. The pragmatic issues were based on Bryman, (2004) considerations that in most
cases, determining sample size is related to time and cost resources.

25
According to Roscoe (1975) proposes the sample size in multi-variety research (including
multiple regression analysis, the sample size should be several times (preferably 10 times or
more) as large as the number of variables in the study.
When the population is less than 1000, we can take a sample of 30% of the total population. If
the population is greater than 1000, a sample size of 10-20% can be a representative of the
population (Gay and Airasian, 2003).

The total population of this study are consisted around 64 management team members and
supervisors who have been working directly in the construction of the Addis Ababa – Djibouti
railway project.
Therefore, Sample size of respondents of this thesis research is total sampling population*30% =
64*0.3= 19 respondents were participated. This means, for the quantitative and qualitative data
collection of the study, a sample size of 19 members has been selected from a population of 64.
the sample size the researcher used in this research is believed that sufficient would be treated as
parametric. The limitedness of the sample size came from, one the target population in itself is
limited other they were difficult to be located.

3.5. Data Collection Method

No single source has a complete advantage over all others for this particular study. Hence, both
primary and secondary sources have been used. In order to perform relevance, effectiveness and
efficiency analysis, information about estimated and actual cost, planned and actual duration for
different project activities are necessary, current operational cost and actual income such like
information‘s being collected through document review of project plan documents and different
reports. Because this research is comparison in nature (it compares actual and progress of the
project with its intended plan) most of the data has been covered from secondary sources.

Again, in addition to secondary sources some other data was need to collect through primary
sources. The actual status and operational progress of the project in order sustain and give the
expected benefit for the countries has been collected through in the form of semi-structured
interview, when we say semi-structured interview is a technique which follows a framework in
order to address key themes rather than specific questions. At the same time, it allows a certain

26
degree of flexibility for the researcher to respond to the answer of the interviewee and develop
the themes and issues as they arise (MacDonald & Headlam). Therefore, the interview was held
with those who involved in the management and supervision areas which directly or indirectly
involved in the project work to make the information more valuable and reliable.

3.6. Method of Data Analysis

Under this thesis the researcher also used a quantitative approach of analysis technique, Earned
Value Analysis (EVA) is used as a systematic approach to the integration and measurement of
Cost, Schedule, and technical (Scope) accomplishment on a project. EVA technique is used to
show past performance of the project, current performance of the project and predict the future
performance of the project in statistical techniques.

Calculating Earned value: Earned value management measures progress against a baseline. It
involves calculating three values for each activity in the work breakdown structure (WBS):

 The Planned Value (PV): the portion of the approved cost estimate planned to be spent on
the given activity during a given time.
 The Actual Cost (AC): the total costs incurred in accomplishing work on the activity in a
given period. This actual cost must correspond to whatever was budgeted for Planned
Value and the Earned Value ( all labor, material, equipment, and indirect cost).
 The Earned value: the value of the work actually completed.

These three values combined to determine that point in time or not work is being accomplished
as planned. The most commonly used measures are cost variance;

Cost variance (CV) = EV – AC

And the schedule variance:

Schedule Variance (SV) = EV - PV

These two values converted to efficiency indicators to reflect the cost and schedule performance
of the project. The most commonly used cost-efficiency indicator is the cost performance index
(CPI). Its calculated that:

27
CPI = EV/ AC

The sum of all individual EV budgets divided by the sum of all individual ACs is known as the
cumulative CPI, and is generally used to forecast the cost to complete a project.

The schedule performance index (SPI) calculated:

SPI = EV/PV is often used with the CPI to forecast overall project completion estimates.

A negative schedule variance (SV) calculated at a given point in time means the project is behind
schedule. While a negative cost variance (CV) means the project is over budget.

Calculation of Forecasting Indexes

Forecasting of schedule and cost is done by calculating the forecasted values, which are
expressed as (Gapaldo & Volpe, 2010):

Estimation of Complete (ETC) expresses to forecast the expected cost required to complete all
the remaining works. Though the intended evaluation research will consider the budget at
completion (BAC), Earned Value (EV), and indicators of the performance of cost and plan.

ETC can be calculated

ETC = EAC – AC

Estimation at Completion (EAC) expresses the expected total cost requires to finish all the works
of the engineering projects.

EAC = BAC/ CPI

Variance at completion (VAC); expresses the variance of the total cost of the work and expected
cost. It can be expressed by:

VAC = AC + ETC - BAC

Meanwhile, Descriptive analysis technique is also used for those data collected from interview
and document review to analyze and interpret into meaning full information for better result.

28
 CHAPTER FOUR
Result and Discussions

4.1. Introduction

Under this chapter presents the result of the collected data from document analysis and the
relevant discussion through interview questions. The research aimed at evaluating the overall
performance the Addis Ababa – Djibouti railway project through OECD criterion to assess
and analyze the relevance of the project, how efficient were the project activity in terms of
cost and schedule, to examined the effectiveness of the project considering its objective and
how the project will sustain for a while with its benefits.

Most of the data has been collected from secondary sources basically from project documents
and other relevant journals and reports.

The semi-structured interview questions have been addressed and interviewed for 8 out of 19
target populations that is for project managers, coordinators and site supervisors.

4.2. Respondent’s profile

Under this section try to provide the general information about the respondents in terms of
their education, work experience and position of the project.

Table 4.1 Characteristics of Respondents

Respondents Education Work experience Position

R1 Civil Engineer 8 yrs Project Manager
R2 Civil Engineer 9 yrs Project Manager
R3 Mechanical Engineer 5 yrs Project coordinator
R4 Civil Engineer 7 yrs Site Manager
R5 Management 5 yrs HR coordinator
R6 Civil Engineer 6 yrs Project coordinator
R7 Project Management 4 yrs Project coordinator
R8 Economics 6 yrs Project coordinator

29
 4.3. OECD Evaluation Results of Addis Ababa–Djibouti railway
project

4.3.1. Relevance of the project

Relevance is the first OECD criteria of evaluation. It evaluates the purpose, overall goal,
meaningfulness and the extent to which its objectives are consistent with recipient‘s needs.
Relevance analysis is very important, because if the project does not help to address present
needs or problems, then it does not matter how effective, efficient or coherent it is – it is no
longer appropriate (―European commission publication‖, n.d).

The researcher evaluated the purpose and importance of the project by referring to the intended
plan of the project. Thus, most of the data are collected from the project's plan and reports. First,
when relevance of the project is evaluated based on the major promises of the project, and the
intended to achieve, it will be evaluated the extent to which the objectives of the project are
consistent with local community and country needs, and global priorities.

According to Australia Development Agency (ADA, 2009) guidelines for project and program
evaluations, it suggests to evaluate the objective of the project in terms of its correspondence
with the most recent objective of the country‘s development plans and Sustainable Development
Goals (SDGs), therefore from the country‘s development plan and priority angle, the Growth and
Transformation Plan (GTP) is selected as a reference to evaluate either the project is relevant,
because GTP is the grand plan of the country that locate where the country is going. Also,
Sustainable Development Goals (SDGs) are used as reference to develop the relevance of the
project. Therefore, by rising OECD questions used for evaluating relevance of the project going
to examine the importance of Addis Ababa- Djibouti railway project:

30
Logical Framework Approach to ensure the relevance of the project

The LFA is a widely adopted methodology used by most multilateral and bilateral agencies
working in development or technical cooperation. (The Technical Cooperation Strategy, 2002)
according to TCS, the LFA helps stakeholders to think through and analyze the ‗logic‘ of a
project in a systematic and structured way, first by conducting a detailed analysis of a number of
elements, and second by relating the results of these analyses to each other and to the project‘s
overall objective. This ensures a sound project proposal and a high-quality project. The LFA
provides a project structure in which major components are explicitly and clearly interrelated,
and interrelationships are clarified. The LFA plays a particularly critical role in project planning
and design, but it can also be used throughout the project cycle, including implementation,
monitoring and evaluation (USAID, 2011).
Applying the LFA in any of the project plan as a tool has several additional advantages. A key
advantage is that it creates a dialogue amongst the project team, helping to clarify their roles
during implementation, as well as how they can ensure project sustainability and maximize
results (USAID, 2011). This dialogue also establishes and expands ownership of the project.
Another critically important advantage is that applying the LFA clarifies both the project scope
and what it can realistically achieve. This supports a better understanding of how the project will
complement other projects with the same or similar aims. A good project design will anticipate
possible constraints during the project implementation phase and will thus contribute to smoother
implementation.

The LFA can be used in a flexible manner according to the context and scope of the project.
While it is commonly used in major complex projects —for example, it is possible to organize a
3–4-day project design workshop for all project stakeholders using the LFA to gain a common
understanding of, and common agreement on, all aspects of the project —it can also be used in
small groups or even by an individual team member thinking through the project‘s logic.

31
Relevance of the project against the local beneficiaries (OECD 2000):

Under the initial analysis of the project reflecting that the Ethiopia –Djibouti railway project is
realized by the changes brought out by it in the transport sector of the economy. A high
proportion of freight traffic from road is diverted to railway transportation. Reduction in numbers
of truck on road reduces the annual road maintenance. The vehicular pollution gets reduced with
this project as the rail will be run on electricity and there is less use of gasoline and diesel. Thus,
there will also be high amount of fuel savings and saving in foreign exchange. This can help to
regenerate the countries‘ economies in to economic catalyst, compared with road transportation
especially fright the railway would provide less expensive means of transportation for Ethiopia
and will augmented the country‘s overall transport capacity. This can express that an investment
of Addis Ababa – Djibouti railway line could maintain the superiority between the costs of
transport by truck to the tune of USD 42.80 per tonn to the cost of USD 15.30 to USD 35.60. As
per the report of the United Nations Conference on Trade and Development (UNCTAD, 2003),
the cost of transportation represented on average 13.8% of the value of imports in Sub-Saharan
Africa, and it goes up to as high as 20.7% for landlocked countries like Ethiopia. Thus the
reduction in transportation costs is therefore necessary to reduce the import price of goods in
Ethiopia.

Consequently, under such mega project benefit comes in many ways such as major savings could
be realized through road maintenance and rehabilitation works, let‘s say that 100,000 heavy
trucks weighing 30 tonnes each causes road surface damage equivalent to 240,000 cars
(Cabanius, 2003) requiring an investment to the tune of USD 26 million to rehabilitate 100 km of
road in Ethiopia. Further, rail requires full rehabilitation every 15 to 20 years, compared to 7 to
10 years for roads. The rail link would also reduce the volume of petrol imports for freight
carriers and limit carbon dioxide emission levels. This could be reduced up to 75% for a
railroad‘s energy consumption and 85% carbon footprint could compared to that for highway
(Pozzo di Borgo, 2011)

Benefits for local beneficiaries are also come in terms of employment generation for unskilled
labor during construction period of LRT (2011 -2015) as well as to the skilled and unskilled

32
professionals during the operation phase. The freight and fare box revenues are the financial
benefits from this railway project. The benefits accrue to the local economy due to operation of
this rail projects are many.

Thus, as per the initial assessment made by TAG (2007) advised that developing transportation
are links the factors of production in a complex web of relationships between producers and
consumers. The outcome is commonly a more efficient division of production by the exploitation
of geographical comparative advantages, as well as the means to develop economies of scale and
scope. Nation‘s economic growth is increasingly linked with transport developments, Therefore,
it‘s clear that planned project objectives are very useful, relevant and realistic to the situation on
the ground. The project has manifold advantage for the local beneficiaries.

To what extent does the project comply with development policy and planning of the
country? (OECD, 2000)

Infrastructure is a crucial component of the government‘s long-term development strategy, and

an integral part of Ethiopia‘s Growth and Transformation Plans (GTP I and II), which aim to
structurally transform the economy from an agricultural base to an industrial and manufacturing
powerhouse over five-year intervals. Railways are one part of this industrial strategy. By
lowering the economic and transaction costs of logistics, Ethiopia seeks to facilitate industrial
zone development and its export and manufacturing sectors, in turn promoting the long-term
structural transformation of the economy towards higher technology, value-added sectors. Rail
investment is also connected to the concept of ―Transit-Oriented Development‖, where urban
and cross-national railways raise land values and stimulate the development of industrial and
commercial corridors.

This goal of the project also attains one of the major development objectives of the Government
in GTP II is reducing poverty and generating employment for the expanding labor force. Other
this project also contributes for achievement of Growth and Transformation Plan II special focus
of infrastructure development; large scale energy, transport and telecommunication infrastructure
development programs and social sector development; increase national potable water supply
coverage, expansion of the education service coverage and make essential health services

33
accessible to all citizens. (GTP II, 2015/16). As the data‘s show the project is in line with
development policy and planning of the country. It‘s may be because of the project is intended
by the government and they make it compatible with their national plans of the country.

Developing railway projects for country like Ethiopia that has a relatively high social and
economic growth rate in Africa. Since introducing of the new millennium, Ethiopia has
witnessed a stable development of national economy and achieved double-digit growth of GDP
for consecutive years. However, Ethiopian economy is based on agriculture, and its industry is
still at an early development stage. Due to lack of basic production bases, Ethiopia‘s social and
economic development mostly depends on the international trade. As an inland country, its
international trade has to rely on ports of other countries. Ethiopia‘s import/export goods are
mainly transported through Port of Djibouti. Regarding the goods imported via Port of Djibouti,
most of them are transported to Ethiopian central regions. The feasibility study done by CIECC
states that the freight traffic volume of these regions‘ accounts for 75% of the total, and the
freight traffic volume of the north area and east area accounts for 15% and 10% respectively.
Therefore, construction of Addis Ababa-Djibouti Port railway corridor is an important transport
infrastructure project for Ethiopia. Construction of the Project allows Ethiopia to build a short
and convenient international transport corridor to the sea and so it is of great significance for
sustainable social and economic development of this country.

The extent to which, the construction of a project is consistent with global Environmental
priorities?

There was a meter-gauge railway connected Addis Ababa to the Port of Djibouti, which is jointly
owned by Ethiopia and Djibouti and run by Chamin de Fer Djibouti- Ethiopia (CDE). It was
completed in 1910 and has 37 stations in total. Due to subgrade settlement, rail deformation, and
rails blocked by highway in some sections, this railway cannot be put into operation. Although
there is a European company performing partial repair for this railway but this seems ineffective.
As this railway is nearly abandoned, goods transport between Addis Ababa and Djibouti Port
mostly depends on the highway of which the construction standards are low. Along the railway
line, the climate is dry and the eco-environment is fragile. Highway transport has such

34
disadvantages as low transport capacity, high transport cost, and large energy consumption, and
in addition its exhaust emission results in environmental pollution.

High technical standards for standard-gauge railway have been applied to the Project. Compared
with expressway transport, railway transport is a ―green‖ high-capacity transport mode that is
fast and convenient, land-saving, energy-saving, environmental-friendly, and safe. The land
occupied by the project just roughly accounts for 1/2 of that occupied by highway project; if
electric traction is adopted, the energy consumption will only accounts for 1/4-1/6 of that of
highway project and also the project reduces the volume of petrol imports for freight carriers and
limit carbon dioxide emission levels. This could be reduced up to 75% railroad‘s energy
consumption and 85% carbon footprint could compared to that for highway. Therefore, through
construction of the Project, a green transport corridor of Ethiopia is comes into being, and the
birth of the project is imminent in terms of environment.

4.3.2. Efficiency Analysis of the Addis Ababa – Djibouti railway project

Efficiency is the other OECD evaluation criteria; it has been defined by OECD/DAC as the
determination of whether project use ―the least costly resources possible in order to achieve the
desired results‖ (OECD, 1992, p. 1). From the beginning of a project and throughout all its
stages, different stakeholders of the project have to address many questions. The most common
questions are those who deal with the time schedule and the projected cost of the project. For
instance, are we ahead or behind schedule? How efficiently are we using time? When will
likely finish the project? Are we under or over budget? How efficiently are we using our
resources? How efficiently must we use our remaining resources? How much is the project likely
to cost? Will we be under or over budget at the end of the project? (PMI, 2005). EVM is a very
powerful tool which is able to address the above questions and because it integrates cost, time
and the work done (or scope) and can be used to forecast future performance and completion
dates and costs, and better answer OECD questions related to efficiency, based on that let‘s
evaluate the efficiency of the Addis Ababa – Djibouti railway project as the researcher cited in
chapter three to using those formulas.

35
Table 4.2: Results of Earned Value Analysis

BCWS ACWP BCWP

Main Investment Planned Actual %Progress Earned
Value Cost Value
Railway construction $4,737.00 $4,140.00 100% $4,737.00

Power Supply $62.20 $70.87 100% $62.20

Land Requestion and
$29.50 $35.60 100% $29.50
demolition

Rolling Stock $292.00 $342.00 100% $292.00

Social Environmental $7.60 $6.40 95% $7.22

Grand Total $5,128.30 $4,594.87 $5,127.92

Source: Field data Jun 2021

Cost Variance (CV = EV - AC) $533.05

Cost Performance Index (CPI =
EV/AC) 1.12
Schedule Variance (SV = EV - PV) -$0.38
Schedule Performance Index (SPI =
EV/PV) 1.00

Source: Field data Jun 2021

BAC $5,128.30

PV $5,128.30

AC $4,594.87
EV $5,127.92

Source: Field data Jun 2021

36
 1) Cost Variance (CV) and Cost Performance Index (CPI)

Cost Variance (CV) It is a very important factor to measure project performance. It indicates
how much over budget or under budget the project is. If the project cost variance is positive
project is under budgeted, therefore, the cumulative cost variance of the Addis Ababa – Djibouti
railway project indicated that the project completed at $533.05 under budgeted which are less
spend compared to budgeted cost means that the project had been constructed efficiently in
utilization of the budget.

The cost performance index (CPI) it indicates an index showing the efficiency of the
utilization of the resources on the project. If the spending less on the work performed than was
budgeted, the CPI will be greater than 1. If not, and spending more than was budgeted for the
work performed, then the CPI will be less than 1. The cumulative Cost Performance Index (CPI)
of the Addis Ababa – Djibouti railway project is 1.12, this tells us the project were completed
with efficient utilization of the resource.

2) Schedule Variance & Schedule Performance Index

The schedule variance (SV); indicates how much ahead or behind schedule the project is. In
this regard the SV of the Addis Ababa – Djibouti railway project is -0.38, based on the result the
project was slightly behind the schedule which need to be taken as a lesson for the other similar
projects.

The schedule performance index (SPI); is a measure of how close the project is to performing
work as it was actually scheduled. If we are ahead of schedule, EV will be greater than PV, and
therefore, the SPI will be greater than 1. Obviously, this is desirable. On the other hand, an SPI
below 1 would indicate that the work performed was less than the work scheduled. Which
indicated bad move of the project. So now the SPI of the Addis Ababa – Djibouti railway project
is 1 this means that the project work completed by the project team was about a head of the
planned schedule this results a positive impact in the completion of the project and it leads
efficient utilization of the budget and project resource.

37
 4.3.3. Effectiveness of the Addis Ababa – Djibouti railway project

Evaluation effectiveness is central to project development and its evaluation. The OECD (2002)
Glossary of Terms defines development effectiveness as ―the extent to which a given project
objectives were achieved, or are expected to be achieved, taking into account their relative
importance.‖ It evaluates to what extents are the reason for project existence immediate
objectives is (most likely) achieved? And it‘s the target group reached? When the effectiveness
of the project executed, it follows the appropriate means of verification for tracking progress,
performance and achievement of indicator values from the intended plan and compare with the
current status of the project.

The extent to which the project objectives as defined are achieved? (OECD, 1992)

The immediate objective of the Addis |Ababa – Djibouti railway project planned in 2007 was:

Ethiopia became a landlocked country following Eritrea‘s independence in 1993, which

constrained Ethiopia‘s trade. Consequently, in the early 2000s, the Port of Djibouti became
Ethiopia‘s main port and a gateway for over 90% of its international trade.

Investments in road and rail along the Djibouti corridor had the potential to significantly reduce
transport costs and time. They were a key element encouraging greater interest from investors in
developing manufacturing export capabilities in Ethiopia.

In 2007, a TAG was established under the Ethiopian Ministry of Transport to define a framework
for the development of a railway corridor. This included a pre-feasibility study on socio-
economic and macro-economic benefits, and a detailed corridor analysis. The TAG
recommended the introduction of a railway system throughout Ethiopia as the primary national
mass transportation system. The study emphasized the importance of modernization and
expansion of the existing 1 m gauge (1,000 mm) railway to a standard gauge (1,435 mm) line to
provide faster access to the Port of Djibouti from inland Ethiopia. In the same year, the Council
of Ministers established the ERC by regulation No. 141/2007 to develop the railway
infrastructure and provide freight and passenger transport services in Ethiopia.

38
In June 2010, the Transport Ministers of both countries signed an MoU in Djibouti on the
Development and Operation of a Standard Gauge Railway Line between Ethiopia and Djibouti.
The Ethiopian Government subsequently adopted its five-year GTP to achieve economic
structural transformation and sustainable accelerated growth towards Ethiopia‘s longer-term
vision of being a middle-income country by 2020–2023. The GTP included the development of
dry ports, rail and road networks, and air transport. It aimed to develop the standard gauge
railway line for Ethiopia and Djibouti and a 34 km light rail system for Addis Ababa as priority
projects.

Since 2010, Ethiopia has been striving to build an extensive rail network in line with the GTP.
Railway transport services are regulated through a series of bilateral agreements to:

 guarantee Ethiopia access to the sea (transit transport service)

 outline management of the railway (as well as a minimum volume guarantee by Ethiopia)
 specify rates as freely negotiated between shipper and carrier
 deal with customs arrangements for the Port of Djibouti.

In 2011, the ERC awarded an EPC (engineering, procurement and construction) contract for the
railway line from Addis Ababa to the Port of Djibouti to two Chinese state-owned companies,
CREC and CCECC.

In 2012, the governments of Ethiopia and Djibouti signed a bilateral agreement for the
development and operation of the standard gauge network. In 2016, the two governments agreed
on the development, operation and management of the railway network. ERC and Djibouti´s
Minister of Equipment and Transport signed commercial contracts with CREC and CCECC
respectively. In the same year CREC and CCECC formed a consortium to operate the entire
railway line for six years.

In October 2016 in Ethiopia and in January 2017 in Djibouti, the passenger railway services were
opened. The official commercial operation commenced in January 2018.

39
Current profile of the project:

The Addis Ababa–Djibouti Railway runs roughly parallel to the old meter-gauge Ethio–Djibouti
Railway for most of its length. However, the standard-gauge railway is built on a new, straighter
right-of-way that allows for much higher speeds. New stations have been built outside city
centers, and most of the old railway stations have been decommissioned. There are 68 viaducts
and bridges, comprising 3% of the railway's total length.

The total length of the railway line is 759 km of which 754 km are running between the two
terminal stations at Sebeta and the Port of Doraleh and the remaining five kilometers are for
shunting operations. A total of 666 km of the railway line is in Ethiopia, while a total of 93 km is
in Djibouti. The 115 km section from Sebeta to Adama is the only double-track section of the
line, and it also has the highest grades with a net elevation loss of 650 meters. It features several
viaducts with lengths of up to 800 meters. The remainder of the railway is single-track, with
passing loops distributed evenly along its length.

The railway begins at Sebeta, just outside of Ethiopia's capital of Addis Ababa. The city is
served by two stations in its southern outskirts, at Furi-Lebu and Indode. The line then runs
southeast to Modjo and Adama, At Modjo, a railway junction exists for the planned Modjo–
Hawassa Railway. In addition, at Modjo the railway is connected to the Modjo Dry Port,
Ethiopia's most important inland dry port and also Ethiopia's main hub for domestic and
international freight services.

There are 21 dedicated railway stations along the railway, all of them can serve as passing loop
stations, as they have three tracks or more (except the Adigala station which has only two
tracks). Four of the 21 railway stations are designed as passing loops only, so there is no freight
loading / unloading or passenger service. Two of the remaining 17 stations are freight yards only
and two others for passengers only. The remaining 13 stations can handle both passenger
services as well as freight loading / unloading.

The 15 passenger stations usually have a single boarding platform, with a station building
attached to it. The platforms are about 200, 300, or 400 meters long. The Awash station, the only
one with three platforms, is also located along the railway but also at the junction point with the

40
Awash–Hara Gebeya Railway. The Furi-Labu and Dewale stations have two platforms. All
station buildings along the line contain facilities for ticketing and refreshment, and they even
have prayer rooms. The architecture of the station buildings (except that of Awash station) is
eclectic, featuring traditional Ethiopian elements with some Chinese interpretation.

The operators of the railway consider an annual freight tonnage that is far below the railway
limits. As a rule of thumb, the operators foresee an annual freight tonnage increase of about one
million tonne per year, starting at 1–2 million tonnes in the first year of operations.

Although road traffic in Ethiopia drives on the right, trains drive on the left in the double-track
sections. This is consistent with Chinese railway practice. The single-track sections are equipped
with passing loops, each of which is triple-track so that two trains can wait for the main line to
clear. Each station also serves as a passing loop.

The railway line is almost fully electrified. Power is transmitted at 230 kV and 130 kV to eight
substations. Traction power is supplied at 35.8 km intervals, with 18+1 stations in Ethiopia and
three in Djibouti. General electrification ends after the Djibouti–Nagad passenger station. Trains
are pulled by diesel locomotives to reach the Port of Doraleh and cargo terminals at inland dry
ports. This is necessary to avoid interference between the overhead catenary and loading cranes.

An official trial service began on the railway after inauguration in October 2016. On 9 May
2017, the first thorough stress test was conducted, with more than 30 trains on the line at the
same time. Passenger trains ran between Furi-Labu and Dire Dawa, while freight services ran
between the Port of Doraleh and the Modjo Dry Port railway station and the railway finally
began commercial operations on 1 January 2018.

Financial Effectiveness

The total cost of the project was USD5.17 billion at completion.

The Governments of Ethiopia and Djibouti altogether financed 30% of the project and currently
own the railway assets. The other 70% of the project cost was financed through concessional

41
loans from China‘s Exim Bank (Exim-bank), the China Development Bank, and the Industrial
and Commercial Bank of China. These loans were supported by market capitalization of nearly
USD3.3 billion. The Governments of Ethiopia and Djibouti have both purchased credit guarantee
insurance for their loans.

Financing for both construction and the power transmission lines were provided by China‘s
Exim-bank in two separate packages. Project construction was financed through a US$ 2.49
billion commercial loan. The Ethiopian government, notably, divided the construction contract
into two sections, from Addis Ababa to Mieso, and from Mieso to Dewele at the border with
Djibouti, in order to foster competition and faster completion. As contractor for the LRT, CREC
was a competitive choice for the Addis-Mieso section, with the idea that the railway would
eventually be linked to the urban LRT; meanwhile CCECC won the eastern section from Mieso
to the Djiboutian border. Contracts were signed at the end of 2011 and construction began in
2012. After winning the Ethiopian contract, CCECC promptly crossed over the border to lobby
the government of Djibouti for the Djiboutian segment of the contract as well, and facilitated
finance from China‘s Exim-bank. A separate loan package for rolling stock procurement was
arranged with the China North Industries Group (Norinco), who later signed a contract with
Ethiopia‘s METEC to assemble wagons domestically.

Considering the move and care-fullness of the government and other stakeholder having loan
from Chinese banks under commercial loan are relatively advantageous than other sources, the
major advantage to Chinese loans has been financial flexibility in the post-construction phase.
One ERC respondent noted the Chinese were more ―flexible‖ and ―willing to support you‖. In
this case, the Ethiopian government has struggled to repay external debts due to the ongoing
shortage of foreign reserves. Poor export performance and years of internal instability left the
country with dollar reserves shortages. The shortage has also constrained railway-related
expenditures on spare parts, locomotives, and management fees for the railway‘s O&M. On the
Exim-bank loans itself, after the expiration of the grace period, Ethiopia has reportedly struggled
to repay the interest, let alone principal, on the commercial loans. On this, China appears to have
been remarkably lenient. Ethiopia was able to default on its loan repayments to China for one
year, mutually agreed to and with no penalty. Additionally, in late 2018 after high-level bilateral
talks and via the FOCAC platform, the original SGR loan terms were renegotiated from a 10 to

42
20-year tenor, or repayment period. On the part of the contractors, the SOEs have had to swallow
their costs and the optics of the Belt and Road demand that the projects continue to run, even if
the Ethiopian state cannot pay for them.

Another major problem with the financial part of the project is associated with lower traffic
volumes than predicted in the transport forecast and currency exchange rate fluctuations – as the
project‘s debt was structured in US dollars, while construction and operation cost as well as
revenues were granted in Ethiopian Birr.

In response to some repayment risks, the Chinese banks restructured the Ethiopian debt and
extended the repayment period from 15 to 30 years.

Effectiveness in Management perspective

New railway technology can foster inter-firm spillovers by encouraging industrial clustering, but
also has the potential to generate a broader supply chain in services, retail, and equipment and
parts manufacturing. The presence of foreign or transnationals firms can be a source of
knowledge spill-over essential for this process. Most directly, the operation and maintenance of
new imported railway technology also creates demand for skilled labor and necessitates large-
scale development of human capital—what is sometimes known as the skill-enhancing trade
hypothesis. For the operation of new railway projects, a key technology transfer is of the skills
and ―know-how‖ needed to operate and maintain assets. Construction contractors may provide
technical, but also managerial skill transfers. A new railway system requires technical staff,
including drivers and engineers, but also entails a parallel ―soft‖ infrastructure in the form of
management structures to administrate, regulate, and maintain new assets.

Under project supervision and control functions on behalf of the Governments of Ethiopia and
Djibouti are performed by a Joint Railway Commission of the two state-owned companies ERC
and SDCF.

CREC and CCECC are to operate and maintain the railway for six years from the launch of
operation and will provide two additional years of technical support during the gradual handover
of management duties to EDR from 2024.

43
The Bilateral Agreement signed on 16 December 2016 between the two governments established
EDR, with shareholdings divided between Ethiopia (75%) and Djibouti (25%). Its main purpose
is to manage the Addis Ababa–Djibouti Railway (including the maintenance and renewal of all
infrastructure and equipment) and to operate freight and passenger transport services on the line
from 2024.

4.3.4. Sustainability of the Addis Ababa – Djibouti project

Assessing sustainability allows evaluators to determine if an intervention‘s benefits will last

financially, economically, socially and environmentally. Sustainability should be considered at
each point of the results chain and the project cycle of an intervention. Evaluators should also
reflect on sustainability in relation to resilience and adaptation in dynamic and complex
environments. This includes the sustainability of inputs (financial or otherwise) after the end of
the intervention and the sustainability of impacts in the broader context of the intervention
(OECD, 2013).

Evaluating sustainability provides valuable insight into the continuation or likely continuation of
an intervention‘s net benefits in the medium to longer term, If these various aspects of
sustainability are carefully considered by an evaluation, it can lead to important insights into how
interventions can plan and implement for change that ensures sustainable development in the
future.

Economic sustainability

The new railway is expected to bring considerable advantages for long-haul transport of freight,
including massively reduced travel times, from up to 50 hours down to 10 hours. On the cost
side, the temporary passengers and freight tariffs adopted by the two governments are very
competitive compared to road transport (USD0.017 per passenger-km, an import rate of
USD0.046 per ton-km and an export rate of USD0.023 per ton-km,) giving the railway a
competitive cost advantage.

44
The reduced transport costs and delivery times are forecast to increase trade volumes between
the two countries. The estimated market share of rail transport is set at 75%. However, meeting
the traffic forecasts does depend to an extent on the overall expansion of the Port of Djibouti.
The new line currently has a capacity of 11.2 million tons of freight, rising to 24.9 million tons
by 2025. The ERC anticipates the line will carry approximately 4 million tons of cargo by 2035,
growing from an expected 2.3 million tons in 2025 and 3.1 million tons in 2030. To achieve
these figures, the line requires an expansion of the handling capacity at the Port of Doraleh (an
extension of the Port of Djibouti for handling oil, bulk cargo and containers with an annual
turnover capacity of 8.2 million tons), with the aim of reaching 10 million tons by 2022.

The new railway line boosts the performance of the international trade corridor and significantly
contributes to strengthened economic ties between the two countries. For Ethiopia, it helps
accomplish its strategic goal of sustainable and stable economic development towards a middle-
income country, with an annual GDP growth of 8.3% in 2019 and the target of an average GDP
growth of 11% annually. For Djibouti, the better and more competitive railway service supports
port and transit cargo operations, which are essential sources of income and employment for the
country. The annual GDP growth for Djibouti is 5.9%, driven by public investment in rail and
port infrastructure. In 2019, the service sector, of which transport forms the largest component,
accounted for 76% of GDP.

Social sustainability

The line, designed as Ethiopia‘s main transport corridor, strengthens development opportunities
for rural communities. New railway stations built on the outskirts of the urban settlements
require, however, further investments to connect them to city centers and other interchange
nodes by means of public transport. A strategic Transport Master Plan for Addis Ababa,
currently under preparation, addresses the issues of integrated transport and urban development.

The transfer of knowledge from the project development and first years of operation will benefit
job creation in the local communities.

45
Management and Technical sustainability

As part of the investment propositions, the two engaged Chinese enterprises committed
themselves to training local personnel to operate and maintain the railway. More than 300
employees of the ERC were sent to technical universities and schools in Beijing, Tianjin and
Chengdu to further their professional knowledge of railway engineering, train driving and track
maintenance. In addition, they provided regular training to promote compatibility of the working
culture between both parties. Upon completion of the project, approximately 2,000 local workers
were hired for infrastructure and rolling stock maintenance

46
 Chapter Five
Summery, Conclusion and Recommendations

The primary aim of this thesis is to assess and evaluate the Addis Ababa – Djibouti grand
railway project under OECD criterion. This chapter seeks to situate the findings from chapter
four in to the context of the aim and objectives, which represent the original motivation of the
study. Accordingly, the following section presents concluding statements and then makes
recommendations.

5.1. Summary of Findings

After thoroughly analyzing the information gathered through document analysis and interview
the following major findings are presented. The general objective of this study is to evaluate
different aspects of the project. In such a case the researcher has taken four OECD criteria‘s
being selected; Relevance, Effectiveness, Efficiency and Sustainability of the project being
evaluated.
The project is Relevant, because the objectives are consistent with beneficiaries‘ requirements,
country needs, and global priorities. It‘s completed at budget and time, and meet the intended
objective with the extended expected benefits. This Project is the most important international
railway transport corridor of Ethiopia and the most important inland corridor of collection,
distribution and transport for Djibouti Port. Construction of the Project was a great significance
for improvement of the transport system in Ethiopia and Djibouti and promotion of the social
and economic development of the two countries. Therefore, construction of the Project is
imminent.

From the interview and secondary data analysis found that the Addis Ababa–Djibouti line has
provided visible socio- and macro-economic benefits. Export and import in Ethiopia and
Djibouti were clearly improved, increasing economic performance in both countries. The
transformative aspect of railways in Ethiopia and Djibouti played an important role in land
development. The project has fostered trade between the two countries by boosting the

47
performance of the international trade corridor, although there are still improvements to the Ports
of Djibouti and Doraleh that need to be completed to fully realize the project‘s benefits. For
Ethiopia, the railway helps accomplish the country‘s strategic goal of sustainable and stable
economic development towards a middle-income country.

The political relations between the two countries were strengthened through the signed bilateral
agreements and competitive tariffs for passengers and freight adopted by the two governments.
The new line connects rural regions and helps reduce regional disparities. Local communities
have been provided with development opportunities. The transfer of knowledge from the project
development and first five years of operation will benefit job creation in the local communities.

Integration of the rail line with the existing freight and port networks appears to be low and is
likely influencing the low usage of the line. This have been improved through better planning at
the inception of the project to address inter-country coordination. The project fostered
technology transfer, local personnel have been trained and the companies agreed to hand back
the operations and maintenance by 2023. Local employees from the ERC have been sent to
technical universities and schools in Beijing, Tianjin and Chengdu in order to enhance their
professional knowledge of railway maintenance and operation.

5.2.Conclusion

The Project are meet the requirements of construction of National Railway Network of Ethiopia,
and it provide a fast and convenient international transport corridor to the sea for Ethiopia and
meanwhile accelerate sustainable social and economic development of Ethiopia. Through
construction and completion of the Project, a green transport corridor of Ethiopia has come into
being, and the railway line being complementary to the current highway transport upon
completion of the Project. In addition, the Project has also improved the inland transport system
of Djibouti Port and facilitate development of the port service industry and national economy of
Djibouti. Therefore, construction of the Project was vital.

48
 5.3.Recommendations

The researcher learned that during the construction of the Addis Ababa- Djibouti railway project
lacks usage of any local resources that could be contributed to the local economy through the
acquisition of local materials for the construction of the project. According to world Bank report
(2012) local sourcing encourages local manufacturing and service activities that could have
promote local‘s employment. In other way such practice could have close the balance of trade
between the two nations in the global trade. Therefore, in order to promote such balance the
government should set a cap on which kind of material imported and which are use from local
resource.

Another issue that arose during the construction of the Addis Ababa-Djibouti Railway was
related to land grabbing and locals‘ complaints for unjust compensations. As some point the
Chinese companies in charge of the construction of the project argue that the Ethiopian and
Djiboutian authorities were in charge of carrying out land valuations. Consequently, after the
completion of the project both nation governments faced some kind of security issue on the
project materials. This suggests the need for local governments to listen to the demands of local
communities and adequately compensate them in the event of land loss will guarantee the long
last benefit of such kind of mega project.

Another main issue during the construction of the project was local labor treatment issue. There
was many allegations of unfair wages and treatment of managers in the Addis Ababa – Djibouti
railway project. In such regards, the local authorities should set guidelines to ensure Chinese
companies favor the employment of local workers and contribute to job creation in the country.
This might contribute some kind of insurance for the workers and work under appropriate
salaries and conditions.

49
Reference

Abednego O.G. (2015)..Development of A Project Management Evaluation Model for the

Construction Industry in Kenya. Unpublished PhD dissertation, Jomo Kenyatta University of
Agriculture and Technology, Kenya.

Ahmed, S.M., Azhar, S., Kappagntula, P. and Gollapudil, D. (2003), ―Delays in construction: a
brief study of the Florida construction industry‖, Proceedings of the 39th Annual ASC
Conference, Clemson University, Clemson, SC, pp. 257-66.

Ahsan, K. & Gunawan I. (2010) Analysis of cost and schedule performance of international
developmental projects. International Journal of Project Management, Vol. 28, No. 1; pp. 68–78

Ayman Ahmed & Ezzat Othman, (2013) ―Challenges of mega construction projects in
developing countries‖ research paper of The British University in Egypt (BUE)

Belout, A., Gauvreau, C., (2004). Factors influencing project success: the impact of human
resource management. International Journal of Project Management, 1–11.

Bryman, A. (2004). Social Research Methods.2nd ed..Oxford: Oxford University Press.

Bryman, A. & Bell, E. (2003). Business Research Methods. Oxford: Oxford University Press.

Bourgault, M., Drouin, N., & Hamel, E. (2008). Decision making within distributed project
teams: An exploration of formalization and autonomy as determinants of success. PMI Project
Management Journal, 39, Supplement, S97-S110.

Cambridge Systematic, Inc. (2007). National Rail Freight Infrastructure Capacity and Investment
Study. Washington DC: Association of American Railroads, 2007.

Chen, H-t. (2004). Practical Program Evaluation: Assessing and Improving Planning,

50
Implementation, and Effectiveness. Thousand Oaks, CA: Sage.

Chew Tai, C. (2011). Sustainable Railway Development through Careful Planning, Design and
Implementation, honk Kong.

Creswell, J. W. (2013). Research Design: Qualitative, Quantitative, and Mixed Methods

Approaches. United States of America: SAGE Publications.

Dehar M, Casswell S, and Duignan P. (1993). Formative and Process Evaluation of Health
Promotion and Disease Prevention Programs. Evaluation Review 17 #2: 204-220,

Drummond MF, et al. (1987) Methods for the Economic Evaluation of Health Care Programs.
Oxford University Press, Vol 15. pp 0-19.

Ethiopian Railway Corporation. (2014) Progress reports on the execution of project

Ethiopian Railway Corporation. (2018) Final Actual Report on Project

European Commission. (2013a): The Fourth Railway Package – Completing the single European
railway area to foster competitiveness, and growth, COM (2013) 25 final, Brussels.

Farbey B., Land F. & Targett D. (1992) Evaluating investments in IT. Journal of Information
Technology, 7 (2), 109 -122.

Final Feasibility Study Report (2013), CIECC

Fisher RJ and Peters L. (1982) the Role of Evaluability Assessment in Mental Health Program
Evaluation. Canadian Journal of Community Mental Health, 1,34.

51
Flyvbjerg, B., Bruzelius, N. and Rothengatter, W. (2003). Megaprojects and Risk: An Anatomy
of Ambition. Cambridge: Cambridge University Press.

Fortmann SP, William PT, Hulley SB, Maccoby N, Farquhar JW. Does Dietary Health
Education Reach Only the Privileged? Circulation 661: 77-82, 1982.

Fugar, F. D. K. & Agyakwah‐Baah, A. B. (2010) Delays in building construction projects in

Ghana. Australasian Journal of Construction Economics and Building, Vol.10, No.(1/2); pp.103-
116

Growth and Transformation Plan (GTP) 2010/11-2014/15. Ministry of Finance and Economic
Development 2010, Ethiopia.

Gyimah Boadi, E. (2002) Confronting corruption in Ghana and Africa, briefing paper: Ghana
Centre for Democratic Development (CDD‐Ghana), Vol.4, No.2, pp. 16.

Hair, J., Money, A., Samuel, P and Page, M. (2007). Research methods for business Chichester,
England: John Willey & Sons, Ltd

https://www.un.org

J. Price Gittinger. (1972). Economic Analysis of Agricultural Projects, Baltimore: The Johns
Hopkins University Press, p. 1

Kothari, C. R. (2004), Research Methodology: Methods and Techniques, (Second Edition), New
Age International Publishers.

52
Kazaz, A., Ulubeyli, S. and Tuncbilekli, N.A. (2012).Causes of delays in construction projects in
Turkey.Journal of Civil Engineering and Management, 18(3): 426–435.

Kerzner, H.(2009). Project Management: A systems Approach to Planning, Scheduling, and

Controlling. (10thedn). Hoboken, NJ: John Wiley.

Kerzner, H. (2013). Project management: A systems approach to planning, scheduling and

controlling. New York: Wiley & Blackwell.

Leedy, P.D. and Ormrod, J.E. (2005). Practical Research: Planning and Design.8th Ed. Pearson
Educational International.

Mok, K. Y.,Shen, G. Q. & Yang, J. (2015) Stakeholder management studies in mega

construction projects: A review and future directions. International Journal of Project
Management, Vol.33, No.2; pp.446-457

Morten Welde. (2016). Ex post evaluation of transport projects – experiences from Norway
Norwegian University of Science and Technology; Concept Research Programme. Trondheim;
Norway

National Institute on Drug Abuse. (1991). Handbook for Prevention Evaluation. Department of
Health and Human Services. Thousand Oaks: Sage publisher.

Ninson. (2018). Evaluating Community Development Projects Using the OECD/DAC

Evaluation Criteria. Journal of Multi-Disciplinary Evaluation. Vol. 14.

Organization for Economic Cooperation and Development (OECD). (1998). Review of the DAC
principles of development assistance. Paris: DAC Working Party on Aid Evaluation.

Patton, M.Q. (1997). Utilization-focused Evaluation. CA, Thousand Oaks: Sage publisher.

53
PMBOK Guide.3rd ed. Project Management Institute, Inc., (2008). A Guide to the Project
Management Body of Knowledge. Newtown Square, PA, USA: Project

Management Institute (PMI).

Ministry of Finance and Economic Development of Ethiopia.(2008).Guidelines for monitoring
and evaluation of public sector projects. Addis Ababa, Ethiopia.

Mutijwaa, P., &Rwelamila, D. (2007). No Title Project Management Competence in Public

Sector Infrastructure Organization. Construction Management and Economics.

Naomi, B, and Giorgio, L. (2015).A Megaproject Research Framework. University of Leeds,

Published by the University of Leeds, April 2015 ISBN 978-0-9576805-7-9

PMI. (2005). A Guide to the Project Management Body of Knowledge. 5th ed. USA: Project
Management Institute, Inc.

Rossi PH, Freema HE. Programs, Policies, and Evaluations. Evaluation: A Systematic Approach.
Newbury Park: Sage publisher.

Sanders, & Cunningham, D. J (1974). Techniques and procedures for formative evaluation. In G.
D. Borich (Ed.), Evaluating educational programs and products. Englewood Cliffs, N. J:
Educational Technology.

Saunders, M., Lewis, P. and Thornhill, A. (2009). Research Methods for Business Students. 5th
Edition, Prentice Hall, Harlow.

Smith M. Evaluability Assessment: Reflections on the Process. Evaluation and Program

Planning 13: 359-364, 1990.

Stuart MacDonald & Nicola Headlam. (n.d). Research Methods Handbook: Introductory Guide
to Research Methods for Social Research. Manchester: The Centre for Local Economic

54
Strategies (CLES).

Samy Tayie (2005) Research Methods and Writing Research Proposals, Path Way to Higher
Education Project, PP 19, 31,35 & 38.

Thomaz Chianca. (2008). The OECD/DAC Criteria for International Development Evaluations:
An Assessment and Ideas for Improvement. Journal of Multidisciplinary Evaluation, Volume 5,
Number 9.

Tessmer M. (1984). Formative Evaluation Alternatives. Beverly Hills: Sage publisher.

Wholey JS (1994). Assessing the Feasibility and Likely Usefulness of Evaluation. Handbook of
Practical Program Evaluation: 15-39.

Wideman, R. M. (2002). Wideman comparative glossary of common project management terms

v3.1.Retrieved from http://www.maxwideman.com/pmglossary/index.htm.

Zhang, Y., & Zhang, L. (2003). Study on Reasons for Delays in Civil Engineering Project in
China. In Conference Proceeding, Sustainability and Innovation in Management and
Technology, 10¬12 2003.

55
APPENDEX

Interview Guideline.

Dear Respondent,

My name is Eskinder Adinew, I am a post graduate student at St. Mary‘s University at the
department of Project Management. I am conducting Evaluation research about Addis Ababa –
Djibouti railway project using OECD Criteria. You are here by kindly requested to answer the
questions listed below sincerely.

The data collected from the interview and the result of the research will be used strictly for an
academic purpose and will be kept confidential. Thank you in advance for taking your time to
answer the questions.

Kindly introduce your role/position in this project……………………………

Interview questions;

 To what extent does the project comply with development policy and planning of the
nation?
 In what extent were the project objective and implementation strategies consistent with
national and sub-regional needs?
 To what extent were the project implementation mechanisms outlined in the project
document effective in delivering the project outputs and outcomes?
 Did the project apply anytime or cost saving mechanisms in order to achieve results
within the approved timeframe and budget?
 Were the project objective and components clearly practicable and feasible within its
time frame?
 How successful was the project in achieving its planned output in terms quality,
sequencing, timeliness and usefulness?

56
  Are there any social or political factors that influence positively or negatively the
sustenance of the project result and impact?

To what extent is the continuity of project result dependent on continued financial and technical
support?

57