Gibi
Gibi
Gibi
June ,2018
Addis Ababa, Ethiopia
1
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Declaration
I, the undersigned, declare that this research thesis is my original work and has not been
presented for a degree in any university, and that all source of materials used for the thesis
have been duly acknowledged.
Declared by:
Signature: _______________________
Date: ____________________________
Confirmed by Advisor:
Signature: ___________________________
Date: _______________________________
2
Assessment of Monitoring and Evaluation Practice and Challenges 2018
This is to certify that this thesis prepared by Asnat Mulugeta, entitled: Assessment of
Practice and Challenges of Monitoring and Evaluation: The Case of Local NGOs
Executing Health Projects and submitted in partial fulfilment of the requirements for the
Master of Business Administration (MBA) Degree complies with the regulation of the
University and meets the accepted standards with respect to originality and quality.
3
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Acknowledgements
My deepest gratitude goes out to my advisor, Ethiopia Legesse Segaro (D. Sc), for her
advice, guidance and constructive feedback in undertaking this study. I wish to acknowledge
the management of The Kaizen company and my colleagues for their active support, and
encouragement. Similarly, I would like to express my heartfelt thanks to subjects of the study
who gave me invaluable information about their respective local nongovernmental
organizations monitoring and evaluation practice and challenge. Finally, I would like to thank
all my family and friends for their care, support and invaluable advice in all my walks of life.
i
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Table of Content
Contents Page
Acknowledgements ................................................................................................................................ i
List of tables.......................................................................................................................................... iv
List of figures ......................................................................................................................................... v
Acronym ............................................................................................................................................... vi
Abstract................................................................................................................................................ vii
CHAPTER ONE ................................................................................................................................... 1
1. Introduction ....................................................................................................................................... 1
1.1 Background of the Study ............................................................................................................ 1
1.2 Statement of the Problem ........................................................................................................... 3
1.3 Research Question ...................................................................................................................... 4
1.4 Objectives of the Study ............................................................................................................... 4
1.4.1 General Objective ................................................................................................................ 4
1.4.2 Specific Objective ................................................................................................................ 5
1.5 Scope of the Study ....................................................................................................................... 5
1.5.1 Geographical and Respondent Scope ................................................................................. 5
1.5.2 Content Scope ....................................................................................................................... 5
1.6 Limitation of the Study ............................................................................................................... 5
1.7 Significance of the Study ............................................................................................................ 5
1.8 Operational Definition ................................................................................................................ 6
1.9 Organization of the Study .......................................................................................................... 6
CHAPTER TWO .................................................................................................................................. 7
2. REVIEW OF RELATED LITERATURE ...................................................................................... 7
2.1 Monitoring and Evaluation ........................................................................................................ 7
2.1.1 Monitoring ............................................................................................................................ 7
2.1.2 Evaluation ........................................................................................................................... 10
2.2 Collecting and Analysing M&E data ...................................................................................... 11
2.3 M&E System.............................................................................................................................. 11
2.4 Monitoring and Evaluation Framework ................................................................................. 13
2.4.1. Laying the Foundation for M&E Framework................................................................ 13
2.4.2 Types of Monitoring and Evaluation Frameworks ......................................................... 15
2.5 Overview of NGOs .................................................................................................................... 16
2.5.1. International NGOS .......................................................................................................... 17
2.5.2 Ethiopian local NGOs ........................................................................................................ 17
ii
Assessment of Monitoring and Evaluation Practice and Challenges 2018
2.6 Code of Conduct and Regulatory Framework For Ngos In Ethiopia .................................. 18
2.7 The challenges of NGOs ........................................................................................................... 19
2.7.1 External Challenges ........................................................................................................... 20
2.7.2 Internal Challenges ............................................................................................................ 20
2.8 Conceptual Model ..................................................................................................................... 24
CHAPTER THREE ............................................................................................................................ 25
RESEARCH METHODOLOGY ...................................................................................................... 25
3.1 Introduction ............................................................................................................................... 25
3.2 Research Design ........................................................................................................................ 25
3.3 The Target Population .............................................................................................................. 25
3.4 Sampling Strategy ..................................................................................................................... 25
3.5 Data Collection Tools and Techniques .................................................................................... 26
3.6 Validity and Reliability............................................................................................................. 27
3.7 Data Presentation and Analysis ............................................................................................... 27
3.8. Ethical Issues ............................................................................................................................ 27
CHAPTER FOUR............................................................................................................................... 28
DATA PRESENTATION, ANALYSIS AND INTERPRETATION.............................................. 28
4.1 Introduction ............................................................................................................................... 28
4.2 Questionnaire Response rate.................................................................................................... 28
4.3: Presentation, analysis and interpretation of data ................................................................. 28
4.3.1: Background information .................................................................................................. 28
4.3.2: Employees knowledge status regarding M&E ............................................................... 30
4.3.3 Monitoring and evaluation practice ................................................................................. 33
4.3.4 Challenges Local NGOs face in executing M&E ............................................................. 42
4.3.5 Coping Mechanism ............................................................................................................ 46
CHAPTER FIVE ................................................................................................................................ 48
SUMMARY, CONCLUSION AND RECOMMENDATION ......................................................... 48
5.1 Introduction ............................................................................................................................... 48
5.2 Summary of findings................................................................................................................. 48
5.3 Conclusion ................................................................................................................................. 50
5.4. Recommendation...................................................................................................................... 51
5.5 Suggestions for Further research ............................................................................................ 51
Annex ............................................................................................................................................... 58
iii
Assessment of Monitoring and Evaluation Practice and Challenges 2018
List of tables
iv
Assessment of Monitoring and Evaluation Practice and Challenges 2018
List of figures
v
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Acronym
AAU Addis Ababa University
vi
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Abstract
Nowadays, NGOs increasingly play a prominent role in the development sector by filling
gaps the developing world face. The study emphasizes on the assessment of practice and
challenges of monitoring and evaluation in local NGO’s within Addis Ababa, Ethiopia. i.e.
case of local NGOs working on health projects under the USAID local capacity development
project. The purpose of this research is to asses this practice and challenges. The population
of this research is the 288 local NGOs operating within Addis Ababa, Ethiopia while the
target population is 50 NGOs that are registered and operational under the USAID local
capacity development program implementing health projects. A questioner is distributed to
all 50 local NGOs. Finally, 34 respondents filled in and returned the questioner properly.
This research employs descriptive research design for acquisition of quantitative data. The
data is analysed using SPSS and interpreted in percentage, and frequency. The findings of
this study reveal that: a large majority of the subjects (94.1%) confirm encountered
challenges such as policy/legal framework, inadequate baseline data, lack of fund and
deficiency of expertise to monitor and evaluate projects effectively. They adopt mechanisms
such as introducing participatory M&E approach, relocating budget for M&E and limiting
M&E activities to mitigate the challenge. This study in general shows that although local
NGOs have good M&E practice they also face numerous challenges when implementing
M&E. The implications of the study and relevant recommendations is forwarded in this study.
vii
Assessment of Monitoring and Evaluation Practice and Challenges 2018
viii
Assessment of Monitoring and Evaluation Practice and Challenges 2018
CHAPTER ONE
Introduction
This study examined the monitoring and evaluation practice and challenges of local NGOs
implementing health projects in Addis Ababa, Ethiopia.
This chapter outlines the background to the study, statement of the problem, general
objectives, specific objectives, research questions, conceptual framework, significance of the
study, justification of the study, scope of the study.
Since the mid-2000s, monitoring and evaluation has taken on a far greater role in
international development. The aid effectiveness agenda has brought about a major change in
development agencies‘ motivation to focus on results and impact, and to provide evidence of
their effectiveness. In order to respond to this move, monitoring and evaluation has been
given much more prominence in many organisations. This in turn has led to a greater
understanding of the challenges faced when attempting to collect and access the right data
that improves the work outputs, at the same time as demonstrating accountability to both
donors and beneficiaries.
1
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Ethiopia has been one of the major recipients of international aid in recent times. According
to OECD-DAC statistics, net Official Development Assistance (ODA) to Ethiopian amounted
to US$3.26billion in 2012‘, making it the 5th largest recipient among 169 aid receiving
developing countries. This is the official aid channelled through bilateral and multilateral
relationships with international donors and agencies. In addition to this, there is substantial
amount of money remitted through unofficial channels through NGOs, which commonly
referred to as channel Three (OECD annual report 2012).
In accordance to the newly enacted federal charities and societies proclamation 621/2009 an
important premise of allowing development partners‘ to channel development assistance
through NGOs/CSOs is the organizations ability to manage funds efficiently and effectively,
and to deliver and document results. As a result, in order to meet this expectations, the
proclamation demands these organizations to systems for monitoring and evaluation.
During the past 15 years, NGOs have been increasingly pressured by all types of funders to
demonstrate their effectiveness and document their programs outcomes as the current
political and funding environment continues to stress the importance of accountability and
measuring performance (Walker & Grossman, 1999; Salamon, 1999). Donors are demanding
more formal accountability requirements from NGOs to ensure that their donations are being
used to benefit society.
Studies of local NGOs performance gave rise to increasing skeptism about their assumed
comparative advantage. However, fragmented evaluation and progress reports show that
short-term project objectives are achieved with excellent positive result but with limited
sustainable change. Some of the major reasons that put issue of sustainability in question are
highly related to monitoring and evaluation such as: human capacity, data quality and
analysis, lack of clarity about the precise objective of projects and beneficiaries‘ involvement.
The new legislation on Charities and Societies, Proclamation No. 621/2009 promulgated in
February 13, 2009 has introduced new challenges to most of the NGOs operating in the
country and thus, having apparent potential in affecting SCS and its local partners operation
in Ethiopia. It hence, appears significant for the organization to examine its programmes and
working methods in view of the new law and identify the implications thereof to make
2
Assessment of Monitoring and Evaluation Practice and Challenges 2018
appropriate decisions and devise well calculated ways to aptly cope within the current legal
framework.
The local capacity development program is a 5-year program funded by USAID with the aim
of creating an increased and evolving pool of capable Local Implementing Partners (who are
legally registered local NGOs and private companies) by strengthening management capacity
to be fully compliant with USAID‘s requirements and regulations. The program is being
implemented by the Kaizen company who resides in United states of America as a social
enterprise. At the beginning of the project 28 local NGOs joined the program and received
support on the areas of financial management, program management, monitoring and
evaluation and sustainability and leadership via trainings, technical assistances and peer to
peer experience sharing. Then after the 2 years in to the project, 60 other local NGOs and
private organization joined the program. The program is now on the beginning of its 4th year,
with 88 organizations composing different sectors i.e. 50 Health, 11 Education, 9 Democracy
& Governance, 8 Economic Growth, 4 Environment, 6 agriculture. (Source: Local
Implementing Partners Orientation Program presentation, 11/7/2017). Therefore, the aim of
this study is to assess monitoring and evaluation practice and challenges of local
nongovernmental organizations that operate health programs in Addis Ababa.
3
Assessment of Monitoring and Evaluation Practice and Challenges 2018
According to the Organization for Economic Cooperation and Development (OECD) 2018
annual forum, major donors' aids to developing countries fell by nearly 5% in 2017 thus
breaking a long upward trend since 1997 due to the global recession. It is predicted that
continuing tight budgets in OECD countries will put pressures on aid levels over the coming
years.
Due to this fall in international funds, international donors are all demanding more formal
accountability requirements to ensure that their funds are being used to benefit society and
meet population needs (Andrew et al., 2009). According to the World Bank (2010), having
M&E polices provides an environment in which aid is highly effective and produces very
high results. Leeuw (2001) emphasized that M&E is seen as a critical component of more
effective aid and the need for it has accelerated to the extent that it has been described as a
growth industry and a public good (Leeuw, 2001).
Therefore, it is important to ask what the existing practice and challenges looks like in these
local NGOs and also asses the possible coping mechanisms that are being used by these
NGOs to ensure their transparency and accountability and enhance their project performance.
4
Assessment of Monitoring and Evaluation Practice and Challenges 2018
5
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Challenge: A challenge is something new and difficult which requires great effort
and determination to overcome.
6
Assessment of Monitoring and Evaluation Practice and Challenges 2018
CHAPTER TWO
This chapter presents the related literatures on the study to have an insight in to the research
topic and briefly expose the readers to some of the major areas of the subject matter under
consideration. The chapter is presented under the following sections:
Monitoring and evaluation serve several purposes. In the absence of effective monitoring and
evaluation, it would be difficult to know whether the intended results are being achieved as
planned, what corrective action may be needed to ensure delivery of the intended results, and
whether initiatives are making positive contributions towards human development.
Monitoring, as well as evaluation, provides opportunities at regular predetermined points to
validate the logic of a programme, its activities and their implementation and to make
adjustments as needed. Good planning and designs alone do not ensure results. Progress
towards achieving results needs to be monitored. Equally, no amount of good monitoring
alone will correct poor programme designs, plans and results. Information from monitoring
needs to be used to encourage improvements or reinforce plans. Information from systematic
monitoring also provides critical input to evaluation. It is very difficult to evaluate a
programme that is not well designed and that does not systematically monitor its progress.
(UNDP, 2006)
2.1.1 Monitoring
Monitoring can be defined as a continuing function that aims primarily to provide the
management and main stakeholders of an ongoing intervention with early indications of
progress, or lack thereof, in the achievement of results. An ongoing intervention might be a
project, programme or other kind of support to an outcome. (UNDP,2002)
7
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Monitoring is the day-to-day management task of collecting and reviewing information that
reveals how an operation is proceeding and what aspects of it, if any, need correcting.
Monitoring is a continuing function that uses the systematic collection of data on specified
indicators to inform management and the main stakeholders of an ongoing International
Federation or national society operation of the extent of progress and achievement of results
in the use of allocated funds. (IFRC:2002)
Reporting is an integral part of monitoring. Monitoring information is
Compiled in standard and ad hoc reports;
Shared with implementing partners, donors and beneficiaries
Used to draw conclusions in evaluations
Computerised systems for monitoring offer opportunities for the following: efficient data
storage, flexibility and speed of analysis, cross-comparisons, trend analysis, and preparation
of simple graphs. However, before deciding on what computer programme to use you should
check the following:
Do existing manual systems work efficiently? If yes, then computerisation may not be
an immediate concern.
Will data be collected extensively for a significant period of time, and be analysed
quantitatively? If yes, then computerisation is likely to offer considerable efficiency
gains. What is the best programme or software to use? This will depend on the staff
skills, equipment and funds available, the type of data required, and the type of
analysis planned. Relatively simple computerised systems using Microsoft Excel or
Access exist and information on existence, strengths and weaknesses of such systems
can be accessed.
Whatever system is chosen, the organization should ensure detailed plans for computerisation
should be prepared as part of the monitoring and evaluation system design, to ensure that the
necessary physical and financial resources are provided for and ensure provision for back up
to the system in case of computer breakdown. In addition, skilled staff will be required to
operate and maintain the system, and to undertake the necessary analysis. (UNDP:2002)
8
Assessment of Monitoring and Evaluation Practice and Challenges 2018
9
Assessment of Monitoring and Evaluation Practice and Challenges 2018
2.1.2 Evaluation
Evaluation is the systematic and objective assessment of an on-going or completed operation,
programme or policy, its design, implementation and results. The aim is to determine the
relevance and fulfilment of objectives, as well as efficiency, effectiveness, Impact (overall
Goal) and sustainability. An evaluation should provide information that is credible and
useful, enabling the incorporation of lessons into management decision-making. (IFRC,
2002).
Types of Evaluation
2. Mid-term evaluation: This type of evaluation takes place while the implementation of
the planned project is on-progress. Such evaluations are conducted relatively early in the mid-
way of the project life and are usually external assessments. What distinguishes it from
terminal and ex-post evaluations is that correction to the current project still can be made on
the basis of findings and recommendations (EMI, 2014).
10
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Monitoring and evaluation data collection methods could generate better results if they are
simple, clear, short and focused. Hence appropriate methods have to be identified and used
based on the extent and the type of information expected.
11
Assessment of Monitoring and Evaluation Practice and Challenges 2018
bilateral aid agencies also regularly apply M&E to measure development effectiveness as
well as demonstrate transparency (Briceno, 2010).
Monitoring and Evaluation is a combination of two processes which are different yet
complementary (Gorgensand Kusek, 2009). It is therefore a process of systematically
collecting and analyzing information of ongoing project and comparison of the project
outcome/impact against the project intentions (Hunter, 2009). An M&E system, on the other
hand is a set of components which are related to each other within a structure and serve a
common purpose of tracking the implementation and results of a project (SAMDI, 2007). It is
therefore an integrated system of reflection and communication that support project
implementation. An M&E system is made up of four interlinked sections, which are: setting
up of the M&E system, implementation of the M&E system, involvement of the project
stakeholders, and communication of the M&E results (Guijt et al., 2002). Theoretically, ‗an
ideal M&E system should be independent enough to be externally credible and socially
legitimate, but not so independent to lose its relevance‘ (Briceno, 2010). It should therefore
be able to influence policy making from recommendations of lessons learned as well as be
sustainable overtime for it to be responsive to the needs of the stakeholders.
12
Assessment of Monitoring and Evaluation Practice and Challenges 2018
A theory of change defines the pieces and steps necessary to bring about a given long-term
goal. A theory of change describes the types of interventions (whether a single programme or
a comprehensive community initiative) that bring about the results hoped for. A theory of
change includes the assumptions (often supported by research) that stakeholders use to
explain the process of change.
13
Assessment of Monitoring and Evaluation Practice and Challenges 2018
A theory of change:
• demonstrates the pathway of how to get from here to there (i.e. what is needed for goals to
be achieved)
• requires underlying assumptions to be detailed out in a way that they can be tested and
measured
• puts the emphasis first on what the organization wants to achieve rather than on what the
organization is doing
Source: Adapted from Theory of Change by Act Knowledge (http://theoryofchange.org)
Agreed among the key stakeholders at the end of the planning stage, is essential in order to
carry out monitoring and evaluation systematically. This framework serves as a plan for
monitoring and evaluation, and should clarify:
What is to be monitored and evaluated
The activities needed to monitor and evaluate
Who is responsible for monitoring and evaluation activities
When monitoring and evaluation activities are planned (timing)
How monitoring and evaluation are carried out (methods)
What resources are required and where they are committed
14
Assessment of Monitoring and Evaluation Practice and Challenges 2018
In addition, relevant risks and assumptions in carrying out planned monitoring and evaluation
activities should be seriously considered, anticipated and included in the M&E framework.
(USAID,2012)
2.4.2 Types of Monitoring and Evaluation Frameworks
Though there is no ideal framework and different frameworks are used for different
situations, three of the most common are conceptual frameworks, results frameworks and
logical frameworks/logic models. (Frankel and Gage, 2007)
1. Conceptual framework
Conceptual frameworks are diagrams that identify and illustrate relationships among relevant
organizational, individual and other factors that may influence a programme and the
successful achievement of goals and objectives. They help determine which factors will
influence the programme and outline how each of these factors (underlying, cultural,
economic socio-political etc.) might relate to and affect the outcomes. They do not form the
basis for monitoring and evaluation activities, but can help explain programme results.
(Frankel and Gage, 2007)
2. Results Frameworks
Results frameworks sometimes called strategic frameworks illustrate the direct relationships
between the intermediate results of activities all the way to the overall objectives and goals.
They show the causal relationship between programme objectives and outline how each of
the intermediate results/ outputs and outcomes relates to and facilitate the achievement of
each objective, and how objectives relate to each other and the ultimate goal. Results
frameworks do form the basis for monitoring and evaluation activities at the objective level.
(Frankel and Gage, 2007)
3. Logical Frameworks
15
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Nongovernmental organizations are hard to define due to the inconsistent use of the term.
Non-profit organizations and private voluntary organizations are the types of organization
that are labelled as a nongovernmental organization, although they do not fit squarely within
this label. The United Nations defined the nongovernmental organization as one that does not
form part of a government and is also not a conventional profit business. Some
nongovernmental organizations are funded by governments and maintain their
nongovernmental status by not allowing government representatives to be members of the
16
Assessment of Monitoring and Evaluation Practice and Challenges 2018
There are approximately about 3,056 re-registered civil society organizations of which 2,650
are local and the remaining 406 are international organizations operating in different parts of
the country (ChSa, 2014). Likewise the Addis Ababa city government in the same period host
about 700 NGOs of which 224 are local ones that signed formal project operational
agreement with the respective bureaus of the city government (AABoFED, 2014).. The
international NGOs vary widely in their interest in and in skill at constructing mutually
beneficial partnerships with local counterparts. CARE, Catholic Relief Services, World
Vision, and Save the Children are United States–based examples of the larger international
relief and development groups carrying out programs in the country. Many are increasingly
forging partnerships with various national NGOs and supporting efforts to increase the
institutional capacity of these partner groups. (Jeffery,2000)
International NGOs trace their Ethiopian roots to the catastrophic famine crises of 1973–74
and 1984–85. The NGOs of those years were overwhelmingly focused on emergency relief
operations and were largely foreign entities. Local church-affiliated agencies also played a
very significant role in these operations. NGOs were instrumental in preventing even greater
loss of life during both catastrophic episodes for various reasons. During the initial famine of
1973–74, various groups engaged in relief operations formed what became known as CRDA
(Christian Relief and Development Association), the first NGO umbrella organization in
Ethiopia. CRDA was organized by a coalition of Catholic charities, other religious affiliates,
and a few outside, secular NGOs. Its formation also marked the first organized cooperation
between the government (that of Haile Selassie) and the NGO sector in the country.
(Jeffery,2007)
However, this cooperation was not well established. For the most part, NGOs that formed or
surfaced immediately after the Derg overthrow were ill prepared to have much impact. With
few resources, untrained staff, and limited exposure to the non-profit world, many
17
Assessment of Monitoring and Evaluation Practice and Challenges 2018
A clear indicator of a more sophisticated carriage on the part of the NGO community in
Ethiopia is provided by the adoption of the Code of conduct for NGOs at the culmination of a
collaborative effort on the part of diverse leaders of the sector. The code is meant as a
proactive statement of principles by the sector and serves as a symbol that it is capable of
self-regulation, monitoring, and evaluation (Jeffrey, 2007).
The code of conduct for NGOs in Ethiopia was formally adopted in March 1999, when the
overwhelming majority of NGOs operating in the country swore to uphold its principles and
its formation is considered one of the major achievements for the sector since the onset of the
contemporary era for NGOs in 1991. On January 6, 2009, the Charities and Societies
18
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Proclamation No. 621/2009 of Ethiopia was enacted and defines two categories of formal
CSOs in Ethiopia: Charities and Societies. (Chasa,2011)
Charities are institutions established exclusively for charitable purposes and provide public
benefit. Societies, on the other hand, are associations or persons organized on a non-profit
making and voluntary basis for the promotion of the rights and interests of their members and
to undertake other similar lawful purposes as well as to coordinate with institutions of similar
objectives. Charities and Societies are given one of three legal designations, Ethiopian
Charities or Societies, Ethiopian Resident Charities or Societies or Foreign Charities, based
on where the organization was established, its source of income, composition of
membership, and membership residential status (Chasa, 2014).
Ethiopian Charities or Societies are institutions formed under the laws of Ethiopia, whose
members are all Ethiopians, generate income from Ethiopia and are wholly controlled by
Ethiopians. These organizations may not use foreign funds to cover more than 10% of their
operational expenses. Similar institutions that receive more than 10% of their resources from
foreign sources or whose members include Ethiopian residents are designated
Ethiopian Resident Charities or Societies. Foreign Charities, on the other hand, are those
formed under the laws of foreign countries, or whose membership includes foreigners, or
foreigners control the organization, or the organization receives funds from foreign sources
(Chasa, 2011).
The provisions of the Proclamation are applicable to charities or Societies that operate in
more than one regional state or Societies whose members are from more than one regional
state; foreign Charities and Ethiopian Resident Charities and Societies even if they operate
only in one regional state; and, charities or Societies operating in the City Administration of
Addis Ababa or Dire-Dawa. (Chasa, 2011).
The Ethiopian civil society, especially the NGOs sector has been engulfed with various
external and internal problems for a long time. The challenges may be categorized into
two broad parts; external and internal. This thesis will look at the challenges in relation to
the NGOs monitoring and evaluation.
19
Assessment of Monitoring and Evaluation Practice and Challenges 2018
20
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Frequent data collection means more data points; more data points enable managers to track
trends and understand intervention dynamics hence the more often measurements are taken,
the less guess work there will be regarding what happened between specific measurement
intervals. But, the more time that passes between measurements, the greater the chances that
events and changes in the system might happen that may be missed (Gebremedhin et al.,
2010). Guijt (1999) concurs that to be useful, information needs to be collected at optimal
moments and with a certain frequency. Moreover, unless negotiated indicators are genuinely
understood by all involved and everyone‘s timetable is consulted, optimal moments for
collection and analysis will be difficult to identify.
According to Cornielje, Velema and Finkenflugel (2008), only when the monitoring system is
owned by the users the system is it likely to generate valid and reliable information.
However, all too often the very same users may be overwhelmed by the amount of daily work
which in their view is seen as more important than collecting data and subsequently the
system may become corrupted. A system of data collection should be self-organizing and
evolving as it gathers information from the environment where the staff would then generate
the information in the course of their daily activities (Innes &Booher, 1999: 415).
21
Assessment of Monitoring and Evaluation Practice and Challenges 2018
formal training, in-service training, mentorship, coaching and internships. Lastly, M&E
capacity building should focus not only on the technical aspects of M&E, but also address
skills in leadership, financial management, facilitation, supervision, advocacy and
communication.
Building an adequate supply of human resource capacity is critical for the sustainability of
the M&E system and generally is an ongoing issue. Furthermore, it needs to be recognized
that ―growing‖ evaluators requires far more technically oriented M&E training and
development than can usually be obtained with one or two workshops. (Acevedo et al., 2010).
Monitoring and evaluation carried out by untrained and inexperienced people is bound to be
time consuming, costly and the results generated could be impractical and irrelevant.
Therefore, this will definitely impact the success of projects (Nabris, 2002). In assessment of
CSOs in the Pacific, UNDP (2011) discusses some of the challenges of organizational
development as having inadequate monitoring and evaluation systems. Additionally, the lack
of capabilities and opportunities to train staff in technical skills in this area is clearly a factor
to be considered.
Staff need to be trained not only on collecting descriptive information about a program,
product, or any other entity but also on using something called ―values‖ to determine what
information and to draw explicitly evaluation inferences from the data, that is inferences that
say something about the quality, value or importance of something (Davidson, 2004).
Players in the field of project management like project and program managers, M and E
officers, project staff and external evaluators will require specialized training not just in
project management and M and E; but specifically in areas like Participatory monitoring and
evaluation and results based monitoring and evaluation (Murunga, 2011).
22
Assessment of Monitoring and Evaluation Practice and Challenges 2018
23
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Policy Environment
- Donors requirement
-Competency
- Training received
-Existing M&E system
- M&E experience
-Methods of evaluation
-Frequency of
Insufficient baseline data
monitoring
&
-Data collection
method Lack of Fund
Coping Mechanism
24
Assessment of Monitoring and Evaluation Practice and Challenges 2018
CHAPTER THREE
RESEARCH METHODOLOGY
3.1 Introduction
This chapter outlines how the research was conducted. It focuses on the research design,
target population, sampling strategy, data collection tools and techniques and data analysis
used in this study.
The study utilized a descriptive research design. Descriptive research design is used to
describe an event or phenomena as it exists at present and is appropriate when the study is
concerned in specific predictions, narrative of facts and characteristics concerning individuals
or situations (Kothari, 2003). This research used a quantitative research and attempts to find
out existing challenges and practice of M&E in selected Ethiopian local NGOs. From all
local NGOs, only 50 local NGOs are currently implementing or have implemented a health
project between 2015 and 2017 under the support of the USAID Local Capacity
Development program will be considered for this study. The quantitative methods were used
to generate numerical data, which is statistically manipulated to meet required objectives
through descriptive statistics (frequencies and percentages).
Population refers to the entire group of people; event or organizations that a researcher wants
to study. The population of this research is the 288 NGOs operating within Addis Ababa,
Ethiopia. The sample population is 50 NGOs that are registered and operational under the
USAID local capacity development program, who are implementing or have implemented a
health project between 2015 and 2017 and are implementing monitoring and evaluating using
a defined M&E system. The population size is therefore finite.
This is the process of selecting a sufficient number of elements from a population (Raval,
25
Assessment of Monitoring and Evaluation Practice and Challenges 2018
2009). It also refers to the techniques and procedures to be applied in selecting a sample. Out
of 88 NGOs under the support of the USAID local capacity development program 50 of the
local NGOs are working under health projects and all 50 were selected i.e. census will be
exercised to get a more comprehensive picture of the issue under study. In addition, as the
researcher has been working with them over three years, it will help the process much easier
in approaching them and getting the necessary information
The respondents for the target population were project manager and/or M&E staff of these 50
local NGOs as they are responsible of many aspects of the project, including the M&E
system, hence are in a better position to provide the information required by this study.
Chart 1: NGOs operating within Addis Ababa, Ethiopia
Population
PACT
41
Ethiopian Health
CSOs directory
51
CCRDA, 196
A questionnaire was used to collect information on the M&E systems being used by the
NGOs. Primary data was collected through the administration of written questionnaires to the
project manager or M&E staff from each NGO.
To collect data, survey questioner technique was used to distribute the questionnaire for all
50 local NGO‘s single M&E staff from each NGO under the local capacity development
program. The researcher prepared a list of themes and key questions to be covered in the
questioners based on prior literature on M&E. The questionnaire focused on the challenges
and practice of M&E in the selected NGOs. The questionnaire contains both closed and open-
26
Assessment of Monitoring and Evaluation Practice and Challenges 2018
ended questions, which allowed the collection of qualitative and quantitative data. The
questionnaires were designed in a simple manner for the respondents to be able to understand
the questions.
The questionnaires were distributed via email attaching both the word document and a link to
Google form which will automatically forward the response in to Google document for ease
of analysis. For those respondents that couldn‘t fill the questioner on the Google document
due to internet problems, the word document was distributed in person.
To ensure reliability of the research, the research objectives were stated in a precise and
concise manner. Validity of the data collection tool was done through consultations an M&E
specialist and department of the local capacity development program. This established any
built-in errors in the measurement of the questionnaire. The researcher did a pilot test with 10
Local Capacity development staff to check on the reliability of the questionnaire. The staffs
which were part of the pilot test were not part of the main study.
27
Assessment of Monitoring and Evaluation Practice and Challenges 2018
CHAPTER FOUR
DATA PRESENTATION, ANALYSIS AND
INTERPRETATION
4.1 Introduction
This chapter presents findings of the survey data analysed and interpreted in line with the
study objectives. The findings are presented in the form of tables, graphs and charts showing
frequencies and percentages.
This part of the study deals with presenting, analysing and interpreting the data gathered from
questioners. The data analysis is presented in 5 parts: Part 1: Back ground information, Part
2: Human capacity/employees knowledge status, Part 3: Current monitoring and evaluation
practice, Part 4: Challenges in executing M&E and Part 5: Adopted coping mechanisms.
28
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Total 34 100
Highest Secondary 0 0
level of College 1 2.9
education University 33 97.1
Total 34 100
The table further indicates that majority (97.1) of the respondents had university level of
education while the remaining 2.9% had college level education. This indicates that the
respondents were highly educated.
Position Frequency Percent
29
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Question 2.1 sought to determine the professional years of monitoring and evaluation
experience respondent local nongovernmental organizations staff had. The table above shows
the findings as follow:
The table above shows that majority (38.2%) of the respondents had 2-5years of professional
monitoring and evaluation experience. 26.5% and 20.6% of the respondents had less than
2years and 6-9years of professional monitoring and evaluation experience. The remaining
14.7% of the respondent had senior level status with over 10years of monitoring and
evaluation experience. This indicates that majority of the respondents are classified under the
middle level management position.
30
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Question 2.2 sought to determine the type of trainings respondent local nongovernmental
organizations employees had received so far while question 2.3 tracked the importance of this
trainings received by the respondents. The table below cross tabulates the two questions:
27 3 2 1 1 34
Total
Table 4 M&E trainings received * Relevance of the trainings received to M&E knowledge
Crosstabulation
The table above shows that majority of the respondents (14) received in-service training only.
Of the 14 respondents 9 of them stated that the training was very important in enhancing their
monitoring and evaluation knowledge while 2 of them labelled the training as important. Two
other respondents (each) ranked the trainings as slightly important and nor important. The
second type of monitoring and evolution training that is taken by respondents was a blend of
both formal and in-service training with 12 respondents. Of these 12 respondents, 11 ranked
the training as very important while one respondent ranked it as important. The table further
shows that 7 of the respondents had received formal training only. Of these 7 respondents, 6
ranked the training as very important while the remaining 1 ranked it as moderately
important. In the contrary, 1 respondent indicated that he/she had not received any
monitoring and evaluation training so far. Therefore, it is possible to conclude that almost all
of the respondents had received a training on monitoring and evaluation through formal, in-
31
Assessment of Monitoring and Evaluation Practice and Challenges 2018
service or both form of training and as a result this has enabled them to enhance their
monitoring and evaluation knowledge.
Question 2.5 sought to determine the competence of other relevant staff members in handling
monitoring and evaluation tasks. The findings are presented in the figure below as follow:
Incompetent
17%
Competent
62%
The figure above shows that majority of the respondents (62%) ranked he competency of
other relevant staff as competent while 9% ranked them as very competent. This indicates
that a more than half of the respondents show there are capable and competent staff that can
properly handle a given monitoring and evaluation task.
To the contrary, 17% of the respondents rate the competency of other relevant staff members
as incompetent while 12% said they don‘t know the competency level of other staff members.
Question 2.6 sought to determine whether there is a system that assist staff in analysing,
capturing and managing data which in turn will help build the knowledge of the M&E staff.
The Table below shows the findings as follow:
Frequency Percent
Yes 27 82.4
No 6 17.6
Total 34 100.0
32
Assessment of Monitoring and Evaluation Practice and Challenges 2018
The table above shows that 82.4% of the respondents indicated that there is a system that
assists staff to capture , analyze and manage data while the remaining 17.6% stated that there
is no such system within the organization.
The respondents were probed for the existing monitoring and evaluation practice. The first
question for this section sought to determine which stakeholders were involved in monitoring
and evaluation practices. Figure 4.5 shows the findings.
All project staff Only M&E staff Donors Community Beneficiary Government
8%
6%
6%
9%
67%
20%
As shown on figure 4.2 all project staff were involved in about 67% of monitoring and
evaluation practices of projects executed by local NGOs, followed by the only monitoring
and evaluation staff (20%). 9% reported that donors were involved as they were the one who
finance projects of these local NGOs, and they were there to track use of their resources.
Figure 4.2 also shows that equal number of respondents reported that the major stakeholder
involved in the monitoring and evaluation of projects were community, and beneficiaries
each with 6% respondent rate while government was involved 8%. This shows that projects
executed by respondents did not fully demonstrate strong downward accountability to the
beneficiaries, government and community as a result this could also deter sustainability of
project results.
33
Assessment of Monitoring and Evaluation Practice and Challenges 2018
The figure further illustrates that while significant number of respondents (20%) reported that
only the projects monitoring and evaluating staff is involved in the projects M&E activities,
this implies that there is a huge burden on the monitoring and evaluation staff as M&E is a
group effort and not a one department function.
Yes 13 38.2
No 21 61.8
Total 34 100.0
Question 3.3 sought to determine the role of management towards the implementation of the
monitoring and evaluation system. Figure 4.3 shows the finding as follow:
2.90%
14.70%
20.60%
61.80%
34
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Majority (61.8) of the respondent local nongovernmental organizations indicate that there is
an adequate role of management in their organizations monitoring and evaluation. The table
also shows that 14.7% of the respondents rate the role of management as very adequate.
With over 74% (61.8%+14.7%) level of adequacy, it indicates that there is a good practice
of engaging the top-level management in the monitoring and evaluation practice of the local
nongovernmental organizations.
The table further shows that 20.6% and 2.9% of the respondents rate the role of
management involvement towards the implementation of the monitoring and evaluation
system as inadequate and do not know. Question 3.4 sought to determine the most common
method of monitoring and evaluation data collection tool used by these local
nongovernmental organizations. The figure below shows the findings as follows:
70
60
50
40
30
20
10
0
Questioner Interview Attendance forms Focus group
discussion
Focus group discussion method is a qualitative data collecting method that enables the project
managers to have an in-depth understanding of the issues pertaining to the implementation of
35
Assessment of Monitoring and Evaluation Practice and Challenges 2018
their projects, the study found that 17.6% of the respondents use focus group discussion as a
monitoring and evaluation data collecting method for their projects.
The figure further shows that 14.7% of respondents use attendance forms to collect
monitoring and evaluation data of their projects. Consistent use of attendance form will
enable the project manager and other decision makers to the reach of the project activities in
terms of the number of peoples.
In addition, figure 4 also shows that only 5.9% of the respondents were use in depth
interviews as monitoring and evaluation data collection method for their projects. Although,
this method could have given the project managers an in-depth understanding of project
implementation, the study found that about 94% of the respondents fail to employ it.
Question 3.5 sought to determine whether the local nongovernmental organizations have a
written M&E plan while question 3.6 rates the adoptability of the plan.
No 5 14.7
Total 34 100.0
Table 7: Availability of written M&E plan
Majority of the respondents indicated that their organization has an M&E plan for all its
projects while 38.2% responded that they have an M&E plan for some of their projects. The
remaining 14.7% respondent local nongovernmental organizations had no monitoring and
evaluation plan.
36
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Figure 5 shows that 58.6% of the respondent local organizations indicated that the M&E plan
their organization has for all projects and for some of its projects is easy to adopt while
10.4% indicated that the plan they had is easy to adopt. The remaining 31% indicated that the
monitoring and evaluation plan is difficult to adopt.
The number of respondent local nongovernmental organizations that rated the plan to be
difficult (31%) is a significantly high number as a result it puts the practicality and the
implementation of the plan in questionable state.
Question 3.7 sought to determine the reason why some of the local nongovernmental
organizations had no written M&E plan. The table below shows the findings as follow:
37
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Lack of budget 4 80
It is irrelevant 1 20
Total 5 100
Question 3.8 sought to determine the type of planning and monitoring and evaluation tools
that are used by the respondent‘s local nongovernmental organizations. The figure below
shows the findings as follows:
6%
20%
6%
68%
38
Assessment of Monitoring and Evaluation Practice and Challenges 2018
NGOs was the results frame work. The figure above further shows that equal number (6%) of
respondent local NGOs use the theory of change and outcome mapping as a planning,
monitoring and evaluation tool. Question 3.9 sought to address three questions with regard
whether choice of indicator affects the M&E system, whether knowledge of impacts,
outcome and inputs influence performance of M&E system and whether respondent local
NGOs experience challenges when applying the M&E system. Respondents were asked to
use a Likert scale to rate their choice. The tables below show the findings as follows:
Agree 17 50.0
Disagree 1 2.9
Total 34 100.0
Majority of the respondents 97.1 (50%+ 47.1%) agreed that their knowledge of impacts,
outcome, outputs and inputs influence performance of monitoring and evaluation systems
while only 2.9% disagreed. This indicates that the respondent local NGOs know that they
need to understand various components of the planning and M&E tools use.
Response Frequency Percent
Agree 12 35.3
Not sure 1 2.9
Disagree 3 8.8
Total 34 100.0
39
Assessment of Monitoring and Evaluation Practice and Challenges 2018
their performance while 35.3% agree. While only 8.8% disagree with the idea and 2.9%
indicated they are not sure about the influence of indicators on M&E performance.
This indicates that the design of monitoring and evaluation systems should include the right
indicators as indicators provide critical information on performance of projects.
Agree 17 50.0
Disagree 2 5.9
Total 34 100.0
Table 11: Experience challenge when applying M&E system
Majority of the respondents (50%) agreed that they experience challenges when applying the
M&E system and tool while 44.1% strongly agreed to the idea. This indicates that respondent
local NGOs have difficulty measuring their M&E work using the designed M&E tool.
Question 3.10 sought to determine how often the local NGOs monitor their activities. The
figure below shoes the findings:
Bi-annually
18%
Daily
32%
Quarterly
12%
Monthly
38%
40
Assessment of Monitoring and Evaluation Practice and Challenges 2018
activities were carried out daily. The findings also indicate that 18% of the respondents were
conducting monitoring and evaluation activities bi-annually followed 12% monitored
activities quarterly. None of the respondents reported that they never monitored their
activities.
Failure to carry out continues and proper monitoring means that respondent local
nongovernmental organizations were unable to identify the progress of the projects they
implement that could lead to failure of the overall development objective of the NGOs could
occur.
Question 3.11 sought to determine the form of evaluation that the local NGOs had been a part
of while question 3.12 sees the type of evaluations conducted. The figure below shows the
findings through crosstabulation.
Of the total 34 organizations, 11.8% had no evaluation experience. 17.6% had a practice of
conducting internal evaluation for their projects while 11.8% had their projects evaluated by
external consultant. However, 58.8% of the organizations had the experience of conducting
both internal and external evaluations.
41
Assessment of Monitoring and Evaluation Practice and Challenges 2018
The graph above shows that only one organizations had the experience of conducting ex-ante
evaluation thorough inhouse capacity and using consultant. 47.1% had the chance to conduct
mid-term evaluation through internal and external evaluators. While 20.6% and 8.8% of the
organizations had conducted terminal and ex-post evaluation respectively.
Question 3.13 sought to determine whether the respondent local NGOs use inputs from M&E
findings for various decision making. The figure below shows the fining as follow:
18%
3%
3%
76%
Majority of respondents (76%) reported that they always utilize monitoring and evaluation
findings as input for decision making while 18% responded they sometimes use the findings.
The figure further shows that only 3% reported that they don‘t use M&E findings as input for
decision making while remaining 3% reported that they don‘t know.
This indicates that respondent local NGOs refer to monitoring and evaluation findings for
various decisions making as a indicating that inputs from M&E are valuable.
42
Assessment of Monitoring and Evaluation Practice and Challenges 2018
This section outlines the findings from the respondents through identifying the challenges
local NGOs face in the process of monitoring and evaluating their projects.
Question 4.1 sought to identify the major barriers that hinder local NGOs ability to
effectively implement their M&E activities. The figure below shows the finding as follow:
3% Policy/legal framework
26%
Lack of expertise
32%
In addition, the study found that 3% of the respondents were challenged due to a not friendly
M&E tool.
Question 4.2 sought to determine the extent to which the charities and societies agency
proclamation (621/2009) of 70/30 rule negatively affected their organizations M&E practice
and system. Figure 4.9 shows the findings.
43
Assessment of Monitoring and Evaluation Practice and Challenges 2018
6%
3%
38%
24%
29%
Figure 11: Effect of charities and societies agency proclamation number 621/2009
Figure 11 shows that 38.2% of respondent local nongovernmental organizations reported that
the ChSA (charities and societies agency) proclamation number 621/2009 negatively affects
their M&E performance while 29.4% and 23.5% indicate that the proclamation had high and
moderate effect respectively. On the contrary, 2.9% and 5.9% of the local nongovernmental
organizations reported that the proclamation had low and no effect to their M&E performance
respectively.
No effect
Low effect
Moderately
Highly
Very highly
44
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Figure 12 shows that 35% of the respondent local nongovernmental organizations indicated
that donors reporting format had low effect in the implementation of their M&E plan. While
35 % (6% + 29%) indicated that donors requirement format has very high or high negative
effect on their M&E implementation. Although this shows a major difference between the
two extremes, the remaining 6% of the respondents indicate that donor‘s requirement has no
effect on their M&E implementation. Therefore, this high tendency of requiring different
reports for the same work by different donors could create excessive burden on local
nongovernmental organizations to conform to these different requirements.
The last question for this section address issue related to what the overall trend looks like
over the past 5 years with regards to the challenges in conducting M&E. For this, the figure
below shows the findings:
Becoming more
About the same challenging
as it was 26%
33%
Improving
41%
Majority of the respondents (41%) indicated that the existing challenges in conducting M&E
compared to the past 5 years is improving while 32% indicated that the challenges are about
the same as it was. The figure further shows that 27% of the respondents confirmed that it is
becoming more challenging compared to the past 5 years. This finding indicates that most
respondents believe M&E challenges are decreasing through time and thus it can be argued
that there is a promising future for M&E.
45
Assessment of Monitoring and Evaluation Practice and Challenges 2018
This section of the finding aims to identify the coping mechanisms used by local NGOs to
address challenges related to M&E. Question 5.1 sought to determine opinion of respondent
local nongovernmental organizations on coping methods used by their respective
organizations to curb challenges related to monitoring and evaluation. The figure below
shows the finding as follow:
Adopt
participatory
Limit M&E
apporach for
activities
M&E
32%
47%
Figure 14 shows that 47% of the total respondents choose adopting a participatory approach
for M&E as the prior method of coping with their existing M&E challenges. The remaining
32% and 15% of the respondent local nongovernmental organizations indicated that their
organization chooses to limit its M&E activities and allot more budget respectively. The
figure, further indicate that 6% choose other coping mechanism such as: adopt to the learning
and male necessary adjustments/modifications, implement systemic data collection method
and minimize M&E staff size.
The last question pertaining to part 5 of the questioner sought to determine the possible
solutions that could positively contribute to enhance respondent local nongovernmental
organizations M&E system. For this, the figure below shows the findings as follow:
46
Assessment of Monitoring and Evaluation Practice and Challenges 2018
0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0
Figure 15 shows that 44.1% of the respondent local nongovernmental organizations choose
building staff capacity as the major solution to positively enhance their organizations M&E
system. 20.6% and 17.6% of the total respondents opt to choose minimization of the burden
of data collection and reporting and development of an M&E plan as the possible solutions to
enhance their organizations M&E system. The figure further indicates that 11.8% of
respondents believe increased role of management could be a potential solution to enhance
M&E system while the remaining 5.9% choose to a computerized M&E system as a possible
solution for enhanced M&E system.
47
Assessment of Monitoring and Evaluation Practice and Challenges 2018
CHAPTER FIVE
5.1 Introduction
This chapter presents and discusses briefly the summary of findings, then offers a conclusion
and recommendations, and finally gives suggestions for further research.
As outlined in chapter one, the primary aim of this thesis is to examine the practice and
challenges of Addis Ababa based local nongovernmental organizations in monitoring and
evaluating their executed projects. The research objectives were used to guide the collection
of required data from the respondents.
The next sub section presents summary of findings, concluding statements and then makes
recommendations.
Relevant M&E staff working in these NGOs had received the necessary training in
monitoring and evaluation either formally or through in-service training besides having
several years of experience working with monitoring and evaluation systems. Moreover,
these M&E staff indicated that the trainings they received played an important role in
enhancing their M&E knowledge.
In addition, the findings indicate that this local NGOs have competent staff (other than the
M&E department) that can properly handle a given monitoring and evaluation task. It is also
founded that these local NGOs have a system that assists staff to capture, analyse and manage
data.
Generally, most local NGOs did not have a computerized M&E system in place. This
indicates that the information obtained is likely to be inaccurate and not timely.
The findings show that these local NGOs did not engage all relevant stakeholders such as
beneficiaries, government, donors and community in their M&E activities. Beneficiaries‘,
48
Assessment of Monitoring and Evaluation Practice and Challenges 2018
The findings of this study support the idea that M&E activities need commitment of
management. Over 74% of these NGOs have a good practice of engaging the top-level
management in the monitoring and evaluation practice adequately.
The research question further helped to identify that most NGOs have a written M&E plan
for their projects. However, significant number of these NGOs had the plan only for some of
their projects. The NGOs with no written M&E plan indicated that this was due to lack of
budget. In addition, more than half of these NGOs experience challenge when applying their
M&E system.
Majority (68%) of these local NGOs use the logical framework as an M&E tool next to the
result framework. Therefore, the logical framework was found to be popular as a monitoring
and evaluation tool relied on throughout the stages of the project life cycle. In addition, it
was found that the choice of indicators in setting up monitoring and evaluation systems
influence the performance of the tool.
In addition, it was found that most local NGOs had a good practice of monitoring their
activities daily and monthly. More than half of the local NGOs also had the experience of
conducting internal and external evaluations.
It was also founded that these local NGOs refer to monitoring and evaluation findings for
various decision making as a indicating that inputs from M&E are valuable.
The findings present the main challenges in chapter four as follows: lack of sufficient
funding, stringent legal frame work/policy issue, insufficient baseline data lack of expertise,
lack of expertise, and not friendly M&E tool respectively. It was clear that each of these
challenges had a huge effect on their M&E practice.
In addition, it was found that most of these NGOs organizations find the ChSA (charities and
societies agency) proclamation number 621/2009 is affecting their M&E performance. To the
contrary, it was found that donors reporting format had low effect in the implementation of
their M&E plan.
49
Assessment of Monitoring and Evaluation Practice and Challenges 2018
It was found that most NGOs choose adopting a participatory approach for M&E as the prior
method of coping with their existing M&E challenges. While the remaining local
nongovernmental organizations choose to limit their M&E activities and allot more budget. It
was also found that other coping mechanism such as: adopt to the learning and male
necessary adjustments/modifications, implement systemic data collection method and
minimize M&E staff size were used as means of coping.
The research question further helped to identify possible solutions that local NGOs could opt
for while monitoring and evaluating their executed projects. It is evident that given the
intensity of the challenges, mitigating the challenges is something that local NGOs alone
cannot overcome.
5.3 Conclusion
The intention of this research was to examine the practice and the challenges Addis Ababa
based local nongovernmental organizations faced by while monitoring and evaluating their
executed projects. In order to address the primary aim of this research, the below stated key
research conclusions can be discerned.
Human capacity/employees M&E knowledge, use of M&E tools and utilization of
monitoring and evaluation information improve the implementation and use of the
monitoring and evaluation system.
Relevant staff had monitoring and evaluation experience and training, utilized monitoring and
evaluation information adequately and carried out regular data collection from various
sources. Furthermore, the role of management in monitoring and evaluation was adequate.
These local NGOs don‘t fully demonstrate strong downward accountability to the
beneficiaries, government and community as a result this could also deter sustainability of
project results.
Projects implemented by the local nongovernmental organizations in Addis Ababa were not
effectively monitored and evaluated. This is due to various obstacles such as: lack of
sufficient funding, stringent legal frame work/policy issue, insufficient baseline data lack of
expertise, lack of expertise, and not friendly M&E tool respectively. These challenges could
pose serious consequences both for projects and the wider NGO sector.
50
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Finally, local NGOs can curb challenges related to monitoring and evaluation through
adopting a participatory approach to M&E, by allotting more budget to M&E activities and
limiting M&E activities. In addition, it can be concluded that building staff capacity,
minimization the burden of data collection and reporting and development of an M&E plan as
the major possible solution to enhance local NGOs M&E system.
5.4. Recommendation
Based on the findings of this study and the conclusion made, the study makes the following
recommendations to address some of the key findings of the study:
It is important for local NGOs to continue enhancing their staff capacity through the
provision of various formal and in-service trainings.
There is need to combine the use of the logical framework with outcome mapping.
Outcome mapping lies in the shift away from assessing the development impact of a
programme and toward changes in the behaviour, relationships, actions or activities of
the people, groups, and organizations with whom a development programme is
working directly and seeking to influence.
The findings of the research also highlight the fact that there is not much involvement
of beneficiaries, government and community in monitoring and evaluation activities
of NGOs executed projects. As a means of fostering sustainability these relevant
stakeholders should be more involved in activities of the NGOs.
The study result shows a critical lack of budget in monitoring and evaluation of
projects implemented by the NGOs. There is need for soliciting fund from donors and
other income generating activities of monitoring and evaluation. Hence it is advisable
if concerned parties could help in filling the gap.
51
Assessment of Monitoring and Evaluation Practice and Challenges 2018
I) Further research to identify the human capacity of local NGOs and its influence
on monitoring and evaluation systems
II) Further research would be required to determine the actual impact of in
appropriate monitoring and evaluation on the performance of local NGO executed
projects in the city
III) Since the legal framework of the Ethiopian CSO has a huge impact on the M&E
activities of the local NGOs, further research to identify the specific gaps would
be required
52
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Reference
Acevedo, G. L., Rivera, K., Lima., L, & Hwang., H. (Eds.). (2010). Challenges in
monitoring and evaluation: An opportunity to institutionalize M &E systems. Fifth
conference of the Latin America and the Caribbean Monitoring and Evaluation
Network. Washington DC, World Bank 2010
Andrew, P., Rebecca, H., Georgina, M. (2009). Monitoring and evaluation of UN-assisted
Communication for Development Programs:11th UN Inter-Agency Roundtable
on communication for Development, 18-20.
Briceño, Bertha. 2010. Defining the type of M&E system: clients, intended uses, and actual
utilization. DC
Conner, R.F, Altman, D.G. & Jackson, C. (1984). 1984: A brave new world for evaluation.
In R.F.Conner, D.G. Altman & C. Jackson (Eds.). Evaluation Studies Review
Annual (Vol. 9). (pp. 12-22). Beverly Hills, CA: Sage
CRDA. (2006). Good practices of NGO‘s Urban Development Interventions. Addis Ababa:
Master printing press.
CCRDA. (2014). Good practices of NGO‘s Urban Development Interventions. Addis Ababa:
Master printing press.
Charities and Societies Agency. (2011). Charities and Societies Proclamation No.621/2009.
Addis Ababa: Brehanena Selam printing press.
53
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Debebe,H. (2010). Report on laws and regulations governing civil society organizations in
Ethiopia. Addis Ababa: unpublished.
Guijt, I. (1999). Participatory monitoring and evaluation for natural resource management
and research. Socio-economic Methodologies for Natural Resources Research.
Chatham, UK: Natural Resources Institute.
Gebremedhin, B., Getachew, A. & Amha, R. (2010). Results based monitoring and
evaluation for organizations working in agricultural development: A guide for
practitioners. International Livestock Research Institute, Nairobi, Kenya.
Frankel, L. & Gage, D. (2007). A practical guide to planning, monitoring and evaluation and
impact assessment. London, Save the children, UK.
Gorgenese et all. (2010). Beyond the Log frame: A new tool for examining health and peace
building initiatives. Development in practice, 18(1), 66-81
Gorgens, M.,& Kusek, J.Z. (2010). Making Monitoring and Evaluation Systems Work: A
Capacity Development Toolkit. Washington D.C, World Bank
Griffine, N. (2005). Beyond the Log frame: A new tool for examining health and peace
building initiatives. Development in practice, 18(1), 66-81
54
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Hunter, J. (2009). Monitoring and evaluation: are we making a difference? Namibia Institute
for Democracy John Meinert Printing, Windhoek, Namibia
Hogan, J. (2007). NGO Capacity Building: The Challenge of Impact Assessment. Paper
presented to the New Directions in Impact Assessment for Development Methods &
Practice Conference. IDPM University of Manchester
IFRC. (2002). Project level monitoring and evaluation: who really wants to know. The
annual report on results and impact of IFRC operations. Office of Evaluation
Innes, J.E. & Booher, D.E. (1999) A consensus building and complex adaptive systems: A
framework for Evaluating collaborative planning. Journal of the American planning
Association, 65 (4), 412-423
Jack,,E. (2014). Response rate and Responsiveness for surveys: Standards and the
Jeffery, C (2000). Civil Society, NGOs, and Development in Ethiopia A Snapshot View.
―CSOs and local capacity development‘ Reprinted in Jeffery 2007 The World Bank
1818 H Street N.W. Washington, D.C. 20433 USA Copyright © 2007
Kothari, C.R. (2005). Research Methodology: Methods and techniques. Daryaganj, New
Delhi: New Age International (P) Ltd.
Kusek, J. Z., & Rist, C. R. (2004). Ten steps to a Results-based Monitoring and Evaluation
Leeue,C. Lawrence M, & Kaith M.(2010). Research Methods in Education (6th ed.). New
York: Rout ledge publisher.
55
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Scriven C. (1996), Monitoring and evaluation framework for Waverley Action for Youth
Service, Social Policy Research Centre Report, University of New South Wales
Rachel, H., Thoms, L., Angela, C., Tlina, K., Joan, O., Brain, P. (2013). Legal frameworks
and political space for nongovernmental organizations: An overview of six
countries: Unpublished.
Rist, R. C., Boily M. H., & Martin F. (Eds). (2011). Influencing change: building evaluation
capacity to strengthen governance. Washington DC, World Bank.
Samuel T., Biraj, S., Merga, A., Gadissa, B. (2010). Evaluation and design of Social
accountability component of protection of Basic services project ,Ethiopia.
Addis Ababa: Unpublished.
Samuel, J., Mantel, Jr., Jack, M., Margaret M.(2001). Core Concepts of Project
Management. River Street, Hoboken-USA: Johnwiley&sons.inc.
56
Assessment of Monitoring and Evaluation Practice and Challenges 2018
TECS. (2013, April). Guideline to determine operational and administrative: Early evidence
of impact. Tracking Trends in Ethiopia‘s Civil society, Policy brief 5 , 2-5.
UNDP. (2002), (2006) and (2009). Hand book on planning, Monitoring and Evaluation for
development results. New York,USA: Colonial communications corp.
USAID. (2012). Hand book on planning, Monitoring and Evaluation for development
results, Ethiopia, Addis Ababa.
UNAIDS (2008). Organizing framework for a functional national HIV Monitoring and
Evaluation System. Geneva
UNDP Pacific Centre (2011). A Capacity Development Plan for CSOs in the Pacific. World
Bank. (2009): Project appraisal document. Washington: unpublished World Bank.
(2004). Ethiopia General Education Quality Improvement Program: Project
appraisal document. Washington: unpublished
Walker K., Grossman L., and salamaon N. (1999). M&E as learning: Rethinking the
dominant paradigm. World Association of Soil and Water Conservation
Wegayehu H., (2014): Monitoring and evaluation practices and challenges of local
nongovernmental organizations executing education projects in Addis Ababa
57
Assessment of Monitoring and Evaluation Practice and Challenges 2018
Annex
Questioner
Code:_____________
The purpose of this questionnaire is to gather information on practices and challenges local
nongovernmental organizations face in monitoring and evaluating their projects. The
information collected through this questionnaire will be treated with confidentiality and used
for academic purpose only. Kindly take a moment to answer all the questions as accurately as
possible.
58
Assessment of Monitoring and Evaluation Practice and Challenges 2018
A. Yes
B. No
B. 2 - 5years
C. 6-9 years
D. Over 10 years
D. None
4. How would you rate the trainings importance in enhancing your M&E knowledge?
A. Very Important
B. Important
C. Moderately Important
D. Slightly Important
E. Not Important
5. What is the competence of other relevant staff members in handling M&E tasks?
59
Assessment of Monitoring and Evaluation Practice and Challenges 2018
A. Very competent
B. Competent
C. Incompetent
D. Very incompetent
E. Don‘t know
6. Is there a system that assist staff in capturing, managing and analyzing data?
A. Yes
B. No
C. If no, why?_______________________________
1. Does your organization have any M&E experience in the past 4 years:
1. Yes
2. No
2. Who are the major stakeholders involved in M&E of your projects? (Possible to circle more than
1. All project staff
2. Only M&E staff
3. Donors
4. Community
5. Beneficiary
6. Government
7. Other________
3. Does your organization use computerized M&E system?
1. Yes
2. No
4. How would you rate the role of management towards the implementation of the M&E system?
1. Very adequate
2. Adequate
60
Assessment of Monitoring and Evaluation Practice and Challenges 2018
3. In adequate
4. Very inadequate
5. Don‘t know
5. What is the most common method of M&E data collection?
1. Questioners
2. Interviews
3. Attendance forms
4. Focus group discussion
5. Other: ____________
6. Does your organization have written M&E plan that guide project execution?
1. Yes, for all projects
2. Yes, for some projects
3. No
7. How would you rate the adoptability of this M&E plan?
1. Very easy
2. Easy
3. Difficult
4. Very difficult
8. If your answer is no to Q. 6, what is the reason behind?
1. Lack of budget
2. It is irrelevant
3. Lack of expertise
4. Other, specify: ______________________
9. Which of the following planning and M&E tools does your organization use?
A. Logical framework
B. Theory of change
C. Result framework
D. Outcome mapping
E. Most significant change
F. Others, specify:______________________
10. -Please tick next to the appropriate column in the table below.
61
Assessment of Monitoring and Evaluation Practice and Challenges 2018
62
Assessment of Monitoring and Evaluation Practice and Challenges 2018
2. Yes, sometimes
3. No
1. What are the major barriers that hindered your organizations ability to effectively and efficiently
implement M&E? 1. Indicate Very highest barrier, 2. Highest barrier, 3. Medium barrier, 4. Least
barrier and 5. Not a barrier (possible to rank more than one choice)
2. To what extent do you think the charities and society‘s proclamation (621/2009) of 70/30 rule
negatively affects your organizations M&E practice?
1. Very highly
2. Highly
3. Moderately
4. Low effect
5. No effect
3.To what extent does donors reporting requirement and format negatively affect the
implementation of M&E?
1. Extremely unlikely
2. Unlikely
3. Neutral
4. Likely
5. Extremely likely
4, Over all, the existing challenges in conducting M&E compared to the past 5 years is:
63
Assessment of Monitoring and Evaluation Practice and Challenges 2018
1. Which of the following methods does your organization opt for in coping with M&E
challenges?
1. Allot more budget for M&E
2. Limit M&E activities
3. Adopt participatory approach for M&E
4. Other: __________________________________
2. Which of the following possible solutions could contribute to positively enhanced your
organizations M&E system?
1. Building staff capacity
2. Minimize the burden of data collection and reporting
3. Develop an M&E plan
4. Computerize M&E system
5. Increased role of management
6. Other, specify: ____________________________________
What recommendation/suggestion would you give that could improve M&E practice and
curb the challenges?
___________________________________________________________________________
_______________________________________________________________________.
End
64