Background Paper on

Conceptual Issues Related to Health Systems Research

to Inform a WHO Global Strategy on
Health Systems Research

Steven J. Hoffman
John-Arne Røttingen
Sara Bennett
John N. Lavis
Jennifer S. Edge
Julio Frenk

____________________________________________

A Working Paper in Progress

Last Revised 29 February 2012
Preface

This paper was commissioned to provide a conceptual underpinning for the WHO Global Strategy on
Health Systems Research that is currently under development. It reviews existing definitions, terms,
conceptual models, taxonomies, standards, methods and research designs which describe the scope of
health systems research as well as the barriers and opportunities that flow from them. It addresses each
of the five main goals of the WHO Strategy on Research for Health, including organization, priorities,
capacity, standards and translation.1 Any feedback would be greatly appreciated and can be sent by
email to Steven Hoffman ([email protected]).

Acknowledgements

Thank you to Lucy Gilson for sharing an advanced copy of the Methodological Reader (2012) she edited,
which is now available at http://www.who.int/entity/alliance-hpsr/resources/alliancehpsr_reader.pdf.2
Thank you to Julia Olesiak at the McMaster Health Forum for layout design.

Disclaimer

This working paper is intended for a restricted audience only. It should not be abstracted, quoted,
reproduced, transmitted, translated or adapted, in part or in whole, in any form or by any means.

Contact Information

Steven J. Hoffman, BHSc MA JD

McMaster University
1280 Main Street West, MML-417
Hamilton, Ontario, Canada, L8S 4L6
[email protected]

Steven J. Hoffman1-4 | John-Arne Røttingen5-6 | Sara Bennett7

John N. Lavis1-2,8-9 | Jennifer S. Edge10 | Julio Frenk5,11
1
Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Ontario, Canada
2
McMaster Health Forum, McMaster University, Hamilton, Ontario, Canada
3
Munk School of Global Affairs, University of Toronto, Ontario, Canada
4
Interfaculty Initiative in Health Policy, Harvard University, Cambridge, MA, USA
5
Harvard Kennedy School, Harvard University, Cambridge, MA, USA
6
Department of Health Management and Health Economics, Institute of Health and Society, Faculty of
Medicine, University of Oslo, Norway
7
Department of International Health, John Hopkins School of Public Health, Baltimore, MD, USA
8
Centre for Health Economics & Policy Analysis, McMaster University, Hamilton, Ontario, Canada
9
Department of Political Science, McMaster University, Hamilton, Ontario, Canada
10
Department of International Development, University of Oxford, Oxfordshire, UK
11
Harvard School of Public Health, Harvard University, Boston, MA, USA

1
World Health Organization. 2010. WHO’s Role and Responsibilities in Health Research: Draft WHO Strategy on Research for
Health. Sixty-Third World Health Assembly. A63/22. Geneva: World Health Organization. Available at
http://apps.who.int/gb/ebwha/pdf_files/WHA63/A63_22-en.pdf
2
Gilson L, ed. 2012. Health Policy and Systems Research: A Methodological Reader. Geneva: Alliance for Health Policy & Systems
Research. Available at http://www.who.int/alliance-hpsr/resources/alliancehpsr_reader.pdf.

2
Table of Contents

Abstract ......................................................................................................................................................... 5

1. Conceptualizing Health Systems ............................................................................................................... 6

2. The Promise of Health Systems Research ................................................................................................. 9
3. Conceptualizing Health Systems Research .............................................................................................. 10
3.1 Characteristics of Health Systems Research ..................................................................................... 11
3.2 Boundaries of Health Systems Research ........................................................................................... 13
3.3 Defining Health Systems Research .................................................................................................... 18
3.4 Study Designs and Methods Used in Health Systems Research........................................................ 18
4. Conceptual Barriers for Health Systems Research to Improve Health Outcomes .................................. 19
4.1 Generalizability Challenge ................................................................................................................. 19
4.2 Comparativity Challenge ................................................................................................................... 20
4.3 Applicability and Transferability Challenge ....................................................................................... 20
4.4 Standards Challenge .......................................................................................................................... 21
4.5 Priority-Setting Challenge.................................................................................................................. 21
4.6 Community Diversity Challenge ........................................................................................................ 22
5. Strengthening the Contributions of Health Systems Research ............................................................... 22
5.1 Supporting Health Systems Research as a Field of Scientific Endeavour .......................................... 22
5.1.1 Need for a Common Language ................................................................................................. 22
5.1.2 Need for Cross-Disciplinary and Cross-Jurisdictional Learning ................................................ 23
5.1.3 Need for an International Society for Health Systems Research ............................................. 23
5.2 Building National Capacity for Health Systems Research ................................................................. 23
5.2.1 Individual-Level Capacities ....................................................................................................... 24
5.2.2 Organization-Level Capacities .................................................................................................. 24
5.2.3 System-Level Capacities............................................................................................................ 25
5.3 Embedding Health Systems Research as a Core Function of Health Systems .................................. 25
6. Conclusion ............................................................................................................................................... 26

Appendix 1: Tabular Comparison of 41 Health System Frameworks .......................................................... 27

Appendix 2: Brief Summaries of 41 Health System Frameworks ................................................................ 40
Appendix 3: Institutional Partners of the Alliance for Health Policy & Systems Research ......................... 62
Appendix 4: Do You Know Your Health Systems Definitions?..................................................................... 69
Appendix 5: Taxonomy of the Health Systems Evidence Database ............................................................ 70

3
“Something is wrong. For the first time,
public health has commitment, resources,
and powerful interventions.
What is missing is this: the power of these
interventions is not matched by the power
of health systems to deliver them to those
in greatest need, on an adequate scale, in
time. In part, this lack of capacity arises
from the failure of governments all around
the world to invest adequately in basic
health systems. It also arises, in part, from
the fact that research on health systems has
been so badly neglected and underfunded.
The two go together. So long as
investments in health systems are given
low priority, research in this area will also
be neglected. In the absence of sound
evidence, we will have no good way to
compel efficient investments in health
systems.”
Dr. Margaret Chan, Director-General of the World Health Organization
Beijing, China, October 29, 2007
4
Abstract
Health systems research is widely recognized as essential for strengthening health systems, getting cost-
effective treatments to those who need them, and achieving better health status around the world.
However, there is significant ambiguity and confusion in this field’s characteristics, boundaries, definition
and methods. Adding to this ambiguity are major conceptual barriers to the production, reproduction,
translation and implementation of health systems research relating to both the complexity of health
systems and research involving them. These include challenges with generalizability, comparativity,
applicability, transferability, standards, priority-setting and community diversity. Three promising
opportunities exist to mitigate these barriers and strengthen the important contributions of health
systems research. First, health systems research can be supported as a field of scientific endeavour, with
a shared language, rigorous interdisciplinary approaches, cross-jurisdictional learning and an
international society. Second, national capacity for health systems research can be strengthened at the
individual, organizational and system levels. Third, health systems research can be embedded as a core
function of every health system. Addressing these conceptual barriers and supporting the field of health
systems research promises to both strengthen health systems around the world and improve global
health outcomes.

Figure 1: World Map of the 9,818 MEDLINE Records Containing the Term “Health Systems”

Source: GoPubMed, which reports the frequency that terms appear in MEDLINE indexes for publications,
which include titles, abstracts, journal names and corresponding author’s affiliation. Many regions of
the world will be underrepresented in this figure given the popularity of other indexes, such as
LILACS for Spanish-language literature. This data was obtained on 25 February 2012.

5
1. Conceptualizing Health Systems
Pragmatic solutions already exist to address many of the greatest global health challenges, yet
progress remains frustratingly slow because many health systems are constrained and cannot fully
operationalize them. Eliminating two-thirds of child mortality and three-quarters of maternal mortality
would be possible if only the world knew how to effectively support the widespread implementation of
the simplest of existing interventions.3 Achieving better health internationally thus requires new
knowledge for both the discovery of biomedical innovations as well as the health policies and systems
necessary to actually deliver them. Achieving the Millennium Development Goals and nearly every global
health priority depends on it.4

Health systems have been defined in many ways. The most widely-used definition is from the
World Health Organization’s World Health Report 2000, which defines health systems functionally as “all
the activities whose primary purpose is to promote, restore or maintain health.”5 These activities are
often grouped into six categories or “building blocks”, namely 1) service delivery, 2) health workforce, 3)
health information systems, 4) medical products, vaccines and technologies, 5) health systems financing
and 6) leadership and governance.6 Health systems have also been defined at least in part in terms of
contributing actors. The European Observatory for Health Systems & Policies, for example, defines
health systems as the “people, institutions and resources, arranged together in accordance with
established policies, to improve the health of the population they serve, while responding to people’s
legitimate expectations and protecting them against the cost of ill-health through a variety of activities
whose primary intent is to improve health.”7 The Tallinn Charter from the 2008 WHO European
Ministerial Conference on Health Systems defines health systems as the “ensemble all public and private
organizations, institutions and resources mandated to improve, maintain or restore health” which
“encompass both personal and population services, as well as activities to influence the policies and
actions of other sectors to address the social, environmental and economic determinants of health.”8

Health systems have also been conceptualized in numerous ways. We conducted a

comprehensive search of the academic and grey literature in Google Scholar, Science Direct, PubMed
and Web of Science and consulted key informants in an attempt to find as many conceptual frameworks
as possible that were published over the last 20 years which describe the functions, actors, goals and/or
reform opportunities of health systems. Our search led us to 41 different frameworks, which we have
classified as either system frameworks (i.e., focused on the whole health system), sub-frameworks (i.e.,
focused on particular parts of the health system) or supra-frameworks (i.e., focused on how other
societal systems interact with the health system). Although since our search only targeted system
frameworks, the lists of sub-frameworks and supra-frameworks have been included for illustrative and
comparative purposes only. We also categorized the frameworks according to whether they were

3
Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS, and the Bellagio Child Survival Study Group. 2003. How many child
deaths can we prevent this year? The Lancet 362: 65-71.
4
Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, Pielemeier NR, Mills A, Evans T. 2004. Overcoming health-systems
constraints to achieve the Millennium Development Goals. The Lancet 364(9437): 900-906.
5
World Health Organization. 2000. World Health Report 2000. Health Systems: Improving Performance. Geneva: World Health
Organization.
6
World Health Organization. 2007. Strengthening Health Systems to Improve Health Outcomes. Geneva: World Health
Organization.
7
European Observatory for Health Systems and Policies. Observatory Glossary, 2007. Available at
http://www.euro.who.int/observatory/Glossary/Toppage.
8
WHO European Ministerial Conference on Health Systems. 2008. Tallinn Charter: Health Systems for Health and Wealth.
Resolution EUR/RC58/R4. Geneva: World Health Organization.

6
developed to offer a better understanding of health systems, to offer a way of comparing them, to help
with informing changes to health systems, or to outline a method of evaluating their performance or
changes to them (see Table 1).

Table 1: Categorization of 41 Health System Frameworks

Type of Framework
Goal Sub-Framework Framework Supra-Framework
 Yett, et al., Univesity of  Evans, University of British Columbia, 1981  Hsiao & Heller, International
Southern California, 1972† (“Actors Framework”)* Monetary Fund, 1997†
 Kutzin, WHO Regional Office for  Roemer, University of California, 1991 (“Basic  Atun & Menabde, Imperial
Understanding

Europe, 2001† Interactions Framework”)* College, 2008 (“Systems Thinking

 Mills, et al., World Bank, 2006†  WHO, 2000 (“Health Systems Performance Framework”)†
Framework”)*  Veillard, et al., Canadian Institute
 Khaleghian & Das Gupta, World Bank, 2004† for Health Information, 2011*
 WHO, 2007 (“Building Blocks”)*
 Global Fund, 2008*
 Mikkelson-Lopez, et al., Geneva Health Forum,
2010†
 Feldstein, et al., Harvard  Rechel, et al., European Observatory on Health  Hurst, OECD, 1992 (“Fund Flows
University, 1972* Systems and Policies, 2010 (“HiT Template”)* and Payment Framework”)*
 Feldstein & Friedman,  Anell & Willis, Swedish Institute
Department of Health, for Health Economics, 2000*
Comparing

Education and Welfare, 1976*  Hurst & Jee-Hughes, OECD, 2001*

 Nixon & Ulmann, University of  Docteur & Oxley, OECD, 2003*
York, 2006†  Anand & Bärnighausen, University
of Oxford and Harvard University,
2004*
 Siddiqi, et al., WHO Regional
Office for the Eastern
Mediterranean, 2009*
 WHO, 2008 (“Primary  Frenk, Mexican Health Foundation, 1994  Cassels, 1995†
Healthcare”)* (“Reform Framework”) †  World Bank, 2007 (“Healthy
 Savel, et al., Centers for Disease
 Londoño & Frenk, Inter-American Development Development”)†
Informing Change

Control and Prevention, 2010† Bank & Mexican Health Foundation, 1997†
 Sicotte, et al., University of Montreal, 1998
(“Integrated Performance Framework”) †
 Mills & Ranson, London School of Hygiene and
Tropical Medicine, 2001*
 Population Health and Wellness, British Columbia
Ministry of Health Services, 2005*
 Commonwealth Fund, 2006*
 Van Olmen, et al., Institute of Tropical Medicine
Antwerp, 2010†
 Ergo, et al., USAID, 2011  Aday, et al., University of Texas, 1998  Arah, et al., University of
Evaluating

(“Behavioural Healthcare Framework”)* Amsterdam, 2006*

 Roberts, et al., Harvard University, 2003 (“Control  Shakarishvili, et al., The Global
Knobs Framework”)* Fund to Fight AIDS, Tuberculosis
 Ramagem & Raules, Pan American Health and Malaria, 2011*
Organization, 2008*
 International Health Partnership, 2008†
Note: *indicates a descriptive framework and † indicates an interactive framework. Full citations listed in Appendices 1 and 2.

For example, Milton I. Roemer of the University of California, Los Angeles developed a basic
interactions framework in 1991 as a way to understand health systems (see Table 1, first row, middle
column). In his framework, a health system is “the combination of resources, organization, financing and
management that culminate in the delivery of health services to the population.” Resources include
health professionals, facilities, commodities and knowledge. Organization includes one principal
authority of government (at several levels), other governmental agencies with health functions,

7
voluntary health agencies, enterprises and a private health care market. Management includes health
planning, administration, regulation and legislation. Economic support includes governmental tax
revenues (at different levels), social insurance (statutory), voluntary insurance, charity, personal
households and foreign aid (where relevant). Finally, delivery of services include primary health care,
secondary care and tertiary care.9

One example of a framework for comparing health systems comes from the European
Observatory on Health Systems & Policies (see Table 1, second row, middle column). In their latest
template for authors writing a Health Systems in Transition (HiT) profile, key components of a health
system are to be presented in separate chapters. These include: 1) an introduction that outlines the
broader context of the health system; 2) organization and governance, which explains how a health
system is organized, the main actors, their decision-making powers, historical evolution, regulation, and
the level of patient empowerment in the areas of information, rights, choice, complaints procedures,
safety and involvement; 3) financing, which provides information on the level of expenditure, who is
covered, what benefits are covered, the sources of health care finance, how resources are pooled and
allocated, the main areas of expenditure, and how providers are paid; 4) physical and human resources,
which deal with the planning and distribution of infrastructure, IT systems, and health professional
registration, training, trends and career paths; 5) provision of services, which concentrates on patient
flows, organization and delivery of services; 6) principal health reforms, which reviews policy and
organizational changes that have had or will have a substantial impact on health care; 7) assessment of
the health system, which provides an evaluation based on the stated objectives of the health system and
other indicators; and 8) conclusions, which highlight lessons learned from health system changes,
remaining challenges and future prospects.10

The Commonwealth Fund’s Framework for a High Performance Health System for the United
States offers an example of a framework for informing changes to health systems (see Table 1, third row,
middle column). Their framework, developed in 2006, identifies four goals and priorities for performance
improvement, namely 1) high-quality care, 2) efficient care, 3) access and equity for all, and 4) system
and workforce innovation and improvement. Various policy options and indicators are outlined for each
goal.11

Finally, the “Control Knobs Framework” developed by Marc J. Roberts and colleagues at Harvard
University in 2003 and adopted by the World Bank Institute’s Flagship Program on Health Systems
Strengthening offers an example of a framework for evaluating changes to health systems (see Table 1,
fourth row, middle column). In their framework, health systems are conceptualized as “a set of
relationships where the structural components (means) and their interactions are associated and
connected to the goals the system desires to achieve (ends).” This framework identifies five major
“control knobs” of a health system which policymakers can use to achieve health system goals: 1)
financing, 2) organization, 3) payment, 4) regulation, and 5) behaviour. These knobs influence the
achievement of efficient, quality and access as intermediate performance measures and ultimately
performance goals of improved health status, customer satisfaction and risk protection.12

9
Roemer MI. 1991. National health systems of the world. Vol 1: The countries. Oxford: Oxford University Press; Roemer MI.
1993. National health systems throughout the world: lessons for health system reform in the United States. American
Behavioral Scientist 36(6):694-708.
10
Rechel B, Thomson S, van Ginneken E. 2010. Health systems in transition: Template for authors. United Kingdom: European
Observatory on Health Systems and Policies.
11
Commonwealth Fund Commission on a High Performance Health System. 2006. Framework for a High Performance Health
System for the United States. New York: Commonwealth Fund.
12
Roberts MJ, Hsiao WC, Berman P, Reich MR. 2003. Getting Health Reform Right. New York: Oxford University Press.

8
 The diversity of existing frameworks highlights the great variety of ways in which health systems
are understood by different people, disciplines and regions, and how health systems have been
conceptualized differently over time. Such discrepancies may represent a lack of coherence,
inefficiencies and untapped opportunities for collaboration, as well as the large number of conceptual
issues for which there is no consensus and for which greater research and deliberation is necessary.
Alternatively, the plethora of frameworks may further highlight the continued need for diversity in
health systems research, its context-specificity, opportunities to build on work in other fields, and how
such frameworks may need to be fit for purpose (see Appendices 1 and 2 for tabular comparisons and
brief summaries of 41 health system frameworks).

2. The Promise of Health Systems Research

Despite ambiguity in their exact definition and conceptualization, there is arguably now a global
consensus that strengthening health systems is necessary for achieving better global health and that this
issue requires greater attention. There is also arguably now a global consensus that research on health
systems is essential for making this happen (see Panel 1; see Appendix 3 for institutional supporters).

While not perfect, research is the best way currently known to systematically search for new
knowledge and generate new evidence. It is a process by which sources, objects and processes are
studied to establish facts, test hypotheses, explore ideas, evaluate interventions, develop theories and
advance new conclusions. At its core, research involves posing questions, gathering information, and
proposing answers. As such, it represents the best starting point for decision-making, or at least one key
input, especially for decisions concerning public policy.13

Recognition for the instrumental and conceptual value of health systems research is not new. It
has been repeatedly called for and prioritized over the past decades in a series of important events and
documents, including the:
 Commission on Health Research for Development (1990);
 Ad Hoc Committee on Health Research Relating to Future Intervention Options (1996);
 Alliance for Health Policy and Systems Research (1999);
 WHO World Health Report 2000 – Health Systems: Improving Performance (2000);
 WHO Task Force on Health System Research Priorities for Equity in Health (2004);
 World Report on Knowledge for Better Health (2004);
 Ministerial Summit on Health Research (2004), Mexico Statement on Health Research (2004),
and the related World Health Assembly Resolution A58/22 (2005);
 High Level Task Force on Scaling up Research and Learning for Health Systems (2008), Global
Ministerial Forum on Research for Health (2008) and the Bamako Call to Action on Research for
Health (2008);
 WHO Strategy on Research for Health (2009/2010); and
 First Global Symposium on Health Systems Research in Montreux, Switzerland (2010).

13
Lomas J. 1997. Improving Research Dissemination and Uptake in the Health Sector: Beyond the Sound of One Hand Clapping.
Hamilton, Canada: Centre for Health Economics & Policy Analysis, McMaster University.

9
 Future important milestones for health systems research are likely to include WHO’s World
Health Report 2012: No Health Without Research (2012) and the upcoming Second Global Symposium on
Health Systems Research in Beijing, China (2012).

Panel 1: Ten Opportunities for Health Systems Research

 Enhance the image of health systems. Interventions for strengthening the governance, financial
or delivery arrangements of health systems do not engage important stakeholders the way visible
or emotive topics such as child mortality or HIV/AIDS might engage them.
 Enhance the image of health systems research. Other forms of research like biomedical science
and drug discovery are prestigious, whereas health systems research is often perceived as “fluffy,”
“pedestrian,” and “applied.”
 Publicize the types of health systems questions amenable to scientific enquiry. Some believe
that health system problems are primarily political, and therefore best solved using common
sense or ideology rather than research evidence.
 Emphasize the long-term nature of health systems research. Answers from such research can be
slow to arrive and uncertain. Health systems development is a long-term process and there are
complex and indirect links between changes and final outcomes.
 Help generalize research findings across contexts as possible. The effects of interventions
crucially depend on the environment in which they are implemented such that it is important to
know the extent to which any research findings may be applicable in different contexts.
 Encourage dedicated funding for health systems research. Health systems research usually does
not have a disease-specific focus such that it can be difficult to secure funding when it is often
dedicated to certain illnesses or conditions.
 Educate about the complex nature of health systems research. Health system interventions are
part of large, messy reforms with strong political imperatives, such that systematic evaluations are
difficult to design and may be difficult to defend.
 Take every opportunity to evaluate health system reforms. There are only approximately 200
national health systems in the world and they rarely undergo large-scale changes.
 Expand research capacity and build on the best of each disciplinary tradition. It will be important
to address the dearth of obvious institutional homes with clear career structures.
 Ask the right questions. Improved understanding is needed about the types of research that really
change the way decision-makers think and will inform their work.

Adapted from Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, Pielemeier NR, Mills A, Evans T. 2004. Overcoming health-systems
constraints to achieve the Millennium Development Goals. The Lancet 364(9437): 900-906.

3. Conceptualizing Health Systems Research

It is clear that health systems can be defined and conceptualized in different ways, that better
knowledge about health systems is essential for improving health, and that investing in health systems
research is the best way to generate this knowledge. From a functional point of view, health systems
research can therefore be understood as a search for knowledge which contributes to health systems

10
strengthening and our understanding of health systems. However, there is a need for better describing
and defining what constitutes health systems research. It is assumed that health systems research is a
field of study that should be equally relevant for health systems across low-, middle- and high-income
countries. Indeed, it is clear that health systems research is conducted in every part of the world (see
Figure 1 on page 5). However, this assumption deviates from the reality that the various domains of
research related to health systems seem to have developed separately in low- and middle-income
countries and in high-income countries and across different disciplines.

It has been said for the term “quality” that it is difficult to define but easy to detect when it is
present. The same may be true for health systems research. There is probably a rather high degree of
agreement when deciding whether a study should be considered health systems research or not;
however, there is greater difficulty in precisely describing and defining the field. But in an attempt to do
so, the field and its characteristics will first be described, then its boundaries examined, and finally, a
definition for health systems research will be proposed.

3.1 Characteristics of Health Systems Research

Health systems research is a field of study that can largely be characterized by the questions it
poses and the answers it provides that can help strengthen health systems or better understand the
context in which they function. One of the field’s greatest strengths is how multiple disciplines,
knowledge paradigms, research designs and methods are all contributing to this endeavor (see Table 2).
However, this is also perhaps the field’s greatest conceptual challenge. Health systems research is a truly
multidisciplinary field, but to position itself as a mature field of research it also needs to develop a
stronger interdisciplinary culture. A truly interdisciplinary ethos would mean that researchers from
different traditions are not only informed about others’ positions, but also acknowledge and understand
those complementary perspectives and judge research according to authors’ own disciplinary standards.
There is a need for crossing traditionally great divides between, for example, a positivist paradigm often
utilizing quantitative methods and, say, a constructivist or relativist paradigm that usually relies on
interpretations of qualitative data.

However, there is increasing understanding of the importance of health systems research and of
its academic complexity and scientific challenges. In fact, recently there has been a crowding-in effect
from below and from the side which is promising and can be fostered. From below, researchers and
health professionals with experiences from clinical medicine and program delivery have realized that the
biggest barrier for improving health is not the development of new health technologies or bundles of
clinical interventions, but how these technologies and interventions can be better implemented,
integrated and delivered within organizational and system structures. From the side, social scientists
have seen how the health sector is expanding, representing power relationships in society, and
presenting an opportunity for addressing inequities and relieving poverty. This crowding-in may
inadvertently lead to turf wars over who owns the field and decides its paradigms and priorities, and may
even end up with battles like in the “science wars” of the 1990’s14 or divides as described in C.P. Snow’s
The Two Cultures.15 There have been tendencies for conflict where one group asks for more robust,
rigorous and generalizable research (that, for example, can answer whether system interventions work
and how they should be implemented) and another group argues for emphasizing context and asking

14
Brown JR. 2001. Who Rules in Science? An Opinionated Guide to the Wars. Cambridge, MA: Harvard University Press; Parsons
K, ed. 2003. The Science Wars: Debating Scientific Knowledge and Technology. Amherst, NY: Prometheus Books.
15
Snow CP. 1959. The Two Cultures. London: Cambridge University Press.

11
why interventions work in specific settings, how they impact the system as a whole, and how their
implementation was negotiated among relevant stakeholders.

Table 2: Characteristics of Health Systems Research

Characteristic Categories Comment

Disciplines Epidemiology, public health, HSR is both a field of multidisciplinary and interdisciplinary
medicine, psychology, economics, research with contributions from both health sciences and social
political science, sociology, sciences.
geography, anthropology, history,
law, ethics/philosophy, management
Paradigms Positivist, realist or The positivist view of the natural sciences used in epidemiology
relativist/constructivist/interpretivist and economics meets the relativist view of other social sciences.
Perspective Reductionist/single factors or Research may aim to understand single causal relationships or
holistic/complexity/systems science the interconnectedness of the system as a whole.
Designs Fixed or flexible Research designs can be fixed or be changed during the research
process.
Approaches External, internal or Researchers can be external and either observe or intervene, or
participatory/action research the approach can be more participatory like in action research.
Methods Quantitative, mixed methods The use of methods partly follows the knowledge paradigms,
(combined) or qualitative designs and approaches.
Aim Evaluative/normative, descriptive, The purpose of the research can be descriptive, explanatory or
explanatory or explorative explorative.
Intention Instrumental/applied, Research can have a short term applied intention with potential
theoretical/fundamental or direct impact or purposes which may have more indirect impact
critical/emancipatory on the system in the long run.
Level of study Macro (system as a whole), meso This is the classical way of dividing the different types of research
(institutions) or micro (individuals) questions.
Vertical/ Disease program or system oriented Research may focus on improving a delivery of a specific program
horizontal or the performance of delivery over all.
Focus Outcomes/outputs/content or Research can focus on the outputs or outcomes of policies and
processes decisions or on the processes of how they were developed and
implemented.
Country focus Low, middle and high income Research and its methods have developed in parallel instead as
countries one community.
Actors Academic/researcher or policy Unlike in clinical medicine where the roles of researcher and
maker/practitioner practitioner often meet in one individual the actors are separate.
Standards Diversity of standards relevant for No uniform standard is possible in the same way as in clinical
the different paradigms, designs and areas. However, more work can be done to develop relevant
methods standards that are intellectually coherent and fit for purpose.
Sources: Gilson L. 2012. Health Policy and Systems Research: A Methodology Reader. Alliance on Health Policy and Systems Research: Cape
Town; Sheik K, Gilson L, Agyepong IA, Hanson K, Ssengooba F, Bennett S. 2011. Building the field of Health Policy and Systems Research: framing
the questions. PLoS Medicine 8(8): e1001073; Gilson L, Hanson K, Sheikh K, Agyepong IA, Ssengooba F, Bennett S. 2011. Building the field of
Health Policy and Systems Research: social science matters. PLoS Medicine 8(8): e1001079; Bennett S, Agyepong IA, Sheikh K, Hanson K,
Ssengooba F and Gilson L. 2011. Building the field of Health Policy and Systems Research: an agenda for action. PLoS Medicine 8(8): e1001081;
Mills A. 2011. Health Policy and Systems Research: Defining the terrain; identifying the methods. Health Policy and Planning 2011: 1-7; Howard
S. 2011.. Background paper to UK funders’ workshop on health policy and systems research in low and middle income countries. ESRC,
Wellcome Trust, MRC, UKCDS and UKAID: London.

These difficult issues need not be fully resolved. The field as a whole should probably embrace
such discussions, acknowledge that there is no single approach or solution, and help seemingly
competing paradigms co-exist and contribute to increased learning and understanding. Answering
specific questions on effectiveness or feasibility serves an instrumental goal and may have a short-term
direct impact on policy formulation. However, more fundamental questions and development of theory
or a more critical perspective may have more indirect effects and possibly higher impact on the system in

12
the longer term. If health systems research is to be a multi- and interdisciplinary field of inquiry, it will
need to promote and preserve its diverse characteristics (see Table 2). The field must avoid being
captured by any single paradigm, tradition or discipline, or excluding any perspective that may be
important or helpful.

3.2 Boundaries of Health Systems Research

Health systems research is clearly a field within the larger domain of health research. Most
would agree that it overlaps a bit with clinical and behavioral and population health research, but not
with biomedical research. The overlap with clinical and behavioral research may be most apparent in
sub-domains related to improving the delivery of services, such as improvement science and
implementation science. The overlap with population health research, however, is less clear, but likely
includes research on the public health system and the delivery of non-personal public health programs
and interventions. Excluded from health systems research would be population health research’s focus
on measuring or describing health, examining the determinants of health status and outcomes, and
assessing the effects of specific health promotion interventions. Most of health policy research16 is also
part of the health systems research field, but not the health policy research that is purely relevant to the
clinical or population health domains such as policies on the safety of a prescription drug or the built
environment, respectively (see Figure 2).

There is also general consensus that the meso- and macro-level questions addressed in health
services research falls within the remit of health systems research. There is less agreement on whether
micro-level questions related to the delivery of health services, programs and interventions should be
considered health systems research. Some may argue these micro-level questions are purely within the
domain of health services research, while others say that all of health services research is part of health
systems research, especially giving the overlapping focus on delivery arrangements and to a lesser extent
on financial and governance arrangements. Then again, the terms “health systems research” and “health
services research” have been used independently in ways that suggest they are either two separate
fields of research or the same field of research but denoted by different terms, yet they have clearly
evolved together in parallel. There are also some indications of the use of the term health systems
research for addressing the needs of low- and middle-income countries, and health services research
being the term used by the community of researchers focusing on developed countries.

Within the health service delivery area there are also various sub-domains including
implementation science, quality improvement science, delivery science, operations research and
management science (see Figure 3). These five sub-fields of research all aim to improve the delivery of
services, but with somewhat different approaches. Research on implementation issues is of course
relevant both when conducting policy analysis or studying health system interventions, such as those
addressing financing or the health workforce. However, implementation science has been defined more
narrowly as research to promote the uptake and successful implementation of evidence-informed

16
Health policy research has been defined as a multidisciplinary field involving instrumental analyses of policy alternatives that
affect the health care system or the health of the general public, and scholarly inquiries into the process of health policy
making and how it is shaped by ideas, interests, and institutional arrangements. (Canadian Association for Health Services and
Policy Research. What is ‘HSPR’? 2011. Available at https://cahspr.ca/en/resources, which was adapted from McMaster
University. 2007. Strengthening Health Policy Scholarship in Canada).

13
 Figure 2: Health Systems Research as a Multidisciplinary Field of Health Research

Figure 3: Examples of Sub-Fields in Health Systems Research and Health Services Research

14
clinical interventions,17 and this field of study often has an external or top-down perspective by
examining interventions which facilitate implementation into the system. Implementation science
defined this way is closely related to and partly overlaps with knowledge translation research. Quality
improvement science usually has a more internal (or bottom up) perspective addressing what kind of
approaches actors within the system themselves can utilize to improve quality of care.18 Operations
research is an older term which generally is a discipline that applies analytical methods for making better
decisions (i.e., decision science), but has been used more broadly within health to denote on-the-ground
timely knowledge generation relevant for continuously improving performance of health programs.19
More recently the term “health care delivery science” has been introduced and can be seen to
encompass all three of these approaches.20 In the United Kingdom, the term “service delivery research”
has also been used within the Service Delivery and Organization program which recently merged into the
Health Services and Delivery Research program.21 Management science applied to health care is in
general having an organizational and leader-driven focus, but there are no clear definitions, and
academic departments on health management cover a diverse program of research.22 In general there is
need for more work on clarifying the scope of these five different sub-fields of health services research
to better understand where there is overlap, where there are gaps, and where they serve as terms
describing the same areas of research. This issue will eventually need to be resolved for better clarity in
understanding what constitutes the knowledge base for health systems.

Given the existential centrality of service delivery to health systems, it only makes sense to
include all of health services research within the domain of health systems research. This would include
research on the delivery of services related to specific conditions or diseases so long as the research
question is related to delivery and not purely a clinical issue.

While these boundaries for health systems research are likely most defensible from academic
and theoretical perspectives, health systems research leaders must acknowledge the reality that the
term used for their field is still not widely adopted as indicated when measured in number of

17
Implementation research has been defined as the scientific study of methods to promote the systematic uptake of clinical
research findings and other evidence-based practices into routine practice, and hence to improve the quality and
effectiveness of health care. It includes the study of influences on healthcare professional and organizational behaviour.
(Available at www.implementationscience.com/info/about/.)
18
The concept of improvement science recently emerged to provide a framework for research focused on healthcare
improvement. The primary goal of this scientific field is to determine which improvement strategies work as we strive to
assure effective and safe patient care. (Improvement Science Research Network. 2012. Available at
http://www.improvementscienceresearch.net/about/improvement_science.asp.)
19
Operations research has been defined as a field that “aims to develop solutions to current operational problems of specific
health programmes or specific service delivery components of the health system, e.g., a health district or a hospital. It is
characterized by a strong problem-solving focus and an urgency to find solutions. Its demand-driven nature and close
association with health care delivery and routine health care operations ensure operational relevance of the research
activities and rapid uptake and local utilization of research findings.” (Remme JHF, Adam T, Becerra-Posada F, D’Arcangues C,
Devlin M, et al. (2010) Defining Research to Improve Health Systems. PLoS Med 7(11): e1001000.
doi:10.1371/journal.pmed.1001000).
20
Health Care Delivery Science has been defined as “how we bring best practices of care to every patient, every time.”
(Available at http://www.dartmouth.edu/~tdc/overview-step.html.)
21
The HS&DR programme aims to produce rigorous and relevant evidence on the quality, access and organisation of health
services, including costs and outcomes. The programme will enhance the strategic focus on research that matters to the NHS
including research on implementation and a range of knowledge mobilisation initiatives. (Available at
http://www.netscc.ac.uk/hsdr/index.html.)
22
“Health management comprises activity around the development and implementation of policy and the organization of
services aimed at improving health. The focus is on delivery and effecting change in organizations concerned with improving
population health.” (Hunter DJ, Brown J. 2007. A review of health management research. European Journal of Public Health
17(suppl 1): 33-37. doi: 10.1093/eurpub/ckm061.)

15
researchers, papers produced or investments. For example, the term “health systems research” only
appears in MEDLINE records 192 times (nearly half of which had corresponding authors based in Canada)
whereas the term “health services research” appears 37,894 times (with just over half of these
corresponding authors in either the United States or United Kingdom) (see Figure 4). The term “health
services research” also appears far more frequently in books than the term “health systems research”
(see Figure 5). Finally, “health systems research” is so rarely used as a Google search term and so rarely
appears in news reports that Google Trends did not even have enough data to compare it to the much
more popular “health services research” and “health policy research” terms (see Figure 6). The
suggestion of including the more popular sub-field of health services research as part of health systems
research is clearly imperialistic, but it is justified by the promise of future benefits including conceptual
clarity, greater coherence, and new opportunities for collaboration. We would propose engaging with
influential institutions, organizations and leaders within the health services research community to
discuss these issues with the aim of coming to a common way of using and understanding these terms.

Figure 4: Use of the Term “Health Systems Research” in MEDLINE Records

Health Services Research Health Policy Research Health Systems Research

Total 37,894 1,140 192
Top Year 2003 2010 2008
First Publication 1951 1975 1990
Top Countries United States (15,442) United States (654) Canada (95)
(Total number of United Kingdom (4,560) Israel (197) South Africa (52)
publications, based Canada (1,659) Taiwan (75) United States (9)
on location of Australia (1,262) Canada (59) Thailand (9)
primary author) Netherlands (913) Nigeria (22) Mexico/Belgium (7)
Source: GoPubMed, which reports the frequency that terms appear in MEDLINE indexes for publications, which include titles, abstracts, journal
names and corresponding author’s affiliation. Results are mostly driven by the presence of these terms in the corresponding author’s affiliation,
such as the Health Systems Research & Consulting Unit of Canada’s Centre for Addition & Mental Health, the Health Systems Research Unit of
South Africa’s Medical Research Council, and the Health Systems Research Center of Mexico’s National Institute of Public Health. This data was
obtained on 25 February 2012.

16
 Figure 5: Use of the Term “Health Systems Research” in Books

Source: Google Books Ngram Viewer, which compares terms based on the frequency in which they are found in the millions of books that they
have scanned. This data was obtained on 25 February 2012.

Figure 6: Too Few Searches and News Reports on “Health Systems Research” for Google Trends to
Compare to “Health Services Research” and “Health Policy Research”

Rank Health Services Research Health Policy Research

1 Canada Canada
2 New Zealand South Africa
3 Singapore Australia
4 Australia New Zealand

Source: Google Trends, which compares the frequency in which people input terms into the Google search engine and the frequency in which
these terms appear in news articles. There were too few Google searches and new reports on “health systems research” for Google Trends to
compare it to “health services research” and “health policy research.” The five countries that most frequently searched these terms on a per
capita basis are listed. This data was obtained on 25 February 2012.

17
3.3 Defining Health Systems Research

A review of existing definitions for “health systems research”, “health services research” and
“health policy research” – and permutations of them – highlighted the existence of overlapping terms,
great ambiguity and lots of confusion (see Appendix 4 for a quiz to test your knowledge of health
systems research definitions). Based on the research conducted for this paper, we propose a new
definition that describes the field, its main foci, and much more clearly delineates boundaries:

Health systems research is a multidisciplinary field of health

research which studies governance, financial and delivery
arrangements for health care and public health services,
implementation considerations for reforming or strengthening
these arrangements, and broader economic, legal, political and
social contexts in which these arrangements are negotiated and
operate. The purpose of health systems research is to improve the
understanding and performance of health systems. Health
systems research includes all of health services research, most
health policy research, and some clinical and population health
research, but does not include any biomedical research.

(See Figure 2)

We suggest this definition could be used for the Global Strategy for Health Systems Research,
but that for the long-term, the proposed International Society for Health Systems Research should
facilitate a consultative process through its (hopefully) large and diverse membership to revise this
definition and finalize it.

The broad range of issues addressed in health systems research when using this definition can
be explored through the Health Systems Evidence Database [www.healthsystemsevidence.org] – the
world’s most comprehensive, free access point for synthesized health systems research – and the
taxonomy it uses to categorizes its thousands of records (see Appendix 5 for the database’s taxonomy).

3.4 Study Designs and Methods Used in Health Systems Research

Health systems research features a broad range of study designs and methods. The
Methodological Reader recently published by the Alliance for Health Policy & Systems Research in 2012
classifies research strategies into two main areas: fixed designs that are established before data
collection, and flexible designs that evolve during the study process. Fixed strategies typically use more
positivist approaches to study design; data is generally quantitative and investigators primarily seek to
measure the impact of a phenomenon under highly specified and controlled conditions. Experimental
designs and modeling are typically coupled with statistical analysis. Common data collection techniques
include surveys, structured and semi-structured interviews, and routine record reviews.23

23
Gilson L, ed. 2012. Health Policy and Systems Research: A Methodological Reader. Geneva: Alliance for Health Policy &
Systems Research. Available at http://www.who.int/alliance-hpsr/resources/alliancehpsr_reader.pdf.

18
 Flexible designs, on the other hand, are more interpretivist in nature and deal primarily with
qualitative data. Overarching designs include case study, grounded theory, ethnographic, life histories
and phenomenological research. Qualitative interviews, focus group discussions, observation and
document review serve as primary means of data collection. Data is analyzed in iterative and interpretive
processes.

Building on this fixed-flexible dichotomy, the Methodological Reader outlines seven research
strategies that capture the breadth of health systems research. Impact evaluation and cross-national
analyses are two types of fixed strategies involving rigorous, large-scale quantitative procedures. Flexible
strategies include single and multiple case study, ethnographic study and action research designs that
involve in-depth qualitative analysis of an exploratory nature. Finally, cross-sectional approaches and
studies tracing policy and system change over time fall into both fixed and flexible design types, often
employing structured quantitative analysis and detailed qualitative techniques like process tracing. 24

4. Conceptual Barriers for Health Systems Research to Improve Health Outcomes

The challenge in defining health systems research, its characteristics, boundaries and methods is
demonstrative of the many barriers that prevent its production, reproduction, translation and
implementation for improving health systems. These barriers stem from the complexity of health
systems and from the complexity of the health systems research field.

Health systems are extraordinarily complex social structures. Like all complex systems, they are
multi-layered, nonlinear and highly sophisticated. Despite embracing rapid technological innovation and
constant reorganization, health systems are strongly resistant to planned change, if only as a
consequence of the sheer number of independent players, established policies, zealously guarded
interests, entrenched professional silos and divergent cultures that together help characterize its
complexity. This web of elements – and the unpredictable interactions among them – often limit the
usefulness of mechanistic “cause-and-effect” approaches, including in the study of health systems and
evaluation of changes to them. Research questions often cannot be answered with methods like
randomization and control groups which are commonly used in biomedical and clinical research, and
researchers must often conduct their work in difficult political environments and in contexts that are
constantly evolving.

The complexity of health systems leads to three challenges facing researchers who study health
systems: generalizability, comparativity, and applicability and transferability. A fourth challenge relates
to the complexity of health systems research and resulting community differences.

4.1 Generalizability Challenge

First, research findings often depend on the particular context in which studies were conducted,
which makes it difficult to generalize learning beyond the specific context of the health systems that
were studied. Deepening confusion is how generalizability is approached differently across disciplines
and methods.

24
Gilson L, ed. 2012. Health Policy and Systems Research: A Methodological Reader. Geneva: Alliance for Health Policy &
Systems Research. Available at http://www.who.int/alliance-hpsr/resources/alliancehpsr_reader.pdf.

19
 Researchers categorized as positivists, for example, tend to focus on making sure their findings
can be statistically generalized beyond the specific study population and setting, as can be done in
randomized control trials due to strong internal validity and reliability. Case study researchers, on the
other hand, use analytic or theoretical generalizability, where they extract broad insights from one or
more cases and thoroughly examine and thoughtfully analyze the context and processes that produced
their findings in each setting.25 Comparativists abstract the specifics of one case to ideas and theories
that accurately describe multiples cases.26 All of these methods for enhancing external validity and
promoting generalizability are subject to scrutiny from different factions of researchers with different
methodological backgrounds.

4.2 Comparativity Challenge

Second, it is often difficult to compare health systems or conduct large-scale cross-system

studies. Unlike other areas of inquiry, health systems researchers have relatively few opportunities to
learn about their objects of study. There are only around 200 national health systems in the world and
they are very rarely reformed on a large scale (yet are constantly experiencing small changes). This low
sample size problem means that certain empirical methods like cross-sectional analyses can be less
powerful than if larger samples were possible. It also means that it is particularly important to seize
every opportunity to study health system reform when it does occur. This goal, however, is difficult to
achieve given the different timelines of the health system leaders who make changes to their systems
and the health services researchers who would evaluate them. Reforms are also not usually rolled-out in
ways that are amendable to rigorous study, with Oregon’s health insurance lottery and Mexico’s Seguro
Popular segmented regional implementation representing notable exceptions.27

Unfortunately there are far too few examples of actually embedding research into the process of
health system reform or otherwise studying health system changes as they happen. There are even
fewer that do so using common designs across systems, which would otherwise facilitate drawing cross-
national lessons. These few opportunities for natural and quasi-experiments are also especially
important given how difficult it is to isolate the effect of changes through other means, such as
randomization and systematic experimentation.

4.3 Applicability and Transferability Challenge

Third, the fact that health systems research is often highly context-specific means that studies
conducted in one jurisdiction may not be applicable or transferable to another jurisdiction. Applicability
is the likelihood that an intervention could be implemented in a new specific context, and transferability
is the likelihood that a study’s findings could be replicated in a new specific context (i.e., that any
evidence of effectiveness would remain the same).28 This barrier, in many respects, is conceptually

25
Robson C. 2002. Real World Research: A Resource for Social Scientists and Practitioner-Researchers, 2nd ed. Oxford: Blackwell
Publishing.
26
Gilson L, ed. 2012. Health Policy and Systems Research: A Methodological Reader. Geneva: Alliance for Health Policy &
Systems Research. Available at http://www.who.int/alliance-hpsr/resources/alliancehpsr_reader.pdf.
27
Finkelstein A, Finkelstein A, Taubman S, Wright B, Bernstein M, Gruber J, Newhouse JP, Allen H, Baicker K, Oregon Health
Study Group. 2011. The Oregon Health Insurance Experiment: Evidence from the First Year. NBER Working Paper No. 17190.
Available at http://www.nber.org/papers/w17190; King G, Gakidou E, Imai K, Lakin J, Moore RT, Nall C, Ravishankar N, Vargas
M, Téllez-Rojo MM, Eugenio J, Ávila H, Ávila MH, Llamas HH. 2009. Public policy for the poor? A randomised assessment of the
Mexican universal health insurance programme. The Lancet 373(9673): 1447-1454.
28
Wang S,Moss JR, Hiller JE. 2006. Applicability and transferability of interventions in evidence-based public health. Health
Promotion International 21:76-83.

20
similar yet opposite to the generalizability challenge, which is the likelihood that a study’s findings can be
generalized to a broader context.29

The challenge of applying and transferring health systems research from one context to another
means that research users often have to undertake the difficult task of trying to assess and adapt
research from foreign jurisdictions to their own local context. Translating health systems research into
practice is already a difficult task, given the political nature of health systems decision-making,
multiplicity of stakeholders, and the importance of other inputs such as public opinion and ethical
considerations. This challenge only makes knowledge translation efforts even more important.

This applicability and transferability challenge, however, also means that researchers have to
expend great efforts to learn about the environment in which they are working and may need to repeat
experiments in different contexts. They do have help: a recent review found 25 external validity,
applicability and transferability frameworks to aid in this effort, but none of them had been validated or
assessed for their effectiveness or reliability.30 Unfortunately few researchers, policymakers or funders
are trying to push beyond current recognition that “context matters,” and few people are trying to figure
out how to actually increase the global applicability of studies (rather than just assessing their
applicability) and actually increase the adaptability of research evidence to different contexts. The
importance of such cross-jurisdictional learning is exponentially compounded by how there is limited
capacity in many countries to produce health systems research of their own. Overcoming this challenge
will also be essential for bridging the communities of researchers who focus on health systems in high-
income countries and in low- and middle-income countries – who currently are often split into different
departments, attend different conferences and publish in different journals.

4.4 Standards Challenge

In addition to health system complexity, the field of health systems research is also itself
complex. As previously noted, there is a lack of definitional agreement on key terms, a multitude of
conceptual frameworks and paradigms, and disagreements on how different methods fit together and
the circumstances in which they may be useful in answering different types of questions. Compared to
fields like clinical medicine, there are also relatively few standards by which research can be reported
and evaluated. While this comparison to clinical medicine may not be fair given the diversity of
approaches brought to bear on health systems issues, the health systems research field could probably
do a better job of evaluating research based on the standards of authors’ own paradigms, traditions or
disciplines.

4.5 Priority-Setting Challenge

The health systems research community also lacks widely agreed-upon processes for dynamically
setting priorities for the type of research that should be conducted and funded. Whereas the clinical and
population health research communities can set priorities based on the global burden of disease or risk
factors, each health system is run differently, serves different functions, achieves different goals, and has

29
Burchett H, Umoquit M, Dobrow M. 2011. How do we know when research from one setting can be useful in another? A
review of external validity, applicability and transferability frameworks. Journal of Health Services Research and Policy 16(4):
238-244.
30
Burchett H, Umoquit M, Dobrow M. 2011. How do we know when research from one setting can be useful in another? A
review of external validity, applicability and transferability frameworks. Journal of Health Services Research and Policy 16(4):
238-244.

21
different opportunities for improvement. Given the context-specificity of health systems research, this
challenge is compounded by the limited transferability of priority-setting processes from one jurisdiction
to another and thus requires national capacity for prioritization. Timelines are also much shorter and less
predictable in the health systems research field – with decision-makers often unsure of exactly what
evidence they want except for that they needed it yesterday.

4.6 Community Diversity Challenge

Further, in many respects, the health systems research community is actually not really a
community. As a field of inquiry that is devoted to strengthening health systems and understanding the
context in which they function – in addition to advancing particular theoretical debates and developing
particular methods – health system researchers come from different places, were trained in different
disciplines, hold different traditions, speak different languages, prefer different methods and focus on
different questions. Other than the common goal of strengthening health systems, understanding the
context in which they function, and improving health outcomes on a population-scale, there is little that
binds health systems researchers together and much that pulls them apart. It is currently a disparate
field rather not a cohesive community.

5. Strengthening the Contributions of Health Systems Research

Fortunately, the challenges presented by these conceptual barriers point to opportunities for
strengthening the contributions of health systems research. Indeed, in addition to numerous options for
mitigating any negative effects, aspects of these conceptual barriers may actually serve as the field’s
greatest strengths.

5.1 Supporting Health Systems Research as a Field of Scientific Endeavour

The different backgrounds, disciplines, methods and questions of health systems research, and
the perceived lack of community among health systems researchers, suggests the need to build health
systems research as an important and coherent field of scientific endeavour. Such marked differences,
when viewed as diversity, can actually be the field’s greatest strength and the basis on which its
participants can make substantial contributions to improving health around the world. Such unity amidst
diversity can be nurtured through collaborative forums and by directly addressing the many legitimate
methodological and disciplinary issues that need to be problematized.

5.1.1 Need for a Common Language

But the advantages of diversity require a platform on which to build. First, the field could be
supported in developing a common language for health systems researchers to use and authoritative
textbooks from which students and experts alike can draw. This can be facilitated through deliberation
and consensus on the scope, boundaries and definitions related to health systems research, as well as
further theory development and conceptual understandings. Such a development may be supported by
the use of clear taxonomies of both health systems and of health systems research like the International
Classification of Diseases or the UKCRC Health Research Classification System.31

31
International Classification of Diseases (ICD). Available at http://www.who.int/classifications/icd/en; UKCRC Health Research
Classification system. Available at http://www.hrcsonline.net.

22
5.1.2 Need for Cross-Disciplinary and Cross-Jurisdictional Learning

Second, the field would benefit from mechanisms that help it take advantage of the best that
each traditional academic discipline has to offer and then promote additional cross-disciplinary learning.
This could include acknowledging the value of different research methodologies, developing standards
within different research paradigms, promoting mixed-methods research, facilitating opportunities for
health systems researchers to come together, and evaluating rigor based on available alternatives. Also
important is greater attention to strengthening the cross-jurisdictional applicability of health systems
research which requires having clearer frameworks for assessing transferability and enhanced research
methods that lead to more generalizable findings. Journals could assist by publishing richer descriptions
of both methodology and context, and education institutions can offer graduate and postgraduate
training programs – in addition to the current short course offerings that are typically offered.

5.1.3 Need for an International Society for Health Systems Research

Third, as was recommended at the First Global Symposium on Health Systems Research (2010),
an International Society for Health Systems Research could be created to start developing the
institutions, bells and whistles typical of any other discipline, track the state of the field, and nurture it
into a cohesive yet diverse whole. This would be an important milestone for the field’s development
which has largely advanced through individual papers and efforts rather than through any systematic or
collective initiatives. Such a professional association could also help bridge developing and developed
country perspectives into a single community of practice, which are currently two separate fields
artificially fragmented along countries of focus. It could also help encourage the uptake of its
participants’ efforts by developing methodological standards, supporting guidelines development, and
establishing classification schemes as is common among clinical specialities.

The benefits of supporting health systems research as a field of scientific endeavour include the
emergence of a common corpus of health systems knowledge, cross-jurisdictional learning, avoiding
duplication, and the bridging of different disciplines and cultures of each sub-group to allow for more
rigorous and helpful research outputs. Achieving these benefits, however, will require systematic,
collective and coordinated approaches, not simply one-off collaborative efforts.

5.2 Building National Capacity for Health Systems Research

The context-specificity of health systems research emphasizes the need to build capacity for it in
every jurisdiction, particularly in the low- and middle-income countries that currently have the least
capacity. Capacity-building is one of the five main strategies of the WHO Strategy on Research for Health
and was the focus of a flagship report by the Alliance for Health Policy & Systems Research in 2007.32
This report emphasized that multiple actors, including researchers, policymakers, health providers and
civil society organizations play an important role in the production, assessment and application of
research findings, and that an effective environment for health systems research needs to build capacity
– albeit different types of capacity – among multiple actors and their networks.

32
World Health Organization. 2010. WHO’s Role and Responsibilities in Health Research: Draft WHO Strategy on Research for
Health. Sixty-Third World Health Assembly. A63/22. Geneva: World Health Organization. Available at
http://apps.who.int/gb/ebwha/pdf_files/WHA63/A63_22-en.pdf; Green A, Bennett S, eds. 2007. Sound Choices: Enhancing
Capacity for Evidence-Informed Health Policy. Geneva: World Health Organization. Available at http://www.who.int/alliance-
hpsr/resources/Alliance_BR.pdf.

23
 Capacity frameworks frequently distinguish between individual, organizational and system level
capacities.33 This organizing framework is used to reflect on priority areas for capacity development in
health systems research with the aim of building sufficient capacity for health systems research across
low- and middle-income countries so as to facilitate context-specific and generalizable health systems
research.

5.2.1 Individual-Level Capacities

The fact that individuals engaged in health systems research, either as research producers or
users, are disparate with diverse backgrounds and training requires a tailored approach to capacity
development. Some individuals may aspire to become truly interdisciplinary health systems researchers,
familiar with a broad range of the common methods used in health systems research. Such individuals
will most likely need Masters or Doctoral training programs with a primary focus on health systems
research. Others may come to health systems research with strong disciplinary backgrounds, but require
exposure to the particular challenges of health systems, and need to acquire fluency in engaging with
researchers from different traditions. Health professionals, program managers, and policy advisors may
become important health systems researchers in their own right, and/or facilitators of health systems
research, providing guidance on key systems questions, enabling access to settings where research
needs to be conducted, and helping translate findings into health system reforms. At a minimum they
will require training and support to build the necessary skills to acquire, assess, adapt and apply health
systems research.34

For the dedicated health systems researcher, individual skills and expertise will probably best be
addressed through the systematic development of primary training courses and curricula, not through
ad-hoc short courses. Many of the measures described above to build the field will also be critical to the
development of appropriate training curricula. While many post-graduate training programs have run
courses on health systems for some years, it appears that the development of courses on health systems
research applicable to LMICs is a much more recent phenomena, and there is substantial scope for
sharing of curricula, teaching materials and even faculty so as to facilitate the rapid development of
capacity for health systems research across the world.

5.2.2 Organization-Level Capacities

Organizational homes for researchers reflect the heterogeneity of the field. Many health systems
researchers are found throughout academic institutions, including schools of public health, medical
schools, business schools, law schools, and a variety of disciplinary departments such as anthropology,
economics, geography, history, political science and sociology. But many health systems researchers
work outside of an academic setting in policy analysis institutes or think-tanks that have the mission of
informing government policy and decision-making.

A balance is needed between allowing diversity to flourish, and building recognized institutional
homes for health systems research. In order to develop a critical mass of researchers, as well as establish
health systems research as a legitimate field of scientific endeavor, there is a need for departments or
units within academic and policy analysis institutions that are dedicated to health systems research. Such

33
Bennett S, Agyepong IA, Sheikh K, Hanson K, Ssengooba F, Gilson L. 2011. Building the Field of Health Policy and Systems
Research: An Agenda for Action. PLoS Medicine 8(8): e1001081. doi:10.1371/journal.pmed.1001081.
34
Canada Health Services Research Foundation. 2001. Is Research Working for You? A Self-Assessment Tool for Health Service
and Policy Organizations. Ottawa: Canada Health Services Research Foundation.

24
organizational settings may be effective in both developing the profile of the field and for efficient
organization of the production, reproduction, translation and implementation of health systems
knowledge.35

5.2.3 System-Level Capacities

Several systems interface with health systems research, including national research systems
(that prioritize, fund and ethically review health and other research), health systems (which are often the
subject of study but also where primary users of health systems research reside), and systems of higher
education. Each raise particular challenges: how can national research systems be adequately funded
and have sufficient capacity to effectively identify and support critical health systems research? How can
health systems utilize research and knowledge management functions so that actors have access to
appropriate evidence in a timely fashion? And finally, with respect to systems for higher education, how
can career paths for health systems researchers be developed and how can adequate funding for
postgraduate health systems research training be secured?

While the primary goal of such capacity development is to build capacity for health systems
research at the local and national level, global-level actions are important to support the achievement of
this goal. The proposed International Society for Health Systems Research could establish a sub-
committee for training and methods, with a mandate to better document standards for health systems
research. For example, this group could identify which types of research methods are best designed to
address different types of research questions, and what is best practice in terms of employing such
methods. There is also substantial scope for the coordinated development and sharing of high-quality
teaching curricula, readers on research methods, and online learning resources about widely used
research designs. While to some extent this is already occurring informally, a more coordinated and less
fragmented approach – as seen in other more established fields – could make a substantial impact on
meeting the capacity development challenge.

5.3 Embedding Health Systems Research as a Core Function of Health Systems

Given there are relatively few health systems in the world, even fewer large-scale reform efforts
each year and complex processes unfolding at every turn, it would be helpful if every effort was made to
study health systems when opportunities present themselves. One of the most promising ways to
achieve this goal is to embed health systems research as a core part of every health system and simply
make research a necessary function like financing, service provision, stewardship and resource
generation.

There are at least three mechanisms required for this goal of embeddedness to be achieved.
Health system leaders and their trusted advisors must: 1) learn to acquire, assess, adapt and apply health
systems research in their decision-making processes; 2) request the preparation of evidence syntheses to
inform their decisions; and 3) mandate rigorous monitoring and evaluation of health system
performance and reforms to aid future learning. Health system leaders could also require health impact
assessments to inform changes before they are made and impact evaluations to assess any changes after
they are implemented.

35
Frenk J. 1993. The new public health. Annual Review of Public Health 14: 469-490.

25
 Opportunities to embed research within health systems are numerous. For example, Ministries
of Health can establish internal technical departments staffed by researchers or they can collaborate
with researchers based at universities or think-tanks, especially those that already have strong ties with
government. Government managers can be trained in using research and data and managing evaluations
of their programs. In fact, every mission of technical assistance by an international agency or a non-
governmental organization can be turned into an opportunity for fostering embeddedness if such
technical assistance is integrated into government and local organizations rather than offered without
local collaboration.

But ultimately, embedding health systems research as a core health systems function is about
trust. For the successful translation of research to action, policymakers must trust the findings of
researchers, and in turn, researchers must trust that policymakers will not misuse (or abuse) their work.
The benefits of such trust in developing, prioritizing and embedding health systems research also extend
beyond health systems: such actions can be a key component of reform agendas promoting efficiency,
good governance and accountability.

6. Conclusion
Health systems research is widely recognized as essential for strengthening health systems,
getting cost-effective treatments to those who need them, and achieving better health status around the
world. However, there is significant ambiguity and confusion in this field’s characteristics, boundaries,
definition and methods. Adding to this ambiguity are major conceptual barriers to the production,
reproduction, translation and implementation of health systems research relating to both the complexity
of health systems and research involving them. These include challenges with generalizability,
comparativity, applicability, transferability, standards, priority-setting and community diversity. Three
promising opportunities exist to mitigate these barriers and strengthen the important contributions of
health systems research. First, health systems research can be supported as a field of scientific
endeavour, with a shared language, rigorous interdisciplinary approaches, cross-jurisdictional learning
and an international society. Second, national capacity for health systems research can be strengthened
at the individual, organizational and system levels. Third, health systems research can be embedded as a
core function of every health system. Addressing these conceptual barriers and supporting the field of
health systems research promises to both strengthen health systems around the world and improve
global health outcomes.

26
 Appendix 1: Tabular Comparison of 41 Health System Frameworks

Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
*Descriptive framework E.g. Performance evaluation, comparative E.g. Leadership, Governance, Financing, E.g. levels, blocks, web, tiers, actors,
†Interactive framework framework for analysis Resources, Service delivery systems, across or between elements

1. Feldstein, et al., To assess the distributional impact of Structure of insurance coverage A model simulating the distribution of
Harvard University, alternative national health insurance (deductibles, coinsurance rates, etc.), health expenses with different
1972* options using a simulation model. The income and family composition, mix of insurance coverage and price elasticity
A sub-framework model assists in calculating the revenue sources (income-related is developed. The model is applied
comparing the actuarial value of benefits of different premiums, payroll tax, general tax across a large sample of expenditures
distributional impact of insurance coverage and different price revenue, etc.) to derive distributions for families of
national health insurance elasticities of demand different composition
options in the USA
2. Yett, et al., Univesity of Two econometric models to study Health service institutions: voluntary/ Follows flows of demand and supply
Southern California, health manpower policies are proprietary short-term hospitals; between providers, consumers and
1972† presented: a macro-econometric governmental hospitals; skilled nursing available health manpower;
A sub-framework for model using aggregate data to homes; outpatient clinics of non-federal consumers, providers, and manpower
understanding how investigate comprehensive health hospitals; private sector offices of medical are linked through services and labor
manpower is distributed planning at the national, state, and and surgical specialists. Health manpower: market
across health systems sub-state levels, and a micro- medical specialists and general
simulation model treating the practitioners in private practice; surgical
interactions of individuals, health specialists in private practice; physicians
manpower personnel, health service employed by hospitals; hospital interns
institutions, and educational and residents; registered nurses; allied
institutions in the analysis of health health professionals and technicians;
manpower policies for a nation non-medical personnel
3. Feldstein & Friedman, An operational method combining Equations representing annual health care Follows the impact of two NHI plans
Department of Health, stochastic micro-simulational models expenditures of a group of families with on total expenditure at baseline values
Education and Welfare, of household demand with aggregate the same demographic composition, against changing demand elasticities.
1976* supply and price determination income, and insurance coverage; formulas
A sub-framework equations is used to calculate for expenditure distribution, net out-of-
comparing the effects of equilibrium quantities and prices for pocket expenses, etc.; quantities and
alternative national two proposed NHI options prices of hospital and medical care
health insurance (NHI)
plans

27
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
4. Evans, University of To highlight the resource allocation Five classes of transactors: consumer- Traces degrees of government
British Columbia, 1981 and administrative organization of the patients (who utilize care), first-line regulation and the pervasiveness
(“Actors Framework”)* health care sector as distinct from providers (contacted directly by public and private insurance in
A framework for typical “market” mechanisms and how consumers), second-line providers (whose modifying market structures between
understanding five key that health systems possess unique output is either used by consumers under transactors
groups of actors in the patterns of incomplete vertical the direction of first-line providers or
health care market. integration supplied as intermediate products to first-
line or other second-line providers),
insurers and governments (exercise or
dele-gate regulatory authority)
5. Hurst, OECD, 1992 To determine optimal health system Health expenditure of GDP, consultations Based on flow of funds from central
(“Fund Flows and payment schemes that protect with general practitioners and specialists, government to health care providers
Payment Framework”)* consumers from financial risk while medicines prescribed per capita, acute
A supra-framework for minimizing cost; helps compare hospital admissions, perinatal mortality
comparing health system government spending on health
reforms in seven Western services to gross domestic product
European countries (GDP) and helps assess adequacy,
equitable access, income protection
and cost containment in health
systems performance
6. Roemer, University of To study a health system as “the Five components of any health system: Management and economic inputs
California, 1991 (“Basic combination of resources, resources (human resources, facilities, produce resources that can be
Interactions organization, financing and commodities and knowledge), organized into health programs to be
Framework”)* management that culminate in the organization (principal authority of delivered to populations; management
A framework for delivery of health services to the government, other governmental agencies and financial support is required in
understanding how four population” with health functions, voluntary health organizing and delivering programs
elements of health agencies, enterprises, private health care
systems contribute to the market), management (health planning,
delivery of services administration, regulation and legislation),
economic support (governmental tax
revenues, social insurance, voluntary
insurance, charity and personal
households) and delivery of services
(primary health care, secondary care and
tertiary care)

28
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
7. Frenk, Mexican Health To outline the relational perspective Exchanges between providers, the Demonstrates how governments serve
Foundation, 1994 between providers and populations by population, the state (as collective as collective mediators interacting
(“Reform Framework”) † specifying principal actors, their mediator), organizations that generate with providers, resource generators
A framework informing exchanges and the basis of resources and other sectors that produce and other sectors to provide services
change that outlines the interrelationships services with health effects at systemic to populations
fundamental features of (institutional arrangements),
health system reform programmatic (setting priorities),
organizational (production of services)
and instrumental (intelligence generating)
levels constitute reform processes
8. Cassels, 1995† To clarify what constitutes health Six key institutional components: the Political decisions determine reforms
A supra-framework for sector reform and consider the state, service providers, resource in six main programmes: improving the
comparing and informing context-specific nature of institutional institutions, institutional purchasers, other civil service, decentralization,
change for health sector reforms for health in less developed sector agencies that produce health strengthening national health
reform in developing countries; frame health sector reform benefits indirectly and populations. ministries, broadening financing,
countries as a process that evaluates how managed competition and
existing policies, institutions, engagement with the private sector
structures and systems manage issues
of efficiency, access, cost-containment
and responsiveness to demand
9. Londoño & Frenk, To promote a framework of structured Four basic functions: financing, delivery, Modulation is the central mission of
Inter-American pluralism for increased equity, quality modulation (setting transparent and fair the ministry of health, Financing is the
Development Bank & and efficiency in health that organizes rules of the game) and articulation main function of social security
Mexican Health the health system by functions rather (managing and organizing transactions institutes, articulation is managed via
Foundation, 1997† than social groups. between groups). the establishment of “organizations for
A framework informing health services articulation” and
change in Latin American delivery is open to pluralism adapted
health care reform to differential needs of urban and rural
populations

29
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
10. Hsiao & Heller, To demonstrate how fluctuations in Health status of population, Population health has a direct effect
International Monetary population health influence the macro- microeconomic variables (labour on demand for health services and is
Fund, 1997† economy productivity, poverty rates, demand for influenced by government policy and
A supra-framework for medical care), macroeconomic variables the provision of public goods;
understanding how health (inflation rate, wage and exchanges rates), population health has consequences
status affects the macro- demands on health care system, for microeconomic variables (e.g.
economy government policies labour productivity) which have larger
implications for macroeconomic
activity
11. Aday, et al., To define a prevention- and outcomes- Based on type and extent of affected Identifies a tripartite continuum of
University of Texas, 1998 oriented continuum of healthcare groups’ participation in formulating and program evaluation based on structure
(“Behavioural Healthcare integrating health services research implementing policies and programs, (the availability, organization, and
Framework”)* methods and policy analysis for availability and utilization of services and financing of behavioral healthcare
A framework for assessing healthcare system flows of payment; measures effectiveness programs), processes (transactions
evaluating health systems performance (how structure, process or both contribute between patients and providers in the
that lists clinical and social to outcomes of healthcare at the course of care delivery) and outcomes
determinants of mental community, system, institution or patient (ultimate outcomes of health care
health outcomes along a level), equity (participation and freedom services to enhance the health of
continuum of behavioural of choice) and efficiency (the combination individuals)
health programs of goods and services with the highest
attainable total value be produced given
limited resources and technology) are
identified within the health system
12. Sicotte, et al., To develop a comprehensive Four functional dimensions of action: Organizational performance is
University of Montreal, framework grounded in Parsons' social two internal functions (maintaining values determined by the dynamic
1998 (“Integrated system action theory to overcome the and stabilizing production) and two equilibrium resulting from the
Performance current fragmented approach to external functions (adapting to the continuous exchange and interaction
Framework”)† health care organizations’ environment to acquire the necessary between the four functions
A framework informing performance management resources and attaining the valued goals
change in the performance of the system)
management of health care
organizations

30
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
13. Anell & Willis, To derive a simple framework for Measured health expenditures: Traces and compares domestic health
Swedish Institute for comparing data underlying health care percent GDP, expenditures per capita, spending per capita across select
Health Economics, 2000* systems by analyzing health care drug expenditures per capita, MRI units measured expenditures
A supra-framework resource profiles for Denmark, France, per capita, CT scanners per capita,
comparing health care Germany, Sweden, the United number of hospital beds per capita, health
resource profiles across Kingdom, and the USA care employment per capita, number of
multiple national health
physicians per capita, number of nurses
systems
per capita, and health care employment
as percentage of total employment
14. WHO, 2000 (“Health To outline key functions of health Four key functions: resource generation, Four key functions serve as inputs that
Systems Performance systems that influence how inputs are financing, service provision and synergistically promote positive health
Framework”)* transformed into health system stewardship; three central goals: health, outcomes, responsiveness and
A framework for outcomes responsiveness and financial protection financial protection
understanding the key
functions and inputs that
drive health system
performance
15. Mills & Ranson, To examine previous conceptual Key players: governments, populations, Focus on increasing the role of the
London School of Hygiene frameworks for health to understand financing agents and providers; key areas state and regulation, increasing public
and Tropical Medicine, how health systems work and how for reform: regulation, financing, resource control over financing, greater
2001* they can be changed in low- and allocation and the provision of services decentralization of management and
A framework informing middle-income countries greater involvement of the private
change in health systems sector in service provision
reform for low- and middle-
income countries
16. Hurst & Jee-Hughes, To compare key indicators of health Three goals: health improvement and Focuses on the rate of development
OECD, 2001* system performance arrangements outcomes; responsiveness and access; and for indicators of performance in health
A supra-framework adopted by WHO, OECD, Australia, financial contributions and health outcomes, equity, efficiency and
comparing methods for Canada, UK and USA in hopes of expenditure; each goal has two responsiveness across countries
performance measurement conceptualizing performance components of assessment: the average
and management across
measurement level and the distribution of each goal
OECD countries

31
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
17. Kutzin, WHO Regional To clarify the policy levers that are Four key functions: revenue collection, Follows funding and benefit flows
Office for Europe, 2001† available to enhance the insurance pooling of funds, purchasing of services between individuals and the funding
A sub- framework for function for the population as and the provision of services and collection of health services
understanding the insurance efficiently as possible given the
function of health systems ‘starting point’ of a country’s existing
institutional and organizational
arrangements
18. Docteur & Oxley, To give policymakers a better Policy goals: ensuring access to needed Focuses on the degree to which policy
OECD, 2003* understanding of the state of reforms health-care services; improving the goals have been achieved through
A supra-framework across OECD countries and to inform quality of health care and its outcomes; health system reforms in OECD
comparing health system them of policy orientations that may allocating an “appropriate” level of public countries
reforms across OECD potentially have greater payoffs sector and economy-wide resources to
countries
health care (macroeconomic efficiency);
and ensuring that services are provided in
a cost-efficient and cost-effective manner
(microeconomic efficiency)
19. Roberts, et al., To provide a framework for Institutional drivers underpinning the “A set of relationships where the
Harvard University, 2003 policymakers to use when striving to control knobs: financing, payment, structural components (means) and
(“Control Knobs”)* achieve health system goals regulation, organization and behaviour; their interactions are associated and
A framework evaluating intermediate performance measures: connected to the goals the system
changes in health systems efficiency, quality and access; and goals: desires to achieve (ends)”
that enables policymakers to health status, customer satisfaction and
focus on the relationship
risk protection
between structural
components and goals
20. Khaleghian & Das To evaluate the impact of new public EPHFs include disease surveillance, health Follows health system capacity in light
Gupta, World Bank, management strategies for public education, monitoring and evaluation, of true market reforms (those
2004† health (e.g. decentralized, contract- workforce development, enforcement of involving user charges and provider
A framework for based delivery of services) on essential public health laws and regulations, public competition), pseudo-market reforms
understanding how reforms public health functions (EPHFs) across health research, and health policy (e.g., purchaser-provider splits,
in public management industrialized and developing development. contracting and other market-
impact health systems
countries. simulating reforms), decentralization
and other reforms to health
management

32
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
21. Anand & To investigate the link between human Dependent variables: maternal mortality Traces the effect of variation in human
Bärnighausen, University resources for health and health rate, infant mortality rate, and under-five resources for health density across
of Oxford and Harvard outcomes mortality rate; independent variables rates of maternal mortality, infant
University, 2004* across sets: aggregate density of human mortality and under-five mortality
A supra-framework resources for health; doctor and nurse across countries
comparing the impact of densities separately; controlled variables:
health worker density on income, female adult literacy, and
health outcomes across absolute income poverty
multiple countries
22. Population Health To identify key public health services Core programs: long-term programs Core programs are implemented
and Wellness, British that health authorities can provide to representing the minimum level of across a series of programs that build
Columbia Ministry of strengthen the link between public services provided, public health strategies: health system capacity while being
Health Services, 2005* health, primary care, and chronic strategies to implement core programs, monitored for equitable access and
A framework informing disease management lenses: population and inequality lenses to quality
change in British Columbia’s ensure health needs for all are met,
public health renewal efforts system capacity: information systems,
staff training, quality assessment, etc.
23. Mills, et al., World To examine disease-specific and health Seven key constraints: financial, physical Traces disease-specific and health
Bank, 2006† system responses to common inaccessibility, poorly skilled staff, poorly system responses to each of seven key
A sub-framework for constraints experienced in less motivated staff, weak planning and constraints to foster capacity-building
understanding how health developed countries to deliver services management, lack of intersectoral in developing countries
services can be strengthened more effectively, efficiently and action/partnership, poor quality care in
in less developed countries
equitably private sector
24. Nixon & Ulmann, To determine whether increased Inputs: lifestyle, environmental and Econometric analyses using a fixed
University of York, 2006† expenditure on health is causally occupational factors; outputs: life effects model are conducted on a
A sub-framework comparing linked to improved health outcomes expectancy and infant mortality panel data set tracing the effect of
the relationship between Analysis considers health spending and variation in health spending on infant
health spending and health outcomes in 15 European Union countries mortality and life expectancy
outcomes in 15 EU countries
between 1980 and 1995
25. Arah, et al., To develop a set of indicators that can Four tiers: health, non-health care Places quality of care within larger
University of Amsterdam, be used to investigate quality of health determinants of health, health care performance framework; consists of
2006* care across countries using system performance, health system four interconnected tiers arranged by
A supra-framework comparable data design/context; core quality dimensions: potential causality
evaluating health systems effectiveness, safety and
that assists in comparably responsiveness/patient-centeredness
measuring quality of health
care across countries

33
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
26. The Commonwealth To achieve system goals by meeting Goal of system: to deliver effective, safe, Suggests that high quality, efficient
Fund, 2006* high quality performance well-coordinated, patient-centred care for care coupled with equitable access and
A framework informing improvement priorities long, healthy and productive lives of the system-wide innovation and
change for performance population improvement will support system
improvement by meeting goals
high quality health priorities
27. WHO, 2007 (“Building To practically organize health systems Six building blocks: service delivery, health The six building blocks converge to
Blocks”)* into six operational “building blocks” workforce, information, medical products, provide highly accessible, safe, quality
A framework for for health systems strengthening vaccines and technologies, financing, care with great coverage to
understanding how health stewardship; system goals: improved populations ultimately achieving
systems “building blocks” health, responsiveness, social and system goals
converge for system-wide
financial risk protection, improved
strengthening
efficiency
28. World Bank, 2007 To outline a new strategic vision for Four goals: improve system performance, Presents a plan of action and internal
(“Healthy the World Bank in improving its financial protection from poverty, functional adjustments for
Development”)† capacity to respond to complex health financial sustainability, governance and implementation to improve the impact
A supra-framework issues globally and with a accountability; five directions: renew of Bank interventions
informing change in the country focus focus, support client-country efforts to
World Bank’s strategic strengthen health systems, balance
approach to Health, Nutrition
systems strengthening with priority-
and Population
disease interventions and foster strategic
engagement.
29. Ramagem & Raules, To create a common performance Eleven Essential Public Health Functions Countries continuously move from
Pan American Health measurement tool that respects the (EPHF): indispensable set of actions, under performance measurement to action
Organization, 2008* organizational structure of each the primary responsibility of the state that through the framework, developing
A framework evaluating country’s health system are fundamental to achieving public interventions that effectively deliver
health systems performance health; three goals: strengthen public EPHFs and meet system goals
using 11 essential public health practice, improve capacity of the
health functions
national health authority to execute the
EPHF and develop public health
infrastructure

34
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
30. WHO, 2008 (“Primary To structure primary health care (PHC) Four broad policy areas for essential Four reform policy areas work
Healthcare”)* reforms that converge on what is changes: moving towards universal synergistically and converge around
A sub-framework informing needed for an effective response to coverage, putting people at the centre of improved health for all
change in primary health the health challenges of today’s world, service delivery, integrating health into
care service delivery the values of equity, solidarity and public policies across sectors and
social justice that drive the PHC providing inclusive leadership for health
movement and the growing governance
expectations of the population in
modernizing societies
31. International Health To help monitor and evaluate scale-up Six guiding principles: collective action, To reduce duplication and
Partnership, 2008† efforts for better health to ensure that alignment with country processes, balance fragmentation of data collection,
A framework evaluating accountability and results from single between country participation and management and reporting and to
changes in health systems donors and joint initiatives are independence, harmonized approaches to maximize country benefits and the
through increased translated into well-coordinated performance assessment, capacity quality of evaluation; five goals with
monitoring and evaluation of
efforts to monitor performance and building and health information system proposed actions and principal actors
scale-up efforts in health
evaluate progress and results in strengthening, adequate funding are presented
system reforms
country
32. Atun & Menabde, To take into account the context Four levers available to policy-makers Proposes “health system behaviour”
Imperial College, 2008 within which the health system managing the health system: stewardship focused on complex interactions
(“Systems Thinking functions, namely, the demographic, and organizational arrangements, between health systems elements and
Framework”)† economic, political, legal and financing, resource allocation and contextual factors; “systems thinking”
A supra-framework for regulatory, epidemiological, socio- provider payment systems and service perceives interrelationships and
understanding health demographic and technological provision; intermediate goals: equity, repeated events, seeing patterns of
systems in a broader social contexts (“DEPLESET”) efficiency (technical and allocative change rather than static “snapshots”
context that applies “systems
efficiency), effectiveness and choice;
thinking”
system goals: health, financial risk
protection and consumer satisfaction
33. The Global Fund, To derive a list of key components Components that are key to building a Key functions operate independently
2008* within effective health systems based well-functioning health system: effective to support effective health systems
A framework for on the Global Fund’s experience in healthcare delivery system, health
understanding the key health systems strengthening workforce, health information system,
components of well- equitable access, financing system and
functioning health systems
leadership and governance

35
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
34. Siddiqi, et al., WHO To assess health systems governance Ten principles for HSG: strategic vision, Ten principles of HSG are assessed at
Regional Office for the (HSG) at the national and sub-national participation and consensus orientation, the national and sub-national level as
Eastern Mediterranean, levels using common parameters in rule of law, transparency, responsiveness, indicators of effective governance in
2009* the Eastern Mediterranean; considers equity and inclusiveness, effectiveness health
A supra-framework role of the state vs. the market, role of and efficiency, accountability, intelligence
comparing ten principles for the ministries of health vs. other state and information and ethics
health systems governance ministries, role of actors in
assessment across countries governance, static vs. dynamic health
in the Eastern Mediterranean systems and health reform vs. human
region
rights-based approaches to health
35. Mikkelson-Lopez, et To address governance issues from a Builds on the WHO Building Blocks Traces governance throughout the
al., Geneva Health broader systems perspective across all approach, adding governance dimensions WHO Building Blocks framework,
Forum, 2010† levels of the health system to each block. Governance inputs applying checks and balances at the
A framework for (participation, strategic vision, consensus level of inputs, intermediary functions
understanding how orientation), attributes (control of and outcomes
governance parameters exist corruption, accountability, transparency)
at all levels of the WHO
and outcomes (responsiveness, equity,
Building Blocks model
efficiency) are presented
36. Savel, et al., Centers To present the public health Interconnects public health departments, Smaller discs represent “nodes” which
for Disease Control and community a robust technology regional health information organizations, are connection (access) points to
Prevention, 2010† infrastructure for secure and timely providers and federal agencies; fosters an resources (services) which are
A sub-framework informing data, information, and knowledge open collaborative effort involving the maintained (controlled) by that local
change in health exchange, not only within the public public health information entity; each node is essentially a
communications technology health domain, but between public network community, clinical partners, technology connection point that is
though a grid-based health
health and the overall health care academia and industry to provide installed within an organization or
information network
system scientific and public health rigor, partner site to share their resources
collaborative (and well-defined) and/or services with appropriate and
governance/oversight and long term authorized members of the PHGrid
return on investment

36
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
37. Van Olmen, et al., To assist in the description and Ten elements: goals and outcomes, values Relations between the elements are
Institute of Tropical analysis of any health system at and principles, service delivery, the reciprocal and interconnected
Medicine Antwerp, 2010† national, intermediate or local levels population, the context, leadership and (including feedback loops, emergent,
A framework informing for purposes of health systems governance and the organisation of generative and nonlinear processes,
change at national and sub- strengthening resources (finances, human resources, dynamic equilibriums between
national levels for health infrastructure and supplies, and operating forces); context encircles the
systems strengthening
knowledge and information system and the population touches on
all elements of the system, indicating
its omnipresence
38. Rechel, et al., To provide detailed guidelines and Template consists of nine chapters: Traces health system performance at
European Observatory on specific questions, definitions, introduction, organization and the country level across nine factors
Health Systems and suggestions for data sources, and governance, financing, physical and
Policies, 2010 (“HiT examples needed to compile HiTs human resources, provision of services,
Template”)* principal health reforms, assessment of
A framework for comparing health system, conclusions and
health system performance appendices
across countries
39. Shakarishvili, et al., To comparatively analyse donors’ Four pre-requisite factors: harmonization Traces each donor’s financial
The Global Fund to Fight contributions to strengthening specific of conceptual and operational contribution to strengthening
AIDS, Tuberculosis and aspects of countries’ health systems in understanding of what constitutes HSS, individual elements and components
Malaria, 2011* multi-donor supported funding development of a common set of criteria of country’s health system
A supra-framework environments to define health expenditures as
evaluating national health contributors to HSS, development of a
systems by tracing the common HSS classification system and
impact of donors’ health harmonization of HSS programmatic and
systems strengthening (HSS) financial data for inter-agency
expenditures
comparative analyses
40. Veillard, et al., To propose an operational approach Six stewardship functions: define the Applies the six functions of
Canadian Institute for scoping out different health system vision and strategy for better health, exert stewardship to the health
Health Information, stewardship functions and relating influence across all sectors for better sector in the context of national values
2011* them in practice to national contexts health, govern health systems in a way and socioeconomic constraints within
A supra-framework for and various health system goals that is consistent with prevailing values, which the stewardship role of national
understanding the concept of ensure system design is aligned with health ministries takes place
stewardship and its health system goals, leverage available
applications to the health legal and regulatory instruments and
sector compile, disseminate and apply
intelligence

37
 Framework Name Overview, Goals & Main Functions Health System Components Health System Interactions
41. Ergo, et al., USAID, To offer a structure that organizes Three essential components: the health The interactions within and between
2011† information and assesses how various care sector (governance and service components, sub-components,
A sub-framework evaluating health systems strengthening delivery), the community (physical and constitutive elements and MNCH
health systems approaches initiatives might cause changes that social environments) and households interventions will determine the
to maternal, neonatal and result in improved maternal, neonatal (household characteristics and individual coverage and quality of MNCH; four
child health as it relates to
and child health (MNCH) factors); four control knobs represent the control knobs analyze how health
the broader health system
types of ‘tools’ available to address systems strengthening initiatives
weaknesses in the system: financing, trigger changes in the health system,
organization, regulation and and what the impact is on MNCH
communication morbidity and mortality
1
Feldstein M, Friedman B, Luft H. 1972. Distributional aspects of national health insurance benefits and finance. National Tax Journal;497-510.
2
Yett DE, Drabek L, Intriligator MD, Kimbell LJ. 1972. Health manpower planning: an economic approach. Health Services Research;134-147.
3
Feldstein M, Friedman B. 1976. The Effect of National Health Insurance on the Price and Quantity of Medical Care. [Monograph] In: Rosett RN, ed. The Role of Health Insurance
in the Health Services Sector:505-541. New York: National Bureau of Economic Research.
4
Evans RG. 1981. Incomplete vertical integration: the distinctive structure of the health-care industry. In: van der Gaag J and Perlman M, eds. Health, economics, and health
economics. North Holland.
5
Hurst JW. 1991. Reforming health care in seven European nations. Health Affairs;10(3):7–21.
6
Roemer MI. 1991. National health systems of the world. Vol 1: The countries. Oxford: Oxford University Press.; Roemer MI. 1993. National health systems throughout the
world: lessons for health system reform in the United States. American Behavioral Scientist;36(6):694-708.
7
Frenk J. 1994. Dimensions of health system reform. Health Policy;27:19–34.
8
Cassels A. 1995. Health Sector reform: some key issues in less developed countries. Journal of International Development;7(3):329–348.
9
Londoño JL, Frenk J. 1997. Structured pluralism: towards an innovative model for health system reform in Latin America. Health Policy;41:1-36.
10
Hsiao WC, Heller PS. 1997. What Should Macroeconomists Know about Health Care Policy? IMF Working Paper:WP.07.13. International Monetary Fund.
11
Aday LA, Begley CE, Lairson DR, Slater CH, Richard AJ, et al. 1998. A framework for assessing the effectiveness, efficiency, and equity of behavioral healthcare. American
Journal of Managed Care;5:25-44.
12
Sicotte C, Champagne F, Contandriopoulos AP, et al. 1998. A conceptual framework for the analysis of health care organizations performance. Health Services Management
Research;11:24-48.
13
Anell A, Willis M. 2000. International comparison of health care systems using resource profiles. Bulletin of the World Health Organization;78(6):770-778.
14
World Health Organization. 2000. World health report 2000—health systems: improving performance. Geneva: World Health Organization.
15
Mills AJ, Ranson MK. 2001. The Design of Health Systems (Chapter 3). In: Merson M, Black R, Mills A. (Eds.) International Public Health, Diseases, Programs, Systems and
Policies. Gaithersburg MD: Aspen Publications.; Mills AJ, Ranson MK. 2006. The Design of Health Systems (Chapter 11). In: Merson M, Robert E. Black, Anne J. Mills. (Eds.)
International Public Health: Diseases, Programs, Systems and Policies (2nd ed.)., Sudbury MA: Jones and Bartlett Publishers.
16
Hurst J. Jee-Hughes M. 2001. Performance measurement and performance management in OECD health systems. OECD Labour Market and Social Policy Occasional Papers.
No.47. OECD Publishing.
17
Kutzin J. 2001. A descriptive framework for country-level analysis of health care financing arrangements. Health Policy;56:171–204.
18
Docteur E, Oxley H. 2003. Health-Care Systems: Lessons from the Reform Experience. OECD Health Working Paper: DELSA/ELSA/WD/HEA(2003)9. Paris: Organization for
Economic Co-operation and Development Publications.
19
Roberts MJ, Hsiao WC, Berman P, Reich MR. 2003. Getting Health Reform Right. New York: Oxford University Press.
20
Khaleghian P, Das Gupta M. 2004. Public Management and the Essential Public Health Functions. World Bank Policy Research Working Paper 3220. World Bank.

38
21
Anand S, Bärnighausen T. 2004. Human resources and health outcomes: cross-country econometric study. The Lancet;364:1603–09.
22
Population Health and Wellness. 2005. A Framework for Core Functions in Public Health. Resource Document. British Columbia: Ministry of Health Services.
23
Mills A, Rasheed F, Tollman S. 2006. Strengthening health systems. In: Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, et al. (Eds.) Disease Control Priorities in
Developing Countries. Washington DC: World Bank.
24
Nixon J, Ulmann P. 2006. Relationship between health expenditure and health outcomes: evidence and caveats for a causal link. European Journal of Health Economics 7:7-18.
25
Arah O, Westert G, Hurst J, Klazinga N. 2006. A conceptual framework for the OECD health care quality indicators project. International Journal for Quality in Health Care;5–13.
26
The Commonwealth Fund Commission on a High Performance Health System. 2006. Framework for a High Performance Health System for the United States.
27
World Health Organization. 2007. Strengthening health systems to improve health outcomes. Geneva: World Health Organization.
28
World Bank. 2007. Healthy development: the World Bank strategy for health, nutrition and population results. Washington DC: The World Bank.
29
Ramagem C, Raules J. 2008. The Essential Public Health Functions as a Strategy for Improving Overall Health Systems Performance: Trends and Challenges since the Public
Health in the Americas Initiative, 2000-2007. Washington DC: Area of Health Systems and Services, Pan American Health Organization.
30
World Health Organization. 2008. The World Health Report 2008: Primary Health Care – Now More Than Ever. Geneva: World Health Organization.
31
International Health Partnership (IHP). 2008. A common framework for monitoring performance and evaluation of the scale-up for better health. Briefing Note for H8.
32
Atun R, Menabde N. 2008. Health systems and systems thinking. In: Cocker R, Atun R, McKee M. (Eds.) Health Systems and the Challenge of Communicable Disease. Open
University Press.
33
The Global Fund. 2008. The Global Fund's Strategic Approach to Health System Strengthening. Background Note 4.
34
Siddiqi S, Masud TI, Nishtar S, Peters DH, Sabri B, Bile KM, Jama MA. 2009. Framework for assessing governance of the health system in developing countries: Gateway to good
governance. Health Policy;90(1):13-25.
35
Mikkelson-Lopez I, Baez-Camargo C, Wyss K, de Savigny D. 2010. Towards a new approach for assessing health systems governance. Geneva: Geneva Health Forum.
36 st
Savel T, Hall K, Lee B, McMullin V, Miles M, et al. 2010. A public health grid (PHGrid): architecture and value proposition for 21 century public health. International Journal of
Medical Informatics;79(7):523-529.
37
Van Olmen J, Criel B, Van Damme W, Marchal B, Van Belle S, Van Dormael M, et al. 2010. Analyzing health systems to make them stronger. In: Studes in Health Services
Organisation and Policy. Antwerp: ITGpress.
38
Rechel B, Thomson S, van Ginneken E. 2010. Health systems in transition: Template for authors. United Kingdom: European Observatory on Health Systems and Policies.
39
Shakarishvili G, Lansang MA, Mitta V, Bornemisza O, Blakley M, et al. 2011. Health systems strengthening: a common classification and framework for investment analysis.
Health Policy and Planning;26:316–326.
40
Veillard J, Brown A, Barıs E, Permanand G, Klazing N. 2011. National health ministries in the WHO European region: concepts, functions and assessment framework. Health
Policy;103:191–199.
41
Ergo A, Eichler R, Koblinsky M, Shah N. 2011. Strengthening Health Systems to Improve Maternal, Neonatal and Child Health Outcomes: A Framework. Washington DC: United
States Agency for International Development (USAID).

39
 Appendix 2: Brief Summaries of 41 Health System Frameworks36

1. Feldstein, et al., Harvard University, 1972

Source: Feldstein M, Friedman B, Luft H. 1972. Distributional aspects of national health insurance
benefits and finance. National Tax Journal;497—510.

The authors develop a framework to assess the distributional impact of alternative national health
insurance options. The framework considers the probabilistic character of health expenditures and the
joint importance of income and family characteristics. Authors employ a simulation method to calculate
the actuarial value of the benefits of various insurance coverage plans and different price elasticities of
demand. The framework is used to generate distributions for families of various compositions. These
distributions are combined with an analysis of the incidence of alternative financing plans.

2. Yett, et al., University of Southern California, 1972

Source: Yett DE, Drabek L, Intriligator MD, Kimbell LJ. 1972. Health manpower planning: an economic
approach. Health Services Research;134-147.

Yett et al. present macro-econometric and the micro-simulation models to evaluate policies’ ability to
efficiently allocate health manpower and related resources. The macro-economic model is oriented
toward comprehensive health planning in regional and sub-regional areas and deals with aggregate
behaviour while the micro-simulation model focuses on the individual and emphasizes health man-
power and health professions education at the national level.

Consumers, providers and manpower are linked through services, s, and labor markets, m (diamonds).
Lettered arrows represent groups of equations for demand (right-pointing) and supply (left-pointing).
Variables consist of quantities (Q) and prices (P) of services and manpower.

36
Health system frameworks were identified through an extensive purposive search of the research literature conducted
between October and December 2011 in electronic databases including Google Scholar, Science Direct, PubMed and Web of
Science. Key search terms included "health system framework," "health system model," "health system + definition," "health
system + function," "health system + processes" and "health system + components." Roughly half of the frameworks included
in our research were frequently cited in papers by others, suggesting their importance to the field. Key informants and
experts on health systems research were consulted to find additional frameworks that were not found in the search.

Frameworks marked with (*) have descriptions adapted from Shakarishvili G, Atun R, Berman P, Hsiao W, Burgess C, et al.
2009. Building on Health Systems Frameworks for Developing a Common Approach to Health Systems Strengthening.
Prepared for the World Bank, Global Fund and the GAVI Alliance Technical Workshop on Health Systems Strengthening:
Washington DC.

Frameworks marked with (**) have descriptions adapted from Papanicolas I, Smith P. 2010. EuroREACH Framework for
Health System Performance Assessment. Draft Report: EuroREACH Project.

40
3. Feldstein & Friedman, Department of Health, Education, and Welfare, 1976
Source: Feldstein M, Friedman B. 1976. The Effect of National Health Insurance on the Price and
Quantity of Medical Care. [Monograph] In: Rosett RN, ed. The Role of Health Insurance in the Health
Services Sector:505-541. New York: National Bureau of Economic Research.

Feldstein and Friedman use a micro-simulation model of the supply and price response in the markets
for hospital care and physicians' services as to estimate the effects of alternative national health
insurance policies in the United States. The authors show that an aggregate model of supply and price
response can be combined with a micro-simulation model of demand. The model emphasizes that any
analysis of the effects of alternative national health insurance plans should consider the effect of
insurance on the prices and supply of health services. The authors use an operational method to
combine stochastic micro-simulation models of household demand with aggregate supply and price
determination equations.

4. Evans, University of British Columbia, 1981 (“Actors Framework”)

Source: Evans RG. 1981. Incomplete vertical integration: the distinctive structure of the health-care
industry. In: van der Gaag J and Perlman M, eds. Health, economics, and health economics. North
Holland.

Evans interprets health systems as possessing different patterns of incomplete vertical integration
among five classes of transactors: consumer-patients (who utilize care), first-line providers (contacted
directly by consumers), second-line providers (whose output is either used by consumers under the
direction of first-line providers or supplied as intermediate products to first-line or other second-line
providers), insurers and governments (exercise or dele-gate regulatory authority). Evans sketches the
more common forms of linkages among transactors in the health care market, showing how the
observed or alleged patterns of industry performance in different systems can be traced to differences
in structure.

5. Hurst, OECD, 1991 (“Fund Flows and Payment Framework”)

Source: Hurst JW. 1991. Reforming health care in seven European nations. Health Affairs;10(3):7-21.*

Hurst and colleagues defined health systems in terms of fund flows and payment methods between
population groups and institutions. They identified seven major subsystems of financing and delivery of
health care, namely three voluntary insurance systems (private reimbursement, contract and integrated
models), three compulsory insurance- or tax-funded models (public reimbursement model, contract and
integrated models) and the direct, voluntary out-of-pocket payment model.

The following common objectives can be discerned across health systems:

1. Adequacy and equity in access: All citizens should have access to at least a basic minimum of health care,
and there should be equal treatment for equal need where services are financed publicly.
2. Income protection: Patients should be protected from payments for health care that represent
catastrophic threats to their income or wealth, and the payment for such protection should be related to
individuals’ ability to pay. This will involve at least three types of transfer: insurance (the need for care is
unpredictable); saving (the elderly use more services than the young); and income redistribution (the sick
are often the poor).

41
 3. Macroeconomic efficiency: Health expenditure should consume an appropriate fraction of gross domestic
product (GDP);
4. Microeconomic efficiency: Health outcome and consumer satisfaction should be as high as possible for
the available share of GDP spent on health services. This implies that costs should also be minimized for
the appropriate mix of health care activities.
5. Freedom of choice for consumers: Consumers should be free to choose their doctors under both public
and private insurance, and, with the advice of their doctors, they should be able to exercise some choice
over subsequent treatments and referrals to other providers.
6. Appropriate autonomy for providers: Doctors and other providers should be given the maximum freedom
compatible with the attainment of the above objectives, especially in matters of medical and
organizational innovation.

6. Roemer, University of California, 1991 (“Basic Interactions Framework”)

Source: Roemer MI. 1991. National health systems of the world. Vol 1: The countries. Oxford: Oxford
University Press.* Roemer MI. 1993. National health systems throughout the world: lessons for health
system reform in the United States. American Behavioral Scientist;36(6):694-708.

Some national level systems can also be viewed through a basic descriptive model. A classic example is
one defined by Roemer who described a health system as, “the combination of resources, organization,
financing and management that culminate in the delivery of health services to the population”. There
are five principal components to any health system:

1. Resources: Human resources (personnel), facilities (hospitals, health centres), commodities (drugs,
equipment) and knowledge.
2. Organization: One principal authority of government (at several levels), other governmental agencies with
health functions, voluntary health agencies, enterprises and a private health care market.
3. Management: Health planning, administration (supervision, coordination), regulation and legislation.
4. Economic Support: Governmental tax revenues (at different levels), social insurance (statutory), voluntary
insurance, charity and personal households. Foreign aid may apply in less developed countries.
5. Delivery of Services: Primary health care (preventative and curative), secondary care and tertiary care.

7. Frenk, Mexican Health Foundation, 1994 (“Reform Framework”)

Source: Frenk J. 1994. Dimensions of health system reform. Health Policy;27:19–34.

Abstract: This paper presents the health system as a set of relationships among five major groups of
actors: the health care providers, the population, the state as a collective mediator, the organizations
that generate resources, and the other sectors that produce services with health effects. The
relationships among providers, population, and the state form the basis for a typology of health care
modalities. The type and number of modalities present in a country make it possible to characterize its
health system. In the last part, the paper proposes that health system reform operates at four policy
levels: systemic, which deals with the institutional arrangements for regulation, financing, and delivery
of services; programmatic, which specifies the priorities of the system, by defining a universal package
of health care interventions; organizational, which is concerned with the actual production of services
by focusing on issues of quality assurance and technical efficiency; and instrumental, which generates
the institutional intelligence for improving system performance through information, research,
technological innovation, and human resource development.

42
8. Cassels, 1995
Source: Cassels A. 1995. Health Sector reform: some key issues in less developed countries. Journal of
International Development;7(3):329–348.

Key institutional components of health system:

1. The state: Government institutions responsible for the financing, regulation, purchasing and provision of
health care.
2. Service providers: In the public, private, NGO and traditional sector. Most work in some kind of
institutional setting such as a hospital, health centre or GP practice. Services include clinical care as well
as support services.
3. Resource institutions: Produce the human and material resources f o r health care-concerned with basic
and in-service training of health personnel and health-related research and development (these will
include universities, medical schools, schools of public health, R & D departments of private companies,
foundations etc).
4. Institutional purchasers: Organizations such as insurance funds, district health authorities or health
maintenance organisations which define health needs f o r discrete populations and purchase clinical and
support services from providers using a variety of contractual mechanisms
5. Other sectoral agencies: Produce health benefits indirectly as result of the goods or services they provide
(e.g. agriculture, education, housing, employment, communications, water supply).
6. Populations: People acting individually or as households can produce health benefits through individual or
collective action, lifestyle or behaviour; they are recipients of health care; they can purchase health care,
and can be employed to provide services. Individuals form or join a variety of different organisations
which aim to influence the form, content, cost or quality of services (e.g.: trade unions, political parties,
user groups, village health committees, community health councils and so forth).

Areas of health sector reform programmes:

1. Improving the performance of the civil service: Reducing staff numbers, new pay and grading schemes
(including performance related incentives and salary decompression), better job descriptions and
appraisal systems, improved financial disbursement and accounting, establishing executive agencies.
2. Decentralization: Decentralizing responsibility for the management and/or provision of health care to
local government or to agencies within the health sector. Establishing self-governing hospitals or
autonomous district boards.
3. Improving the functioning of national ministries of health: Through organizational restructuring,
improving human and financial resource management, strengthening policy and planning functions,
setting standards for health care provision and developing systems for monitoring performance, defining
national disease priorities and cost-effective clinical and public health interventions.
4. Broadening health financing options: Through the introduction of user fees, community finance, voucher
systems, social insurance schemes and private insurance.
5. Introducing managed competition: Promoting competition between providers of clinical care and/or
support services through single or multiple purchasers.
6. Working with the private sector: Establishing systems for regulating, contracting with or franchising
providers in the private sector including NGOs and for-profit organizations.

9. Londoño & Frenk, Inter-American Development Bank & Mexican Health Foundation,
1997
Source: Londoño JL, Frenk J. 1997. Structured pluralism: towards an innovative model for health system
reform in Latin America. Health Policy;41:1-36.

Londono and Frenk offer an innovative model to promote equity, quality, and efficiency. They
conceptualize health systems in terms of the relations between populations and institutions. Health

43
systems must perform four basic functions: financing, delivery, modulation (setting transparent and fair
rules of the game) and articulation (managing and organizing transactions between groups). They
propose a model of structured pluralism that organizes the health system by functions rather than social
groups. Modulation is the central mission of the ministry of health (rather than the direct provision of
personal health services). Financing is the main function of social security institutes, which is gradually
extended to protect the entire population. The articulation function would be made through the
establishment of “organizations for health services articulation,” which would perform a series of crucial
activities, including the competitive enrollment of populations into health plans in exchange for a risk-
adjusted capitation, the specification of explicit packages of benefits or interventions, the organization
of networks of providers so as to structure consumer choices, the design and implementation of
incentives to providers through payment mechanisms, and the management of quality of care. Finally,
the delivery function would be open to pluralism that would be adapted to differential needs of urban
and rural populations.

10. Hsiao & Heller, International Monetary Fund, 1997

Source: Hsiao WC, Heller PS. 1997. What Should Macroeconomists Know about Health Care
Policy? IMF Working Paper:WP.07.13. International Monetary Fund.

From report: The report focuses on how issues of health influence the macro-economy. One starts with
the basic recognition that the health status of a population is fundamentally influenced by its age
structure, its exposure to various epidemiological vectors (in part due to geographic factors), its degree
of affluence, its behavior (concerning nutrition and exposure to adverse epidemiological factors), and its
demographic characteristics (e.g., high or low fertility). This basic starting point will obviously influence
the demands placed on a country’s health care system. The health status of a population is also
influenced in part by the nature of the health policies pursued by a government—the provision of public
goods (such as immunizations and vaccinations); the quality of the regulatory policies with respect to
pharmaceuticals; and the extent of activism in the control of public “bads” (such as antidrug or tobacco
policies)—and by the quality and quantity of the medical services available to the population (whether
from the public or private sectors). How a society organizes itself in terms of the implementation of its
health policies and in the financing of the provision of health care is also likely to have a direct and
independent impact on macroeconomic variables, recognizing that the extent of impact will differ across
countries (depending on the size and relative importance of the health care sector).

11. Aday, et al., University of Texas, 1998 (“Behavioural Healthcare Framework”)

Source: Aday LA, Begley CE, Lairson DR, Slater CH, Richard AJ, et al. 1998. A framework for assessing the
effectiveness, efficiency, and equity of behavioral healthcare. American Journal of Managed Care;5:25-
44.**

The Behavioural Healthcare Framework is organized in terms of the structure, process and outcomes of
the healthcare system. Where:
- Structure refers to the availability, organization, and financing of behavioural healthcare programs; the
characteristics of the populations to be served by them; and the physical, social and economic
environment to which they are exposed
- Process refers to the transactions between patients and providers in the course of actual care delivery, as
well as the environmental and behavioural transactions exacerbating behavioural health risks

44
 - Outcomes consist of the ultimate outcome of health care services is to enhance the health of individuals
and communities, however this goal is conceptualized as an ongoing process what can be evaluated
through the intermediate outcomes of effectiveness, efficiency and equity.

12. Sicotte, et al., University of Montreal, 1998 (“Integrated Performance Framework”)

Source: Sicotte C, Champagne F, Contandriopoulos AP, Barnsley J, Béland F, Leggat SG, Denis
JL, Bilodeau H, Langley A, Brémond M, Baker GR. 1998. A conceptual framework for the analysis of
health care organizations performance. Health Services Management Research;11:24-48.**

The Integrated Performance Model for the Health Care System considers the goals and functions of the
health system in addition to other external and internal factors (e.g. socio-economic determinants and
the culture of the health system itself). The framework conceptualises health systems as organized
systems of action with four functional dimensions of action:

Two Internal Functions:

 Maintaining values: Maintaining values and producing meaning
 Production: Integrating and stabilising processes for production

Two External Functions:

 Adapting: Interacting with the environment to acquire the necessary resources and adapting
 Achieving goals: Attaining the valued goals of the system.

The goal attainment function refers to the ultimate goals which the system aims to achieve; these are
health status, responsiveness, financial fairness and efficiency. The production function represents the
processes which are undertaken in order to achieve the system goals: these are often represented
through the dimensions of accessibility, quality and technical efficiency. These two functions are present
in most existing frameworks. The adaptation function considers external influences on the system, and
how the health system adapts to these influences in order to best serve the system’s needs. Finally, the
value maintenance function considers the motivation the actors in the system have in order to maintain
and improve the health system. This includes the organizational culture, worker satisfaction etc. These
four functions can be studied independently but their interactions and trade-offs must also be
considered, allowing for a more dynamic representation of the system.

13. Anell & Willis, Swedish Institute for Health Economics, 2000
Source: Anell A, Willis M. 2000. International comparison of health care systems using resource profiles.
Bulletin of the World Health Organization;78(6):770-778.

A simple framework for comparing data underlying health care systems is presented in this article. It
distinguishes measures of real resources, for example human resources, medicines and medical
equipment, from measures of financial resources such as expenditures. Measures of real resources are
further subdivided according to whether their factor prices are determined primarily in national or
global markets. The approach is illustrated using a simple analysis of health care resource profiles for
Denmark, France, Germany, Sweden, the United Kingdom, and the USA. Comparisons based on
measures of both real resources and expenditures can be more useful than conventional comparisons of
expenditures alone and can lead to important insights for the future management of health care
systems.

45
Measured health expenditures:
 % GDP = % gross domestic product;
 Exp/cap = expenditures per capita;
 Drugs/cap = drug expenditures per capita;
 MRIs = MRI units per capita;
 CT Scanners = CT scanners per capita;
 Beds/cap = no. of hospital beds per capita;
 Emp/cap = health care employment per capita;
 Phys/cap = No. of physicians per capita;
 Nurses/cap = no. of nurses pers capita;
 % Emp = health care employment as % of total employment

14. WHO, 2000 (“Health System Performance Framework”)

Source: World Health Organization. 2000. World health report 2000—health systems: improving
performance. Geneva: World Health Organization.*

The Report defined a health system as one that includes all actors, institutions and resources whose
primary intent is to improve population health in ways that are responsive to the populations served,
and seeks to ensure a more equitable distribution of wealth across populations. It outlined four key
functions of a health system which drive the way that inputs are transformed into health system
outcomes: resource generation, financing, service provision and stewardship.

15. Mills & Ranson, London School of Hygiene and Tropical Medicine, 2001
Source: Mills AJ, Ranson MK. 2001. The Design of Health Systems (Chapter 3). In: Merson M, Black R,
Mills A. (Eds.) International Public Health, Diseases, Programs, Systems and Policies. Gaithersburg MD:
Aspen Publications. Mills AJ, Ranson MK. 2006. The Design of Health Systems (Chapter 11). In: Merson
M, Robert E. Black, Anne J. Mills. (Eds.) International Public Health: Diseases, Programs, Systems and
Policies (2nd ed.)., Sudbury MA: Jones and Bartlett Publishers.

Mills and Ranson examine previous conceptual frameworks for health to understand how health
systems work and how they can be changed in low- and middle-income countries. Governments,
populations, financing agents and providers are identified as key players in each system. The authors
examine regulation, financing, resource allocation and the provision of services when considering key
areas for health sector reform in low- and middle-income countries. Going forward, the authors propose
that reformers focus on increasing the role of the state and regulation, increasing public control over
financing, greater decentralization of management and greater involvement of the private sector in
service provision.

16. Hurst & Jee-Hughes, OECD, 2001

Source: Hurst J, Jee-Hughes M. 2001. Performance measurement and performance management in
OECD health systems. OECD Labour Market and Social Policy Occasional Papers. No.47. OECD.*

The OCED adopts a narrower definition of health system boundaries than that used by the WHO. Their
definition is limited to include only the boundaries to the performance of the health care system, not

46
encompassing public health activities or other wider issues. The set of objectives defined in the OECD
framework are based upon the WHO’s 2000 framework, but include some modifications. When defining
the health system objectives, the OECD argues that access should be a component of responsiveness,
unlike the WHO, which considers access to be a determinant of responsiveness. This allows the OCED
framework to consider questions of equity of access in its framework. The OECD framework also adds
the level of health expenditure as an objective, allowing them to the issue of desirable health spending.
This makes the three goals of the OCED framework are: health improvement and outcomes;
responsiveness and access; and financial contributions and health expenditure. For each of these goals,
there are ‘two components of assessment’, the average level and the distribution of each goal.

In order to relate health system architecture to performance, the OECD framework also includes a
dimension of efficiency in its measurement, and similarly to the WHO 2000 framework this dimension is
not an intrinsic goal as such but reflected in the attainment of the goals. However, the OECD separates
efficiency in its framework into microeconomic efficiency and macroeconomic efficiency. The
microeconomic efficiency dimension is very similar to the WHO’s efficiency concept and involves
comparing the measured productivity of a health system to its maximum attainable productivity.
Productivity is defined as the ratio of outputs to inputs (health outcome and responsiveness per dollar),
a measure of technical efficiency. Macroeconomic efficiency relates to total spending on health,
involving an examination of the benefit of health spending relative to other goods and services, a
concept of allocative efficiency. The OECD framework does not envisage rankings of health systems, and
does not require any weighting or combination of the goals.

17. Kutzin, WHO Regional Office for Europe, 2001

Source: Kutzin J. 2001. A descriptive framework for country-level analysis of health care financing
arrangements. Health Policy;56:171–204.

Abstract: Health financing policies are marked by confusion between policy tools and policy objectives,
especially in low and middle income countries. This paper attempts to address this problem by providing
a conceptual framework that is driven by the normative objective of enhancing the ‘insurance function’
(access to needed care without financial impoverishment) of health care systems. The framework is
proposed as a tool for descriptive analysis of the key functions, policies, and interactions within an
existing health care system, and equally as a tool to assist the identification and preliminary assessment
of policy options. The aim is to help to clarify the policy levers that are available to enhance the
insurance function for the population as efficiently as possible, given the ‘starting point’ of a country’s
existing institutional and organizational arrangements. Analysis of health care financing systems using
this framework highlights the interactions of various policies and the need for a coherent package of
coordinated reforms, rather than a focus on particular organizational forms of ‘health insurance’. The
content of each main health care system function (revenue collection, pooling of funds, purchasing of
services, provision of services) and the market structure with which the implementation of each is
organized are found to be particularly important, as are policies with respect to the benefit package and
user fees.

47
18. Docteur & Oxley, OECD, 2003
Source: Docteur E, Oxley H. 2003. Health-Care Systems: Lessons from the Reform Experience. OECD
Health Working Paper: DELSA/ELSA/WD/HEA(2003)9. Paris: Organization for Economic Co-operation and
Development Publications.

The report aims to give policymakers a better understanding of the state of reforms across OECD
countries and to inform them of policy orientations that may potentially have greater payoffs. Reforms
are assessed according to their impact on the following policy goals in OECD countries: ensuring access
to needed health-care services; improving the quality of health care and its outcomes; allocating an
“appropriate” level of public sector and economy-wide resources to health care (macroeconomic
efficiency); and ensuring that services are provided in a cost-efficient and cost-effective manner
(microeconomic efficiency). To improve access to care and health outcomes, Docteur and Oxley offer
recommendations such as:
- Assuring universal and comprehensive health insurance coverage
- Ensuring adequate and equitable access to needed health services
- Increasing the effectiveness of health systems by initiating focused public health programmes,
establishing new health-care delivery arrangements and supporting public reporting of information on
health-care quality

19. Roberts, et al., Harvard University, 2003 (“Control Knobs Framework”)

Source: Roberts MJ, Hsiao WC, Berman P, Reich MR. 2003. Getting Health Reform Right. New York:
Oxford University Press.*

Roberts, Hsiao, Berman, and Reich (2003) conceptualized a health system as “a set of relationships
where the structural components (means) and their interactions are associated and connected to the
goals the system desires to achieve (ends)”. The framework identifies five major “control knobs” of a
health system which policymakers can use to achieve health system goals: financing, macro-
organization, payment, regulation and education/persuasion. This framework has been used as the
basis for the World Bank Institutes Flagship Program on Health Sector Reform and
Sustainable Financing, now renamed Health System Strengthening.

The institutional drivers underpinning the control knobs framework:

1. Financing: Who pays and who benefits from health care, as well as generating funding for the system as a
whole;
2. Payment: The ways in which money is transferred to health care providers, creating financial incentives
influencing how they behave;
3. Regulation: The use of state coercion to control the behaviour of other actors within the system;
4. Organization: The incentives for the organization; and the incentives, authority, skills and attitudes of
both managers and workers; and
5. Behaviour: Information provision and marketing, incentives and coercion shaping how patients and
providers act in relation to health and health care (addressing treatment seeking behaviours, health
professional behaviours, and patient compliance, lifestyle and prevention behaviours).

48
20. Khaleghian & Das Gupta, World Bank, 2004
Source: Khaleghian P, Das Gupta M. 2004. Public Management and the Essential Public Health Functions.
World Bank Policy Research Working Paper 3220. World Bank.

The authors provide an overview of how different approaches to improving public sector management
relate to so-called core or essential public health functions (EPHFs) such as disease surveillance, health
education, monitoring and evaluation, workforce development, enforcement of public health laws and
regulations, public health research, and health policy development. Key lessons from their examination
propose the following considerations for health sector reform:
 User fees are not an option for the EPHFs because of their public goods characteristics
 Promoting competition among agencies responsible for public health functions does not improve
efficiency; on the contrary, it impedes collaboration and technical assistance and can therefore
compromise the effectiveness of activities such as surveillance and health promotion
 Managerial autonomy is important for the EPHFs as a way of promoting adaptation and innovation
 Decentralizing the EPHFs is a risky strategy, since local governments have little incentive to invest in public
goods and systematically neglect them.
 Public sector norms and rules (the institutional environment) that impede effective administration should
be changed where possible
 Strengthening hierarchical accountability within the public health system is essential to strengthening the
EPHFs

21. Anand & Bärnighausen, University of Oxford and Harvard University, 2004
Source: Anand S, Bärnighausen T. 2004. Human resources and health outcomes: cross-country
econometric study. The Lancet;364:1603–09.

The authors investigate the link between human resources for health and health outcomes by
conducting a cross-country multiple regression analyses with maternal mortality rate, infant mortality
rate, and under-five mortality rate as dependent variables. Aggregate density of human resources for
health was an independent variable in one set of regressions; doctor and nurse densities separately
were used in another set. Authors controlled for the effects of income, female adult literacy, and
absolute income poverty. Anand and Bärnighausen conclude that the density of human resources for
health is important in accounting for the variation in rates of maternal mortality, infant mortality, and
under-five mortality across countries.

22. Population Health and Wellness, British Columbia Ministry of Health Services, 2005
Source: Population Health and Wellness. 2005. A Framework for Core Functions in Public Health.
Resource Document. British Columbia: Ministry of Health Services.

From Report: A Framework for Core Functions in Public Health is part of this public health renewal. This
document provides a framework to help strengthen public health and improve population health in
British Columbia. It is the intent of the Ministry of Health Services that Core Functions in Public Health
(Core Functions) will identify the key set of public health services that health authorities will provide and
will strengthen the link between public health, primary care, and chronic disease management.

 Core programs: Long-term programs, representing the minimum level of public health services
that health authorities would provide in a renewed and modern public health system. Core

49
 programs are organized to improve health; they can be assessed ultimately in terms of
improved health and well-being and/or reductions in disease, disability, and injury.
 Public health strategies: strategies by which core programs are implemented, no matter what
the intended health outcome, e.g. health promotion.
 Lenses: the Population Lens and the Inequalities Lens are in place to ensure the health needs of
specific populations are addressed.
 System capacity: The health information systems, quality management, research and knowledge
development, and staff training and development capacity needed to apply public health
strategies and implement core programs.

23. Mills, et al., World Bank, 2006

Source: Mills A, Rasheed F, Tollman S. 2006. Strengthening health systems. In: Jamison DT, Breman
JG, Measham AR, Alleyne G, Claeson M, et al. (Eds.) Disease Control Priorities in Developing Countries.
Washington DC: World Bank.

The authors review how health systems can be strengthened in differing country contexts to deliver
interventions effectively, efficiently, and equitably. The chapter is mainly concerned with strengthening
health services and looks to disease-specific and health system responses to common constraints
experienced in less developed countries. Six key points can be identified in relation to improving health
systems:
1. Health systems are the basis for the long-term future of sustained health improvements. The
health of the system must be carefully considered whenever major programs are put in place.
2. If capacity constraints are such that a focused disease- or program-specific effort is desirable to
address an urgent problem, the effort should be designed to contribute to the long-term system
strengthening, rather than detracting from it. Countries must avoid having multiple vertical
programs competing for limited human resources and managerial capacity.
3. Reforms affecting organizational structures and human resource management are likely to play
an important role in improved performance. However, emerging evidence suggests in most
settings that changes are most likely to be successfully implemented if they are incremental and
gradual rather than "big bang" reforms.
4. Linking financial incentives to performance, whether through contracts with health care
providers or through performance-related pay, may bring rewards if careful monitoring is
possible; however, evidence on the sustainability of such arrangements is lacking, and effective
monitoring may require long-term external involvement.
5. Organizational reforms must keep the goal of improved health outcomes, equity, and
responsiveness in sight. Special attention to users' demands, to primary care and first-level
hospitals, to quality of care, and to technical backup for disease control programs is required.
6. Capacity-strengthening efforts in most settings must encompass action at all levels, from
increasing leadership of the ministry of health at the national level through strengthening
support for peripheral levels.

50
24. Nixon & Ulmann, University of York, 2006
Source: Nixon J, Ulmann P. 2006. The relationship between health expenditure and health outcomes:
evidence and caveats for a causal link. European Journal of Health Economics;7:7-18.

Abstract: The relationship between health care expenditure and health outcomes is of interest to
policymakers in light of stead increases in health care spending for most industrialized countries. This
study reviews key findings and methodological approaches in this field and reports the results of our
own empirical study of countries of the European Union. Our analysis examines life expectancy and
infant mortality as the ‘output’ of the health care system, and various life-style, environmental and
occupational factors as ‘inputs’. Econometric analyses using a fixed effects mod el are conducted on a
panel data set for 15 members of the European Union over the period 1980–1995. The findings show
that increases in health care expenditure are significantly associated with large improvements in infant
mortality but only marginally in relation to life expectancy.

25. Arah, et al., University of Amsterdam, 2006

Source: Arah O, Westert G, Hurst J, Klazinga N. 2006. A conceptual framework for the OECD health care
quality indicators project. International Journal for Quality in Health Care;5–13.**

The HCQI project was initiated in 2001 with the long-term objective of developing a set of indicators
that could be used to investigate quality of health care across countries using comparable data. In 2006,
Arah and colleagues published the conceptual framework which defined ‘quality of health care’, placing
it within a wider performance framework that acknowledged the key healthy policy goals adopted by
the OECD and its member states. The authors adhere to the WHO definition of a health system in terms
of health actions, and define ‘heath care’ as the combined functioning of public health and personal
health care services.

Their ‘health system’ framework, thus considers not only heath care but the other activities that have a
primary purpose of promoting, restoring or maintaining health. This framework has four interconnected
tiers (connected in a fashion that denotes potential causal pathways) representing:
1. Health: this tier denotes society’s broader health as in influenced by health care and non-health care
factors;
2. Non-health care determinants of health: This tier denotes the mostly society-wide, non-heath care factors
that also influence health;
3. Health care system performance: The tier denotes the processes, inputs, and outcome of the health care
system as well as its efficiency and equity, recognizing that these may sometimes influence health care
determinants. Note that the link between the third tier and the second is captured by primary care
/prevention and health promotion;
4. Health system design and context: This denotes pertinent country and health system policy and delivery
characteristics, which will influence the health system in terms of its costs, expenditure and utilization
patters that must be considered in order to contextualise the findings of the health performance tier.

Within this health system framework, a certain section of the health care system performance tier
denotes the core quality dimensions to be measured in the HCQI project, effectiveness, safety and
responsiveness/patient-centeredness.

51
26. Commonwealth Fund, 2006
Source: Commonwealth Fund Commission on a High Performance Health System. 2006. Framework for
a High Performance Health System for the United States. New York: Commonwealth Fund.

Four Goals of a High Performance Health Care System High Quality Safe Care:
1. Patients get health care that is known to be effective – as needed for treatment, prevention or palliation.
2. Health care provided is safe, delivered in a manner that achieves higher reliability in care processes and
minimizes medical errors
3. Health care is coordinated over time.
4. Care is patient-centered; provided in a timely way with compassion, effective communication, and
excellent services. Patients are informed and active participants of their care.

Access to care for all people:

 There is universal participation.
 Everyone has available to them a minimum level of financial protection, as well as established benefits.
 Care is affordable, from the patient’s and the nation’s perspective.
 Care is provided equitably according to medical need, regardless of race/ethnicity, insurance status,
income, age, sex, or geographical location.
Efficient, high value care:
 Care delivery and insurance administration are efficient.
 Care is delivered at the right time and in the right setting
 There is a system whereby new technologies, devices, producers, laboratory testing, and pharmaceuticals
can be evaluated for both effectiveness and value, including defined processes for their introduction,
surveillance, retesting and re-evaluation over time.

System capacity to improve:

 There is significant investment in innovation and research.
 There is an interoperable information infrastructure that supports integration and continuity of care,
transparency of information on the price and quality of care, and accountability.
 The educational system adequately prepares the next generation of health care providers and leaders,
and the nation develops a stable, competent workforce committed to providing ill Americans with
patient-centred, high quality care.
 The health system responds quickly, at both the individual and population levels, to major health threats
and disasters.
 There is a culture of improvement and professional satisfaction among health care professionals.
 There is an appropriate balance between autonomy and accountability.

27. WHO, 2007 (“Building Blocks Framework”)

Source: World Health Organization. 2007. Strengthening health systems to improve health outcomes.
Geneva: World Health Organization.*

Another important contribution to technical debates on health systems frameworks from WHO was the
2007 report “Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes” which
proposed practical ways to organize health systems into 6 operational “building blocks”: service
delivery, health workforce, information, medical products and technologies, financing, and leadership
and governance. The building blocks (as described by WHO):
1. Service delivery: Good health services deliver effective, safe, quality personal and non-personal health
interventions to those who need them, when and where needed, with a minimum waste of resources.

52
 2. Health workforce: A well-performing health workforce is one which works in ways that are responsive, fair
and efficient to achieve the best health outcomes possible, given available resources and circumstances.
3. Information: A well-functioning information system is one that ensures the production, analysis,
dissemination and use of reliable and timely information on health determinants, health systems
performance and health status.
4. Medical products, vaccines and technologies: A well-functioning health system ensures equitable access
to essential medical products, vaccines and technologies of assured quality, safety, efficacy and cost-
effectiveness, and their scientifically sound and cost-effective use.
5. Financing: A good health financing system raises adequate funds for health, in ways that ensure people
can use needed services, and are protected from financial catastrophe or impoverishment associated with
having to pay for them.
6. Leadership and governance (Stewardship): Leadership and governance involves ensuring strategic policy
frameworks exist and are combined with effective oversight, coalition-building, the provision of
appropriate regulation and incentives, attention to system design, and accountability.

28. World Bank, 2007 (“Healthy Development”)

Source: World Bank. 2007. Healthy development: the World Bank strategy for health, nutrition and
population results. Washington DC: The World Bank.*

The World Bank Strategy for Health, Nutrition, and Population (HNP) Results defined health systems in
terms of functionality. To contribute to improving life and health conditions of the poor and the
vulnerable, the Bank will focus on client-country efforts to achieve results in four areas or Bank Strategic
Objectives for HNP functionality:
1. Improve the level and distribution of key HNP outcomes, outputs, and system performance at country and
global levels in order to improve living conditions, particularly for the poor and the vulnerable.
2. Prevent poverty due to illness (by improving financial protection).
3. Improve financial sustainability in the HNP sector and its contribution to sound macroeconomic and fiscal
policy and to country competitiveness (revenue collection, risk pooling, and strategic purchasing).
4. Improve governance, accountability, and transparency in the health sector (oversight).

The Bank’s concept of stewardship resembles that of the WHO, in that it involves establishing the policy
framework to govern the entire health system; the institutional framework in which the many actors in
health must interact; coordination with non-health sectors; and the generation of data for decision-
making.

Five new Strategic Directions are specified to improve Bank capacity to assist client countries in
achieving the HNP Strategic Objectives in the coming decade:
1. Renew Bank focus on HNP results.
2. Increase the Bank contribution to client-country efforts to strengthen and realize well-organized and
sustainable health systems for HNP results.
3. Ensure synergy between health system strengthening and priority-disease interventions, particularly in
LICs.
4. Strengthen Bank capacity to advise client countries on an intersectoral approach to HNP results.
5. Increase selectivity, improve strategic engagement, and reach agreement with global partners on
collaborative division of labor for the benefit of client countries.

53
29. Ramagem & Raules, Pan American Health Organization, 2008
Source: Ramagem C, Raules J. 2008. The Essential Public Health Functions as a Strategy for Improving
Overall Health Systems Performance: Trends and Challenges since the Public Health in the Americas
Initiative, 2000-2007. Washington DC: Area of Health Systems and Services, Pan American Health
Organization.

From Report: The Pan American Health Organization/World Health Organization (PAHO/WHO) defines
the Essential Public Health Functions (EPHF) as the indispensable set of actions, under the primary
responsibility of the state, that are fundamental for achieving the goal of public health which is to
improve, promote, protect, and restore the health of the population through collective action. The EPHF
performance measurement instrument offers a common framework for measuring EPHF performance
while respecting the organizational structure of each country’s health system. Countries are encouraged
to go from measurement to action through the development of interventions with the goal of: (i)
strengthening public health practice; (ii) improving the steering role capacity of the national health
authority to execute the EPHF; and (iii) developing public health infrastructure. The framework shows
the relationship between the EPHF measurement, the objectives to be pursued and the intervention
areas.

30. WHO, 2008 (“Primary Healthcare Framework”)

Source: World Health Organization. 2008. The World Health Report 2008: Primary Health Care – Now
More Than Ever. Geneva: World Health Organization.*

Identified four broad policy areas for essential changes:

1. Dealing with health inequalities by moving towards universal coverage,
2. Putting people at the centre of service delivery,
3. Integrating health into public policies across sectors, and
4. Providing inclusive leadership for health governance.

From Report: This report structures the PHC reforms in four groups that reflect the convergence
between the evidence on what is needed for an effective response t o t he health challenges of today’s
world, the values of equity, solidarity and social justice that drive the PHC movement, and the growing
expectations of the population in modernizing societies.
 Reforms that ensure that health systems contribute to health equity, social justice and the end of
exclusion, primarily by moving towards universal access and social health protection – universal coverage
reforms;
 Reforms that reorganize health services as primary care, i.e. around people’s needs and expectations, so
as to make them more socially relevant and more responsive to the changing world while producing
better outcomes – service delivery reforms;
 Reforms that secure healthier communities, by integrating public health actions with primary care and by
pursuing healthy public policies across sectors – public policy reforms;
 Reforms that replace disproportionate reliance on command and control on one hand, and laissez-faire
disengagement of the state on the other, by the inclusive, participatory, negotiation-based leadership
required by the complexity of contemporary health systems—leadership reforms.

54
31. International Health Partnership, 2008
Source: International Health Partnership (IHP). 2008. A common framework for monitoring performance
and evaluation of the scale-up for better health. Briefing Note for H8.

To help monitor and evaluate scale-up efforts for better health to ensure that accountability and results
from single donors and joint initiatives are translated into well-coordinated efforts to monitor
performance and evaluate progress and results in country. The framework for evaluation of the scale-up
in the spirit of the Paris declaration can be translated into the following six principles:

1. Collective action: The primary focus should be on the contribution of the collective efforts to scale-up the
health sector response in countries.
2. Alignment with country processes: Monitoring performance and evaluation should build upon national
processes that countries have established to evaluate and review progress in the implementation of
national health sector plans.
3. Balance between country participation and independence: Evaluation processes should be driven by
country needs but conducted in a manner which maintains their independence.
4. Harmonised approaches to performance assessment: Evaluations of the scale-up should use common
protocols and standardized outcome indicators and measurement tools, with appropriate country
adaptations and leadership, minimizing the separate evaluation efforts of individual initiatives, grants and
programmes.
5. Capacity building and health information system strengthening: Systematic involvement of country
institutions in performance monitoring and evaluation is necessary to strengthen health information
systems and promote local capacity for analysis and application of information and evidence.
6. Adequate funding: As a general guide between 5% and 10% of the overall scale-up funds need to be set
aside for monitoring performance, evaluation, operational research and strengthening health
information system

The IHP’s emphasis is on how to map the monitoring and evaluation actions to the framework, rather
than to define the boundaries, functions, goals or domains. To implement the framework in 2008
several key issues need to be addressed, to reduce duplication and fragmentation of data collection,
management and reporting and to maximize country benefits and the quality of evaluation. This
requires coordination and collaboration among the major partners, both at global and country levels.

32. Atun & Menabde, Imperial College, 2008 (“Systems Thinking Framework”)
Source: Atun R & Menabde N. 2008. Health systems and systems thinking. In: Cocker R, Atun R, McKee
M. (Eds.) Health Systems and the Challenge of Communicable Disease. Open University Press.**

In the “systems thinking” approach to health systems analysis, Atun (2008) further expanded other
health system frameworks to take into account the context within which the health system functions,
namely, the demographic, economic, political, legal and regulatory, epidemiological, socio-demographic
and technological contexts (“DEPLESET”). He also introduced the concept of “health system behaviour”
and focused on complex interactions between health systems elements and between these and
contextual factors. He proposed “systems thinking for seeing the whole” - a framework for seeing
interrelationships and repeated events rather than things, for seeing patterns of change rather than
static “snapshots”. The systems framework identified four levers available to policy-makers when
managing the health system: stewardship and organizational arrangements, financing, resource
allocation & provider payment systems, and service provision. The intermediate goals identified in the
framework (equity, efficiency (technical and allocative efficiency), effectiveness and choice) are

55
frequently cited in other frameworks, sometimes as end goals in themselves. The Systems framework
has been extended to develop a Systemic Rapid Assessment (SYSRA) toolkit which allows simultaneous
and systematic examination of the broad context, the health care system and the features of health
programs (such as communicable disease control programs).

33. Global Fund, 2008

Source: Global Fund. 2008. The Global Fund's Strategic Approach to Health System Strengthening.
Background Note 4.

The Global Fund’s experience in supporting health systems strengthening and recent independent
research highlighted several components that are key to building a well-functioning health system
capable to effectively address priority health objectives. These include, but are not limited to:
 An effective healthcare delivery system, capable to efficiently deliver high quality personal and
public health services to those who need them;
 Easy access to a skilled, motivated and supported health workforce that is responsive, fair and
efficient in achieving the best health outcomes possible, given available resources and
circumstances;
 A well-functioning health information system that ensures the production, analysis,
dissemination and use of reliable and timely information on critical health determinants, health
systems performance and health status;
 A well-functioning procurement, supply chain management and logistics system for providing
equitable access to quality medical products and technologies;
 A strong health financing system to raise and equitably distribute adequate funds for health,
and to ensure populations’ protection from health-related financial risks;
 An effective leadership and governance system to ensure that strategic policy frameworks exist
which enable and support effective oversight, coalition-building, identification of areas of
responsibility and development of appropriate regulations, incentives and accountability
mechanisms.

34. Siddiqi, et al., WHO Regional Office for the Eastern Mediterranean, 2009
Source: Siddiqi S, Masud TI, Nishtar S, Peters DH, Sabri B, Bile KM, Jama MA. 2009. Framework for
assessing governance of the health system in developing countries: Gateway to good governance.
Health Policy;90(1):13-25.

Governance of the health system is the least well-understood aspect of health systems. A framework for
assessing health system governance (HSG) at national and sub-national levels is presented, which has
been applied in countries of the Eastern Mediterranean. In developing the HSG framework key issues
considered included the role of the state vs. the market; role of the ministries of health vs. other state
ministries; role of actors in governance; static vs. dynamic health systems; and health reform vs. human
rights-based approach to health. Four existing frameworks were considered: World Health
Organization’s (WHO) domains of stewardship; Pan American Health Organization’s (PAHO) essential
public health functions; World Bank’s six basic aspects of governance; and United Nations Development
Programme (UNDP) principles of good governance. The proposed HSG assessment framework includes
the following 10 principles:

56
 1. Strategic vision: Leaders have a broad and long-term perspective on health and human development,
along with a sense of strategic directions for such development. There is also an understanding of the
historical, cultural and social complexities in which that perspective is grounded
2. Participation and consensus orientation: All men and women should have a voice in decision-making for
health, either directly or through legitimate intermediate institutions that represent their interests. Such
broad participation is built on freedom of association and speech, as well as capacities to participate
constructively. Good governance of the health system mediates differing interests to reach a broad
consensus on what is in the best interests of the group and, where possible, on health policies and
procedures
3. Rule of law: Legal frameworks pertaining to health should be fair and enforced impartially, particularly the
laws on human rights related to health
4. Transparency: Transparency is built on the free flow of information for all health matters. Processes,
institutions and information should be directly accessible to those concerned with them, and enough
information is provided to understand and monitor health matters
5. Responsiveness: Institutions and processes should try to serve all stakeholders to ensure that the policies
and programs are responsive to the health and non-health needs of its users
6. Equity and inclusiveness: All men and women should have opportunities to improve or maintain their
health and well-being.
7. Effectiveness and efficiency: Processes and institutions should produce results that meet population
needs and influence health outcomes while making the best use of resources
8. Accountability: Decision-makers in government, the private sector and civil society organizations involved
in health are accountable to the public, as well as to institutional stakeholders. This accountability differs
depending on the organization and whether the decision is internal or external to an organization
9. Intelligence and information: Intelligence and information are essential for a good understanding of
health system, without which it is not possible to provide evidence for informed decisions that influences
the behaviour of different interest groups that support, or at least do not conflict with, the strategic vision
for health
10. Ethics: The commonly accepted principles of health care ethics include respect for autonomy, non-
maleficence, beneficence and justice. Health care ethics, which includes ethics in health research, is
important to safeguard the interest and the rights of the patients

35. Mikkelson-Lopez, et al., Geneva Health Forum, 2010

Source: Mikkelson-Lopez I, Baez-Camargo C, Wyss K, de Savigny D. 2010. Towards a new approach for
assessing health systems governance. Geneva: Geneva Health Forum.

There is a need to address governance from a broader systems perspective across all levels of the
system. Starting with the WHO Building Blocks framework, the authors observe that governance is
broadly applicable to all building blocks.

36. Savel, et al., Centers for Disease Control and Prevention, 2010
Source: Savel T, Hall K, Lee B, McMullin V, Miles M, et al. 2010. A public health grid (PHGrid):
architecture and value proposition for 21st century public health. International Journal of Medical
Informatics;79(7):523-529.

Abstract:
Purpose: This manuscript describes the value of and proposal for a high-level architectural framework
for a Public Health Grid (PHGrid), which the authors feel has the capability to afford the public health
community a robust technology infrastructure for secure and timely data, information, and knowledge

57
exchange, not only within the public health domain, but between public health and the overall health
care system.
Methods: The CDC facilitated multiple Proof-of-Concept (PoC) projects, leveraging an open-source-
based software development methodology, to test four hypotheses with regard to this high-level
framework. The outcomes of the four PoCs in combination with the use of the Federal Enterprise
Architecture Framework (FEAF) and the newly emerging Federal Segment Architecture Methodology
(FSAM) was used to develop and refine a high-level architectural framework for a Public Health Grid
infrastructure.
Results: The authors were successful in documenting a robust high-level architectural framework for a
PHGrid. The documentation generated provided a level of granularity needed to validate the proposal,
and included examples of both information standards and services to be implemented. Both the results
of the PoCs as well as feedback from selected public health partners were used to develop the granular
documentation.
Conclusions: A robust high-level cohesive architectural framework for a Public Health Grid (PHGrid) has
been successfully articulated, with its feasibility demonstrated via multiple PoCs. In order to successfully
implement this framework for a Public Health Grid, the authors recommend moving forward with a
three-pronged approach focusing on interoperability and standards, streamlining the PHGrid
infrastructure, and developing robust and high-impact public health services.

37. Van Olmen, et al., Institute of Tropical Medicine Antwerp, 2010

Source: Van Olmen J, Criel B, Van Damme W, Marchal B, Van Belle S, Van Dormael M, et al. 2010.
Analyzing health systems to make them stronger. In: Studes in Health Services Organisation and Policy.
Antwerp: ITGpress.

The framework presented is developed for the analysis of any health system at national, intermediate or
local levels. Furthermore, it can be loaded with specific values and principles so that it becomes
normative. As such, it can contribute to the development of strategies for action. Ten elements or
functions are identified as essential and constitutive of any health system: 1) goals & outcomes; 2)
values & principles; 3) service delivery; 4) the population; 5) the context 6) leadership & governance;
and 7-10) the organisation of resources (finances, human resources, infrastructure & supplies,
knowledge & information). The arrows in the framework indicate that the relations between the
elements are reciprocal and interconnected. The context encircles the HS, able to influence whatever
part of the HS. And the population touches on all elements of the system, indicating its omnipresence.
Indeed, HSs are complex adaptive systems. This implies interdependence and interaction between its
elements, including feedback loops, emergent, generative and nonlinear processes, leading to dynamic
equilibriums between operating forces and to sometimes or partly unpredictable results.

38. Rechel, et al., European Observatory on Health Systems and Policies, 2010 (“Health in
Transition (HiT) Template”)
Source: Rechel B, Thomson S, van Ginneken E. 2010. Health systems in transition: Template for authors.
United Kingdom: European Observatory on Health Systems and Policies.

The Health Systems in Transition (HiT) profiles are produced by country experts in collaboration with the
Observatory’s research directors and staff. The profiles are based on a template that, revised
periodically, provides detailed guidelines and specific questions, definitions, suggestions for data

58
sources, and examples needed to compile HiTs. The HiT template was revised in 2010 and consists of 9
chapters:

1. Introduction: Outlines the broader context of the health system, including geography and socio-
demography, economic and political context, and population health.
2. Organization and governance: Provides an overview of how the health system in the country is
organized and outlines the main actors and their decision-making powers; discusses the
historical background for the system; regulation; and describes the level of patient
empowerment in the areas of information, rights, choice, complaints procedures, safety and
involvement.
3. Financing: Provides information on the level of expenditure, who is covered, what benefits are
covered, the sources of health care finance, how resources are pooled and allocated, the main
areas of expenditure, and how providers are paid.
4. Physical and human resources: Deals with the planning and distribution of infrastructure and
capital stock; the context in which IT systems operate; and human resource input into the health
system, including information on registration, training, trends and career paths.
5. Provision of services: Concentrates on patient flows, organization and delivery of services,
addressing public health, primary and secondary health care, emergency and day care,
rehabilitation, pharmaceutical care, long-term care, services for informal carers, palliative care,
mental health care, dental care, complementary and alternative medicine, and health care for
specific populations.
6. Principal health reforms: Reviews reforms, policies and organizational changes that have had a
substantial impact on health care, as well as future developments.
7. Assessment of the health system: Provides an assessment based on the stated objectives of the
health system, financial protection and equity in financing; user experience and equity of access
to health care; health outcomes, health service outcomes and quality of care; health system
efficiency; and transparency and accountability.
8. Conclusions: Highlights the lessons learned from health system changes; summarizes remaining
challenges and future prospects.
9. Appendices: Includes references, further reading and useful web sites.

39. Shakarishvili, et al., The Global Fund to Fight AIDS, Tuberculosis and Malaria, 2011
Source: Shakarishvili G, Lansang MA, Mitta V, Bornemisza O, Blakley M, Kley N, Burgess C, Atun R. 2011.
Health systems strengthening: a common classification and framework for investment analysis. Health
Policy and Planning;26:316–326.

Significant scale-up of donors’ investments in health systems strengthening (HSS), and the increased
application of harmonization mechanisms for jointly channelling donor resources in countries,
necessitate the development of a common framework for tracking donors’ HSS expenditures. Such a
framework would make it possible to comparatively analyse donors’ contributions to strengthening
specific aspects of countries’ health systems in multi-donor supported HSS environments. Four pre-
requisite factors are required for developing such a framework: (i) harmonization of conceptual and
operational understanding of what constitutes HSS; (ii) development of a common set of criteria to
define health expenditures as contributors to HSS; (iii) development of a common HSS classification
system; and (iv) harmonization of HSS programmatic and financial data to allow for inter-agency
comparative analyses. Building on the analysis of these aspects, the paper proposes a framework for

59
tracking donors’ investments in HSS, as a departure point for further discussions aimed at developing a
commonly agreed approach.

40. Veillard, et al., Canadian Institute for Health Information, 2011

Source: Veillard J, Brown A, Barıs E, Permanand G, Klazing N. 2011. National health ministries in the
WHO European region: concepts, functions and assessment framework. Health Policy;103:191–199.

The operational framework relates six functions of stewardship with national contexts, values and
ultimate goals pursued by health systems: to define the vision for health and strategy to achieve better
health; to exert influence across all sectors for better health; to govern the health system in a way that
is consistent with prevailing values; to ensure that system design is aligned with health system goals; to
better leverage available legal and regulatory instruments; and to compile, disseminate and apply
intelligence. The framework aims to clarify the scope of functions that can be exercised by national
health ministries to achieve health system ultimate goals within the boundaries of stewardship. The
extent of use of these functions will depend on both context and goals such as stated in the national
strategies and policies. Furthermore, the framework proposes strategies to evaluate the completeness
and consistency of the stewardship of national health ministries, in relation to the goals set and within
the context the steward operates; and health system performance assessment as a tool to measure the
achievement of health system ultimate goals.

41. Ergo, et al., USAID, 2011

Source: Ergo A, Eichler R, Koblinsky M, Shah N. 2011. Strengthening Health Systems to Improve
Maternal, Neonatal and Child Health Outcomes: A Framework. Washington DC: United States Agency for
International Development (USAID).

Ergo et al.’s framework is based on three essential components (shown as boxes in their framework) of
any health system:
1. The health care sector, comprising two sub-components: enabling environment and
governance; and service delivery
2. The community, with the sub-components physical environment and social environment
3. The households, which consists of household characteristics and individual factors

Each of these components and sub-components comprises various interconnected elements of the
health system. The sub-component enabling environment and governance under the health care sector
component, for example, includes the following health system elements: leadership; policies and
regulations; financing; and provider payment. MNCH interventions are implemented within the health
system. Even though some of the efforts may focus on only a limited number of elements within the
health system, it is ultimately the system as a whole—i.e., the combination of the different components
and subcomponents, and all the interactions within and between them—that will determine the
coverage and quality of MNCH interventions, and therefore the impact on maternal, neonatal and child
mortality and morbidity. This is shown at the bottom of the framework.
Finally, the four control knobs at the top of the framework allow breaking down HSS initiatives and
analyzing how these initiatives trigger changes in the health system, whether and how these changes
affect the coverage and quality of MNCH interventions, and what the impact is on MNCH morbidity and
mortality. The control knobs represent the types of ‘tools’ available to the different actors—including

60
but not limited to the policymakers—to address weaknesses in the system. These are: financing,
organization, regulation and communication. Note that an HSS initiative could very well consist of a
combination of several of these tools.

61
 Appendix 3: Institutional Partners of the Alliance for Health Policy & Systems Research

Africa Region (AFRO)

Research Center for Applied Economics for Development, Algeria
UER de Santé au Travail et Environnement, Benin
Action pour L'Enfance et la Santé, Burkina Faso
Centre de Recherche en Santé de Nouna, Burkina Faso
Centre MURAZ, Burkina Faso
Institut de Recherche en Sciences de las Santé (IRSS), Biomedical and Public Health, Burkina Faso
Institut National de Santé Publique, Burundi
Ministère de la Santé Publique, Burundi
Ministry of Health, Burundi
University of Burundi, Burundi
OCEAC, Cameroon
Université des Montagnes-Bangangte, Cameroon
Leadership for Environment and Development (LEAD), Côte d'Ivoire
Egyptian Alliance for Health System Support , Egypt
Salem for Health Research Consultants, Egypt
African Medical and Research Foundation (AMREF), Ethiopia
Ethiopian Health & Nutrition Research Institute, Ethiopia
Africa Health Research Organization, Ghana
Centre for the development of People (CEDEP), Ghana
Health Foundation of Ghana, Ghana
Institute of Statistical, Social and Economic Research (ISSER), University of Ghana, Ghana
National Catholic Secretariat, Ghana
School of Medicine and Health Sciences, Ghana
Wuni Zaligu Development Association (WUZDA), Ghana
Ministère de la Santé Publique, Guinea
AMREF Kenya, Kenya
Egerton University, Kenya
Helps Foundation Kenya, Kenya
Kenya Cardiac Society, Kenya
Kenya NGO Alliance against Malaria (KeNAAM), Kenya
University of Nairobi, Kenya
J & F Consulting, Malawi
Malawi Health Equity Network, Malawi
Mauritius Institute of Health, Mauritius
Jhpiego, Mozambique
Action Group on Adolescent Health, Nigeria
African Council for Sustainable Health Development (ACOSHED), Nigeria
Afrihealth Information Projects/Afrihealth Optonet Association, Nigeria
Care Net Nigeria, Nigeria
Centre for Health and Development, Nigeria
Centre for Health Policy and Strategic Studies, Nigeria
Centre for Hospital and Health Services Improvement, Nigeria
Common Heritage Foundation, Nigeria
Contemporary Tasks Solutions, Nigeria
Cooper & Williams Consulting, Nigeria
Department of Community Medicine, University College Hospital, Nigeria
Family Health International, Nigeria
Health Reform Foundation of Nigeria, Nigeria
HIV/AIDS and TB Division, Ministry of Health, Nigeria

62
StatsXperts International Consulting, Nigeria
Susan Ohio Foundation, Nigeria
University of Ibadan, Nigeria
University of Nigeria, Nigeria
University of Uyo Teaching Hospital, Nigeria
AfHea, Senegal
RESEAO, Senegal
Rescue International, Sierra Leone
Centre for Health Systems Research & Development University of the Free State, South Africa
Doso Institute for Health Policy, South Africa
Health and Development Africa, South Africa
University of Kwa-Zulu Natal, South Africa
University of the Western Cape, School of Public Health, South Africa
University of the Witwatersrand School of Public Health, South Africa
Faculty of Medicine, University of Khartoum, Sudan
General Directorate of Pharmacy, Sudan
Health Policy Directorate, National Ministry of Health, Sudan
Khartoum State Ministry of Health, Sudan
Sudan Medical Heritage Foundation, Sudan
Tropical Medicine Research Institute, Sudan
University of Medical Sciences & Technology, Sudan
Amuru District Local Gorvernment, Anaka Hospital, Uganda
Centre for Socio-economic Research and Training (CSRT), Uganda
Gulu Regional Referral Hospital, Uganda
Health Care Management Unit, Lviv Medical University, Uganda
HealthNet Consulting, Uganda
Joint Clinical Research Centre, Uganda
Kabano Research and Development Centre, Uganda
Makerere University, Uganda
Mbarara University of Science and Technology, Uganda
Ifakara Health Institute, United Republic of Tanzania
Tanzania Food and Drugs Authority, United Republic of Tanzania
Ministry of Health, Zambia
University of Zambia, School of Humanities and Social Sciences, Zambia
Zambia Forum for Health Research (ZAMFOHR), Zambia
University of Zimbabwe Clinical Research Centre, Zimbabwe
University of Zimbabwe, Zimbabwe
Zimbabwe Grace Trust, Zimbabwe
Zimbabwe National Family Planning Council, Zimbabwe

Americas Region (PAHO/AMRO)

Center for the Implementation of Public Policies promoting Equity and Growth (CIPPEC), Argentina
Centro de Estudios de Población y Desarrollo (CEPyD), Argentina
Centro de Estudios Distributivos, Laborales y Sociales (CEDLAS), Argentina
Fundación de Investigaciones Económicas Latinoamericanas (FIEL), Argentina
Fundación de Medicina Familiar y Preventiva, Hospital Italiano de Buenos Aires, Argentina
Maestría en Epidemiología, Gestión y Políticas de Salud, Argentina
Univeridad Nacional del Nordeste, Argentina
Ministry of Health, Belize
Centro De Estudos de Cultura Contemporânea (CEDEC), Brazil
Escola Brazileira de Administraçáo Pública, Brazil
Fiocruz, Brazil

63
National School of Public Health, Ministry of Health, Brazil
Network for Health Systems and Services Research in the Southern Cone of Latin America, Brazil
British Columbia Centre for Disease Control, Canada
Canadian Public Health Association, Canada
Canadian Society for International Health, Canada
Edifice Saint Urbain, University of Montreal, Canada
Healthy Child Uganda, Canada
Sprinkles Global Health Initiative, Canada
University of British Columbia, Canada
Universidad Catolica de Chile, Chile
Asociación Centro de Gestión Hospitalaria, Colombia
Colombian Health Association (ASSALUD), Colombia
Fundacion Santa Fe de Bogota, Colombia
Pontificia Universidad Javeriana, Colombia
Santafe de Bogota Foundation, Health studies and research center (CEIS), Colombia
Universidad CES, Colombia
Universidad de Antioquia, Colombia
Universidad de Caldas, Facultad de Ciencias para la Salud, Colombia
Universidad Nacional de Colombia, Colombia
CONEDSA, Costa Rica
International Health Central American Institute Foundation, Costa Rica
Escuela Nacional de Salud Pública, Cuba
Instituto Nacional de Higiene, Epidemiología y Microbiología, Cuba
Intistuto Pedrokouri, Cuba
Fundacion Plenitud, Dominican Republic
Public Health Institute at the Pontifical Catholic University of Ecuador, Ecuador
Veeduría del Buen Vivir, Ecuador
Universidad de El Salvador, El Salvador
Instituto de Salud Incluyente, Guatemala
Instituto Nacional de salud Pública/Centro de Investigación en Sistemas de Salud, Mexico
Red de Investigación en Politicas, Sistemas y Servicios de Salud Nodo Paraguay, Paraguay
Facultad de Salud Publica y Administracion, Universidad Peruana Cayetano Heredia, Peru
Gerente de Proyectos y Captación de Recursos, Peru
Instituto de Investigacion Nutricional, Peru
Universidad peruana Cayetano Heredia, Peru
US Naval Medical Research Center Detachment (NMRCD), Peru
Sir Arthur Lewis Institute of Social and Economic Studies, Trinidad and Tobago
Department of International Health, School of Public Health, Boston University, United States of America
Family Health International, United States of America
Johns Hopkins University, United States of America
Population Council, United States of America
Tufts University School of Medicine, United States of America
UCLA School of Public Health, United States of America
University of Alabama at Birmingham, United States of America
University of New Mexico, United States of America
Yale University School of Public Health, United States of America
Centro de Informaciones y Estudios del Uruguay, Uruguay
GEOPS, Uruguay
Universidad de la República, Uruguay
Pan American Health Organization, Venezuela

64
South-East Asia Region (SEARO)
Bangladesh Institute of Development Studies (BIDS), Bangladesh
Bangladesh Research Initiatives of Environment Society (BRIES), Bangladesh
Bangladesh Women's Health Coalition (BWHC), Bangladesh
BIRPERHT, Bangladesh
Ibrahim Medical College, Bangladesh
ICDDRB, Bangladesh
Institute of Allergy and Clinical Immunology of Bangladesh, Bangladesh
State University of Bangladesh, Bangladesh
ADRA India, India
Aga Khan Health Service, India
Anusandhan Trust (SATHI), India
Centre for Health and Social Justice, India
Centre for Management of Health Services, Indian Institute of Management, India
Centre for Operations Research and Training, India
Chettinad hospitals and Research Institute, India
Foundation for Research in Health Systems, India
Gokhale Institute of Politics and Economics, India
Government Medical College, India
Gram Bharati samiti (GBS), India
Health Vision and Research, India
Indian Institute of Public Health-Hyderabad, India
Indian Institute of Technology (Madras), India
Institute of Health Systems, India
International Institute of Health Management Research (IIHMR), India
Maharashtra Association of Anthropological Sciences, India
Mahatma Gandhi Institute of Medical Sciences, India
National Council of Applied Economic Research, India
National Institute of Epidemiology, India
Prarthana Charitable Trust, India
Public Health Foundation of India, India
Sadhana Insitute for sustainable Development, India
School of Public Health, Postgraduate Institute of Medical Education and Research, India
Seva Mandir India
South Asian Institute of Health Promotion (SAIHP), India
Surat University Campus, India
Tata Institute of Social Sciences, India
The Maharashtra Association of Anthropological Sciences, India
UNICEF India Country Office, India
Urban Health Resource Centre, India
Voluntary Health Association of India, India
Demographic Institute Faculty of Economics University of Indonesia, Indonesia
Gadjah Mada Medical School, Indonesia
Ministry of Health, Indonesia
Asian People's Alliance for Combating HIV & AIDS, Nepal
BP Koirala Institute of Health Sciences, Nepal
CANVAS, Nepal
Forum for Human Rights and Public Health (Friendship Nepal), Nepal
Health Research and Social Development Forum, Nepal
INRUD Nepal, Nepal
Nepal Health Research Council, Nepal
Resource Centre for Primary Health Care, Nepal

65
School of Public Health and Community Medicine, Nepal
South Asian Institute for Policy Analysis and Leadership (SAIPAL), Nepal
Department of Sociology, University of Peradeniya, Sri Lanka
Faculty of Medicine and Allied Sciences, Sri Lanka
Health Policy Research Associates, Sri Lanka
Joint Alliance of Supplementary Medical Professionals, Sri Lanka
Marga Institute, Sri Lanka
Faculty of Medicine, Chiang Mai University, Thailand
Faculty of Nursing, Chiang Mai University, Thailand
Faculty of Public Health, Thammasat University, Thailand
Health Intervention and Technology Assessment Program, Thailand
Mahidol University, Faculty of Social Sciences & Humanities, Thailand
Naresuan University, Thailand
Prince of Songkla University, Faculty of Medicine, Thailand
Prince of Songkla University, Thailand
Thammasat University, Thailand

European Region (EURO)

Karl Landsteiner Institute Quality Assurance and Patient Safety (IQMS), Austria
Call for Health Public Alliance, Azerbaijan
Free University of Brussels, Belgium
Ghent University, Belgium
Institute of Tropical Medicine Antwerp, Belgium
Medical University, Bulgaria
National Center of Health Informatics, Bulgaria
University of Copenhagen, Denmark
University of Tampere, Finland
Centre d'Etudes et de Recherches sur le Développement International (CERDI), France
Curatio International Consulting, Georgia
Curatio International Foundation, Georgia
Welfare Foundation, Georgia
HealthMonitor Research and Consultancy Non-profit Public Purpose Ltd. (EgeszsegMonitor), Hungary
Semmelweis University Budapest, Hungary
UNICEF Israel, Israel
Istituto Superiore de Sanita, Italy
Healthcare Development Institution, Kazakhstan
Institute of Public Health, Kazakhstan
National Research Center of Maternal and Child Care, Kazakhstan
Public Foundation, Health Policy Analysis Center, Kyrgyzstan
Institute for Biomedical Research, Kaunans University of Medicine, Lithuania
Lithuanian University of Health Sciences, Lithuania
European Forum for Primary Care, Netherlands
Norwegian Knowledge Centre for the Health Services, Norway
University of Bergen, Norway
National Scientific and Applied Center for Preventive Medicine, Republic of Moldova
Associacion Comunitaria de Salut Sexual del Camp de Tarragona, Spain
Sabirmedical S.L., Spain
Department of Public Health and Clinical Medicine, Umeå University, Sweden
International Health Systems Research, Sweden
Umeå Center for Global Health Research, Sweden
International Council on Alcohol and Addictions (ICAA), Switzerland
Swiss Tropical Institute, Switzerland

66
Centre for Regional Policy Research Cooperation (Studiorum), Former Yugoslav Republic of Macedonia
Ventio Organisation Ltd., Turkey
AMREF UK, United Kingdom
Brunel University, United Kingdom
Centre for Healthcare Modelling and Informatics, United Kingdom
Centre for Innovation in Health Management, United Kingdom
Imperial College London, United Kingdom
Institute for International Health and Development, Queen Margaret University College, United Kingdom
Institute of Development Studies, United Kingdom
International HIV AIDS Alliance, United Kingdom
London International Development Centre (University of London), United Kingdom
London School of Hygiene and Tropical Medicine, United Kingdom
National Institute for Health and Clinical Excellence, United Kingdom
Public Health and Policy, London School of Hygiene and Tropical Medicine, United Kingdom
University of Aberdeen, United Kingdom
School of Public Health, Tashkent Medical Academy, Uzbekistan

Eastern Mediterranean Region (EMRO)

The Cochrane Collaboration Bahrain Branch, Bahrain
Health Ministry of Iran, Iran
Hormozgan University of Medical Sciences, Iran
Mashhad University of Medical Sciences, Iran
Ministry of Health and Medical Education, Undersecretary for Coordination, Iran
National Public Health Management Center, Iran
Otolaryngology, Head & Neck Surgery Research Center, Iran
Shahid Beheshti University of Medical Sciences, Iran
Vice Chancellor of Food and Drug, Medical University of Isfahan, Iran
American University of Beirut, Lebanon
Faculty of Health Sciences, Lebanon
Ministry of Health, Morocco
Aga Khan University, Pakistan
Centre for Health & Population Studies, Pakistan
Provincial Health Services Academy, Government of NWFP, Pakistan
Center for Strategic Health Studies, Syrian Arab Republic
Faculty of Medicine, Damascus University, Syrian Arab Republic
Thamar University, Yemen

Western Pacific Region (WPRO)

Anton Breinl Centre for Public Health and Tropical Medicine, James Cook University, Australia
Center for Health Policy and Management, Australia
Curtin University, Australia
Deakin University, Australia
Nossal Institute for Global Health, Australia
School of Population Health, University of Queensland, Australia
University of New South Wales, Australia
MoPoTsyo Patient Information Centre, Cambodia
National Institute of Public Health (NIPH), Cambodia
Beijing Municipal Centers for Disease Prevention and Control, China
Centre of Health Management and Policy Research, Shandong University, China
China Health Economics Institute, Peking University of MOH, China
Harbin Medical University, China
Health Human Resources Development Center (HHRDC), Ministry of Health, China

67
Nanjing Medical University, China
Ningxia Medical University, China
Peking University China Center for Health Development, China
Peking University Health Science Center, China
School of Public Health, Peking University, China
Shanghai Health Development and Research Center, China
Shanghai Second Medical University, China
The Chinese University of Hong Kong, China
The Hong Kong Institute of Health Economics, China
The Second Clinic Department of Peking University Third Hospital, China
Weri Fan Medical College, China
WHO Collaborating Centre for Urban Health Development, China
Centre for Health Information, Policy, and Systems Research, Fiji
Health Research Council of the Pacific (HRCP), Fiji
GQ1 Management Group, Malaysia
University of Malaya, Malaysia
University Sains Malaysia, Malaysia
National Centre for Health Development, Mongolia
NMDHB, New Zealand
University of Auckland, New Zealand
Divine Word University, Papua New Guinea
World Health Organization, Papua New Guinea
Department of Health, Center for Health Development, Eastern Visayas, Philippines
Institute of Philippine Culture, Ateneo de Manila University, Philippines
Medical Action Group, Philippines
Philippine Health Social Science Association, Western Visayas, Philippines
Philippine Nurses Association Cebu Chapter Inc., Philippines
Social Health Insurance, Networking and Empowerment (SHINE), Philippines
University of the Philippines, College of Public Health, Philippines
Xavier University, Philippines
Center for Health System Research, Hanoi Medical University, Vietnam
Center for Reproductive and Family Health (RaFH), Vietnam
Central Institute for Medical Science Information (CIMSI), Vietnam
Hanoi Medical University, Vietnam
Hanoi School of Public Health, Vietnam
Ministry of Health, Vietnam

68
 Appendix 4: Do You Know Your Health Systems Definitions?

We invite you to try to match the definitions to the term each one is supposed to define. Our experience shows this
is more difficulty that is probably should be. Correct answers are below.

“… addresses questions that are not disease-specific but concern

systems problems that have repercussions on the performance of the
health system as a whole. It addresses a wide range of questions,
health systems
from health financing, governance, and policy to problems with  
structuring, planning, management, human resources, service
research
delivery, referral, and quality of care in the public and private
A
sector.”

“…all research that underpins improvements in the way health

health services
services are financed, organised, planned and delivered, and includes  
health technology assessments and health policy research.”
B research

“… a problem-oriented field into which people enter from a wide

health services and
range of disciplinary backgrounds to work together to find ways that  
health care can best be organized, financed, and delivered.”
C policy research

“… multidisciplinary field of scientific investigation that studies how

social factors, financing systems, organizational structures and
processes, health technologies, and personal behaviours affect access health services
to health care, the quality and cost of health care, and ultimately our
 
research
health and well-being. Its research domains are individuals, families,
D
organizations, institutions, communities, and populations.”

“… purposeful generation of knowledge that enables societies to

health systems
organize themselves to improve health outcomes and health  
services.”
E research

“… scientific study of the organized social response to health and health systems
disease conditions in populations.”
F  
research

“… a field that seeks to understand and improve how societies

organize themselves in achieving collective health goals, and how
different actors interact in the policy and implementation processes
to contribute to policy outcomes. By nature, it is inter-disciplinary, a
health policy and
blend of economics, sociology, anthropology, political science, public  
health and epidemiology that together draw a comprehensive picture systems research
of how health systems respond and adapt to health policies, and how
health policies can shape − and be shaped by − health systems and
G
the broader determinants of health.”

“… multidisciplinary field of inquiry, both basic and applied, that

examines access to, and the use, costs, quality, delivery, organization,
health services
financing, and outcomes of health care services to produce new  
knowledge about the structure, processes, and effects of health
research
H
services for individuals and populations.”

Answers: The correct terms are directly horizontally across from the definitions. Sources: (A) Remme JHF, Adam T, Becerra-Posada F, D’Arcangues C, Devlin M, et al. (2010) Defining Research to
Improve Health Systems. PLoS Med 7(11): e1001000; (B) Health Services Research Network. 2012. Available at http://www.nhsconfed.org/Networks/HealthServiceResearchNetwork/Pages/
AboutHSRN.aspx; (C) Canadian Association for Health Services and Policy Research. “What is ‘HSPR’?” 2011. Available from https://cahspr.ca/en/resources; (D) Lohr, K. N. & Steinwachs, D. M.
Health services research: an evolving definition of the field. Health Services Research 2002;37(1):7-9; (E) First Global Symposium on Health Systems Research. 2010. Symposium background; (F)
Frenk, J. (1992). The New Public Health, in National Institute of Public Health of Mexico, The First Five Years, 1987 - 1991. National Institute of Public Health. First Printing, pp. 15-33.
Cuernavaca. Mexico; (G) Alliance for Health Policy and Systems Research. 2011; (H) Health Services Research: Opportunities for an Expanding Field of Inquiry - An Interim Statement. Committee
on Health Services Research: Training and Work Force Issues, Division of Health Care Services, Institute of Medicine. National Academy Press: Washington.

69
 Appendix 5: Taxonomy of the Health Systems Evidence Database
Governance arrangement (161) Health savings accounts (Third party contributions) (2)
Policy authority (43) Targeted payments/penalties (Incentivizing consumers) (24)
Centralization/decentralization of policy authority (16)
Accountability of the state sector's role in financing & delivery (1) Delivery arrangement (1517)
Stewardship of non-state sector in financing & delivery (24) How care is designed to meet consumers' needs (475)
Decision-making authority about coverage and available care (10) Availability of care (17)
Corruption Protections (1) Timely access to care (42)
Organizational authority (58) Culturally appropriate care (52)
Ownership (18) Case management (87)
Management approaches (7) Package of care/care pathways/disease management (349)
Accreditation (13) Group care (3)
Networks/multi-institutional arrangements (30) By whom care is provided (742)
Commercial authority (30) System - Need, demand & supply (13)
Licensure & registration requirements (10) System - Recruitment, retention & transitions (25)
Patents & profits (3) System - Performance management (33)
Pricing & purchasing (11) Workplace conditions - Provider satisfaction (22)
Marketing (10) Workplace conditions - Health & safety (21)
Sales & dispensing (13) Skill mix - Role performance (61)
Commercial liability (2) Skill mix - Role expansion or extension (131)
Professional authority (42) Skill mix - Substitution (113)
Training and licensure requirements (22) Skill mix - Multidisciplinary teams (227)
Scope of practice (9) Skill mix - Volunteers (1)
Setting of practice (2) Skill mix - Communication & discussion for distant providers (19)
Continuing competence (7) Staff - Training (18)
Quality & safety (10) Staff - Support (28)
Professional liability (3) Staff - Workload/workflow/intensity (47)
Consumer & stakeholder involvement (53) Staff - Continuity of care (35)
Consumer participation in policy & organizational decisions (16) Staff/self - Shared decision-making (56)
Consumer participation in system monitoring (1) Self-management (191)
Consumer participation in service delivery (33) Where care is provided (422)
Consumer complaints management (3) Site of service delivery (310)
Stakeholder participation in decisions (or monitoring) (19) Physical structure, facilities & equipment (44)
Organizational scale (11)
Financial arrangement (168) Integration of services (55)
Financing systems (61) Continuity of care (36)
Taxation (9) Outreach (3)
Social health insurance (18) With what supports is care provided (415)
Community-based health insurance (9) Health record systems (27)
Community loan funds (1) Electronic health record (42)
Private insurance (14) Other ICT that support individuals who provide care (192)
Health savings accounts (Individually financed) (1) ICT that support individuals who receive care (240)
User fees (30) Quality monitoring and improvement systems (58)
Donor contributions (11) Safety monitoring and improvement systems (33)
Funding organizations (34)
Fee-for-service (Funding) (3) Implementation strategy (675)
Capitation (Funding) (4) Consumer-targeted strategy (448)
Global budget (8) Information or education provision (357)
Prospective payment (Funding) (3) Behaviour change support (209)
Indicative budgets (Funding) (2) Skills and competencies development (75)
Targeted payments/penalties (Funding) (21) (Personal) Support (129)
Remunerating providers (75) Communication and decision-making facilitation (40)
Fee-for-service (Remuneration) (15) System participation (18)
Capitation (Remuneration) (13) Provider-targeted strategy (290)
Salary (11) Educational material (154)
Prospective payment (Remuneration) (6) Educational meeting (154)
Fundholding (1) Educational outreach visit (55)
Indicative budgets (Remuneration) (3) Local opinion leader (10)
Targeted payments/penalties (Remuneration) (63) Local consensus process (7)
Purchasing products & services (33) Peer review (9)
Scope & nature of insurance plans (7) Audit and feedback (80)
Lists of covered/reimbursed providers, services & products (17) Reminders and prompts (105)
Restrictions in coverage/reimbursement rates (9) Tailored intervention (8)
Caps on coverage/reimbursement (7) Patient-mediated intervention (20)
Prior approval requirements for coverage/reimbursement (4) Multi-faceted intervention (105)
Lists of substitutable services & products (6) Organization-targeted strategy (34)
Incentivizing consumers (44)
Premium (level & features) (2) Brackets indicate the number of synthesized research products available on each
Cost-sharing (1) topic as of 21 December 2011 at www.healthsystemsevidence.org

70
71
Background Paper on Conceptual Issues Related to Health Systems Research
to Inform a WHO Global Strategy on Health Systems Research
Steven J. Hoffman | John-Arne Røttingen | Sara Bennett | John N. Lavis | Jennifer S. Edge | Julio Frenk

A Working Paper in Progress

Last Revised 29 February 2012

72