FRAMEWORK FOR NATIONAL

PERFORMANCE INDICATORS
FOR NURSING AND MIDWIFERY
 FRAMEWORK FOR NATIONAL
PERFORMANCE INDICATORS
FOR NURSING AND MIDWIFERY

3
 CONTENTS

FOREWORD BY THE MINISTER FOR HEALTH

FOREWORD BY THE CHIEF NURSING OFFICER

SECTION 1 — INTRODUCTION

1.1 What is the purpose of this Framework? 08

1.2 How will this Framework benefit the health service user? 08

1.3 Performance measurement 08

1.4 What are PIs? 08

1.5 Why do we need Health Service PIs? 08

1.6 Why do we need specific Nursing and Midwifery PIs? 08

1.7 Why do we need National Nursing and Midwifery PIs? 11

1.8 What is the policy context of this Framework? 12

SECTION 2 — FRAMEWORK FOR NATIONAL PERFORMANCE INDICATORS FOR

NURSING AND MIDWIFERY

SECTION 2. PART A — MINIMUM DATA SET

2.A.1 What is a Minimum Data Set? 14

2.A.2 What is a Nursing Minimum Data Set? 14

2.A.3 Why is a minimum data set required for the nursing and midwifery workforce 14

2.A.4 Nursing and midwifery workforce registry 15

2.A.5 Developing Minimum Data Sets for PIs 16

SECTION 2. PART B — PERFORMANCE INDICATORS

2.B.1 Evidence Review 19

2.B.2 Types of indicators 20

2.B.3 Considerations 22

2.B.4 Governance 23

2.B.5 Performance Indicator Prioritisation and Endorsement Process 24

2.B.6 What makes a good indicator? 25

4
 2.B.7 Matrix of National Nursing and Midwifery PIs 26

2.B.8 Process for developing the PIs 28

2.B.9 Pilot Test Performance Indicators 31

2.B.10 Determine review frequency 31

2.B.11 What are Metadata? 31

2.B.12 Data Reporting 31

2.B.13 Tracking Performance Indicators 31

2.B.14 Conclusion 32

SECTION 3 — REFERENCES AND APPENDIX

3.1 References 33

3.2 Appendix 1 35

5
FOREWORD BY THE MINISTER FOR HEALTH

Over the years, we have developed the nursing and midwifery professions and rightly so.
Significant investment has been made and it’s important that we demonstrate its effect.

It is vital, therefore, that we measure the outcomes and contribution of nursing and midwifery
interventions and initiatives on client/patient care. This framework will assist in demonstrating
the impact of nursing and midwifery across key areas of our health services, through the use
of performance indicators.

Currently, we have limited data on the nursing and midwifery workforce to inform workforce
planning, service planning, recruitment and retention, training and development, and disaster
and emergency planning. Therefore, the first priority action from this framework is the
development of a minimum data set for the nursing and midwifery workforce.

Once the minimum data set is implemented, we will know the accurate number of nurses and
midwives employed in our health services.

The significance of this work cannot be underestimated in facilitating the management of the
health services and the development of performance indicators for nursing and midwifery.

The use of performance indicators will further embed an ethos of performance improvement in
nursing and midwifery and help both professions to make visible their contribution to a culture
of quality and safe patient care.

Simon Harris TD
Minister for Health

6
FOREWORD BY THE CHIEF NURSING OFFICER

I am delighted to introduce this Framework for National Performance Indicators for Nursing and
Midwifery.

In 2010, the Health Information and Quality Authority (HIQA) published guidance on the
development of key performance indicators in health care. This framework builds on that
guidance but is tailored to nursing and midwifery and captures issues that predominate in
nursing and midwifery services.

The Strategy for the Office of the Chief Nursing Officer 2015–2017 contains a suite of integrated
policies that are designed to revolutionise the way nursing and midwifery services will be
delivered in the future. The performance indicators developed and endorsed for use at national
level, as a result of this framework, will be incorporated in the HSE National Service Plan and
will measure the impact of these and other policies.

I am confident that by implementing this framework, we will be able to unambiguously

demonstrate the magnitude of the nursing and midwifery contribution to the health service.

I look forward to working with all of our partners, particularly the HSE and NMBI, and other key
stakeholders to implement this framework.

Dr Siobhan O’Halloran
Chief Nursing Officer

7
 SECTION 1 — INTRODUCTION

1.1 What is the purpose of this Framework?

The purpose of this Framework is to (i) set out the policy context for the use of nursing and midwifery performance
indicators (PIs) to demonstrate the relationship between inputs and outcomes/impact; (ii) provide guidance on PIs
including clarifying the relationship between Structure, Process and Outcome PIs; and (iii) ensure that there is a
standardised approach to their development, prioritisation, endorsement and implementation.

1.2 How will this Framework benefit the health service user?
All nursing and midwifery policy is developed with the aim of meeting the needs of the health service user. PIs will
tell us whether or not we are achieving our policy objectives.

1.3 Performance measurement

Performance measurement in healthcare settings has become very important in recent years as a way of ensuring
that the delivery of health services is improving all the time. The Irish health service requires significant funding
and performance measurement is one tool used to inform the allocation of resources in a way that facilitates the
achievement of policy objectives. It also allows healthcare settings to assess and continually improve their own
performance and compare it to similar settings.

1.4 What are PIs?

PIs are measures of performance that are used by organisations to measure how well they are performing against
targets or expectations. PIs measure performance by showing trends to demonstrate that improvements are being
made over time. PIs also measure performance by comparing results against standards or other similar organisations.
This helps organisations to improve the service they provide by identifying where performance is at the desired level
and also identifying where improvements are required. Performance measurement promotes accountability to all
stakeholders, including the public, service users, clinicians and the Government. It does this by facilitating informed
decision-making and safe, high quality and reliable care through monitoring, analysing and communicating the
degree to which organisations meet key goals (HIQA, 2010, updated 2013). There are three types of PIs: Structure,
Process and Outcome and these are explored in further detail in 2.B.2.

1.5 Why do we need Health Service PIs?

In the health service, information plays a pivotal role in promoting improvements in the safety and quality of client
care. The delivery of care that is safe, client-centred, compassionate, effective and efficient is the responsibility of
all health care professionals (HSE, 2015) and requires information that is of good quality, comparable and can be
shared within the health sector.

Health service PIs identify good practice and provide comparability within and between similar services. They also
identify opportunities for improvement and the need for a more detailed investigation of standards. The ultimate goal
of PIs is to contribute to the provision of a high quality, safe and effective service that meets the needs of service
users (HIQA, 2010, updated 2013).

1.6 Why do we need specific Nursing and Midwifery PIs?

PIs for nursing and midwifery identify specific and measurable elements of the quality and quantity of nursing and
midwifery care provided. Nursing and midwifery care is critical to achieving positive client outcomes and PIs are
needed to evaluate nursing interventions and implement initiatives to improve quality and quantity. They are also
important in providing accountability and assurance to the public (Skills for Health, 2016). In addition, they identify
areas of good practice that must be recognised and celebrated (HSE, 2015).

Nursing and midwifery care is delivered in a complex environment and there are multiple factors that influence
nursing and midwifery practice as indicated by the National Council for the Professional Development of Nursing and
Midwifery (June 2010) in the following figure (Figure 1):

8
 text and en
oc ial con viro
er s Values Demographics nm
en
id Society t
W Epidemiology
Legislation Resources
al principles; c
sion on
Belief Systems Research &
s ce
ofe pt
Norms evidence
Policy Pr Caring Ethics Safety
s
Quality Team working Accountability

Competence Knowledge
ocial care
Skills and s se
Expertise
th ttin
He al Administrators

g
Health professionals Lay carers

RCN RNID
RPN RPHN
RM
RGN

Patients;
selection of
interventions;
outcomes

Given this complex environment, it is important to develop PIs that capture the contribution made specifically by
nurses and midwives to the health service. When it comes to the development of these PIs, Griffiths et al. (2008)
state that the aim is to “…identify a relatively small number of indicators that still relate clearly to the multifaceted
and somewhat elusive concept of quality nursing care”.

Evidence shows that the nurse staffing and skill mix resource has a direct impact on patient and staff outcomes
(Aiken et al., 2014), so it’s vital to be able to demonstrate the benefits for patients of investment in this resource.

The pay for nursing in 2015 was €2.2 billion¹. However, the total cost of the nursing and midwifery resource includes
additional costs, for example, education and regulation costs.

The Health Service Personnel Census shows that nursing (including midwifery) is the largest staff category in the
public health service accounting for almost 34% of all staff:

¹ This figure includes both statutory and section 38 agencies.

9
 Table 1
Number (WTE) of staff employed in the public health service, as at July 2016
WTE (excld. career
Staff Category % of Total
break)

Medical/Dental 9,356.62 8.86

Nursing 35,712.36 33.81

Health & Social Care Professionals 15,002.76 14.2

Management/Admin 16,451.36 15.57

General Support Staff 9,487.46 8.98

Patient & Client Care 19,630.39 18.58

Total 105,640.95 100

Since the publication of The Report of the Commission on Nursing in 1998, there has been significant investment in
nursing and midwifery initiatives including:

• the appointment of the Chief Nursing Officer at Assistant Secretary level in the Department of Health and
the establishment of the CNO’s Office;
• the location of undergraduate nursing and midwifery education in the universities and higher education
sector;
• the enactment of legislation and regulation giving prescriptive authority to nurses and midwives;
• the enactment of the Nurses and Midwives Act 2011;
• the development of career pathways, to enable role expansion across clinical management and education
dimensions; and
• the establishment of a national structure for leading and implementing the development of capacity and
capability within the nursing and midwifery workforce.

The Commission on Nursing enabled significant developments in the nursing and midwifery resource and the
monitoring of PIs is essential to demonstrate the contribution of the professions to safe patient care. This can be
achieved through the use of nursing and midwifery PIs at both national and local levels.

The HSE introduced process PIs in the form of Quality Care Metrics (QCM). A range of organisations (across acute
hospitals, midwifery, children’s, older persons, intellectual disability, mental health services and public health/
community nursing services) currently gather data on a monthly basis that provide an indication of care quality
aligned to evidenced based standards, policies and procedures. The HSE is currently identifying a new suite of QCM
using a robust academic evidence-based framework that will determine a range of new process measures sensitive
to the influences of nurses and midwives and agreed through national consensus. This work is aligned to the policy
direction of the CNO’s Office and this Framework.

The Office of the Nursing and Midwifery Service Director (ONMSD) (2015) in the HSE highlights the benefits of
nursing and midwifery QCM as a measuring system that:

• monitors and assesses performance against evidenced based standards;

• quantifies trends and characteristics;
• highlights exceptional care and areas of risk which require immediate attention;
• provides a standardised system to track and benchmark the quality of care;

10
 • offers direction on educational needs for healthcare staff; and
• promotes staff engagement and accountability for the quality of care.

It adds that in addition to providing real time information to nurses and midwives about how clients are benefiting
from quality care delivery, QCM data enables managers to monitor individual ward performance and organisational
progress in delivering safer, quality focused client care.

VanDeVelde-Coke et al. (2012) conclude that “Equipped with the right information, nurses can better demonstrate
their value, advocate for the impact their profession has on the entire system, and focus their efforts on those factors
that have the greatest effect on healthcare outcomes, confirming that nurses make a difference”.

1.7 Why do we need National Nursing and Midwifery PIs?

National nursing and midwifery PIs are needed to evaluate the implementation and effectiveness of policy and to
show the impact of the investment in nursing and midwifery at a national level. The first step in this pathway (Figure
2) is to continue to embed the core values of nursing and midwifery practice, i.e., Compassion, Care and Commitment
(Department of Health, 2016). Then, the nursing and midwifery resource must be stabilised using the Framework
for Safe Nurse Staffing and Skill Mix. This aims to apply a consistent method to determine optimum nurse staffing
and skill mix levels. Nursing and midwifery outputs/outcomes will be maximised by deploying staff appropriately and
maximising their scope of practice. Finally, PIs will be used to measure the impact of these initiatives, informed by
quality assured nursing and midwifery data. Figure 2:

Quality Assured Nursing and Midwifery Data

Measure Impact Nursing and Midwifery Pls

Deploy
Maximise
appropriately/maximise
Output/Outcomes
scope of parctice

Stabilise Resource Framework for Safe Nurse

Staffing and Skill Mix

Embed Nursing and

Compassion, Care,
Midwifery Values
Commitment

National PIs are also needed to allow for benchmarking between hospitals or other healthcare settings. However,
currently there is a lack of quality assured nursing and midwifery data with which to plan the nursing and midwifery
resource to optimise client care.

11
1.8 What is the policy context of this Framework?
The following figure (Figure 3) shows the position of this Framework in the context of overall policy:

• WHO Health 2020 • Strategy for the Office of the Chief

• WHO European Startegic Directions for Nursing Officer 2015-2017
strengthening Nursing and Midwifery • Department of Health Statement
Towards Health 2020 Goals of Strategy 2016-2019
• Department of Health Framework
for Service Plan KPIs

• Patient Safety
• Pay and Numbers Strategy
• Framework for Safe Nursing Staffing and
Skill Mix in General and Specialist Medical
and Surgical Care Settings in Adult
Hospitals in Ireland
• Future nursing and midwifery in the
community capable of driving integration
of primary care and acute nursing and Framework for National Performaance
midwifery services Indicatos for Nursing and Midwifery
• Future development of advanced and
specialist nursing and midwifery practice
• Code of Practice for the Governance of
State Bodies

The Strategy for the Office of the Chief Nursing Officer was informed by the Department’s Statement of Strategy and
relevant World Health Organisation (WHO) reports (Health 2020 and European Strategic Directions for Strengthening
Nursing and Midwifery Towards Health 2020 Goals).

It contains four strategic objectives as follows:

• To provide expert policy input and direction to support government priorities and to optimise public
investment in the health system;
• To strengthen the role of nurses and midwives by optimising the scope of practice across the health service;
• To enhance the impact of nurses and midwives and demonstrate this through the utilisation of robust data
intelligence;
• To enable nurses and midwives to serve as full partners in health care design and improvement by enhancing
leadership, competency and opportunities.

The development of this Framework is one of the priority actions cited in the Strategy and it will facilitate the
achievement of the aforementioned strategic objectives. This Framework has been developed in line with the
Department’s Framework for Service Plan KPIs and in consultation with the ONMSD in the HSE.

In addition, it will assist in monitoring the impact of relevant policies on:

• patient safety (National Patient Safety Office);

• workforce planning (Pay and Numbers Strategy; and Framework for Safe Nurse Staffing and Skill Mix in
General and Specialist Medical and Surgical Care Settings in Adult Hospitals in Ireland);
• nursing and midwifery (Future nursing and midwifery models in the community capable of driving integration
of primary care and acute nursing and midwifery services, and Future development of advanced and
specialist nursing and midwifery practice); and

12
 • regulation (Code of Practice for the Governance of State Bodies).

In other countries, PIs are contained within overarching frameworks for assessing the performance of the health
system. Within these frameworks, PIs are categorised under health system goals (and/or domains or dimensions of
performance). There is currently no analogous health system performance framework in Ireland. Therefore, national
nursing and midwifery PIs should be categorised under the strategic objectives set out in the Department of Health’s
Statement of Strategy 2016–2019:

• Support people to lead healthy and independent lives;

• Ensure the delivery of high quality and safe health and social care;
• Create a more responsive, integrated and people-centred health and social care service;
• Promote effective and efficient management of the health services;
• Develop a high-performing Department to lead and oversee the health and social care sector.

This format will maintain the flexibility required to measure the impact of nursing and midwifery policy as the
preferred model of care emerges.

13
 SECTION 2 — FRAMEWORK FOR NATIONAL
PERFORMANCE INDICATORS FOR NURSING AND
MIDWIFERY

SECTION 2. PART A — MINIMUM DATA SET

2.A.1 What is a Minimum Data Set?

A data set is a set of data that is collected for a specific purpose and a minimum data set (MDS) is the core data
identified as the minimum required for that purpose. Once PIs have been developed, it is necessary to determine
what data needs to be collected for each PI being used to measure performance. This should be achieved by creating
a minimum data set and be based on what data are feasible to collect. (HIQA 2010, updated 2013)

2.A.2 What is a Nursing Minimum Data Set?

A nursing² minimum data set (NMDS) provides a formal structure for electronic data sets to support nursing and
midwifery care in all settings. What distinguishes it from other healthcare data sets is the inclusion of nursing
care elements, i.e., nursing diagnoses, interventions and nursing outcomes (National Council for the Professional
Development of Nursing and Midwifery, 2009).

According to Werley et al. (1991), the purpose of a NMDS is to:

• establish comparability of nursing data across clinical populations, settings, geographic areas, and time;
• describe the nursing care of patients/clients and their families in a variety of settings, both institutional and
non-institutional;
• demonstrate or project trends regarding nursing care provided and the allocation of nursing resources to
patients/clients according to their health problems or nursing diagnoses;
• stimulate nursing research through links to the detailed data existing in nursing information systems and
other health care information systems; and
• provide data about nursing care to influence clinical, administrative, and health policy decision-making.

2.A.3 Why is a minimum data set required for the nursing and midwifery
workforce?
Currently, there is a dearth of data on nursing and midwifery in the Irish health system and there is no NMDS. The
most important NMDS is a MDS for the nursing and midwifery workforce. Such a MDS would ensure the availability
of quality assured data for forecasting and budgeting; distribution of the workforce; training and development; and
recruitment and retention (World Health Organisation, 2015).

In addition, this MDS would allow for the identification of cohorts of nurses and midwives with specialist skills for
emergency or disaster planning. It would also enhance the accuracy of reporting on our nursing/midwifery workforce
to international organisations such as the OECD.

The WHO (2015) describes a health workforce registry as a critical component of the Human Resources for Health
Information System (as shown in the following figure (Figure 4)) which in turn is vital to any Health Information
System.

² The extracts referenced here do not specifically mention midwifery, however, for the purpose of this document ‘nursing’ should be read as ‘nurs-
ing and midwifery’.

14
 Functional Components of Human Resources for
Health (HRH) Information System

VACANCY AND
WORKFORCE RECRUITMENT FINANCE REGISTRATION
PRODUCTION AND LISTENING

MIGRATION PAYROLL BENEFITS

PERFORMANCE HRH
TRAINING
MANAGEMENT WORKFORCE
REGISTRY

RETIREMENT

2.A.4 Nursing and midwifery workforce registry

All nurses and midwives who practise in Ireland must, by law, be listed on the Register of Nurses and Midwives which
is maintained by the Nursing and Midwifery Board of Ireland (NMBI). This means the register is the ideal place
to collect data on the nursing and midwifery workforce (both in both public and private sectors). The data will be
conveniently collected from registrants upon their first registration or renewal of registration. The following is an
example of a minimum data set ³, 4 for the nursing and midwifery workforce:

• Identification number
• Name
• Gender
• Date of birth
• Place of birth
• Home address
• Phone number
• Email address
• Language
• Country of primary nursing/midwifery qualification
• Country of training
• Year of first registration in Ireland
• Highest level of education/training completed to date applicable to nursing/midwifery practice
• Division of the register: General; Midwives; Children’s; Psychiatric; Intellectual Disability; Public Health;
Nurse Tutor; Nurse Prescribers; Advanced Nurse Practitioners; Advanced Midwife Practitioners
• Division of the register in which you are practicing
• Specialist qualifications
• Employed/Unemployed/Employed but not working in healthcare/Retired
• Employed full-time/part-time/temporary/contract basis/agency
• Employment sector: Public/Private/Voluntary
• Employment setting: Primary Care/Acute Care/Social Care
• Employment type: clinical practice/management/administrative/education/research/regulation/sales/Other
(Please specify)
• Grade
• Name of place of employment
• Address of place of employment

This process will be simplified for registrants through the provision of dropdown options and definitions of terms.

The data set will be agreed by key stakeholders (Department of Health (CNO’s Office, Research & Development and
Health Analytics Unit); HSE; and the NMBI)) who will specify their data requirements and the purpose/use of data.

³ The WHO (2015) recommends the following MDS: Identification Number; Full name; Birth history, Citizenship, Country of Residence, and
Language; Address; Contact Information; Education, Professional Licence, and Certification; Employment Status; Employment Address.

4 The Department of Health (2002) recommends the following MDS: Health Board/Authority Region; Place of Employment; Work Address/
Assignment; Sex; Date of Birth; Nationality; An Bord Altranais (Irish Nursing Board) Personal Identification Number; Grade/Job Title; Position Title
(local title); Commitment; Contract; Registrable Qualifications; and Academic Qualifications.

15
2.A.5 Developing Minimum Data Sets for PIs
A minimum data set should be developed based solely on the essential data required to operationalise a PI. As data
collection can involve the use of additional resources it is essential that only the minimum amount of data, required
to enable effective decision making, is collected (HIQA, 2010, updated 2013).

The MDS should be developed using the following steps (Table 2), adapted from the HIQA guidance (2010, updated
2013):

1. Define the Ideally, the required data should be collected as part of routine service delivery.
level of health Whether or not the available data meets the requirements of the measurement
information process should have been determined during feasibility testing. If requirements
are not met there will be a need to collect additional data.

Data are routinely collected during the delivery of healthcare in order to manage
care. These data are then processed at different levels within the healthcare
system according to the needs of the system and the purpose of the information
as follows:

• episode-level: which is necessary to facilitate the management of care for

each individual service contact. Episode level data records details of a service
user’s journey through the health service and includes data such as socio-
demographic details, referral details, and clinical details. Episode level data
are based on the concept of an ‘episode of care’ which commences at the first
contact with the service and is a means of describing and recording relevant
information in relation to the care provided to an individual service user during
a defined period of time. A unique identifier for individuals is necessary in order
to report episode-level information
• case-level: which is necessary to facilitate the management of care for each
individual service user. Case-level data are an aggregate of all the episodes
an individual service user has during a reporting period and is derived from
episode level data
• facility-level: which is necessary to facilitate the management of the service
facility. Facility-level data includes data relating to the facility such as number
of beds, staffing, expenditure and also includes episode-level and case-level
data
• system-level: which is necessary for policy and planning purposes on a system
wide or national basis. System-level data are an aggregate of all data elements
in a particular region and is derived from episode, case and facility-level
information.

Frequently, the PI will require data to be processed from different levels, using a
combination of data during analysis, to achieve a measurement.

2. Define the The urgency of decisions to be made based on the PI or the level of monitoring
frequency of required, will determine the frequency of data collection.
collection

16
3. Document It is necessary to write detailed data collection specifications to ensure that data
the data are collected and measured consistently and to reduce the risk of bias. There
collection should be a data development process which results in data standards that con-
process tribute to a consistent approach to data collection and use. Data standards are
agreements on the representation, format, and definition of common data. These
data standards will then assist in the process of ensuring data collection is of high
quality and enable consistent and comparable reporting of data and information.

Data can be collected manually, electronically or a combination of both. Methods

of data collection need to be explored to determine the feasibility of the PI and
answer the following questions:

• can existing data sources be used? During the feasibility analysis existing data
sources will have been identified and where possible these should be utilised.
However, if an existing data source does not meet the needs of the project,
then it should not be used
• can existing data sources be enhanced? If the existing data source provides
data closely aligned with the required data but not completely fulfilling the
requirements, it may be possible to enhance the existing data source. Before
enhancing an existing data source it is necessary to consult with others using
the data source to ensure the modification does not impact on other uses of
the data
• is a new method of data collection needed? If a new data source is required it
should be determined that the reporting burden does not exceed the benefits
gained from collecting the data.

4. Identify The most efficient way to collect data are to incorporate the collection process into
data sources routine service-user care, which involves standardising documentation to ensure
the required information is already being recorded for operational purposes.

Data sources/methods include:

• administrative databases that are readily available and therefore will involve
minimal expenditure for data collection, however, the information may not be
specific enough and may not be reliable
• medical record data that are also readily available and contain more detail than
administrative data, including diagnosis, treatment and outcome
• prospective data collection, which involves collecting data specifically for quali-
ty measurement purposes - it is more specific and can define exactly what data
are required. It is, however, not readily available and expensive to collect
• survey data, which involves collecting data regarding knowledge, attitudes and
behaviours and is not otherwise available. It is not readily available and is ex-
pensive to collect.

17
5. Identify • Once a decision has been made on a PI that fulfils the performance
data for measurement aims and the MDS has been identified, each data element within
development the MDS should be described in detail. High quality data collection processes in
which the data set is well defined and standardised ensures that the same data
are not collected, counted or reported differently for different purposes. This
results in a reduction in the burden and use of resources for data collection and
facilitates the principle of ‘create once, use often’.
• Data should be clearly defined and standardised for comparability purposes and
should not be reliant on or limited by the capability of one particular system,
organisation or data collection tool.

6. Assess • Healthcare information is sensitive and therefore provision must be made to

compliance ensure security and confidentiality of data held on service users.
with • Information governance is the process whereby organisations and individuals
Information ensure that personal information is handled legally, securely, efficiently and
Governance effectively, in order to deliver the best possible care.
• The data set should comply with data protection legislation and guidelines and
should have respect for service users’ privacy and confidentiality.

7. Plan Data • There should be clear definitions for each data element in the MDS to ensure
Quality Checks data collectors have a good understanding of what, how and when data needs
to be collected.
• There should also be routine data quality checks to minimise the occurrence of
reporting and input errors .
• Quality checks can be introduced at all stages of the measurement process,
such as data collection, processing, analysis, use and dissemination.

18
SECTION 2. PART B — PERFORMANCE INDICATORS

2.B.1 Evidence Review

In 2012, the Department of Health established the National Nursing and Midwifery KPI Steering Committee, the aim
of which was to identify and agree an implementation plan for national nursing and midwifery high level KPIs. The
Steering Committee identified four nursing and midwifery KPIs for national reporting of which two (pressure ulcer
prevention and falls prevention) were included in the HSE’s National Service Plan (NSP) 2015 with a commitment to
their development and associated reporting by Q3 2015. Two other KPIs (no. of nurses prescribing medication and
no. of nurses prescribing ionising radiation) appeared in the NSP 2016.

PIs are invaluable tools that contribute to the performance monitoring process. However, for PIs to be effective,
they need to have clear definitions to ensure that the data collected are of high quality (that is, consistent, reliable
and in keeping with shared definitions) and to enhance their validity and reliability. Valid PIs measure what they
are intended to measure and reliable PIs will consistently produce the same result regardless of who performs the
measurement.

The Department of Health commissioned an evidence review, in 2015, to contribute towards the development of
this Framework. Following a competitive tendering process, Skills for Health was chosen to undertake the review.
A mixed methodological approach (comprising an online survey and telephone survey of a sample of nurses and
midwives at various grades) was used in the review to gather information on the current range and extent of the
use and impact of nursing and midwifery PIs across a range of services nationally, to inform the development of
this Framework. The online survey was distributed to 163 organisations and respondents were invited to respond on
behalf of their organisations (the response rate was 29%). For the telephone survey, twelve 35–60 minute telephone
interviews were conducted.

The key findings of the evidence review are as follows:

• Performance Indicators (PIs) are being collected on a significant scale throughout a range of organisational
types and locations throughout the Republic of Ireland.

• The PIs being collected span Structural, Process and Outcome types.

• In terms of ‘Structure’ PIs of the 39 different unprompted PIs were mentioned. ‘Sickness and
Absence’ rates were the most commonly identified, with 23 mentions in total; ‘Use of Agency Staff’
was mentioned 15 times. Training, including compliance was mentioned 6 times.

• HSE Nursing and Midwifery Quality Care Metrics accounted for a great many of the Process PIs
used.

• The most mentioned Outcome type PIs included Falls with 15 mentions, Pressure Ulcers 14, and
Patient Experience measures.

• However, there is a great deal of confusion amongst respondents about what are the Structural, Process and
Outcome PIs. Some of this is around nomenclature and this needs to be addressed. However, there needs
to be more development around the technical knowhow of PIs. Respondents needed to appreciate the fact
that the use of the terminology reflects the ‘logic chain’ of structural, process and therefore outcomes and
how they interact.

• The most popular reporting frequency of data across all PI types was monthly. However, there were some
important variations between each PI ‘type’.

• There was a reliance on manual collection of data, with some references to handheld devices as well as
other systems of collection.

• Across all PIs (Structural, Process and Outcome), Excel was the system referred to most for compiling the
evidence (67%). Of the others, a number of bespoke/specialist systems were referred to by around 17%
of respondents.

• There is a ‘mixed economy’ of approaches taken to gather, compile and analyse PIs and their associated
metrics. Some of these make sense, e.g., HR might be more appropriate for staffing levels. However, the
diversity of involvement from a wide range of actors might indicate a degree of inefficiency in how metrics
are systematically collected and reported.

19
 • There are strong indications that (i) the application of PIs is having an influence on the delivery of health
care in the Republic of Ireland. The regular reporting of PIs is making such measures familiar to nurses
and midwives and is therefore providing positive motivation; and (ii) PIs are being utilised in ways that
can improve practice. Respondents from both the online survey and the telephone survey cited a range of
improvements that spanned efficiency gains, improvements to staff development and clinical outcomes.

There was a wide range of examples listed by respondents about how the data are used to improve
practice. The examples focused on the ongoing monitoring of conditions and their development. Action
plans developed based on the deficits of individual wards, was a specific example. Risk identification and
management were also mentioned on a number of occasions, and are linked in particular to the assessment
for risk of fall and pressure ulcers, for example. Another also mentioned, ‘tracking’ and ‘trending’ of the
data that will, over time, demonstrate areas that need improvement. So in this example, whilst immediate
actions plans were not in development, the data was being used to provide trends upon which to inform
decisions on necessary actions related to emerging patterns.

• Tangible changes to practice were also highlighted by respondents. One example was related to an initiative
measuring falls risk assessment, whereby when compared with the data on falls incidence, a positive
correlation was revealed, through the reduction in the number of falls. Additionally, based on the overall
level of fall risk measurement, actions that included alarm mats were introduced that alerted staff to ‘at risk
of fall’ patients that were mobilising, and thereby alerting staff to attend to the patient to prevent a fall on
subsequent mobilisation.

• PIs are often being reported upwards to management. The online survey indicated that overall the reporting
of PIs was commonly reported to ‘middle, senior and board level management’. Structural PIs were more
likely to be presented up to board level.

• There is demand for the provision of support and guidance for the development of PIs across all ‘types’ and
also to promote their use more widely. Thirty-eight percent (38%) of respondents were not confident about
developing their own PIs in the future. A significant minority of respondents (40%) indicated they were
unaware of guidance that had been developed for developing PIs.

The evidence suggests that there is a desire among nurses and midwives to measure and improve performance
which is a positive finding. It shows that there is a need for this Framework to assist with a standardised approach
to the development of PIs. There is also a need for PIs to be developed for use at both national and local level. Local
level PIs are necessary to achieve, measure and maintain high performance in specific settings, whereas, national
PIs are needed to monitor the implementation and effectiveness of policy and to show the impact of the investment
in nursing and midwifery. While some local level PIs could have nationwide applicability, not all PIs could or should
be adopted for use at a national level. Therefore, it is necessary to put a system in place to ensure that potential
national PIs can be assessed, prioritised, endorsed and implemented.

2.B.2 Types of indicators

HIQA’s (2010, updated 2013) guidance on PIs states that:

One of the most significant developments in relation to performance monitoring in the last 30 years has been Avedis
Donabedian’s division of healthcare into structure, process and outcome, for the purpose of defining and measuring
quality. Donabedian has contributed significantly to improvements in the quality and safety of health and social
care through his lifelong commitment to the use of performance measures. According to Donabedian, healthcare
quality can be assessed using a three-part model based on the structures, processes and outcomes of the healthcare
system. This division of healthcare has allowed the identification of data across the full spectrum of healthcare that
contributes to monitoring the quality of the various constituents of healthcare delivery.

The evidence review (see 2.B.1) found that there is a great deal of confusion amongst respondents around Structure,
Process, and Outcome PIs. The differences between these PIs can be summarised as follows (Skills for Health, 2016):

Structure PIs
Structure PIs relate to the resources of the healthcare system that contribute to its ability to meet the healthcare
needs of the population. Structural indicators refer to the resources used by an organisation to deliver healthcare
and includes buildings, equipment, the availability of specialist personnel and available finances. Examples of such
indicators include:

20
 • Agency Nurse WTE use
• Sick Absence Rate
• Staff Turnover Rate

Process PIs
Process PIs relate to the care delivered to the service user and how well it is delivered. Process indicators measure
the activities carried out in the assessment and treatment of service users and are often used to measure compliance
with recommended practice, based on evidence or the consensus of experts. Examples of these indicators include:

• Medication administration practice

• Assessment of falls risk
• Assessment of pressure ulcer risk

Outcome PIs
Outcome PIs relate to the stage of health of the individual or population resulting from their interaction with the
healthcare system. It can include lifestyle improvements, emotional responses to illness or its care, alterations in
levels of pain, morbidity and mortality rates and increased level of knowledge. Such examples include:

• Incidence of falls
• Incidence of pressure ulcers
• Patient experience of care

Griffiths et al. (2008) highlight the difference in the use of structure, process and outcome PIs as follows:

• staffing and skill mix are linked to patient outcome but their use as indicators would stifle change and create
perverse incentives. Use of ‘staffing matched to planned staffing’ as part of a suite of indicators including
outcomes has more potential;
• process indicators should be used with caution because of potential for gaming and difficulty in linking
specific processes and patient outcomes; and
• patient experience of compassionate care is an important outcome in its own right and may provide the
best measure of the nursing contributions to shared outcome and evaluation of complex processes that are
otherwise elusive.

EXAMPLE SHOWING THE USE OF EACH TYPE OF PI

If we wanted to assess a potential relastionship between the use of agency nursing staff and
the incident of falls in a particular health care setting, we could examine three PIs: (i) Agency
Nurse Use (a structure PI); (ii) Assessment of Falls Risk (a Process PI); and (iii) Incidence of
Falls (an Outcome PI).

The Agency Nurse Use PI will tell us whether there is a high or low number of agency staff
in that setting. The Assessment of Falls Risk PI will tell us if the required number of
assessments is being undertaken. The Incidence of Falls PI will tell us how may patients have
fallen. All three PIs will cover the same period of time (but this can be compared to other
periods of time to show trends).

If the Incidence of Falls PI shows that a high number of patients have fallen within the
timeframe concerned, the Assessment of Falls Risk PI can be examined. If it shows that
insufficient assessments are being undertaken, action must be taken to ensure that this is
addressed. For example, if a trend emerges that insufficient assessments are undertaken when
there is a high volume of agency staff, a programme/process may need to be introduced for
these staff to ensure compliance. Once the required number of assessments takes place, this
should result in a reduction in the number of patient falls. If it doesn’t other issues will need
to be examined.

21
PIs can also be characterised according to whether they are generic or specific and by both the type and function of
care for which the measurement is intended.

PIs can be targeted to measure performance that is relevant to all service-users or they can measure aspects of a
service that are relevant to a specific service user population:

• generic PIs measure aspects of performance relevant to the majority of service users and do not target a
specific service user population; and
• specific PIs are related to a specific service user population and measure particular aspects of care related
to those service users.

PIs can be classified according to the type of care for which the measurement process was developed (for example,
preventive, acute or chronic care) and according to the function of care, which can be screening, diagnosis, treatment
and follow-up (National Council for the Professional Development of Nursing and Midwifery, 2010 (adapted from
HIQA, December 2010)) as shown in Figure 5:

Structure Process Outcome

Generic Specific

Preventive Acute Chronic

Assessment/ Intervention/
Screening Treatment Structure
Diagnostic

2.B.3 Considerations
Certain factors need to be considered when developing PIs to ensure that they are fit for purpose. The collection of
data should be adequate for the PI without being onerous for the organisation. A mixture of Structure, Process and
Outcome PIs should be developed and each category should contain a suite of PIs to give a more holistic view of the
healthcare being provided.

The care experienced by clients must be monitored as they are the ultimate focus and beneficiaries of improved
nursing and midwifery care. Gubb (2009) cautions against meeting targets at the expense of individual client
experiences, “…performance is determined against crude indicators, not the expectations and experience of those
using the service”.

22
The NHS Outcomes Framework 2015/2016 (Department of Health, 2014) includes the domain ‘Ensuring that people
have a positive experience of care’ and it contains the following indicators:

• Patient experience of primary care

■ GP services
■ GP out-of-hours services
■ NHS dental services
• Patient experience of hospital care
• Friends and family test
• Patient experience characterised as poor or worse
■ Primary care
■ Hospital care

PIs of nursing and midwifery quality alone are insufficient and additional PIs of wider factors that underpin nursing
and midwifery quality should be developed. Maben et al. (2012) suggest the following examples (Table 3):

Indicators of factors that underpin quality Indicators of staff experience

• Staffing levels: supernumerary staff, bank/ • Staff satisfaction and wellbeing
agency staff, staff/patient ratios • Practice environment/perceived quality
• Skill mix: staff experience/knowledge/skills • Perception of adequate staffing
and expertise • Interprofessional relations
• Sickness rates • Staff intention to leave
• Vacancy rate/staff turnover
• Appraisal/induction
• Record of Continuous Professional
Development/learning plan

Using a suite of PIs should prevent the creation of perverse incentives by ensuring that activity is not encouraged
around PIs at the expense of other important outcomes (National Health Performance Authority, 2012). It would also
introduce uncertainty in the way performance is assessed, as suggested by Bevan (2006), which would make some
forms of gaming more difficult.

2.B.4 Governance
PIs can be developed for use at national level or local level. However, a robust system of governance is required
for the prioritisation, endorsement and tracking of PIs developed for use at national level to ensure consistency of
approach and high standards. A PI being proposed for use at national level must first be assessed to ensure that it
meets certain criteria before being included in the HSE’s National Service Plan (NSP). The following figure (Figure
6) sets out the pathway for a PI from development to inclusion in the NSP (where it becomes a KPI) and the KPI
tracking system:

HSE National Service Plan KPI Tracking System

Joint DoH-HSE KPI Oversight Group

National Nursing and Midwifery Endorsement Group

CNO’s Office (DoH) or ONMSD QCM Governance Group (HSE)

23
Once a PI has been identified for development by the CNO’s Office, its development and pilot testing is overseen by
the ONMSD QCM Governance Group. A request for the PI to be endorsed is then submitted to the National Nursing
and Midwifery PI Endorsement Group. The oversight of National Service Plan KPIs is currently under review by the
Department and the HSE. The creation of a joint DoH–HSE KPI Oversight Group is proposed in order to provide
a governance structure around the inclusion and review of KPIs within the National Service Plan (NSP). Once the
National Nursing and Midwifery PI Endorsement Group has endorsed a PI for use at national level, it would then
request the joint DoH–HSE Oversight Group to prioritise the PI and approve its inclusion in the NSP. Any KPI included
in the NSP would be logged in a KPI tracking system.

2.B.5 Performance Indicator Prioritisation and Endorsement Process

PIs will be considered for endorsement by the National Nursing and Midwifery PI Endorsement Group once they have
met the following criteria as set out in the HIQA guidance (2010, updated 2013):

Validity: Does the PI measure what it is supposed to measure?

A valid PI measures what it is supposed to measure and captures an important aspect of quality (and quantity,
in the case of nursing and midwifery PIs) that can be influenced by the healthcare facility or system. Ideally PIs
selected should have links to processes and outcomes through scientific evidence. Measures that have been selected
using scientific evidence possess high content validity and measures selected through consensus and guidelines will
have high face validity. Content validity refers to whether the PI captures important aspects of the quality of care
provided. Face validity can be determined by the PI making sense logically and clinically or from previous usage.

Reliability: Does the PI provide a consistent measure?

The PI should provide a consistent measure in the same population and settings irrespective of who performs the
measurement. Reliability is similar to reproducibility to the extent that if the measure is repeated you should get the
same result. Any variations in the result of the PI should reflect actual changes in the process or outcome. Reliability
can be influenced by training, the PI definition and the precision of the data collection methods. Inter-rater reliability
compares differences between evaluators performing the same measurement. Internal consistency examines the
relationship between sub-indicators of the same overall measurement, and, if reliable, there should be correlation
of the results. Test/retest reliability compares the difference between results when the same evaluator performs the
measurement at different times.

Explicit evidence base: Is the PI supported by scientific evidence or the consensus of experts?
PIs should be based on scientific evidence, the consensus of expert opinions among health professionals or on clinical
guidelines. The preferred method of choosing PIs is through evaluating scientific evidence in support of each PI and
rating the strength of that evidence. One example of a rating system is to give the highest rating to evidence (“A”
evidence) from meta-analysis of randomised controlled trials and give a lesser rating (“B” evidence) to evidence
for controlled studies without randomisation and a further lower rating (“C” evidence) to data from epidemiological
studies. In healthcare, there may only be limited scientific evidence to support a PI and it becomes necessary to
avail of expert opinion. There are a number of methods by which a PI can be developed through facilitating group
consensus from a panel of experts, such as the Delphi technique, the RAND appropriateness method and from
clinical guidelines.

Acceptability: Are the PIs acceptable?

The data collected should be acceptable to those being assessed and to those carrying out the assessment.

Feasibility: Is it possible to collect the required data and is it worth the resources?
There should be a feasibility analysis carried out to determine what data are currently collected and the resources
required to collect any additional required data. The feasibility analysis should determine what data sources are
currently available and if they are relevant to the needs of the current project. This will include determining if there
are existing PIs or benchmarking processes based on these data sources. The reporting burden of collecting the data
contained in the PI should not outweigh the value of the information obtained. Preferably, data should be integrated
into service delivery, and, where additional data are required that are not currently part of service delivery, there
should be cost benefit analysis to determine if it is cost-effective to collect. The feasibility analysis should also include
what means are used to collect data and the limitations of the systems used for collection. It should also outline
the reporting arrangements, including reporting arrangements for existing data collection and frequency of data
collection and analyses.

Sensitivity: Are small changes reflected in the results?

Changes in the component of care being measured should be captured by the measurement process and reflected
in the results. The performance indicator should be capable of detecting changes in the quality of care and these
changes must be reflected in the resulting values.

24
Specificity: Does the PI actually capture changes that occur in the service for which the measure is
intended?
Only changes in the area being measured are reflected in the measurement results.

Relevance: What useful decisions can be made from the PI?

The results of the measurement should be of use in planning and the subsequent delivery of healthcare and contribute
to performance improvement.

Balance: Do we have a set of PIs that measure different aspects of the service?
The final suite of indicators should measure different aspects of the service in order to provide a comprehensive
picture of performance, including user perspective.

Tested: Have national and international PIs been considered?

There should be due consideration given to indicators that have been tried and tested in the national and international
arena rather than developing new indicators for the same purpose.

Safe: Will an undue focus on the PI lead to potential adverse effects on other aspects of quality and
safety?
The indicator should not lead to an undue focus on the aspect of care being measured that may in turn lead to a
compromise in the quality and safety of other aspects of the service.

Avoid duplication: Has consideration been given to other projects or initiatives?

Prior to developing the indicator, due consideration should be given to other projects or initiatives to ensure that
there will not be a duplication of data collection.

Timeliness: Is the information available within an acceptable period of time to inform decision-makers?
The data should be available within a time period that enables decision-makers utilise the data to inform their
decision-making process. If the data are required for operational purposes, then it will be required within a shorter
timeframe than data used for long term strategic purposes.

While the criteria above are applicable to all PIs, a national nursing and midwifery PI would have to meet a further
criterion:

Impact: Does it measure the impact of national nursing and midwifery policy?
National nursing and midwifery PIs should offer a high-level snapshot of the effect of national policy on service
delivery.

Requests for endorsement of PIs for use at national level will be submitted to the National Nursing and Midwifery PI
Endorsement Group along with the completed template shown in Appendix 1.

2.B.6 What makes a good indicator?

Before embarking on the development of PIs, it is useful to consider what makes a good indicator. According to
Griffiths et al. (2008):

• indicators must be measurable with available data at reasonable cost;

• there must be evidence of variability associated with nursing and this variability must be substantial;
• for process or structure measures, evidence must support links to important outcomes;
• the indicator must be recognised as important (by the public, managers and nurses) and the contribution
of nursing must also be recognised by nurses and others;
• nurses must have responsibility for actions that lead to the outcome in terms of legitimate authority, self-
perception and sphere of practice;
• there must be sufficient knowledge to inform remedial action;
• measures should be chosen to minimise the risk of gaming, where improving performance on the indicators
detracts from overall performance;
• measures, especially measures of outcome, generally need to be risk adjusted to ensure comparability
across settings.

They also concede that the development of nursing and midwifery indicators is challenging but necessary to show
the contribution to patient care made by nursing and midwifery. The key challenges identified are:

25
 • defining data and full specification of indicators;
• adjusting for risk;
• improving the quality of clinical coding;
• identifying indicators for nursing’s impacts in mental health, community, primary care and paediatric
settings;
• identifying and defining indicators that cross care pathways and boundaries;
• timely reporting at the nursing unit level;
• delivering action to improve quality.

2.B.7 Matrix of National Nursing and Midwifery PIs

It is important when developing national nursing and midwifery PIs that, as far as possible, a mixture of structure,
process and outcome PIs are developed for the areas specified in the following matrix (Table 4):

26
 MATRIX OF NATIONAL NURSING AND MIDWIFERY PIs

(with sample PIs including indicators from Heslop and Lu (2014) and Griffiths et al. (2008)
Area Structure Process Outcome
Nursing and • Staffing levels • Staff knowledge/ • Safety of nursing job
Midwifery (nursing hours per Skills/Expertise/ • Staff satisfaction and well-
Workforce patient day) Education level being
• Practice • Training • Staff intent to leave
environment • Number of research • Client/patient experience
• Sick absence rates collaborations • Nurse sensitive outcomes
• Nurse recruitment (Pneumonia, Urinary Tract
• Nurse turnover Infection, Pressure Ulcer)
• Staffing bank/ • Number of research papers
agency published
Quality of • Patient • Nursing • Failure to rescue
Nursing and characteristics intervention/nursing • Pressure ulcer incidence
Midwifery • Education level practice • Falls incidence
Care • Years of experience • Nursing • Nosocomial infection
• Hours of nursing documentation incidence
care per patient day • Pressure ulcer • UTI incidence
• Patient acuity prevention
• Patient turnover • Falls prevention
• Workload intensity • Nosocomial infection
prevention
• UTI prevention
Client/Patient • Client/patient • Medication • Client/patient/family
Experience involvement administration satisfaction with nursing
• Client/patient errors care
centred care • Use of restraints • Client/patient/family
• Integrated care • Post-operative satisfaction with pain
complications management
• Length of stay • Unplanned hospital visits
post discharge
• Vital signs status, self-care
ability
• Symptom resolution/
reduction
• Waiting time of nursing care
• Complaints
Regulation • Staffing levels at • Number of • Number of registrations
regulator applications • Outcome of FTP hearings
• Agency staffing at for registration • Regulator staff experience
regulator processed • Nurse/Midwife employer
• Sick absence rates • Number of experience
at regulator complaints referred • Applicant/registrant
• Staff turnover at to Preliminary experience
regulator Proceedings • Time taken to process
• Sensitivity of Committee registration application
nursing and • Number of • Time taken to process
midwifery register to complaints referred complaints referred to
meet workforce and to/heard by Preliminary Proceedings
national reporting Fitness to Practise Committee
needs Committee • Time taken to process
complaints referred to/
heard by Fitness to Practise
Committee

27
The areas in the matrix were chosen to reflect the need to give the fullest picture possible, i.e., including PIs of
quality of nursing and midwifery care, PIs that underpin that quality (Nursing and Midwifery Workforce), and PIs
that indicate what the experience is like from the client/patient’s perspective. ‘Regulation’ has been added to reflect
the vital role of the nursing and midwifery regulator (the NMBI) in nurse and midwife staffing due to its registration
function.

2.B.8 Process for developing the PIs

Certain factors need to be considered when developing PIs to ensure that they are robust and fit for purpose. These
factors, adapted from the HIQA guidance (2010, updated 2013), are as follows (Table 5):

1. Define the • Define the goals of the measurement

audience • Reasons for measurement
and use for • Intended audience for measurement
measurement • Identify the domains for which the measurement is intended
• Identify a balanced suite of PIs

2. Consult with Consult, as appropriate, with all relevant stakeholders (decision-makers;

stakeholders clinicians; service providers; data capture and analysis staff; service users; and
and advisory national/regional committees that have responsibility for health information
group and standards to ensure compliance, etc.) throughout the data development
process bearing in mind that the service user is the most important stakeholder.
Advisory group membership should include the relevant health professionals and
stakeholders for the area being measured. It should include epidemiologists (or
healthcare quality experts with experience in epidemiology) to ensure that the
data collection and analysis methodology is reliable and valid.
3. Choose the Choose the area to be measured based on:
area to measure • the importance of the problem
• service-user safety
• potential for improvement
• controllability by health or social care system/professionals.

28
4. Achieve A number of approaches have been developed to assist in identifying a balanced
a balance in set of PIs including:
measurement • the ‘balanced scorecard’ (developed by Kaplan and Norton) suggests four
perspectives of a performance indicator set to provide a comprehensive view
of the performance of an organisation:
• service user perspective measures how an organisation meets the
assessed needs and expectations of the service user
• internal management perspective measures the key business
processes that have been identified as necessary for a high quality and
effective service
• continuous improvement perspective measures the ability of the
organisation’s systems and people to learn and improve
• financial perspective measures the efficient use of resources to
achieve the organisations objectives. It is important to note that there
are differences between the application of the balanced scorecard for a
health sector organisation and for a healthcare sector, e.g., in the units
of analysis, purposes, audiences, methods, data and results (Zelman et
al., 2003).

• The “Three Es” framework uses the three domains of economy, efficiency and
effectiveness:
• economy which measures the acquisition of human and material
resources of the appropriate quality and quantity at the lowest cost
• efficiency which measures the capacity to provide effective healthcare
using minimum resources
• effectiveness which measures the degree to which the organisation
attains established goals.

• Performance frameworks identify domains of healthcare performance that

can be used as a basis for the development of performance indicator sets

• The process of achieving a balanced set of PIs can be assisted by

incorporating the structure, process and outcome classification into the
methodology for assessing the healthcare system. These classifications are
interdependent and structure can have an impact on processes which in turn
can have an impact on outcomes.
5. Determine • PIs should be chosen based on the judgement and consensus of experts and
selection criteria potential users.
• Section 5 outlines selection criteria for PIs that will also be used by the
National Nursing and Midwifery PI Endorsement Group.
6. Define the A clear definition of the indicator should be prepared to ensure it is appropriately
indicator interpreted by those with responsibility for collecting the data (the template in
Appendix 1 sets out the type of detail required).

29
The process for the development of nursing and midwifery PIs is summarised in the following figure (Figure 7)
(National Council for the Professional Development of Nursing and Midwifery, 2010 (adapted from HIQA, December
2010)):

Step 1.
goal of nursing and midwifery care

Validity Step 2. Choose the area of care to measure

Reliability
Explicit evidence base
Acceptability
Feasibility
Sensitivity Step 3.

Relevance
Balance
Tested
Safe
Step 4.
Avoid duplication
Timeliness

PI 1 PI 2 PI 3
Metadata Metadata Metadata

Minimum Minimum Minimum

data set data set data set

Step 5. Data reporting

Review
Step 6. Pilot test
indicators

Step 7. Commence use of PIs

30
2.B.9 Pilot Test Performance Indicators
As part of its development, a PI must be tested through a pilot to refine it and ensure that it is fit for purpose. A clear
plan for the pilot should be prepared that includes the criteria for selecting the pilot site(s), proposed length of pilot
test, training and education of participants and information to be obtained from the pilot.

The HIQA guidance (2010, updated 2013) suggests that the information to be obtained from the pilot can be posed
as a number of questions, such as:

• are there validity and reliability issues in relation to data collection?

• is the information obtained from the PI of use in decision-making?
• can the PIs contribute to improved service and quality of care?
• have there been any issues identified through quality checks and are data recorded
• consistently?
• what additional measures that were not in place for the pilot, need to be instituted for the PIs to be rolled-
out successfully?
• are there any modifications necessary to the PI specifications?

The pilot test can also be used to validate the PIs against the selection criteria used for prioritising and endorsing
the PIs (see 2.B.5).

2.B.10 Determine review frequency

There should be a plan to review the PI at regular intervals to assess whether it needs to be modified in accordance
with changes in the health service. In addition, PIs should be assessed for removal or replacement. Reeves et al.
(2010) propose assessing indicators based on their recent history of achievement and exception reporting rates.

2.B.11 What are Metadata?

Metadata are data that define and describe other data (HIQA 2010, updated 2013).

2.B.12 Data Reporting

It is envisaged that over time national nursing and midwifery PIs will be included in the HSE’s annual National Service
Plan as KPIs. Data reporting on these KPIs will take place through the HSE’s monthly Performance Report.

2.B.13 Tracking Performance Indicators

KPIs that have been endorsed for inclusion in the National Service Plan, by the proposed Joint DoH–HSE Oversight
Group, will be logged in a KPI tracking system.

The data to be collected for each PI will vary depending on the PI and, therefore, so will the system for its collection.
For example, the HSE’s QCM are currently collected on the Test Your Care (TYC) System, which collects nursing
care process indicators and patient experience questions to monitor and improve standards of patient care. The TYC
System is available nationally to agreed services implementing nursing and midwifery QCM. Users access the QCM
system on TYC using a personalised username and password issued by the regional NMPD Project Officer. Names
of individuals who may access the data entry field and the reporting fields are determined by the clinical service
and supplied to the NMPD Project Officer who arranges the issuing of passwords. Users have access to locations
as determined by their hospital/service governance or as agreed by the relevant Director of Nursing/Director of
Midwifery.

Nursing and midwifery workforce data and patient acuity/dependency data would ideally be collected in hospital/
community settings using an appropriate IT system (or systems, if one is not able to accommodate both hospital
and community and/or nursing and midwifery data). There are systems in existence that manage and utilise nursing
resources through the collection of patient acuity and bed utilisation data. These systems also allow for extensive
reporting.

The minimum data set for the nursing and midwifery workforce would be agreed with the NMBI and collected by
the NMBI, using an augmented version of its current registration system. These data could be provided to the
Department of Health on an annual basis.

31
2.B.14 Conclusion
The evidence review (Skills for Health, 2016) indicated that although PIs are being collected on a significant scale
and are being used to improve a range of areas including efficiency, staff development and clinical outcomes, there is
a demand for the development of appropriate guidance. It is intended that this Framework will provide the required
policy direction and guidance.

The first step in implementing this Framework will be the establishment of a group of key stakeholders to agree the
minimum data set for the nursing and midwifery workforce. The second step will be the establishment of a group of
key stakeholders to prioritise and endorse nursing and midwifery performance indicators for use at national level.
Implementation will involve extensive consultation with stakeholders.

In acknowledgement of the constant evolution of the health services, it is recommended that this Framework be kept
under regular review and updated, where appropriate.

32
SECTION 3 — REFERENCES AND APPENDIX

3.1 References

1. Aiken, LH, Sloane, DM, Bruyneel, L, Heede, KVD, Griffiths, P, Busse, R, Diomidous, M, Kinnunen, J, Kózka,
M, Lesaffre, E, McHugh, M, Morenocasbas, MT, Rafferty, AM, Schwendimann, R, Tishelman, C, Achterberg,
TV, Sermeus, W (2014), Bachelor’s Education for Nurses and Better Nurse Staffing are Associated with Lower
Hospital Mortality in 9 European Countries, The Lancet, 383(9931), p1824-1830

2. Bevan, G (2006), Have targets improved performance in the English NHS?, British Medical Journal, 332:419

3. Department of Health (1998), Report of the Commission on Nursing: A blueprint for the future. Government of
Ireland, Dublin Stationary Office

4. Department of Health (July 2002), The Nursing and Midwifery Resource: Final Report of the Steering Group:
Towards Workforce Planning (http://health.gov.ie/wp-content/uploads/2014/06/The-Nursing-and-Midwifery-
Resource-Towards-Workfoce-Planning-July-2002.pdf)

5. Department of Health (June 2016), Office of the Chief Nursing Officer Position Paper One — Values for Nurses
and Midwives in Ireland (http://health.gov.ie/wp-content/uploads/2016/05/Final-position-paper-PDF.pdf)

6. Department of Health UK (December 2014), The NHS Outcomes Framework 2015/16

7. Griffiths, P, Jones, S, Maben, J, and Murrells, T (2008), State of the art metrics for nursing: a rapid appraisal,
National Nursing Research Unit, King’s College London

8. Gubb, J (2009), Have targets done more harm than good in the English NHS? Yes, British Medical Journal,
338:a3130

9. Heslop, L and Lu, S (2014), Nursing-sensitive indicators: a concept analysis, Journal of Advanced Nursing,
70(11): 2469–2482

10. HIQA (2010, updated 2013), Guidance on developing Key Performance Indicators and Minimum Data Sets to
Monitor Healthcare Quality

11. HSE (Office of the Nursing and Midwifery Services Director) (May 2015), Guiding Framework for the implementation
of nursing and midwifery Quality Care Metrics in the Health Service Executive Ireland

12. Maben, J, Morrow, E, Ball, J, Robert, G, and Griffiths, P (November 2012), High Quality Care Metrics for Nursing,
National Nursing Research Unit, King’s College London

13. National Council for the Professional Development of Nursing and Midwifery (September 2009), Measuring the
nursing and midwifery contribution: update 2

14. National Council for the Professional Development of Nursing and Midwifery (June 2010), Clinical Outcomes —
Promoting patient safety and quality of care: implications for nurses and midwives

15. National Council for the Professional Development of Nursing and Midwifery (December 2010), Key Performance
Indicators: a guide to choosing, developing and using KPIs for clinical nurse/midwife specialists and advance
nurse/midwife practitioners

16. National Health Performance Authority (May 2012), Performance and Accountability Framework, subsection 4.1
(http://www.nhpa.gov.au/internet/nhpa/publishing.nsf/Content/PAF)

17. Reeves, D, Doran, T, Valderas, JM, Kontopantelis, E, Trueman, P, Sutton, M, Campbell, S, Lester, H (April 2010),
How to identify when a performance indicator has run its course, British Medical Journal, 340

18. Skills for Health (2016), Towards a National Framework for Key Performance Indicators, Evidence Report

33
19. VanDeVelde-Coke, S, Doran, D, Grinspun, D, Hayes, L, Sutherland Boal, A, Velji, K, White, P, Bajnok, I, Hannah,
K (2012), Measuring outcomes of nursing care, improving the Health of Canadians: NNQR (C), C-HOBIC and
NQuiRE, Nursing Leadership, 25(2) July 2012: 26-37

20. Werley, H, Devine, E, Zorn, C, Ryan, P, Westra, B (April 1991), The Nursing Minimum Data Set: Abstraction tool
for Standardized, Comparable, Essential Data, American Journal of Public Health, Vol 81, No 4, pp 421–426

21. World Health Organisation (2015), Human Resources for Health Information System — Minimum Data Set for
Health Workforce Registry, WHO, Geneva

22. Zelman, WN, Pink, GH, and Matthias, CB (Summer 2003), Use of the Balanced Scorecard in Health Care, Journal
of Health Care Finance, 29(4): pp 1–16

34
3.2 Appendix 1

Template (adapted from HSE KPI Template 2013, http://lenus.ie/hse/handle/10147/267717) to accompany request
for endorsement of PIs for use at national level:

1. PI Title Exact title of PI.

• This is the short title
2. PI Description Description of the PI including a description of the target population.
• Where definitions exist in other documents these should be included here e.g.
Vision for Change or HIQA report
• Where definitions exist that are very long they can be referenced here
3. PI Rationale Rationale for the measurement of the PI
• Government or HSE priority
• Importance of area e.g.: high incidence, high morbidity, high service-user
volumes, costly to provide. Consequences of poor performance on target
population
• Potential for improvement if performance is known
• Evidence to support outcome improvement if target reached
• Existence of agreed/recognised target or benchmark

Indicator Please tick which Indicator Classification this indicator applies to, ideally choose
Classification one classification (in some cases you may need to choose two):
❑ Person Centred Care ❑ Effective Care ❑ Safe Care
❑ Better Health and Wellbeing ❑ Use of Information
❑ Workforce ❑ Use of Resources
❑ Leadership, Governance and Management
4. PI Target Indicate the target for the PI — a target should be set for the PI to inform
progress towards an acceptable level of performance.
• This may take the form of expected activity or a benchmark against similar
organisations or systems e.g. a HIQA target.
5. PI Calculation Indicate how the PI will be calculated.
The target population is called the denominator and includes all services users
or events that qualify for inclusion in the measurement process (for ratios the
numerator is not included in the denominator).
The subset of the target population that meets the criteria as defined in the
indicator is called the numerator.
• Specify whether PI is expressed as a proportion; ratio; percentage; or count
and how it should be interpreted against target.
6. Data Source Indicate the data source(s) which will be used for the PI.
Data Completeness This should give details of primary data collection, e.g. PHN records, patient
Data Quality Issues charts, administration data bases, survey data. It should indicate the route
through which data are communicated and collated, e.g. provided by PHNs to
LHOs to RDO Business Unit to BIU.
Data Completeness and any Data Quality issues
Specify the % coverage of this PI.
Specify any data quality issues known.
7. Data Collection Indicate how often the data to support the PI will be collected
Frequency ❑Daily ❑Weekly ❑ Monthly ❑Quarterly ❑Bi-annually ❑Annually
❑Other – give details:
8. Tracer Conditions Indicate the terms which should be used to identify what should be included in the
data. This should include synonyms, International Classification of Disease (ICD)
and SNOMED (Systematised Nomenclature of Medicine Clinical Terms) where
applicable.
9. Minimum Data Set Indicate what core data items (with definitions) should be collected for the
purpose of reporting the PI.
• The data lines can be included here or an example appended for information
where there is a definitive minimum data set available.

35
10. International Indicate if this PI is collected in other jurisdictions outside of Ireland and therefore
Comparison allows for international comparison.
11. PI Monitoring Indicate how often the PI will be monitored and by whom
PI will be monitored on a (please indicate below) basis:
❑Daily ❑Weekly ❑ Monthly ❑Quarterly ❑Bi-annually ❑Annually
❑Other – give details:
Please indicate who is responsible for monitoring this PI:
12. PI Reporting Indicate how often the PI will be reported (at a National level this will align with
Frequency the agreed reporting timeframe in the NSP).
❑Daily ❑Weekly ❑Monthly ❑Quarterly ❑Bi-annually ❑Annually
❑Other – give details:
13. PI Report Period Indicate the period to which the data applies
❑ Current (e.g. daily data reported on that same day of activity, monthly data
reported within the same month of activity)
❑ Monthly in arrears (June data reported in July)
❑ Quarterly in arrears (quarter 1 data reported in quarter 2)
❑ Rolling 12 months (previous 12 month period)
❑ Other – give details: _________________________
14. PI Reporting Indicate the level of aggregation – this refers to the combination of results
Aggregation to provide a broader picture of performance for example over a geographical
location.
❑ National ❑ Regional ❑ LHO Area ❑ Hospital
❑ County ❑ Institution ❑ Other – give details:
15. PI is reported in Indicate where the PI will be reported for example:
which reports? ❑ Corporate Plan Report ❑ Performance Report (NSP/CBP) ❑CompStat
❑Other – give details:
16. Web link to data Indicate the web link to the data (where this is available)
17. Additional Include any additional information relevant to the PI
Information
Contact details for Data Name:
Manager/Specialist Lead Email address:
Telephone Number:
National Lead and Name:
Directorate Email address:
Telephone Number:
Directorate: