Wallace, 2009
Wallace, 2009
Wallace, 2009
Standardised
Development of methods for HACCP
standardised HACCP assessment assessment
Carol A. Wallace and Susan C. Powell
Lancashire School of Health and Postgraduate Medicine, 723
University of Central Lancashire, Preston, UK, and
Lynda Holyoak
Department of Psychology, University of Central Lancashire, Preston, UK
Abstract
Purpose – Assessment of HACCP systems is a key element in assuring the effective management of
food safety. However, there is no accepted approach or common methodology available to HACCP
practitioners, auditors or regulatory bodies. This paper seeks to examine this situation
Design/methodology/approach – This paper reviews previous approaches to HACCP audit and
describes developments in audit and audit methods based on a long-term study of HACCP in a
multinational organisation.
Findings – The proposed audit tools provide a useful method for collection of data on the
effectiveness of HACCP plans and their implementation.
Research limitations/implications – Limitations of using this approach are identified and
discussed.
Originality/value – New audit tools for validation and verification of HACCP effectiveness are
proposed.
Keywords Environmental health and safety, Food safety, Quality management
Paper type Conceptual paper
Introduction
Food safety is an increasingly important public health issue. The World Health
Organisation (WHO) estimates that in 2000 2.1 million people died from diarrhoeal
diseases (WHO, 2002). In this context, governments worldwide are intensifying their
efforts to improve the safety of the food supply. The work of the WHO Food Safety
Programme includes implementing and improving food safety systems, promoting
good manufacturing practices and educating manufacturers, retailers and consumers
about appropriate food handling (WHO, 2002).
The hazard analysis and critical control point system (HACCP) is the internationally
agreed approach to food safety control. The reference standard for implementation of
HACCP is published by the Codex Alimentarius Commission of the joint United
Nations Food and Agriculture Organisation (FAO)/WHO Food Standards Programme
(Codex Committee on Food Hygiene, 1993; Codex Committee on Food Hygiene, 1997).
Hazard Analysis Critical Control Point is designed to control significant food safety
hazards, i.e. those hazards that are likely to cause an adverse health effect when
products are consumed. In order for HACCP to be effective, it is essential that it is
supported by Good Manufacturing Practice (GMP) or Prerequisite Programmes that British Food Journal
Vol. 107 No. 10, 2005
control the general hygiene and environmental conditions in a food processing pp. 723-742
operation (ILSI, 1999; Sperber, 1998;Wallace and Williams, 2001). In a manufacturing q Emerald Group Publishing Limited
0007-070X
operation, food safety management is achieved through the application of system DOI 10.1108/00070700510623513
BFJ “building blocks” – safe product design, prerequisite programmes and HACCP –
107,10 operating under the framework of the overall operations management system
(Figure 1).
The systems of the HACCP “building block” are developed through application of
the internationally agreed HACCP Principles (Codex, 1997). For effective food safety
management, all three “building blocks” need to be adequately designed and their
724 implementation verified. Assessment approaches are needed to demonstrate
effectiveness of the prerequisite programmes, safe product design and HACCP
blocks. This paper focuses on the assessment of HACCP.
The HACCP system will prevent food-borne disease outbreaks only if it is
understood and applied correctly. There are very few records of food-borne disease
outbreaks in which a food company operating with full commitment and
understanding of the HACCP system has been implicated. Hence when the HACCP
system is applied correctly, food-borne disease should not occur. In the outbreaks that
have been reported involving industries that have implemented the HACCP system,
serious flaws could be detected in their HACCP plans or the implementation of HACCP,
reflecting a lack of understanding of, or commitment to the HACCP system (Motarjemi
and Kaferstein, 1999).
Figure 1.
Manufacturing food safety
management model
independent assessments carried out by regulators and third-party bodies should be Standardised
undertaken. A further WHO Consultation in Geneva in 1998 considered “The Role of HACCP
Government Agencies in Assessing HACCP” (WHO, 1998) and provided guidance on:
. government roles and responsibilities;
assessment
.
principle activities for regulatory assessment;
.
organisation and planning of assessments; 725
.
the assessment process and its implementation;
.
assessor competencies; and
.
specific problems encountered in assessing HACCP systems.
Whilst the planning and preparation stage is important in laying foundations for
HACCP, effectiveness assessment of the HACCP system generally requires evaluation
BFJ of the outcomes from stages 2), 3) and 4) (Mortimore and Wallace, 1998). Similarly, the
107,10 International Life Sciences Institute (ILSI, 1999) describes the assessment of three
phases of HACCP:
(1) Technical evaluation of the process of performing the hazard analysis and
establishing control measures;
(2) Implementation of the resulting HACCP plan, which includes the process of
726 validation; and
(3) Operation of HACCP, which includes the assessment activities of verification
and auditing (Mortimore and Wallace key stages 2), 3) and 4).
Thus, broad criteria for HACCP effectiveness assessment can be established (Table I).
Valid for control of significant food Validity of the HACCP plan depends on the knowledge and
safety hazards that are likely to occur in skills of the HACCP team in terms of understanding the
the type of operation being studied process, its ingredients and the likely occurrence of hazards;
and ability to understand and apply the Codex HACCP
principles to develop an effective HACCP plan
Implemented into the every-day This is achieved through a hand-over process from the
operation of the food company HACCP team to operations management and line personnel,
where it is important that operations personnel take
ownership for day-to-day operation of the HACCP plan.
Implementation is demonstrated through monitoring of
critical control points, taking corrective action where
necessary and keeping records
Adequately verified and maintained Verification includes audit of the working HACCP system
for compliance with the HACCP plan and review of food
safety records. Maintenance involves ongoing update
Table I. procedures both to evaluate impact of changes in the
HACCP effectiveness operation and to keep up to date on knowledge of food safety
criteria hazards
Effectiveness Audit Worksheet”, as written, would elicit information on the Standardised
effectiveness of corrective action taken at CCPs. HACCP
A number of HACCP Audit Checklists and example questions have been published
on the internet, e.g. the United States National Conference on Interstate Milk Shipments assessment
(NCIMS) Dairy HACCP Audit Report Form (NCIMS, 2004), the Australian Standing
Committee on Agriculture and Resource Management HACCP Plan Audit Checklist
(Standing Committee on Agriculture and Resource Management, 2003; Cooper and 727
Pronk, 1997). The NCIMS Form (NCIMS, 2004) provides a tick-sheet of items to be
covered in assessing both prerequisite programmes and HACCP, while the Australian
Standing Committee checklist gives specific questions to be asked about different
aspects of HACCP plan development and implementation. Cooper and Pronk (1997)
outline methodology that can be used to assess HACCP Plan procedures, e.g. visual
observation and records review, and describe eight questions that can be used to assess
management support for HACCP and three questions for review of the HACCP plan.
Torres (2000) includes a checklist criteria in his discussion of government agency
approaches. He appears to look for presence or absence of HACCP System elements
and does not, as written, challenge the effectiveness of these elements.
Although all of these approaches contain useful guidance on topics/questions to be
covered, it was considered that they do not offer a “complete” approach to the
assessment of HACCP systems.
More recently, (Wilkinson and Wheelock, 2004) published a checklist of questions
used in assessing the effectiveness of HACCP implementation and maintenance in food
production plants on the island of Ireland. This checklist is designed to be applied by
trained auditors. It is a detailed approach, and includes aspects of food safety
management as well as HACCP development, implementation and maintenance.
However, the checklist is written as a series of interview questions and therefore must
be applied at the manufacturing site with assistance from site personnel. Whilst it is
clearly important to verify HACCP implementation and maintenance on-site, it is also
possible to assess the validity of HACCP documentation remotely. This has benefits of
identifying major flaws in the approach without the time and travel expense of a site
visit.
Highly structured sets of interview questions such as those proposed by Wilkinson
and Wheelock (2004) also have the potential to limit auditor flexibility and judgment. It
is therefore important to obtain a balance, including a structured framework whilst still
allowing some subjectivity based on auditor experience. This is consistent with
international guidelines for quality and environmental management systems auditing
(BS EN ISO 19011:2002) which states that “the use of checklists and forms should not
restrict the extent of audit activities, which can change as a result of information
collected during the audit”. Therefore, a checklist framework needs to act as an aide
memoir on the essential points to be assessed, while allowing flexibility to the auditor
in making expert judgements about HACCP status.
Some approaches (Sperber, 1998) use simple scoring systems based on allocating
points to “satisfactory” or “unsatisfactory” ratings whilst others (Gagnon et al., 2000;
Mortimore, 2000; NCIMS, 2004) rely on listing of deficiencies/non-compliances. Cooper
and Pronk (1997) describe scoring systems as a topic of contention, indicating that
some companies find scoring useful to indicate progress whilst others believe that they
are difficult to keep objective. Indeed, this is confirmed by discussions with many food
BFJ safety professionals who are wary of the use of arbitrary scores, particularly
107,10 percentages, since a scoring system could be envisaged where a particular HACCP
Plan achieves a 95 per cent score but still has a major food safety flaw (Palmer, 2005). It
is therefore crucial that any scoring system is carefully designed and that its
limitations are clearly understood.
As part of a research programme investigating the impact of training on the
728 effectiveness of HACCP in a multinational organisation, it was necessary to identify
standard approaches to assess the effectiveness of HACCP Systems. This was to allow
assessment of global data from manufacturing sites that had been collected by trained
HACCP auditors using a consistent checklist format. It was also necessary to identify a
comparative measure of individual manufacturing site HACCP progress so that
priority targets for additional training and HACCP support could be identified.
Therefore a scoring system was considered to be an essential part of the assessment
approach.
The aims of the study pertaining to HACCP assessment were:
.
Establish strategies for the assessment of HACCP effectiveness;
.
Develop and validate tools to measure HACCP effectiveness;
.
Determine the validity of HACCP Plans developed by trained HACCP teams; and
.
Evaluate the implementation, verification and maintenance of HACCP plans;
Methods
The methods described here were developed in 2003 for data collection between
October 2003 and March 2004. At that time, there was no accepted tool that provided a
comprehensive measure of the effectiveness of HACCP system implementation. Hence,
it was important to identify a way of assessing the HACCP systems that had been
developed by the HACCP teams following training. Previously published guidance on
HACCP assessment (WHO, 1998; ILSI, 1999; Mortimore and Wallace, 1998; Mortimore,
2000; Sperber, 1998) was considered and, in order to establish a standardised audit
framework that would cover all required aspects of HACCP assessment, two audit
checklist tools were developed. These two tools provide a step-wise approach to
HACCP Assessment (Figure 2).
The HACCP Audit Tools were designed to assess both the validity of the HACCP
Plan (i.e. will the HACCP Plan control all likely hazards?), and the implementation and
maintenance of the HACCP system (i.e. is the HACCP Plan working in practice in the
factory?). As was recommended practice in previous approaches, the tools were
designed to allow off-site assessment (desk-top audit) of HACCP documentation for
validity as well as on-site assessment of implementation and maintenance
effectiveness. Use of off-site desk-top audit was also important to allow assessment
of a larger sample of sites than could be covered by on-site audit, and thus give an
indication of site positioning for HACCP competence throughout a large
manufacturing group. On-site assessment was necessary to provide more rigorous
assessment of HACCP at a smaller sample of sites.
Tool 1 – A HACCP desk-top audit checklist (Figure 3) was developed by selecting
aspects of HACCP audit that focused on the approach to HACCP plan development and
its validity. Tool 2 – HACCP verification audit checklist (Figure 4) was developed as a
Standardised
HACCP
assessment
729
Figure 2.
Use of audit tools in
HACCP assessment
730
Figure 3.
HACCP documentation
desk-top assessment
checklist
Standardised
HACCP
assessment
731
Figure 3.
BFJ
107,10
732
Figure 3.
Standardised
HACCP
assessment
733
Figure 3.
BFJ
107,10
734
Figure 3.
Questions assessed not only the completeness of the documentation, but also validity
of the content. It was intended that the checklist would be applied by experienced
auditors who had known HACCP expertise. Guidance notes were included, where
appropriate, to ensure consistency of application. Wording of individual questions was
designed to be appropriate to a remote audit of HACCP plan paperwork without access
to further site information.
Questions were not developed for Principles 6 (Verification) and 7 (Documentation)
since, although documentation is considered as part of a desk-top audit, these
principles can only be fully assessed during an on-site audit. They are therefore
included in Tool 2.
In order to produce comparable data from application of the desk-top audit tool, a
scoring system was also developed. Since all the HACCP principles being assessed and
the preparation included in Codex Preliminary steps were deemed equally important to
the development of a valid HACCP system, each of the six question groups was
allocated 20 points. However, weighting of individual questions within each question
Standardised
HACCP
assessment
735
Figure 4.
HACCP verification audit
checklist
BFJ
107,10
736
Figure 4.
Standardised
HACCP
assessment
737
Figure 4.
BFJ
107,10
738
Figure 4.
For verification of HACCP effectiveness, both tools can be applied together on-site,
requiring the trained auditor(s) to visit the site for 2-3 days to perform the
assessment.
Discussion
Both assessment tools have now been used to collect data on effectiveness of HACCP
plans and their implementation status. This has demonstrated that the tools are a
useful way of collecting information about HACCP progress. Throughout this process
a number of benefits and limitations have been identified.
Limitations of desk-top assessment include that the assessment can only be based
on the paperwork submitted so anything not submitted is not assessed. In this study it
was found that further communication with sites may be necessary to clarify details of
information provided or to obtain missing information. The approach is further limited
in that, as an off-site assessment, it cannot give a complete judgment on the validity of
hazard identification but rather an assessment of the approaches taken and likely
hazards for the product group. However, it is similarly questionable whether on-site
assessment by one to two auditors could give a more complete judgement on hazard
identification validity if they are independent from the site.
The auditor plays a vital role in the assessment of HACCP systems and both their Standardised
HACCP and audit expertise is crucial to success. In this study it was concluded that HACCP
training of auditors in the use of the checklist tools is also important to ensure
consistency of application. assessment
Desk-top audit was found to be a useful way to gain a comparative measure of
HACCP development progress and documentation quality at a large number of sites,
however it is important to remember that this assessment is based on paperwork alone 741
and so needs to be triangulated with on-site assessment data. From this study it was
concluded that the combined use of Tools 1 and 2 by trained HACCP auditors could
give a detailed assessment of HACCP Plan validity along with verification of its
implementation and maintenance.
As described previously, the use scoring systems in HACCP assessment has been a
contentious issue. None of the previously described approaches apply weighted
scoring to the differential importance of specific parts of HACCP documentation and
system application. This could be one way of establishing a pragmatic scoring system
that reflects the consequence of flaws in a HACCP system. It was found that scoring
can be used effectively to monitor progress at a particular manufacturing site or to
allow comparison of progress within a group of sites, however the scoring system must
be carefully designed for its specific use. In this study a scoring system was necessary
for Tool 1 in order to assess and compare progress in HACCP at a large number of
manufacturing sites. It would also be possible to use the tool qualitatively without the
scoring system, to identify lists of weaknesses that need to be addressed for effective
HACCP plans.
It is hoped that the tools described here will provide a resource for industry to
assess the effectiveness of HACCP systems being developed and in operation at
manufacturing sites.
References
British Standards Institute (2002), BS EN ISO 19011:2002: Guidelines for Quality and/or
Environmental Systems Auditing, British Standards Institute, London.
Codex Committee on Food Hygiene (1993), “Codex guidelines for the application of the hazard
analysis critical control point (HACCP) system: adopted by the 20th session of the joint
FAO/WHO Codex Alimentarius Commission”, Food Control, Vol. 5 No. 3, pp. 210-13.
Codex Committee on Food Hygiene (1997), HACCP System and Guidelines for its Application.
Food Hygiene Basic Texts, Food and Agriculture Organisation of the United Nations,
World Health Organization, Rome, pp. 33-45.
Cooper, K. and Pronk, I. (1997), “Auditing GMPs and HACCP plans”, Food Quality, October.
Gagnon, B., McEachern, V. and Bray, S. (2000), “The role of the Canadian government agency in
assessing HACCP”, Food Control, Vol. 11 No. 5, pp. 359-64.
ILSI (1999), ILSI Validation and Verification of HACCP, International Life Sciences Institute
Europe, Brussels.
Kvenberg, J., Stolfa, P., Stringfellow, D. and Spencer, G.E. (2000), “HACCP development and
regulatory assessment in the United States of America”, Food Control, Vol. 11 No. 5,
pp. 387-401.
Merican, Z. (2000), “The role of government agencies in assessing HACCP – the Malaysian
procedure”, Food Control, Vol. 11 No. 5, pp. 371-2.
BFJ Mortimore, S. (2000), “An example of some procedures used to assess HACCP systems within the
food-manufacturing industry”, Food Control, Vol. 11 No. 5, pp. 403-13.
107,10 Mortimore, S.E. and Wallace, C.A. (1998), HACCP: A Practical Approach, Aspen Publishers Inc.,
Gaithersburg, MD.
Motarjemi, Y. and Kaferstein, F. (1999), “Food safety, hazard analysis and critical control point
and the increase in food-borne diseases: a paradox?”, Food Control, Vol. 10 Nos 4-5,
742 pp. 325-33.
NCIMS (2004), Dairy HACCP Audit Report Form”, US Food & Drug Administration/Center for
Food Safety and Applied Nutrition, paper presented at the United States National
Conference on Interstate Milk Shipments, Frankfort, KY, available at: www.cfsan.fda.gov/
, comm/daihaud.html
Palmer, G. (2005), personal communication, RSSL Ltd, London.
Sperber, W.H. (1998), “Auditing and verification of food safety and HACCP”, Food Control, Vol. 9
Nos 2-3, pp. 157-62.
Standing Committee on Agriculture and Resource Management (2003), A Guide to the
Implementation and Auditing of HACCP, CSIRO Publishing, Canberra, available at: www.
publish.csiro.au/pid/1498.htm
Torres, D.H. (2000), “Role of government in HACCP audit: a Cuban perspective”, Food Control,
Vol. 11 No. 5, pp. 365-9.
Wallace, C. and Williams, T. (2001), “Pre-requisites: a help or a hindrance to HACCP?”, Food
Control, Vol. 12 No. 4, pp. 235-40.
WHO (1995), Training Aspects of the Hazard Analysis Critical Control Point System (HACCP),
World Health Organization, Geneva, WHO/FSF/FOS/96.3.
WHO (1998), Guidance on Regulatory Assessment of HACCP, World Health Organization,
Geneva, WHO/FSF/FOS/98.5.
WHO (2002), “Food safety and food-borne illness, Factsheet, No. 237, World Health Organization,
Geneva, available at: www.who.int/mediacentre/factsheets/fs237/en/
Wilkinson, J.M. and Wheelock, J.V. (2004), Assessing the Effectiveness of HACCP Implementation
and Maintenance in Food Production Plants on the Island of Ireland, Safefood Food Safety
Promotion Board, Cork.