Clin Chem Lab Med 2015; 53(6): 829–832

Editorial

Mauro Panteghini and Sverre Sandberg

Defining analytical performance specifications

15 years after the Stockholm conference
DOI 10.1515/cclm-2015-0303 The Stockholm Conference in 1999 was a landmark in
trying to achieve a consensus on how quality specifications
should be set and a hierarchy of models was established
The European Federation of Clinical Chemistry and [1]. In his paper, Callum Fraser gives not only a historic
Laboratory Medicine (EFLM) has as part of its mission review of the 1999 Conference, where he served as a co-
“to be the leading organization of Laboratory Medi- chair, but also a general survey of the history of labora-
cine in Europe.” Through leadership, the EFLM strives tory quality specifications [2]. He underlines that time has
to enhance patient care and improve outcomes by pro- come to revisit this hierarchy, investigating to what extent
moting and improving the scientific, professional and it is still valid or if it should be modified or expanded,
clinical aspects of laboratory medicine. To focus on which is the ultimate goal of the EFLM conference.
important aspects in laboratory medicine in which some The conference comprised five sessions. The first
strategic actions and measures should be taken, the three sessions examined the possibility and the pros and
EFLM decided to start a biannual series of conferences. cons to base performance specifications on clinical needs,
The first edition was held in Milan, Italy, on November on biological variation data or on state-of-the-art of the
2014. The conference was entitled “Defining analytical measurement procedure, respectively. In their article on
performance goals 15 years after the Stockholm confer- behalf of the EFLM Working Group (WG) on Test Evalua-
ence” as it was considered timely to address the topic of tion, Horvath et al. [3] discuss the complexity of outcome-
performance specifications both because it was a long related models in a way that allows investigation of the
time since it was previously addressed and because impact of analytical performance on medical decisions
performance specifications are central for the clinical and patient management. Whilst it is acknowledged that
application of test measurements and are also of vital these types of evaluations are difficult and may not be
importance for quality control measures that should possible for all measurands in laboratory medicine, the
be taken in the laboratories. The conference, organized authors consider this approach as the “gold standard” for
in cooperation with the Centre for Metrological Trace- setting specifications. Next, Per Hyltoft Petersen, another
ability in Laboratory Medicine (CIRME) of the University one of the 1999 conference organizers, expanded the dis-
of Milan and the Institute for Reference Materials and cussion to cover the use of simulation studies as a way to
Measurements (IRMM) of the European Commission- model the probability of clinical outcome and the impact
Joint Research Centre, was very successful, with 215 of analytical performance upon them [4]. As an example,
participants from 41 different countries, embracing the the influence of analytical performance is investigated
five continents (the complete list of conference partici- for diagnosing of diabetes using hemoglobin A1c and for
pants is available as electronic Supplementary Material individuals at risk for coronary heart disease as defined by
that accompanies the article at http://www.degruyter. serum cholesterol concentrations. In another article, Thue
com/view/j/cclm.2015.53.issue-6/cclm-2015-0303/cclm- and Sandberg address the topic of performance specifi-
2015-0303.xml?format=INT). The delegates came from cations based on how clinicians use laboratory tests [5].
clinical laboratories, from EQAS providers and other These authors comment, however, that there is a large
professional organizations as well as from the in vitro variability among clinician behaviors that can limit the
diagnostics (IVD) manufacturers. In this issue of Clini- role of this approach and that it merely reflects the expec-
cal Chemistry and Laboratory Medicine all contributions tations of the clinicians rather than the ultimate specifica-
given during the conference are published (the slides of tions that should be set.
presentations are available online at http://www.efcclm. The group of Ricos reviews the rationale for using data
eu/index.php/educational-material.html). on biological variation to derive analytical specifications
830 Panteghini and Sandberg: Defining analytical performance specifications

and shares EQAS data from the Spanish program, which effort [16]. Oosterhuis and Sandberg present a model that
employs this approach [6]. Thereafter Carobene describes combines the “state-of-the-art” with biological variation
the heterogeneity of available biological variation data for the calculation of performance specifications. The
[7]. In general, any time an estimate of intra-individual validity of reference limits and reference change values
CV is > 33.3%, it is likely an indication that the distri- are central to this model [17]. In addition, they remind
bution of the individual variances is not Gaussian and the readers that the estimation of total error by adding
therefore the parametric statistical handling is not appro- the performance specification for maximum allowable
priate [8, 9]. Despite the questions regarding the reliabil- imprecision and maximum allowable bias is in princi-
ity of information, this author confirms the importance ple not valid. Nordin addresses “qualitative” laboratory
of a biological variation database that should, however, tests and help to clarify that performance specifications
include only products of appropriately powered studies. for nominal examination procedures must be set with
On behalf of the EFLM WG on Biologic Variation, Bar- respect to the type of test [18].
tlett et al. presents indeed a critical appraisal checklist Final articles of this series by the Plebani’s group
to enable standardized assessment of papers on biologic [19] and Sikaris [20] cover the remaining parts of the
variation [10]. The checklist identifies key elements to be total examination process (i.e., pre- and post-analytical
reported in this type of studies: method of sample collec- phases), highlighting the limitations that still hamper the
tion, number of subjects and samples, population type, use of quality performance specifications in these parts of
sample analysis and data derivation. This checklist can laboratory processes.
also be used a guide on how to perform studies on bio- In this Special Issue the Scientific Programme Com-
logical variation. mittee of the conference publishes a consensus statement
Haeckel and coworkers give us an overview of statis- that was elaborated in a draft form before the conference
tics that may be used to overcome some disadvantages and distributed to all the participants for further discus-
inherent to the model deriving performance specifica- sion [21]. Although the essence of the hierarchy originally
tions from the state-of-the-art of the measurement [11], established in 1999 was supported, new perspectives have
while Matthias Orth discusses the evolving German expe- been forwarded prompting simplification and explana-
rience and the regulation-driven performance criteria in tory additions. Basically, the recommended approaches
general [12]. for defining analytical performance specifications should
The fourth session of the conference discussed per- rely on the effect of analytical performance on clinical
formance criteria in different situations. In the paper outcomes or on the biological variation of the measurand.
by Schimmel and Zegers, we read about the need that The attention is primarily directed towards the measur-
calibration hierarchies have to be implemented cor- and and its biological and clinical characteristics, some
rectly and parameters contributing to measurement models being therefore better suited for certain measur-
uncertainty and systematic bias controlled and elimi- ands than for others. About the pre- and post-analytical
nated, respectively, by technically improving reference phases, it is time to go further and to include performance
methods and materials [13]. Panteghini and his group specifications for these extra-analytical phases. The spec-
describe the criteria behind the definition of allowable ifications should ideally follow the same models as the
limits for measurement uncertainty across the entire analytical phase.
traceability chain [14]. IVD manufacturers will need The main outcome of the conference has been the cre-
to take more responsibility and report the combined ation of an EFLM Task Force (TF) on Performance Specifi-
(expanded) uncertainty associated with their calibra- cations in Laboratory Medicine (TF-PS). Under the TF-PS
tors when used in conjunction with other components five Task and Finish Groups (TFG) have been established
of the analytical system (platform and reagents). This is dealing with the main topics of the conference (Figure 1).
more than what they are currently providing as informa- The terms of reference of the TFG are the following:
tion; typically they only provide the name of the higher –– TFG 1: To allocate different tests to different models
order reference to which the calibration is traced. Ceri- recognized in the Strategic Conference consensus
otti et al. discuss how to define a significant deviation statement and to give an overview and a reason for
in the internal quality control (IQC) practice by trans- why tests are allocated to the different models;
ferring uncertainty calculations onto the IQC chart [15]. –– TFG 2: To define performance specifications for the
Jones notes that there is an embarrassing variation in the most common measurands that should be used by
performance specifications being used by EQAS provid- EQAS organizers (for category I EQAS according to
ers, which needs harmonization through a collaborative Miller et al. [22]);
 Panteghini and Sandberg: Defining analytical performance specifications 831

EFLM Task force on performance specifications

in laboratory medicine

TFG on allocation of
laboratory tests to TFG on performance
different models for TFG on performance TFG on total error TFG on biological
specifications for the
performance specifications for EQAS variation database
extra-analytical phases
specifications

Figure 1: Structure of the EFLM task force on performance specifications in laboratory medicine.

–– TFG 3: To come up with a proposal for how to use the 4. Petersen PH. Performance criteria based on true and false
total error concept and how to possible combine per- classification and clinical outcomes. Influence of analytical
performance on diagnostic outcome using a single clinical
formance specifications for bias and imprecision;
component. Clin Chem Lab Med 2015;53:849–55.
–– TFG 4: To come up with a general proposal on how to 5. Thue G, Sandberg S. Analytical performance specifications
generate performance specifications for the pre- and based on how clinicians use laboratory tests. Experiences from
post-analytical phases; a post-analytical external quality assessment programme. Clin
–– TFG 5: To use a critical appraisal list to evaluate lit- Chem Lab Med 2015;53:857–62.
6. Ricós C, Álvarez V, Perich P, Fernández-Calle P, Minchinela J,
erature on biological variation and extract essential
Cava F, et al. Rationale for using data on biological variation.
information from the papers as well as summarizing Clin Chem Lab Med 2015;53:863–70.
the selected information on a database on the EFLM 7. Carobene A. Reliability of biological variation data available in
website. an online database: need for improvement. Clin Chem Lab Med
2015;53:871–7.
Acknowledgments: The Conference was generously and 8. Braga F, Ferraro S, Lanzoni M, Szoke D, Panteghini M. Estimate
of intraindividual variability of C-reactive protein: a challenging
unconditionally supported by Abbott Diagnostics, Bio-
issue. Clin Chim Acta 2013;419:85–6.
Rad, DiaSorin, Roche Diagnostics and Siemens Health- 9. Braga F, Ferraro S, Ieva F, Paganoni A, Panteghini M. A new
care Diagnostics. robust statistical model for interpretation of differences in serial
Author contributions: All the authors have accepted test results from an individual. Clin Chem Lab Med 2015;53:
responsibility for the entire content of this submitted 815–22.
10. Bartlett WA, Braga F, Carobene A, Coşkun A, Prusa R,
manuscript and approved submission.
Fernandez-Calle P, et al. A checklist for critical appraisal
Financial support: None declared. of studies of biological variation. Clin Chem Lab Med
Employment or leadership: None declared. 2015;53:879–85.
Honorarium: None declared. 11. Haeckel R, Wosniok W, Streichert T. Optimizing the use of the
Competing interests: The funding organization(s) played ‘state-of-the-art’ performance criteria. Clin Chem Lab Med
no role in the study design; in the collection, analysis, and 2015;53:887–91.
12. Orth M. Are regulation-driven performance criteria still
interpretation of data; in the writing of the report; or in the
acceptable? - The German point of view. Clin Chem Lab Med
decision to submit the report for publication. 2015;53:893–8.
13. Schimmel H, Zegers I. Performance criteria for reference meas-
urement procedures and reference materials. Clin Chem Lab
Med 2015;53:899–904.
References 14. Braga F, Infusino I, Panteghini M. Performance criteria for com-
bined uncertainty budget in the implementation of metrological
1. Kallner A, McQueen M, Heuck C. The stockholm consensus con- traceability. Clin Chem Lab Med 2015;53:905–12.
ference on quality specifications in laboratory medicine, 25–26 15. Ceriotti F, Brugnoni D, Mattioli S. How to define a significant
April 1999. Scand J Clin Lab Invest 1999;59:475–585. deviation from the expected internal quality control result. Clin
2. Fraser CG. The 1999 Stockholm Consensus Conference on Chem Lab Med 2015;53:913–8.
quality specifications in laboratory medicine. Clin Chem Lab 16. Jones GR. Analytical performance specifications for EQA
Med 2015;53:837–40. schemes – need for harmonisation. Clin Chem Lab Med
3. Horvath AR, Bossuyt PM, Sandberg S, St John A, Monaghan PJ, 2015;53:919–24.
Verhagen-Kamerbeek WD, et al. Setting analytical performance 17. Oosterhuis WP, Sandberg S. Proposal for the modification of
specifications based on outcome studies – is it possible? Clin the conventional model for establishing performance specifica-
Chem Lab Med 2015;53:841–8. tions. Clin Chem Lab Med 2015;53:925–37.
832 Panteghini and Sandberg: Defining analytical performance specifications

18. Nordin G. Before defining performance criteria we must agree 22. Miller WG, Jones GR, Horowitz GL, Weykamp C. Proficiency test-
on what a “qualitative test procedures” is. Clin Chem Lab Med ing/external quality assessment: current challenges and future
2015;53:939–41. directions. Clin Chem 2011;57:1670–80.
19. Plebani M, Sciacovelli L, Aita A, Pelloso M, Chiozza ML.
­Performance criteria and quality indicators for the *Corresponding author: Mauro Panteghini, Centre for Metrological
pre-analytical phase. Clin Chem Lab Med 2015;53: Traceability in Laboratory Medicine (CIRME), University of Milan,
943–8. Milan, Italy, E-mail: [email protected]
20. Sikaris K. Performance criteria of the post-analytical phase. Clin Sverre Sandberg: Norwegian Quality Improvement of Primary
Chem Lab Med 2015;53:949–58. Care Laboratories (Noklus), Institute of Global Public Health and
21. Sandberg S, Fraser C, Horvath AR, Jansen R, Jones G, Primary Health Care, University of Bergen and Laboratory of Clinical
­Oosterhuis W, et al. Defining analytical performance Biochemistry, Bergen, Norway
specifications: consensus statement from the 1st Strategic
Conference of the European Federation of Clinical Chemistry Supplemental Material: The online version of this article
and Laboratory Medicine. Clin Chem Lab Med 2015;53: (DOI: 10.1515/cclm-2015-0303) offers supplementary material,
833–5. available to authorized users.