Review Article

Page 1 of 7

Past, present, and future of laboratory quality control: patient-

based real-time quality control or when getting more quality at
less cost is not wishful thinking
Alexander Katayev, James K. Fleming

Department of Science and Technology, Laboratory Corporation of America Holdings, Elon, NC, USA
Contributions: (I) Conception and design: All authors; (II) Administrative support: All authors; (III) Provision of study materials and patients: All
authors; (IV) Collection and assembly of data: All authors; (V) Data analysis and interpretation: All authors; (VI) Manuscript writing: All authors; (VII)
Final approval of manuscript: All authors.
Correspondence to: Alexander Katayev. 112 Orange Drive, Elon, NC 27244, USA. Email: [email protected].

Abstract: It is a well-known fact that clinical laboratory test results are one of major contributors to most if
not all clinical decisions and are one of the most cost-efficient tools in the modern healthcare. However, they
may also be a source of significant waste of resources and contribute to errors in patient management. In
order to minimize the latter, clinical laboratories are responsible for development and implementation of the
most rigorous quality control (QC) processes that should adhere to latest scientific advancements and highest
industry standards. This review is an attempt to analyze some past, present, and future trends in laboratory
QC processes, their drawbacks, limitations, and advantages. It is also intended to share the author’s
experience in development and implementation of patient-based real-time QC processes and algorithms in
the national reference laboratory network.

Keywords: Patient-based; real-time; quality control (QC); clinical laboratory; analytical performance

Received: 03 February 2020; Accepted: 22 July 2020; Published: 30 October 2020.

doi: 10.21037/jlpm-2019-qc-03
View this article at: http://dx.doi.org/10.21037/jlpm-2019-qc-03

The past and reconstituted to liquid state prior to analysis or may be

acquired in the liquid state ready for use. When values from
The history of medical laboratory testing may be traced
these materials fall in the predefined acceptability limits,
back to the ancient Egyptians when skilled healers may
it is assumed that the analytical measuring system is under
have used their tongues to check their patient’s urine for a
control and the patient’s results are reliably accurate for the
sweet taste as an indication for diabetes. The quality control
intended use of the assay.
(QC) of their analysis would lie strictly with their individual
The described above approach is undeniably superior
experience in correlating degrees of urine sweetness to the
than training your taste buds, but still suffers from
symptoms of hyperglycemia; weight loss, excessive urination,
limitations.
and thirst. More recently, advancements in analytical tools
made it possible to develop different QC measurements in
order to monitor the accuracy of laboratory test results and The present
provide more reliable information for patient diagnosis and
LQC matrix
management. Those QC tools typically include the use of
liquid QC (LQC) materials with known value assignments In order to keep LQC materials stable, their production
that are measured during production of patient results in cost-efficient, pricing affordable, and combine as many
selected time intervals to assess the analytical accuracy of different analytes as needed in one preparation without their
the given method. Those LQC materials may be lyophilized interaction between each other, the preparation of nearly

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03
Page 2 of 7 Journal of Laboratory and Precision Medicine, 2020

all LQC available on the market include some degree of timing algorithm will be able to reliably detect systematic
matrix alterations. This occurs by the addition of stabilizers, analytical biases due to the intermittent nature of one-at-
preservatives, protease inhibitors, etc., which makes them a-time measurement control and not a continuous real-
contrived matrixes. These matrixes may not be of human time process. Systematic bias may occur during patient
origin (like human serum or plasma), but rather on readily sample measurement between LQC events and disappear
available and less expensive martials of animal origin (like prior to the next LQC event (5). When systematic bias
horse or bovine serum, etc.) and may include dilution with remains undetected, erroneous patient results may escape
other exogenous compounds. Any, or a combination of these detection prior to the next LQC event and be reported. It is
factors, may lead to matrix effects and non-commutability also possible that LQC procedures will not detect random
of those materials. The lack of commutability indicates that bias within the analytical run as they are better designed to
the observed analytical behavior of LQC material analyzed detect calibration differences between runs, as well as lot-
on more than one lot or more than one measurement to-lot variations, which will be displayed as “random bias”
procedure differs from that of native patient samples. It or imprecision on LQC monitoring charts.
is not unusual to see differences in observed bias between
patient samples and LQC and that the bias may be in
Acceptance criteria and analytical quality goals
opposite directions. Lack of commutability may create a
situation when LQC indicates an unacceptable bias that Historically, LQC acceptance criteria were based on
would prompt rejection of patient results when in fact rules based on the assumption that laboratory test result
patient results are not affected (false rejection); or when distributions are following Gaussian (AKA normal) or
LQC will not demonstrate any bias, but patient results near-Gaussian rules. Normal probability distribution is
are significantly biased (no true error detection). It has presumed to be a cornerstone of statistical methodology.
been reported that statistically significant degrees of non- As with any parametric distribution, normal distribution
commutability were observed in over 40% of commercially implies an assumption of a central limit theorem and uses
available LQC materials tested (1). Non-commutability its key parameters of mean, standard deviation (SD), and
was also observed in certified reference materials that are coefficient of variation (CV) that represent variability
intended to be used for analytical method harmonization and uncertainty of its probabilistic analysis. However,
and standardization (2). This can lead to errors in claims normal distribution does have a limitation that it cannot
of traceability to reference materials with false assurance be reliably applied when a CV exceeds 20%. The other
that different methods are harmonized between each other parameters of mean and SD, also cannot be applied to
and provide comparable test results (2). As a result, the skewed distributions, whereby most analyte LQC result
International Federation of Clinical Chemistry (IFCC) distributions are non-Gaussian (6). In the case of non-
created a Working Group on Commutability that published Gaussian distribution, it is more acceptable to use power
recommendations on the assessment of commutability in transformations (like Box-Cox) and corresponding statistical
laboratory medicine (3). algorithms for probabilistic analysis (7).
Nevertheless, it is a standard laboratory practice to use
normal distribution parameters as acceptance criteria rules
Timing of LQC measurements
for LQC. It has been reported that many laboratories still
There is a plethora of published recommendations on use a statistically unreliable ±2 SD rule as a QC metric (4).
optimizing LQC testing in clinical laboratories which Considering this observation, the use of “Westgard Rules”
include timing and number of samples included between that were established by Dr. James Westgard in early 1980s
LQC measurements in order to improve error detection was a monumental step forward. Those rules were based
and minimize false rejection (4). However, according on the very extensive empirical analysis of probabilities of
to the established guidelines from Clinical Laboratory error detection and false rejection using real laboratory
Improvement Amendment (CLIA), it is sufficient to use only test results and simulation techniques. Later, Westgard
two levels of LQC measured during the production run. rules were complemented with the use of Six-Sigma
As a result, it has been reported that most large academic approach and were undoubtedly “state of art” in laboratory
laboratories are still using this approach established back QC science of the time. The most advanced laboratory
in 1980s (4). It may be conjectured that none of LQC LQC software packages are offering “Westgard Adviser”

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03
Journal of Laboratory and Precision Medicine, 2020 Page 3 of 7

BUN LQC-1 (N=4,518) BUN LQC-2 (N=4,506)

Pearson coefficient of skewness: −0.38 Pearson coefficient of skewness: −0.56

20% 15%

15%
10%

10%

5%
5%

0% 0%
12 13 14   15    16 17 38 40 42 44 46 48 50 52 54
Result Result

Urine creatinine LQC-2 (N=1,676) Urine albumin LQC-1 (N=1,449)

Pearson coefficient of skewness: −0.31 Pearson coefficient of skewness: 0.44

15% 20%

15%
10%

10%

5%
5%

0% 0%
100 110 120 130 140 150 160 10 12 14   16    18 20
Result Result

Figure 1 Distribution graphs of some commercial LQC materials from a national reference laboratory. LQC, liquid quality control.

functionality that allows the use of Six-Sigma, user- analytical quality goals like TEa limits. The historical
defined total error allowable (TEa) for calculations of the approach has been the use of “one-size-fits-all” quality
optimal LQC acceptance rules and provide estimated rates goals such as 15% or 20% allowance for systematic and/or
of false rejection and true error detection for the given random bias. This approach is still used in the 2018 FDA
rule. However, Westgard rules were all based on normal Bioanalytical Methods Validation Guidelines for Industry.
distribution parameters such as mean, SD, CV and there However, this approach is gradually being replaced by the
were no attempts to employ any power transformations IFCC recommended multi-model hierarchical approach
to account for skewedness of result distributions. One outlined by a consensus statement from 2015 Strategic
may argue that laboratory test results distribution in the Conference of the European Federation of Clinical
population is different from the distribution of measured Chemistry and Laboratory Medicine (8) which emphasize
LQC materials and the latter is likely to be Gaussian or the use of analytical quality goals based on analyte-
near-Gaussian, but it has not been proven experimentally. specific requirements that are established by assessment of
Distribution graphs of some commercial LQC materials different objective models like clinical outcomes, biological
from national reference laboratory are presented in Figure 1. variation of analytes, and state of the art. In this approach, a
It is quite evident that normal distribution parameters subjective expert opinion model is deemed as least valuable.
cannot be used for a reliable probabilistic analysis and Unfortunately, many laboratories are still using fixed quality
assessment of LQC performance in those examples. goals criteria for all tests without proper risk and outcome
Another issue is related to the choice of proper analysis for each analyte (4).

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03
Page 4 of 7 Journal of Laboratory and Precision Medicine, 2020

The future the advantages of this new approach and gain respect for its
ability to timely detect systematic errors even in the event
The use of patient’s test results for laboratory QC
when LQC did not detect an error, significantly reduce
monitoring was described by early pioneers in this field
the number of repeated samples, and most importantly,
like Drs. Hoffmann, Waid, and Cembrowski as early as
eliminate erroneous results that were reported. The
the 1960s (9-12). However, it was rarely implemented in
detailed description of algorithms and PBRTQC rules set
production due to limitations of information technology
up processes were described previously (16), but it may be
capabilities, lack of standardization between analytical
worth briefly describing the distinctive feature that was
methods, and low level of motivation among laboratory
used to eliminate erroneous results reporting. This feature
scientists with a notable exception of hematology testing
was called “release from the back” meaning that when the
where Bull’s patient based real-time QC (PBRTQC)
predefined number of patient results (called a “block”)
algorithm was widely accepted for use in routine QC
passes acceptance rules (error threshold limits), the first
practices. With the advance of laboratory science,
result in the string of the block is released for reporting;
information technology, and improved harmonization/ a new result is then added to the block and a new block is
standardization of laboratory methods, the limitations again interrogated, if rules passed again, the first result in
of PBRTQC are now negligible. The common use of the second string is released, but if the rules for the given
“big data” and datamining techniques allowed a shift block fails, no result is released and testing is stopped for
in laboratory scientist’s mindset to apply patient based troubleshooting. This creates a “moving block” of results
QC algorithms and broader acceptance of the need for when only results that pass the acceptance criteria are
improvements in laboratory quality monitoring practices. released and no erroneous results are reported to clients.
This caused a wave of publications on the use of different Figure 2 represents an example of “release from the back”
variations of PBRTQC techniques and a beginning of new approach.
era in the laboratory QC routines. Numerous publications When the PBRTQC protocols performance was
described a variety of flavors for possible PBRTQC assessed after a few years of production, the results were
optimization approaches including, but not limited to gratifying: the utilization of traditional LQC materials was
simulated annealing algorithm (13), moving sum of positive reduced by approximately 75–85% that translated into
patient results (14), and abnormal results distributions (15). significant cost savings, repeat sample analysis was reduced
Some reference and academic laboratories have begun by approximately 50% which translated into significant
experimentation with different PBRTQC algorithms labor savings (16). Over the period of almost four months
and some major laboratory middleware providers started post implementation in our laboratory network across the
to introduce PBRTQC capabilities in their commercial country, a total of 60 delta checks were performed when
software packages. This was a start of a change in the the PBRTQC rules were violated, affected samples were
entire landscape of laboratory quality practices. One of the repeated on a different instrument, and false rejection
major U.S. national reference laboratories in collaboration rates were calculated. Repeated results were compared
with one of the major commercial laboratory middleware with results from the failed blocks and a true failure was
providers developed and implemented custom PBRTQC determined when the observed difference for at least
protocols in their routine chemistry and immunoassay one result was greater than TEa chosen for the given
production practices. Those protocols were successfully analyte. Only a single delta check out of 60 (1.7%) did not
assessed for real-time sensitivity (true error detection rate) confirm the true failure. The same comparison exercise
and specificity (false rejection rate) and later offered by the was performed over the same four-month period when
middleware provider as a commercially available package LQC failure resulted in repeating the affected samples on a
for the entire IVD market (16). different instrument. A total of 81 LQC failures with delta
At first, the PBRTQC implementation process was met checks were performed and 25 (30.9%) were not confirmed
with some skepticism among laboratory personnel. This as a true failure when the result difference did not exceed
was not unexpected as the entire paradigm of traditional the TEa for the given analyte. This indicates that PBRTQC
laboratory QC practice was challenged and prior experience may be superior to LQC in its specificity. It may be more
with conventional LQC did not fit with new action difficult to assess sensitivity of PBRTQC protocols in a
protocols. It took several months for laboratorians to realize production environment. However, during the PBRTQC

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03
Journal of Laboratory and Precision Medicine, 2020 Page 5 of 7

Figure 2 “Release from the Back” approach example using a median calculation for a moving block of 10 results.

rules validation studies, the experimental biases were added few years post implementation, standardized chemistry
to patient’s results with error detection rates of 100% and immunoassay testing was monitored with PBRTQC
with affected results blocked from reporting. The only throughout the laboratory network and observed data was
limitation of this approach is that applied systematic errors used to significantly improve the manufacturer’s reagent
were consistent and equal to 1.2 times TEa for the given performance and minimize imprecision and bias of results
analyte which may not be the case in a real-time production over time. In working with reagent suppliers, the net effect
scenario. improved the quality of their measurement system.
Another important feature of PBRTQC is the ability to The described paradigm change in the future of
connect all instruments in a network and remotely monitor the clinical laboratory QC practices may take time for
performance of each instrument in real-time, which allows acceptance by laboratorians, but the long-term results,
for centralized control of the entire analytical process and improve the over-all quality the test result and ultimately
timely detection of warning signals such as drifts prior to improve the benefit to the patient. It was recently reported
encountering a process failure. An example of a drift that that statistical literacy among academic pathologists is
was detected, the analysis interrupted, and followed by far from being perfect and many of them possess only
analytical problem resolution can be seen in Figure 3. basic level of statistical knowledge and will benefit from
PBRTQC algorithms are also proven to be robust additional statistical training (17). It is not uncommon that
tools in controlling for result shifts due to reagent lot clinicians are confused with the possible root cause of an
changes and changes in reagent components. During first aberrant test result that does not fit the clinical presentation

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03
Page 6 of 7 Journal of Laboratory and Precision Medicine, 2020

Figure 3 Screenshot of serum calcium analysis drift correction during production run monitoring. Note that drift was detected and
corrected prior to process failure (red horizontal lines represent control limits; green horizontal line represent target; arrow, time of
corrective action).

of the patient or expected treatment outcome. In this case, analytical quality goals that so many institutions are still
the burden of explanation is placed on the laboratory to adhering to. It is important to note that IFCC recently
provide an adequate answer. Unfortunately, it is impossible created a subcommittee on PBRTQC as a part of its
to bring the chance of laboratory error to zero, but when Laboratory Medicine Committee on Analytical Quality that
laboratory quality assurance practices are utilizing multiple endorsed the use of PBRTQC in clinical laboratories and
layers of process control and rely on the modern statistical provided first of its kind recommendations on PBRTQC
principles, it is very possible to shrink this chance to a near implementation (18). The new QC era key words are
zero, and PBRTQC is one of the available and effective evidence-based, risk-based, and outcome-based.
tools to achieve this goal. Therefore, it is important that
laboratories will work with the health care providers to
Acknowledgments
help them better understand the role of PBRTQC in the
improvement of healthcare, and provide more accurate Funding: None.
results that are truly reliable for their intended clinical
use. Another critical point that is often missed in the
Footnote
interaction between laboratories and health care providers
is the mutual agreement and understanding on the proper Provenance and Peer Review: This article was commissioned
setting of analytical quality goals for the given analyte that by the Guest Editor (Mark A. Cervinski) for the series
will fit the intended clinical use of the test. This dialog can “Patient Based Quality Control” published in Journal of
aid in providing better quality healthcare for our patients. Laboratory and Precision Medicine. The article has undergone
Perhaps it is time to change the one-size-fit-all practice for external peer review.

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03
Journal of Laboratory and Precision Medicine, 2020 Page 7 of 7

Conflicts of Interest: Both authors have completed the 7. Olivier J, Norberg MM. Positively Skewed Data:
ICMJE uniform disclosure form (available at http://dx.doi. Revisiting the Box-Cox Power Transformation. Int J
org/10.21037/jlpm-2019-qc-03). The series “Patient Based Psychol Res 2010;3:68-75.
Quality Control” was commissioned by the editorial office 8. Sandberg S, Fraser CG, Horvath AR, et al. Defining
without any funding or sponsorship. The authors have no analytical performance specifications: Consensus Statement
other conflicts of interest to declare. from the 1st Strategic Conference of the European
Federation of Clinical Chemistry and Laboratory
Ethical Statement: The authors are accountable for all Medicine. Clin Chem Lab Med 2015;53:833-5.
aspects of this work in ensuring that questions related 9. Hoffmann RG, Waid ME, Henry JB. Clinical specimens
to the accuracy or integrity of any part of this work are and reference samples for the quality control of laboratory
appropriately investigated and resolved. accuracy. Am J Med Technol 1961;27:309-17.
10. Hoffmann RG, Waid ME. The "average of normals"
Open Access Statement: This is an Open Access article method of quality control. Am J Clin Pathol
distributed in accordance with the Creative Commons 1965;43:134-41.
Attribution-NonCommercial-NoDerivs 4.0 International 11. Cembrowski GS, Carey RN. Laboratory Quality
License (CC BY-NC-ND 4.0), which permits the non- Management. Chicago: AACC Press; 1989.
commercial replication and distribution of the article with 12. Cembrowski GS. Use of Patient Data for Quality Control.
the strict proviso that no changes or edits are made and the Clinics Lab Med 1986;6:715-33.
original work is properly cited (including links to both the 13. Ng D, Polito FA, Cervinski MA. Optimization of a
formal publication through the relevant DOI and the license). Moving Averages Program Using a Simulated Annealing
See: https://creativecommons.org/licenses/by-nc-nd/4.0/. Algorithm: The Goal is to Monitor the Process Not the
Patients. Clin Chem 2016;62:1361-71.
14. Liu J, Tan CH, Badrick T, et al. Moving sum of number of
References
positive patient result as a quality control tool. Clin Chem
1. Miller WG, Erek A, Cunningham TD, et al. Lab Med 2017;55:1709-14.
Commutability limitations influence quality control results 15. Katayev A, Fleming JK. Patient results and laboratory test
with different reagent lots. Clin Chem 2011;57:76-83. accuracy. Int J Health Care Qual Assur 2014;27:65-70.
2. Miller WG, Myers GL. Commutability still matters. Clin 16. Fleming JK, Katayev A. Changing the paradigm of
Chem 2013;59:1291-3. laboratory quality control through implementation of real-
3. Nilsson G, Budd JR, Greenberg N, et al. IFCC Working time test results monitoring: For patients by patients. Clin
Group Recommendations for Assessing Commutability Biochem 2015;48:508-13.
Part 2: Using the Difference in Bias between a Reference 17. Schmidt RL, Chute DJ, Colbert-Getz JM, et al. Statistical
Material and Clinical Samples. Clin Chem 2018;64:455-64. Literacy Among Academic Pathologists: A Survey Study
4. Rosenbaum MW, Flood JG, Melanson SEF, et al. Quality to Gauge Knowledge of Frequently Used Statistical Tests
Control Practices for Chemistry and Immunochemistry Among Trainees and Faculty. Arch Pathol Lab Med
in a Cohort of 21 Large Academic Medical Centers. Am J 2017;141:279-87.
Clin Pathol 2018;150:96-104. 18. Badrick T, Bietenbeck A, Cervinski MA, et al. Patient-
5. Yago M, Alcover S. Selecting Statistical Procedures for Based Real-Time Quality Control: Review and
Quality Control Planning Based on Risk Management. Recommendations. Clin Chem 2019;65:962-71.
Clin Chem 2016;62:959-65.
6. Frey CH, Rhodes DS. Quantitative Analysis of Variability
doi: 10.21037/jlpm-2019-qc-03
and Uncertainty in Environmental Data Models. Volume
Cite this article as: Katayev A, Fleming JK. Past, present, and
1. Theory and Methodology Based Upon Bootstrap
future of laboratory quality control: patient-based real-time
Simulation. Report No.: DOE/ER/30250-Vol.1 for
quality control or when getting more quality at less cost is not
U.S. Department of Energy Office of Energy Research.
wishful thinking. J Lab Precis Med 2020;5:28.
Germantown, MD 20874. April 1999.

© Journal of Laboratory and Precision Medicine. All rights reserved. J Lab Precis Med 2020;5:28 | http://dx.doi.org/10.21037/jlpm-2019-qc-03