Designation: E2867 − 14

Standard Practice for

Estimating Uncertainty of Test Results Derived from
Spectrophotometry1
This standard is issued under the fixed designation E2867; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.

1. Scope ISO/IEC 17025 General Requirements for the Competence

1.1 This practice describes a protocol to be utilized by of Calibration and Testing Laboratories
measurement laboratories for estimating and reporting the 2.3 Other Standard:4
uncertainty of a measurement result when the result is derived QS 9000 Quality Systems Requirements Chrysler
from a measurand that has been obtained by spectrophotom- Corporation, Ford Motor Company, General Motors Cor-
etry. poration
1.2 This practice is specifically limited to the reporting of 3. Terminology
uncertainty of color measurement results that are reported as
3.1 Definitions: For definitions of terms used in this stan-
color-differences in ∆E format, even though the measurement
dard refer to Terminology E284.
itself may be reported in other units such as percent reflectance
3.2 Definitions of Terms Specific to This Standard:
or transmittance.
3.2.1 uncertainty, n—a parameter associated with a mea-
1.3 The procedures defined here are not intended to be surement result or test result that reasonably characterizes the
applicable to national standardizing laboratories or transfer dispersion of results attributable to the particular quantity being
laboratories. measured of the particular characteristic being tested.
1.4 This standard does not purport to address all of the 3.2.2 instrument uncertainty conditions, n—of a
safety concerns, if any, associated with its use. It is the measurement, conditions wherein the measurements are made
responsibility of the user of this standard to establish appro- repetitively and carefully over a short timescale, without
priate safety and health practices and determine the applica- replacement of the specimen being measured in the specimen
bility of regulatory limitations prior to use. port of the instrument.
2. Referenced Documents NOTE 1—Instrument uncertainty conditions always include potential
specimen drift due to causes such as theromchromism, photochromism, or
2.1 ASTM Standards:2 bleaching of the specimen. While these may be thought of as character-
D2244 Practice for Calculation of Color Tolerances and istics of the specimen, their effects will be picked up here under instrument
Color Differences from Instrumentally Measured Color uncertainty conditions.
Coordinates 3.2.3 operator uncertainty conditions, n—of a
E29 Practice for Using Significant Digits in Test Data to measurement, conditions wherein the measurements are made
Determine Conformance with Specifications repetitively and carefully over a short timescale, with replace-
E284 Terminology of Appearance ment of the specimen being measured by the operator com-
2.2 ISO Standards:3 pletely withdrawing the specimen from the specimen port and
ISO 9001 Quality Management Systems—Requirements replacing the specimen back in the specimen port prior to the
ensuing measurement so that the specimen aperture samples
the same location on the specimen, and the specimen has the
1
This practice is under the jurisdiction of ASTM Committee E12 on Color and same orientation as previous, to the best of the operator’s
Appearance and is the direct responsibility of Subcommittee E12.04 on Color and ability to accomplish.
Appearance Analysis.
Current edition approved May 1, 2014. Published May 2014. Originally 3.2.4 uniformity uncertainty conditions, n—of a
approved in 2013. Last previous edition approved in 2013 as E2867 – 13. DOI: measurement, conditions wherein the measurements are made
10.1520/E2867-14.
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
repetitively and carefully over a short timescale, with replace-
contact ASTM Customer Service at [email protected]. For Annual Book of ASTM ment of the specimen being measured to an entirely new
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website.
3 4
Available from American National Standards Institute (ANSI), 25 W. 43rd St., Available from Automotive Industry Action Group (AIAG), 26200 Lahser Rd.,
4th Floor, New York, NY 10036, http://www.ansi.org. Suite 200, Southfield, MI 48033, http://www.aiag.org.

Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States

1
 E2867 − 14
location on the face of the specimen with the intent of sampling quality system requirements demand that a statement of the
the entire surface of the specimen, or as much of the surface as uncertainty of the test results accompany every test result.
is practical, by the end of the repetitive sampling run. 5.2 Preparation of uncertainty estimates is a requirement for
3.2.5 instrument uncertainty, n—the results of an uncer- laboratory certification under ISO 17025. This practice de-
tainty analysis of a measurement system made under instru- scribes the procedures by which such uncertainty estimates
ment uncertainty conditions. may be calculated.
3.2.6 operator uncertainty, n—the results of an uncertainty
analysis of a measurement system made under operator uncer- 6. Concepts in Reporting Uncertainty of Test Results
tainty conditions. 6.1 A commonly cited definition (1, 2)5 paraphrased to form
3.2.7 uniformity uncertainty, n—the results of an uncertainty a single citation defines uncertainty as “a parameter, associated
analysis of a measurement system made under uniformity with the measurement result, or test result, that characterizes
uncertainty conditions. the dispersion of values that could reasonably be attributed to
the quantity subject to measurement or characteristic subject to
3.2.8 expanded uncertainty, n—uncertainty reported as a
test.” This definition emphasizes uncertainty as an attribute of
multiple of the standard uncertainty.
an individual test result, not as a property defining statistical
3.2.9 measurement system, n—the entirety of variable fac- variation of test results.
tors that could affect the precision, accuracy, or uncertainty of
a measurement result. These include the instrument, the 6.2 The methodology for classification of uncertainty types
operator, the environmental conditions, the quality of the has been classified as Type A and Type B as discussed in
transfer standard, the specimen aperture size, as well as other references (2) and (3). Type A estimates of uncertainty include
factors. estimates based upon knowledge of the statistical character of
the measurement results, or estimates based upon statistical
3.2.10 standard uncertainty, n—uncertainty reported as the analysis of replicate measurement results. The latter may
standard deviation of the estimated value of the quantity include results from control sample monitoring programs, or
subject to measurement. proficiency testing. Type B estimates of uncertainty include
3.2.11 95 % confidence interval, n—the 95 percentile value estimates from calibration certificates and manufacturer’s
of an ascending-ordered distribution of differences between specifications. Type A are evaluated by statistical methods and
multiple measurement results of a derived parameter charac- Type B by non-statistical methods.
terized by a color measurement system. 6.3 The goal of reporting uncertainty is to account for all
3.2.11.1 Discussion—This value is the cumulative distribu- potential causes contributing to uncertainty in the measurement
tion between zero and the stated value of the measurand that result. Uncertainty for a single measurement result is then
contains 95 % of all the measurement results made by this
procedure. ~ s 12 1s 2 2 1 ...1 s n2 ! 1/2
where s1 is the estimate of the uncertainty of the first factor
4. Summary of Practice contributing to variance, s2 the second, and so on, through all
4.1 This practice establishes a protocol for measurement n components of variance.
laboratories to assess the uncertainty of their measurement 6.4 Uncertainty in this practice shall be reported as the 95%
system from test specimens or from control samples of confidence interval of the largest component of all the compo-
materials similar in both first-surface characteristics and color nents of uncertainty assessed.
to those being measured and reported.
6.5 The minimum components contributing to variance
4.2 Where control samples are used, the process will be to shall be the instrument uncertainty, the operator uncertainty,
establish control samples representative of the type of materials the uniformity uncertainty, and the uncertainty of the traceabil-
to be measured. Control samples will be processed to assess the ity scheme.
various uncertainty components of measurement results, the
results retained in a control chart, and the rolling average of the 7. Procedure
uncertainty components of the control samples used as a
surrogate for assessing the uncertainty of a similar specimen. 7.1 Measure the test specimen a minimum of 20 times, and
preferably as many as 30 times, under instrument uncertainty
4.3 Some of the components of uncertainty for color mea- conditions. Make all measurements in compliance with the
surement result are instrument uncertainty, operator manufacturer’s recommendations including prior standardiza-
uncertainty, and uniformity (of the specimen) uncertainty. tion of the instrument using a white tile, a black tile, or light
trap, and a grey tile, if required.
5. Significance and Use
7.2 There will be n* (n - 1) / 2 possible color-differences
5.1 Many competent measurement laboratories comply with between the n measurement results taken two-at-a-time in all
accepted quality system requirements such as ISO 9001, QS possible combinations.
9000, or ISO 17025. When using standard test methods, the
measurement results should agree with those from other similar
laboratories within the combined uncertainty limits of the 5
The boldface numbers in parentheses refer to a list of references at the end of
laboratories’ measurement systems. It is for this reason that this standard.

2
 E2867 − 14
7.3 Calculate the absolute value of the color-differences determination, it is permissible to substitute control samples of
between each of these combinations and retain the results in a a like material for the material being considered in the test
list. Calculate these color-differences in accordance with a result.
color-difference equation chosen from Practice D2244. 8.2 Values quoted using control samples shall be rolling
7.4 Sort the list in ascending order. The member of the averages of the last four determinations of the uncertainty by
sorted list whose index is Int [0.95 * n * (n-1) / 2] contains the the operations of 7.1 – 7.8 using the same control sample in
value of the 95 % confidence interval of the instrument each of the four determinations. Each of the four determina-
uncertainty s1. The symbol Int means the integer value of the tions must be demonstrated to be ‘in-control’ by maintenance
expression in brackets. of control charts of the uncertainty determinations.
7.5 Measure the test specimen a minimum of 20 times, and 8.3 The control samples should be chosen to be as alike the
preferably as many as 30 times, under operator uncertainty material being reported as possible and special attention should
conditions. Follow the operations of 7.1 – 7.3 using this data be paid to the matter of uniformity uncertainty when control
set to calculate the 95 % confidence interval of operator samples are chosen. Uniformity uncertainty is the most likely
uncertainty s2. component to be discrepant between a control sample and a test
7.6 Measure the test specimen a minimum of 20 times, and specimen.
preferably as many as 30 times, under uniformity uncertainty 8.4 The first surface of control samples involved in mea-
conditions. Follow the operations of 7.1 – 7.3 using this data surements of reflection properties should be as nearly identical
set to calculate the 95 % confidence interval of uniformity as possible to the first surface of the test sample.
uncertainty s3. 8.5 Reports of uncertainty where a control sample is sub-
7.7 Sort the uncertainties obtained from Sections 7.1 – stituted for a test sample shall state the facts of the substitution
7.6 s1, s2, s3 in ascending order with the smallest of the three and identify the control sample utilized.
in s1 and the next larger in s2, and so forth. Let
s '1 5 s 1 (1) 9. Reporting Statement
'
s 5 ~s 2 s !
2 2
2
1
2 1/2
(2) 9.1 Form of the Reporting Statement—In reporting
'
s 3 5 ~s 2 s ! 2 2 1/2
(3)
uncertainty, a statement such as the following may be useful:
3 2
“The uncertainty of the value reported was found to be X.XX
The value of s2 used in Eq 3 is the original experimentally (here report the uncertainty value.) This value was determined
assessed value, not that value which results from the calcula- using XX (here report the number of measurements made in
tion of Eq 2 which is s'2. This isolates the uncertainties, each of the assessment of uncertainty; ninety, for instance) measure-
which has been until now included in each of the measured ments categorizing the instrument, operator and uniformity
uncertainties, into a separate uncertainty contribution attribut- uncertainty. This value is the combined expanded standard
able to each subsequent type of uncertainty considered. uncertainty in color difference units according to the (here
However, consult the cautionary remarks in Appendix X1 to name the color-difference equation used) equation calculated
this Practice at X2.1 – X2.4. from D65 and the CIE 10° Observer, which defines the
7.8 Calculate the combined uncertainty uncertainty of the measured value to a confidence level of 95
percent.”
U 5 ~ s 1'2 1 s '22 1 s 3' 2 ! 1/2 (4)
' ' ' 9.2 The following measurement parameters shall be defined
where the elements s 1, s 2, and s 3 are the uncertainties from and specified in the uncertainty expression: (1) the identity of
7.4. the measured sample being reported, (2) the color difference
The combined uncertainty may be expressed in the form equation used in the uncertainty assessment, (3) the values of
‘mean value 6 U to a 95 % confidence interval.’ The value of l and c, only for DECMC, (4) the Illuminant–Observer combi-
U shall be expressed to no greater number of significant digits nation used, (5) the number of measurements in each of the
than are expressed in the mean, and shall be rounded in uncertainty component assessments, (6) the level expressed as
accordance with Practice E29. All calculations should be a percentage to which the coverage factor raises the confi-
carried out with the full precision of the machine employed at dence.
all times and rounded only when the final values, seen in the
worked example in the table in X1.2.1, are calculated.
10. Applications of Uncertainty
7.9 In some infrequently occurring instances, the length of
10.1 The uncertainty results can be used by those who want
time required to make a measurement may make it impossible
to assess the reliability of a measured value. Without an
to make as many as 30 measurements. Under those circum-
uncertainty indication, measurement results cannot be
stances it is permissible to reduce the number of measurements
compared, either among themselves or to a reference value
made to a smaller number providing that it can be demon-
given in a specification or standard. The uncertainty value may
strated that the results are sufficient to the intended purpose.
be further used to decide whether there is a difference between
8. The Substitution of Control Samples results from different laboratories.
8.1 Under circumstances where it would prove a hardship, 10.2 The uncertainty value is also necessary when compar-
or is infeasible to utilize the test specimen for this ing results to allowable values, for example, tolerance limits or

3
 E2867 − 14
allowable (legal) specifications. To make a correct pass-fail is greater than, or equal to, zero and less than, or equal to, 0.41
decision one needs an uncertainty value which, together with total color difference units, ∆E*ab.
the measured value, is in the range of the total tolerable limit.
10.3 In the provided example found in Appendix X1, the 11. Keywords
data reports the range of the total combined uncertainty as 0.41 11.1 control samples; spectrophotometry; test results; uncer-
total color difference units. Therefore, the range of uncertainty tainty; uncertainty budget

APPENDIXES

(Nonmandatory Information)

X1. A WORKED EXAMPLE OF THE UNCERTAINTY CALCULATIONS

X1.1 Underlying Data precision will be given for this example both before and after
X1.1.1 The underlying data may be found on the ASTM their separation from each other by Eq 1-3 in 7.7 of this
Committee E12 web site and downloaded by any user. The data document. The three calculated values from this example, for
is contained in three files. The first file named INSTRUMENT D65-1964 Observer CIELAB ∆E*ab are
UNCERTAINTY.csv contains 20 measurements of a white Consolidated Unconsolidated
optically-brightened paper made under instrument uncertainty Instrument Uncertainty 0.0513 0.0513
conditions. The second, named OPERATOR Operator Uncertainty 0.0529 0.0129
UNCERTAINTY.csv, contains 20 measurements of the same Uniformity Uncertainty 0.4091 0.4057
sample made under operator uncertainty conditions. The third
file, named UNIFORMITY UNCERTAINTY.csv, contains 20 X1.3 Results
measurements made under uniformity uncertainty conditions. X1.3.1 The following table provides the uncertainty calcu-
X1.2 Intermediate Values lated from these example data.
Instrument Uncertainty 0.0513
X1.2.1 Because the resulting intermediate values are Operator Uncertainty 0.0129
complex, many, and are subject to machine representation Uniformity Uncertainty 0.4057
variation, a limited set of results to the first four digits of Total Measurement Uncertainty 0.41 ∆E*ab D65–1964 10° Observer

X2. ON THE CALCULATIONS

X2.1 The four equations in 7.7 and 7.8 imply the consoli- normal, independent, and have identical standard deviations, or
dation of the components of uncertainty by addition under nearly so.
quadrature. The addition of 95 % confidence intervals obtained
X2.3 That these assumptions are made dictates another
from standard deviations of normal distributions definitely add
limitation of the method. That is, the method adopted is limited
under quadrature to form 95 % confidence intervals of the
to quantification of color-differences expressed in ∆E format.
composite. However, the addition of 95 % confidence intervals
The determination of uncertainty of data sets of other deriva-
of non-normal distributions under quadrature may not obtain
tive parameters, such as whiteness index or others, will have to
an exact 95 % confidence interval of the composite.
wait the determination that the distribution of those differences
are closely enough aligned with the chi distribution that the
X2.2 At the same time, one desires to deconsolidate the
deconsolidation under quadrature is effective.
components of uncertainty to determine which are largest, and
therefore the most likely subjects for reduction of uncertainty. X2.4 Accordingly, while the total 95 % confidence interval
Accordingly, ASTM has adopted the 95 % confidence interval of the largest consolidated component is entirely valid, users
of the largest consolidated component as the definition of should view with caution the values obtained by deconsolida-
uncertainty, and allowed the deconsolidation under quadrature. tion under quadrature. They may serve as a guideline as to
The user must exercise caution that the assumptions employed where further improvements in reduction of uncertainty may be
in doing so are, at least, mostly met. These assumptions include made, but they should likely not be reported or promulgated
that the distributions of differences in L*, a*, and b* are beyond the laboratory deriving them.

4
 E2867 − 14
REFERENCES

(1) International Organization for Standardization, International Vocabu- (3) Taylor, B. N., and Kuyatt, C. E., NIST Technical Note 1297,
lary of Basic and General Terms in Metrology, Geneva, Switzerland, Guidelines for Evaluating and Expressing Uncertainty of NIST
1993 (VIM). Measurement Results, 1994.
(2) JCGM 1000:2008 Evaluation of Measurement Data — Guide to the
Expression of Uncertainty in Measurement, Paris, France, JCGM
2008 (GUM).

ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.

This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.

This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or [email protected] (e-mail); or through the ASTM website
(www.astm.org). Permission rights to photocopy the standard may also be secured from the ASTM website (www.astm.org/
COPYRIGHT/).