Measurement Systems Analysis

VARIABLE GAGE R&R METRICS, APPLICATIONS AND ROOT CAUSE

ANALYSIS

Created and presented by: Gabor Szabo

Objectives

By the end of this presentation we will have covered

• Basics of Measurement Systems Analysis

• Overview of Variable Gage R&R Study Metrics
• Application of Metrics
• Root Cause Analysis, Tools
Measurement Systems Analysis

In the world of quality, there has always been a need for reliable data in order to
make data-based decisions.

• Inspections – accepting or rejecting parts based on inspection results

• Improvement activities and projects – process improvements, Six Sigma projects
• Any other measurement activity that has an impact on quality or the
organization

Questions to ask:
• How do you ensure that you can rely on your data and it reflects reality?
• How does one define a measurement system?
Measurement System Definition

What – How – Who

• What: Characteristic of interest

Examples: length, diameter, tensile Strength, angle, waiting time, weight,
number of cracks/voids (on part surface)

• How: Measurement method – includes the gage and the measurement

procedure/technique
Examples: naked eye, steel ruler, caliper, CMM and automated measurement
program, spectrometer, microscope

• Who: Inspectors/Operators
Examples: receiving inspectors, engineers, technicians
Measurement System Analysis – Accuracy and Precision

Accuracy
• Bias: the difference between the average of
observed measurements and a master value
• Linearity: accuracy through the expected range
Precise Accurate
of measurements Inaccurate Imprecise
• Stability: accuracy over time
LINEARITY/BIAS STUDIES

Precision: Measurement variation

• Repeatability: consistency of measurements Accurate Inaccurate
Precise Imprecise
• Reproducibility: difference between operators
GAGE R&R STUDIES
Accuracy – Precision
Master value
Mean of measurements Master value Mean of measurements

Precise Accurate
Inaccurate Imprecise
Master value
Mean of measurements Master value Mean of measurements

Inaccurate
Accurate
Imprecise
Precise
Gage R&R Studies

• Planned studies to assess variation attributed to the measurement system. Gage R&R Studies
only assess precision (repeatability and reproducibility).
• Study plan: samples, operators, trials
The multiply of the above study elements for a number of opportunities (or study sample
size). Example: 10 samples x 3 operators x 3 trials.
• Types:
• Variable – variable output (continuous numerical values) • Non-Destructive
• Attribute – attribute output (pass/fail, good/bad etc.) • Destructive

• History: Developed by automotive industry in the 1960’s. Initially the Average-Range method
was used; the ANOVA method was developed later on (uses sum of squares to estimate standard
variation, which is a more accurate estimation than what the Average-Range method provides)

• Reference Book: AIAG MSA Reference Manual 4th edition

Components of Variation in a Variable Gage R&R Study Process mean
LSL USL

Process mean ( )
Specification limits (LSL, USL)
Tolerance = USL - LSL

Observed Part-To-Part Variation also called Total Variation

Measurement System Variation (Total GR&R)
True Part-To-Part Variation
True Part-To-Part Variation
Measurement System Variation (Total GR&R)
Observed Part-To-Part Variation
Repeatability
Reproducibility
Minitab Tolerance
%Tolerance metric

%Tolerance metric
Tolerance Sigma Multiplier

Measurement System Variation (Total GR&R)

Repeatability
Reproducibility

Part-To-Part Variation
Total Variation (Part-To-Part + Total GR&R)

Measurement System Variation (Total GR&R) Measurement System Variation (Total GR&R)StdDev x Sigma Multiplier
GR&R [% Tolerance] = =
Tolerance Tolerance
Repeatability Reproducibility
Repeatability [% Tolerance] = ; Reproducibility [% Tolerance] =
Tolerance Tolerance
 %Tolerance metric – Repeatability and Reproducibility

6.89% + 4.34% ≠ 8.14%

Why do Repeatability and Reproducibility not add up to equal Total
GR&R?
• Because they are calculated and expressed in units of standard deviation – standards deviations are not additive; variances are.
• Since standard deviation is the square root of variance, the aggregate of Repeatability and Reproducibility is calculated based on the Pythagorean Theorem

A2 + B2 = C2

System Variation
Measurement

(Total GR&R)

C
B

Part-To-Part Variation

VARIANCES STANDARD DEVIATIONS

Total GR&R2 = Repeatability2 + Reproducibility2 Total GR&R = Repeatability2 + Reproducibility2

Total Variation (Part-To-Part + Total GR&R)2 = Part-To-Part Variation2 + Total GR&R2 Total Variation (Part-To-Part + Total GR&R) = Part-To-Part Variation2 + Total GR&R2
%Tolerance metric - Application Type I or II errors %Tolerance = 15%
LSL USL

• Application: inspections where the inspection Area of Area of

Uncertainty Uncertainty
result is compared to a specification and an
accept/reject decision is made.

Examples: inspection activities (receiving Measurement System Variation Measurement System Variation

inspection, in-process inspections, etc.)

Tolerance

• Sample selection: Since the Total Variation component is not part of the %Tolerance
formula, sample selection does not have an affect on the %Tolerance result.
Measurement System Variation (Total GR&R)
GR&R [% Tolerance] =
Tolerance

• Acceptance criteria guidelines for %Tolerance per AIAG MSA Reference Manual 4th edition:
< 10% Acceptable measurement system.
10 – 30% May be acceptable for some applications. Decision should be based on feature criticality, cost of measurement device, etc.
> 30% Unacceptable measurement system. Every effort should be made to improve the measurement system.
%Tolerance metric - Application
• Effect on process capability index Cp/Pp
%Study Variation metric

%Study Variation metric

Sigma Multiplier

Measurement System Variation (Total GR&R)

Repeatability
Reproducibility

Part-To-Part Variation
Total Variation (Part-To-Part + Total GR&R)

Measurement System Variation (Total GR&R) Measurement System Variation (Total GR&R)StdDev x Sigma Multiplier
GR&R [% Study Variation] = =
Total Variation (Part-To-Part + Total GR&R) Total Variation (Part-To-Part + Total GR&R) StdDev x Sigma Multiplier

Repeatability Reproducibility
Repeatability [% Study Variation] = ; Reproducibility [% Study Variation] =
Total Variation (Part-To-Part + Total GR&R) Total Variation (Part-To-Part + Total GR&R)
%Study Variation metric – Application
%Study Variation = 50%
Areas of Uncertainty
• Application: activities where process changes, shifts or Type I or II errors
drifts need to be identified or monitored.

Examples: process/continuous improvement activities, such

as SPC, Design of Experiments, etc.

• Sample selection: Since the Total Variation component is part of the %Study Variation
formula, the %Study Variation metric is affected by sample selection.
Measurement System Variation (Total GR&R)
GR&R [% Study Variation] =
Total Variation (Part-To-Part + Total GR&R)

• Acceptance criteria guidelines for %Study Variation per AIAG MSA Reference Manual 4th edition:
< 10% Acceptable measurement system. Measurement system able to distinguish parts or detect process shifts.
10 – 30% May be acceptable for some applications. Decision should be based on feature criticality, cost of measurement device, etc.
> 30% Unacceptable measurement system. Every effort should be made to improve the measurement system.
%Study Variation vs. %Contribution metrics
%Study Variation – uses standard deviations, non-additive

Total GR&R
42.09%
90.71%

Part-To-Part Variation

%Contribution – uses variances, additive 17.72% 82.28%

12.69% 5.03%

100%

Acceptance criteria guidelines for %Contribution per AIAG MSA Reference Manual 4th edition:
< 1% Acceptable measurement system. Measurement system able to distinguish parts or detect process shifts.
1 – 9% May be acceptable for some applications. Decision should be based on feature criticality, cost of measurement device, etc.
> 9% Unacceptable measurement system. Every effort should be made to improve the measurement system.
 Number of Distinct Categories
Number of Distinct Categories (also called Discrimination Ratio)*

• It represents the number of non-overlapping confidence intervals that will span the range of product variation, i.e. it defines the
number of groups within your process data that your measurement system can distinguish. “Effective gage resolution”

• The higher the number, the better the measurement system at distinguishing parts from one another

Formula: (rounded down to nearest whole number)

Acceptance criteria guidelines per AIAG MSA Reference Manual 4th edition:
ndc = 1: One part cannot be distinguished from
others.

ndc = 2-4: The data can be split into 2-4 groups:

e.g. high and low (2), low-middle-high (3)

ndc ≥ 5: Recommended. Measurement system

capable of distinguishing parts from each other.
Can be used for process control. * Evaluating The Measurement Process, 1984 by Donald J. Wheeler and Richard W. Lyday
Number of Distinct Categories vs. %Study Variation

• Number of Distinct Categories and %Study Variation metrics are inversely proportional:
the higher the %Study Variation, the lower the Number of Distinct Categories

The relationship between ndc and %Study Variation ndc formula:

%Study Variation formula:

TAKEAWAY: %Study Variation, %Contribution and Number of Distinct Categories all mean the same
thing, expressed in different ways!
What if a Gage R&R Study fails? – Root Cause Analysis

• Potential root causes need to be investigated as to what is causing excess measurement

system variation

• A corrective action needs to be taken based on the root causes identified

• Root causes can be related to:

• Gage
• Method/Procedure
• Sample
• Inspection Fixture
• Environment
• Operators/Inspectors

• Root causes can affect

• Repeatability
• Reproducibility
• Both
Potential Root Causes of Measurement System Variation

Environment Method/Procedure Gage

Parameters Gage linearity
Calibration
Edge detection
Focus
Ambient temp. Pattern detection
Humidity Sample Zoom level
Lighting Gage stability
clamping Initial coordinates Verification
Probe used
Vibration
Measurement PM
location/points Manual/Program
Lighting Accuracy/Bias
conditions
Procedure too
vague / No
procedure Measurement
System Variation
Fixture/Nest Cleanliness Understanding
Design Fixture/Nest Elastic Procedure/Drawing
Build deformation
(Tolerances) Experience

Profile/Finish Attitude
Glossy/Matte
Glossy/Matte Training
Skill
Stress/Pressure
Adequate
Datum(s)
Adequate
Datum(s) Fatigue

Fixture Part Man

What if a Gage R&R Study fails? – Root Cause Analysis
Most Typical Root Causes:
• Measurement method/procedure not defined well enough so
operators may interpret it subjectively
• Measurement location not defined well enough
• Sample positioning not defined well enough
• Measurement parameters not defined well enough

• Too much inherent measurement system variation – measurement

system cannot be used for measurement application

• Inadequate clamping of sample in inspection fixture

• Insufficient gage resolution or rounded/truncated measurement

results (Rule of Ten)

• Difference in operator skills – experience and level of training

received
 What if a Gage R&R Study fails? – Root Cause Analysis Tools
Gage R&R Study graph (this chart was run for Gage R&R Study from earlier)
By Part: all study
measurement arranged by
Graphical representation
sample. Sample averages
of Components of
connected by line. Ideally,
Variation (in relation to
multiple measurements for
%Contribution, %Study
each part show little variation.
Variation, %Tolerance)

Range chart: graphically By Operator: helps assess

displays operator measurement averages and
consistency variability are consistent
(Repeatability). Any points across operators. Ideally, the
outside of the control line if parallel to the X axis.
limits show that the
operator is not measuring
Sample-Operator Interaction:
Displays average
Average chart: compares measurements by each
part-to-part variation to operator for each sample.
the Repeatability Ideally, the lines are virtually
component. Ideally shows identical.
lack of control.
What if a Gage R&R Study fails? – Root Cause Analysis Tools
Multi-Vari chart Multi-Vari Chart for ID by Trial - Operator
1 2 3 4 5 6
Graphical representation of the 1 2 Trial
0.1 978 1
relationships between a response
2
(measurement result) and factors 3
0.1 976 4
(trial, sample, operator).
0.1 974
It can help:

ID
• Identify patterns of variation
0.1 972
(operator-to-operator, trial-to-
trial etc.)
0.1 970
• Identify outliers
• Identify which root causes the 0.1 968
improvement efforts should be 1 2 3 4 5 6
focused on eliminating Sample
Panel variable: Operator (this chart was run for Gage R&R Study from earlier)
 What if a Gage R&R Study fails? – Root Cause Analysis Tools
Multi-Vari chart
Y axis set to span Y axis set to span
study variation Multi-Vari Chart for ID by Trial - Operator tolerance band Multi-Vari Chart for ID by Trial - Operator
(default) 1 2 3 4 5 6 1 2 3 4 5 6
1 2 Trial 0.204 1 2 Trial
0.1 978 1 1
2
USL 2
3 3
0.1 976 4 0.202 4

0.1 974 0.200

ID

ID
0.1 972
0.1 98

0.1 970
0.1 96

0.1 968 LSL

0.1 94
1 2 3 4 5 6 1 2 3 4 5 6
Sample %Study Sample
Panel variable: Operator Panel variable: Operator
%Tolerance
Variation
Always assess statistical vs. practical significance, and keep the measurement application in mind
• Above two charts are from the same study with the Y axes set to span different ranges (part-to-part/study variation vs.
tolerance band)
What if a Gage R&R Study fails? – Root Cause Analysis Tools, Scenarios

• What is the potential issue? Multi-Vari Chart for Dim 9 by Trial - Operator
1. Outlier – operator 2, sample 3:
1 2 3
repeatability issues 1 2 Trial
0.573
2. Reproducibility issues 1
2
0.572 3
4
• What are the potential root causes? 0.571

Typo (can only be removed from 0.570

Dim 9
dataset if proven), sample geometry, 0.569
measurement method
0.568

0.567
• How could the measurement system
be improved? 0.566

Verify sample geometry. Verify if 0.565

1 2 3
measurement procedure needs to be
Sample
improved. Verify operator skills. Panel variable: Operator
What if a Gage R&R Study fails? – Root Cause Analysis Tools, Scenarios

• What is the potential issue? Multi-Vari Chart for Dim 6 by Trial - Operators
Data points from operator “EM” see
1 10 2 3 4 5 6 7 8 9
significantly more variation when EM TN Trial
0.0435 1
compared to those from operator “TN” 2
0.0430 3

• What are the potential root causes? 0.0425

Operator skills. Operator training.
0.0420

Dim 6
Measurement procedure not specific
0.041 5
enough.
0.041 0

• How could the measurement system 0.0405

be improved?
0.0400
Provide adequate training. Improve
1 10 2 3 4 5 6 7 8 9
measurement procedure to be more Parts
specific. Panel variable: Operators
 What if a Gage R&R Study fails? – Root Cause Analysis Tools, Scenarios
• What is the potential issue? Multi-Vari Chart for Width by Trial - Operator
1. Repeatability – too much trial-to-trial 1 2 3 4 5 6 7 8 9 10
variation Jeremy Kenny Miguel Trial
9.85
1
2. Reproducibility – difference between 2
3
operator averages too big
9.84

• What are the potential root causes?

Width
1. Too much inherent measurement 9.83

system variation
2. Operator training, skills 9.82

• How could the measurement system be

9.81
improved?
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
Provide adequate training. Part
Measurement system may not be suitable Panel variable: Operator
for application. Improvements to current
system or implement new system.
Takeaways

• Know your metrics

• Know your measurement application and pick your metric accordingly

• Look for patterns of variation

• Identify Root Causes, Improve Measurement System if necessary

Gabor A. Szabo, CQE, CSSGB

(626) 733-5279
[email protected]