Design Expert 7: Intro duction

Richard Buxton. 2007.

1 Soft
Softw
ware
are for
for Desi
Design
gn of Expe
Experi
rime
men
nts

Design Expert  is a piece of software designed to help with the design and interpretation
of multi-factor experiments. In polymer processing, we might use the software to help us
design an experiment to see how a property such as tensile strength varies with changes
in the processing conditions - e.g. changes in rotor speed or ram pressure.
The software off ers
ers a wide range of designs, including factorials, fractional factorials and
composit
compositee designs.
designs. It can handle
handle both process
process variabl
variables,
es, such as rotor
rotor speed,
speed, and also
mixture variables, such as the proportion of resin in a plastic compound. Design Expert 
off ers
ers computer generated D-optimal designs for cases where standard designs are not
applicable, or where we wish to augment an existing design - for example, to fit a more
flexible model.
The aim of this document is to help you get started with Design Expert . Afte
Afterr a brie
brief 
f 
review of statistical terms and concepts, Section 3 gives some advice on selecting an
appropri
appropriate
ate class of designs.
designs. Section
Section 4 gives
gives an overvi
overview
ew of the main steps involv
involved
ed in
running Design Expert , while Sections 5, 6 and 7 give some more detailed guidance.
If you’re new to the statistical approach to design, you may find it useful to consult some
of the books listed in the reference section at the end of the document.
Warning! Design Expert  is not suitable for designing and analysing repeated measures 
designs of the kind that are often used in the Human Sciences.

2 Stat
Statis
isti
tica
call term
termss and
and con
conce
cept
ptss

This section reviews some statistical terms and concepts that are used in the Design 
Expert  software.

2.1 Data

The factors that we vary in experiment can be divided into Process  variables, such as
the speed of an engine or the thickness of an adhesive layer, and Mixture  variables, such
as the proportion of resin in a plastic compound, or the proportion of fat in a chicken
feed.
feed. With a mixture
mixture variable,
ariable, the eff ect
ect depends on the proportion  of a constituent in
the mixture, rather than on the absolute amount.

1
Process variables can be split into Continuous and Categorical.

• Continuous - can be varied freely over a range

– Engine speed, with speed varying from 1000 to 5000rpm

• Categorical - restricted to a few distinct values

– Filler, with a choice between pulverised fuel ash and glass beads

Categorical variables are sometimes divided into Nominal and Ordinal. An Ordinal vari-
able is a categorical variable in which the categories have a natural order - e.g. Small,
Medium, Large. A Nominal variable is one where no order is implied - e.g. a set of 
categories like colour or make of car.
Note: Design Expert uses the term ‘factor’ to refer to both continuous and categorical
variables - this contrasts with some statistical texts and software that reserve the term
‘factor’ for a categorical  variable. In Design Expert, continuous and categorical variables
are described as ‘Numeric’ and ‘Categoric’ respectively.

2.2 Model

In analysing an experiment, we fit models relating a response  or quality characteristic  to

a set of controllable variables. For continuous control variables, we often use a linear,
factorial or quadratic model - for example. . .
Linear model Y   = β 0 + β 1 X 1 + β 2 X 2 + 
Factorial model Y   = β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 + 
Quadratic model Y   = β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 + β 11 X 12 + β 22 X 22 + 

Apart from the intercept, the terms in these models fall into one of three categories.

• Linear terms (Main eff ects) of the form β i X i

– model the average eff ect of varying a control

• Two-factor interactions of the form β ij X i X  j

– allow the eff ect of changing a control to vary with the setting of another control

• Quadratic terms of the form β ii X i2

– allow for curvature in the eff ect of a control on the response

Models for categorical controls often involve a set  of terms to represent a single main
eff ect or two-factor interaction, but the interpretation of the eff ects is similar.

2
2.3 Resolution

A full factorial design is one where the experiment uses all combinations of the levels
of factors. Many designs involve running only a small fraction of a full factorial. This
makes our experiments more economical, but results in what is known as aliasing  between
diff erent eff ects. If two eff ects are aliased together, we can estimate their combined eff ect,
but cannot separate out the size of each individual eff ect. For example, if  A and BC  are
aliased, we’ll be able to estimate the combined eff ect of  A and BC , but we won’t be able
to obtain a separate estimate of  A, or of  BC .
The resolution  of a design gives an indication of the degree to which we’re going to be
able to separate out individual eff ects.

• Resolution III In a resolution III design, main eff ects are aliased with two-factor
interactions. Unless the two-factor interactions are negligible, our estimates of the
main eff ects will be biased.

• Resolution IV Two-factor interactions will be aliased in pairs, but we’ll be able to

estimate the main eff ects clear of any other main eff ects or two-factor interactions.

• Resolution V All main eff ects and two-factor interactions will be clear of other
main eff ects or two-factor interactions. Unless the higher order interactions are
sizeable, a Resolution V will often be almost as good as a full factorial.

3 Selecting a Design

Design Expert  off e rs a large number of diff erent classes of design. If you’re new to the
software, you may well need some help in selecting a class of designs that’s appropriate
for your particular situation. The following are some suggestions, based on the type of 
control variables that you’re dealing with and on the type of model that you wish to fit.
In Design Expert , the design classes are arranged in tabs on the left hand side of the
screen. In the recommendations below, I give the tab first and then the class within the
tab, for example ‘Factorial/2-Level Factorial’. In some cases, I have suggested more than
one class of design.

3.1 Process variables only

• All continuous

– Linear or Factorial model

Factorial/2-Level Factorial
– Quadratic model
Response Surface/Central Composite

3
 • All categorical

– All factors at two levels

Factorial/2-Level Factorial
– Some factors at more than two levels
Factorial/General Factorial
Factorial/D-Optimal

• Mix of continuous and categorical

– Linear or factorial model for continuous controls

∗ All categorical factors restricted to two levels
Factorial/2-Level Factorial
∗ Some categorical factors have 3 or more levels
Factorial/General Factorial
Factorial/D-Optimal
– Quadratic model for continuous controls
Response Surface/Central Composite
Response Surface/D-optimal

3.2 Mixture variables only

• All components have same range and no constraints on design space

Mixture/Simplex Lattice
Mixture/Simplex Centroid
• Above conditions not satisfied
Mixture/D-optimal

3.3 Mixture and process variables

If you have a combination of mixture and process variables, you’ll need to use the
Combined/D-optimal  class.

4 Running Design Expert

Once you’ve decided on the type of design that you’re going to use, the Design Expert 
software is quite easy to run. There are three main steps. . .

1. Constructing the design

Design Expert  takes you through a series of screens in which you specify the infor-
mation needed to construct the design - e.g. names and ranges of your variables and

4
 degree of replication. At the end of this step, Design Expert  gives the design layout .
This is a list of the experimental settings to be used for each of the experimental
runs. The order of the runs is randomised and this is the order in which they should
be carried out.

2. Evaluating the design

Design Expert  off ers two types of information to help you check whether a design
will meet your requirements.

• Alias pattern
– This shows whether you’ll be able to estimate the eff ects that you’re in-
terested in.
• Precision of the fitted model
– The precision of predictions from a fitted model will depend partly on
the background process variation and partly on the experimental design.
Provided you’ve got some idea of the likely size of the background variation,
Design Expert  can help you to estimate the kind of precision that you’re
likely to achieve. If it looks as though you’re not going to be able to
achieve the required precision, you may need to consider carrying out a
larger experiment.

3. Modelling and interpreting the experimental data

Design Expert  off ers a wide range of analytical and graphical techniques for model
fitting and interpretation.
In the analysis of 2-level Factorial designs, extensive use is made of Normal prob-
ability plots to highlight any active  factors - i.e. factors that aff ect the response.
If you’re not familiar with this approach, see Montgomery(2005), section 6.5. The
idea of this approach is that if none of the factors is active, the variation in the
estimates of eff ects will be purely due to random variation, so that a Normal plot
of the estimates will be roughly linear. Active factors will show up as points that
are separated from the underlying linear pattern.

Design Expert  provides an extensive help system. You can access this from any screen
by clicking on the Help or Screen Tips  buttons on the toolbar - these are indicated by a
question mark and light bulb. You can also obtain help by selecting Help. . . Contents
from the main menu.
In addition to the help system, there is a set of tutorials, which take you through the
design and analysis of particular types of experiment. You can access these as follows.

1. Select Help. . . Contents

2. Click on the Contents  tab

3. Select Web-based Tutorials. . . User Tutorials

4. Select the tutorial that you require

5
5 Specifying the features of a design

This section off ers some help with the information that you’ll need to supply during the
design stage.

5.1 Lower and Upper limits for variables

For each numeric variable, you’ll be asked for lower and upper limits. In specifying the
limits, you should aim to make the range of variation large enough to produce eff ects that
will show up against the background process variation, while avoiding limits that are so
extreme as to prevent the process from operating properly.

5.2 Replication

We say that we are replicating  a run, when we include the same set of processing conditions
more than once during the course of an experiment. There are two reasons for replication.

1. Increases the precision of estimates of eff ects

2. Gives additional information on background process variation

Replication is particularly important with small designs, such as a 22 Factorial, where

there are very few runs at each setting of a factor.
Note: If the initial design is a fractional Factorial, it is generally better to move to a full
Factorial (or a larger fraction of the full Factorial), rather than replicating the original
fractional design. Moving to a less fractionated design will reduce the amount of aliasing,
while giving the same improvement in precision that we’d obtain from pure replication.

5.3 Replication vs Repeated measurements

One common mistake is to confuse genuine replicates with repeated measurements on the
same experimental unit.
Suppose we are producing rubber sealing strips for use with car doors and wish to assess
the eff ect of varying the processing conditions used in mixing the raw materials - e.g. rotor
speed or ram pressure. In our experiment, we’ll try a range of diff erent combinations of 
processing conditions. Each batch of material will pass through a series of processes, such
as extrusion, and we’ll then measure the properties of the finished product. To perform
a genuine replicate, we’ll need to mix up a new batch of material (using the same rotor
speed, ram pressure, etc) and pass it through the intermediate processes such as extrusion,
to give the final door seal. The idea is that the variation from one replicate to another
should reflect all sources of background variation that arise when we go from one set of 
processing conditions to another.
If we simply take several measurements on the output produced from a single batch of 
material, the variation that we observe will reflect only the measurement error and not

6
the variation between diff erent runs of the mixer or extruder. As a result, the variation
between diff erent measurements will seriously underestimate the total process variation
and will probably lead to us highlight far too many factors as active.
Note: Although repeated measurements will not usually give a realistic picture of the
total background variation, it’s often a good idea to take repeated measurements in order
to reduce the measurement error. If you do this, the best way of handling the data is to
find the mean  of the measurements for each run and to enter this value as your response
in Design expert . By doing this, you’ll benefit from the increase in the precision of your
measurements, while avoiding the problem of underestimating the process variation.

5.4 Blocking

Experiments are often subject to sources of variation that are unavoidable, but can be
predicted in advance. For example, we may have to spread the experiment over several
days, possibly with very diff erent ambient temperature and humidity. This day to day
variation may well aff ect our measured response. Many of the designs off ered by Design 
Expert  can be split into two or more blocks, so that we can correct for these unwanted
sources of variation. For a discussion of blocking, see Montgomery(2005).

5.5 Centre points

In constructing 2-level factorial designs, Design Expert  allows you to add centre points ,
at which all continuous controls are set in the middle of their ranges. The inclusion of 
centre points allows us to check whether our fitted model is adequate, or whether we need
to include some quadratic terms to allow for curvature.

5.6 Randomisation

Design Expert  automatically randomises the order of the experimental runs. There are
two reasons for randomisation.

• Reduces the risk of unanticipated sources of variation aff ecting the estimates of 
eff ects

• Helps to meet the assumptions of the statistical methods used in analysing experi-
mental data

Sometimes one or more of the variables are difficult to change and it is necessary to impose
restrictions on the degree of randomisation. This results in a split plot  design, which is
rather more difficult to analyse - see Montgomery(2005), section 14-4. If you feel that
full randomisation is not feasible, you are strongly advised to consult a statistician before 
proceeding with your experiment.

7
6 Looking at the properties of a design

Once a design has been constructed, you’ll see a tree diagram on the left hand
side of the Design Expert  window. To examine the properties of the design, select
Design(Actual). . . Evaluation .

6.1 Alias pattern

To examine the alias pattern for a design, click on the Results  tab. Near the top of the
output, you’ll see a table giving a list of estimated terms, together with their aliases.
You can use this table to check whether you’ll be able to isolate the eff ects that you’re
interested in.
Note: If you’ve selected a full factorial design, all eff ects will be estimable and the table
of aliases will not appear.

6.2 Precision

Design Expert  off ers two ways of assessing the precision that you’re likely to obtain with
a given design.

• Precision of predictions at diff erent points in the design space

In the evaluation window, click on Graphs . Design Expert  will give you a contour
plot showing the standard error for predictions of the response at diff erent points in
the design space. Note that the figures on the plot need to be scaled by an estimate
of the standard deviation of the background process variation. For example, if 
previous experimentation suggests that the standard deviation of the background
variation, σ , is 12, a figure of 0.25 on the plot would indicate a standard error of 
0.25 × 12 = 3.
• Power analysis
In the evaluation window, click on Results . Scroll down the display until you come
to a table giving information about each term in the model. The columns on the
right of this table give a power analysis for eff e cts of size 0.5σ, 1σ and 2σ . The
figures in these columns give the chance of detecting an eff ect of the corresponding
size.

7 Using Design Expert to analyse a design

Once you’ve carried out your experiment, you enter the response values into the appro-
priate columns on the Design Layout Screen. If this screen is not currently visible, you
can switch to it by selecting Design(Actual) (in the tree on the left of the screen).
To analyse a response, click on the response name (in the tree on the left of the screen).
You’ll now see a set of tabs that you can use to access various techniques for analysing

8
and interpreting the fitted models. The following are a few notes on the facilities that are
available under each tab.

• Eff ects

– Half-Normal and Normal plots for highlighting active factors

– Pareto chart for giving a picture of the relative sizes of the diff erent eff ects

• ANOVA

– Analysis of variance
This can sometimes be used as alternative way of highlighting active factors
– Summary statistics.
One useful statistic here is the one labelled ‘Adeq Precision’. This is a
kind of signal-to-noise ratio that measures the ratio of the range of varia-
tion in the predicted response to an estimate of the standard error of the
predictions. A high value indicates that the variation that we’re observing
is large in relation to the underlying uncertainty of the fitted model.
– Coe fficients of fitted model
Unless the interactions are negligible, the numerical coefficients can be
difficult to interpret. It is generally better to examine the model through
graphical plots - see below.

• Diagnostics

– Residual plots
Design Expert  off ers the usual range of residual plots for checking assump-
tions such as Normality and constant variance.
– Box-Cox plot for power transformations
This can help us to decide whether we could improve the fit of the model
by measuring the response on a diff erent scale - e.g. by using the log of the
response values.
– Plots of leverage and influence statistics
These plots show the influence of individual data points on the fitted model.
One of the aims of statistical design is to ensure that our models make good
use of all  observations and are not critically dependent on just a few points.
So for standard designs such as Factorials, these statistics will not usually
be needed.

• Model Graphs

Design Expert  off e rs a wide range of diff erent plots to show how the response
varies with changes in the controls. To change the type of plot, use the View 
menu. Notice that contour and 3D surface plots are only appropriate for con-
tinuous  control variables.

9
 If you have 3 or more controls, you’ll obtain a plot of two of the controls with the
remaining variables held at fixed settings. You can change the variables that are
plotted and you can also change the settings of the fixed variables.
Available plots are . . .

– One Factor
Main e  ff ects  plot showing the average eff ect of shifting a single control,
while holding the other controls constant.
– Interaction
Plot showing how the eff ect of changing one control varies with changes in
a second control
– Contour
– 3-D Surface
Note that the surface plot can be rotated to obtain a better view
– Cube
Gives the numerical response values at each combination of three of the
controls. Any further controls are held at fixed settings.

8 References

For a clear introduction  to design of experiments, see Part 3 of DeVor et al (1992) or

Part 4 of Montgomery (2001).
For a more comprehensive coverage, see Montgomery(2005), Wu and Hamada (2000) or
Myers and Montgomery (1995).

DeVor, R.E. et al (1992). Statistical Quality Design and Control, Macmillan, New
York.

Myers, R.H. and Montgomery, D.C. (1995). Response Surface Methodology, Wi-
ley, New York.

Montgomery, D.C. (2001). Introduction to Statistical Quality Control, Wiley,

New York.

Montgomery, D.C. (2005). Design and Analysis of Experiments, Wiley, Hoboken,

N.J.

Wu, C.F.J. and Hamada, M. (2000). Experiments: Planning, Analysis and pa-
rameter Design Optimization, Wiley, New York.

10