/

,/-

~---

,:

PRJNCIPLES OF STATISTICAL DESIGN WITH SPECIAL REFERENCE

TO EXPERDlENT AND TREATMENT DESIGN

by
Walter T. Federer, Cornell University

Abstract
Statistical design has many facets, two of which are experiment design and
treatment design.

The former refers to the arrangement of treatments

periment, while the latter refers to the selection of

ment.

treatme~ts

The f:r-equently used te:r-m "experimental design" is not

i~

an ex-

for the experi-

;~ed

here because

it has been used in many diffe:r-ent contexts and besides the '':iesign" should :not
be "experimental".

Sir Ronald A. Fisher laid down the three "::lasic principles of

experiment design as replication, randomization, and blocking (local control) in

the 1920's.

Frank Yates added orthogonality and confounding in 1933.

Since then

several other principles have been discussed, viz. sensitivity, efficiency,

balancedness of various types, variance optimality,
and sufficiency.

connecte~Dess,

resolvability,

Many of the above principles have been used for treatment de-

sign and others have been added, e.g., unbiasedness, rotatability, mean squared
error optimality, alias balance, saturation, and symmetry.

To many writers of

statistical literature, "experimental design" is combinatorics, computing, and/or

hypothesis testing.

While the first is useful in the construction of plans and

in attaining certain properties for the plans, the last two are not statistical
design.

Also, many statisticians confuse plans for statistical designs with

planning investigations.

Plans are useful but are only one aspect of planning.

Five axioms are given to aid in planning experiments .

FRINCIPLES OF STATISTICAL DESIGN WITH SPECIAL REFERENCE

TO EXPERThfENT Al'ID TREATMENT DESIGN

by
BU-813-M
Walter T. Federer, Cornell University

May, 1983
Revised July 1983

Introduction
Statistical design for an investigation encompasses many items; these are:
(i)

Complete description of population involved; of the sampling unit

(s.u. ), the elements making up the population; unit of the experiment (e.u. ),
or experiment unit, the smallest unit to which one treatment is applied; of the
variables to be Eeasured, as well as their 1istributions; the response equations
prior to applying treatments; and the observational

Q~it

(o.u. ), the smallest unit

on which one observation is made.

(ii)

Complete description, validation, and standardization of measurements

and measuring instruments for the investigation.

(iii)

Treatment design, the selection of the treatments to be used in the

investigation as well as selection of adequate points of reference (controls,

standards, placebos).
(iv)
ment.

Experiment design, the arrangement of the treatment in the experi-

Randomized designs are preferred over non-randomized ones.

(v)

Survey design.

If the investigation is a survey of what is in a

population, a probability sample survey with proper stratification into relatively

homogeneous subgroups is usually preferred.
(vi)

Sequential design.

In some sequentially conducted experiments,

previous results determine which treatment(s) is(are) applied to the next e.u. (s)

entering the experiment in a sequential manner .

Paper No. BU-813-M in the Biometrics Unit, Cornell University.

- 2 -

(vii)

Modeling and model design.

met~e=atical

formulated to explain a phenomenon in an investigation.

priate model or response equation is often a iifficult
many experiments.

The selection and

expression of results is
Selection of an appro~~d

arduous task involving

of the e.u. 's represents the

arrange~ent

model design.
(viii)

Sample size ani replication

and/or e.u. 's to various categories.

numoe~,

including allocation of s.u. 's

Statistical

crite~ia,

as well as available

determine number of samples and replicates.

reso~ces,

(ix)

Size, shape, ar:d handling of e...:.. 's in an eXJ;,?eriment (often

to as experiment technique
that popular in the field

o~
o~

~oo

plot technique).
investigation.

refe~red

often the technique used is

Cfter: more efficient techniques can

be used, and the experimenter should be aware of this.

(x)

Principles and properties of statistical design.

have been formulated for designing experiments.

principles enunciated by Fisher (1935), i.e.,

Several principles

Some of these are the three basic

~ar-domization,

blocking, and local

control (blocking) and several others such as orthogonality, confounding, sensitivity, balance, efficiency, connectedness,
and sufficiency.
designs.

~esolvability,

variance

optimali~J,

Several of the above and others have been used in treatment

The statistical designs obtained by using a certain principle have the

property associated with the principle.

For example, using the principle of or-

thogonality, experiment and treatment designs have the property of being orthogonal
designs.
Some basic axioms (rules) to follow in designing an experiment are given
below.

Fail~e

to follow

~~y

one of them can leed to meaningless results for the

experimenter, with the study ending up as an

expe~ience

rather than an experiment.

One crucial rule which appears to be universally ignored by statisticians and

experimenters is:

- 3 Axiom I:

complete, precise,

which inferences

fu~d

Tigorous description of the population to

made is essential if inferences

to have any meaning.

It is insufficient to say "the results of this experiment apply to the population of which it is a sample".

What is the population precisely?

Textbooks

and literature, statistical or otheTwise, ignore this fact for the most part.
Any reference to the population is more of a "hand-waving" description.
does not
-~iom

~ow

II:

If one

what the population is, any inferences are meaningless.

Design for the expeTimer:t; do not experiment for the

des~gr:.

All too often ar;. experimer:ter and statistician feel their choice

ac.

c:~

e:JqJeri.ment and/ or treatment des~gr: is limited to those appearing i:: ts.::::.es or in

the literature.
rather

th~~

The experil:lent should be considered as i t is to be coc.iucted

being changed to fit a tabled design.

Treatments need ::ot

or dropped to make the expeTimer;.t fit a tabled design.

Following

~e

added

II means

JLv.io~

that new designs may have to be constructed to meet the requirements o: the experiment.
usi~g

One can easily construct many types of designs that are ::ot

simple construction procedures (see, e.g., Patterson and

i~

Wilii"'m"~

tables,

lsr76,

and Khare and Federer, 1981).

Axiom III:

Use the minimum amount of blocking possible to control heteTogeneous

variation among the e.u. 's.

The main idea in blocking is to group the e.u. 's in such a manner that the
variation among e.u. 's within a group is a minimum and that among groups is a
maximum, and this should be done with as few groups as is possible.

Ttere is no

need to allocate extra degrees of freedom to blocking when they should be allocated to error variance.

For example, if one has relatively uniform groups of

seven e.u. 's and has four treatments, the block size should be seven, not four.
~

Each of the treatments but one can be included twice in each block, ~~d a different
treatment is omitted from each block.

For more than four blocks, the procedure

- 4 (An experiment

can be repeated until the desired number of blocks are obtained.

Also, if several

design of this type does not appear in tables of designs. )

sources of experimental variation can be grouped together, this should be done

in order to minimize
of

tr~~splanting

For example, a block in the greenhouse, the day

blocki~~.

material from greenhouse to field,

~~d

a block in a field can

all be called a block rather than blocking on three separate sources of variation
and using a complex

design.

experime~t

If there is no grouping of e.u. 's into

subpopulations, then there should be no blocking.

avoided if simpler designs

~ill

suffice.

Complex designs should be

Ease of statistical

a~a~'Ses,

when data are lost, and interpretations are additional reasons for

us~ng

especially
a design

with minimum blocking.

Axiom IV:

Treatments with jifferent numbers of randomizations will hace different

numbers of replicates, will have different e.u. 's, and will have
~

different~

variances.
Consider the two sets of data, 100 measurements on one plant and one measurement from each of 100 plants.

Means, variance, and interval estimates are com-

puted using exactly the same formulae for both sets of data.
interested in the

computir~

aspects would fail to distinguish between the two

populations to which the data refer.

of measurements on the

The indi7idual

plant.

The first set refers only to the population

The second set refers to the population which

is a mixture of two populations, plants and measurements.

Thus, if the one plant

was randomly selected from the population of plants, the sample size is one, no
matter if a million measurements on that one plant are made.

In the second case,

100 randomizations were used and the sample size of plants is 100.
in this example should be obvious to almost everyone, but in
distinction may be rather subtle.

m~~y

The differences
situations the

This is especially true when the e.u. 's for

different treatments are different sizes and involve different blocking patterns.

- 5 -

Axiom V:

A valid error variance for the

differe~ce

between two treatment effects

must contain all sources of variation in the e.u. 's except that due to the treatments themselves.
This is Sir R. A. Fisher's

(1935)

definitio~

variance for differences of treatment effects.

of a valid estimate of an error

The erroc variance is not neces-

sarily the last line in an analysis of variance table or

"error" on computer output.

so~e

term designated as

Every sir..gle contrast in an experiment could have a

different error variance because of differences in variances for each treatment,

differences in size of e.u. 's for each treatment or groups of treatments, use
of an incorrect response model, or different units of

measu~ement

for different

treatments.
Many, many experiment designs and classes of designs have been devised by
statisticians and mathematicians, but they have been constructed without any con-

sideration of a particular experiment.

has been provided

for~

For these designs, a statistical analysis

(sometimes erroneously called the) linear model.

It should

be noted that the linear model provided may be incorrect and inappropriate, leading
to incorrect statistical analyses and inferences.
the above five axioms should be kept

L~

In using any experiment design,

mind at all times.

Principles of Experiment Design

Fisher

(1935) first presented the three principles of experiment design

known as randomization, replication, and blocking (see Figure 1).

is an extended version of Figure l as given in Federer and Balaam

Figure 2

(1972).

The principles of orthogonality and confounding were discussed by Yates

(1933, 1937).

section.

Blocking has been discussed to some extent in the previous

Replication refers to the number of e.u. 's on each treatment, not

necessarily the number of observations.

Perhaps the most frequently occurring

- 6 -

Replication

Validity of estimate
of experimental error

Figure 1.

Reduction of
experimental error

Fisher's principle of design of experiments .

- 7 -

randomization

...

valid estimate of
experimental error

'---,--------~

~ ~~

balancing

""'
f?artial

~---------------~Jconfoundinq 1~---

Figure 2 .

reduction of expertmental error variance

for differences of
treatment effects

An expanded version of Fisher's diagram.

- 8 mistake in scientific and textbook literature is to have one block of a randomized

4lt

complete block design with multiple observations on each

consider this as a completely randomized

des~gn.

treat~ent

Tee reason

that the concepts of replication and valid estimate of error

understood.

Unless the e.u. is precisely

of replication remains fuzzy at best.

the above five and

~o

de~ined,

Additional

ar.d then to

this

~stake

va~iance

are not

fo~

is

used, and understood, the idea

added to

p~inciples ~ave bee~

doubt more will be added in the

Some of the

~~ture.

additional ones discussed below are balance, se:r:sitivity, efficiency, coru:ectedness, optimality, resolvability, and

sufficie~cy.

Replication
The replicatio::: principle is a
Experiment

units

necessa~:.- pa~t

are replicated to study

of scientific ir:-.e.s-:igation.

t~eatment

variation

The number of e.u. 's, not the number of s.u. 's or o.u. 's,

4lt

of replications in an experiment.

e._;_. to e. u.

:~~om

dete~mine t~e

number

Entire experiments may need to be replicated

(repeated) by other experimenters to validate the results.

The idea

o~

repeat-

ability of results by other investigators is a basic tenet of scientific inquiry.

Experiment designs wtich have the property of being replicated designs are an
essential tool in the hands of competent scientists.

The person who said "Damn

the duplicate plot, give me one and I know where I am" was not much of a scientist.
He wanted to ignore the universal fact that e.u. 's differ and that repeatability
of results of scientific investigations is a necessity.
Randomization
Randomization is the basis for obtaining a valid estimate of a
effect and valid estimate of error variation.
been unclear for

41t

~~y,

in that they do not

ization can both be used.

some.

The concept of

~~derstand

t~eatment

r~~domization

how blocking and

has

~andom-

The confinement of ~andomization within blocks bothers

This should not be a point of confusion, because the restriction on all

- 9 possible permutations to the subset of

permutatio~s

does not vitiate obtaining valid estimates of

ances.

teat can occur within blocks

treat~ef-L

However, the complete restriction to a single

effects and error vari-

~ermutation

results in a

systematically arranged experiment design, which usually results in biased estimates of treatment effects and of error variances.

Hence, such designs are to

be avoided.
The randomization principle is difficult for certain scientists because they
feel they know how to place treatments in an experiment.
vestigators can
results.

j~diciously

place treatments in

the experimental

~reat~e~ts

~terial

lli~der

investigation,

as :est as possible.

impl~es
t~e

and obtair. unbiased

biasedness.

In order

experimenter should block

Ther., hithin each of the blocks,

treatments are randomly allocated to the e.u. 's.

a~ ex~er~ment

The funiamental natcJre of a human being

to be fair to the

Very few, if any, in-

Random allocation provides

equal fairness to treatments in an experiment.

3locking
Basically, there are four methods of reducing the error variance of a
difference between estimated treatment effects.
one method.

The second procedure is to refine

BlockL~g
experime~t

as discussed above is
techniques.

The

third procedure is to measure related extraneous variables and use covariance

techniques; this last method has often been overlooked and/or misused.

Continuous

variates classified into groups and called blocks, e.g., weight of plants, is an
inefficient method.

Instead, covariance should be used as fewer degrees of

freedom are usually used for covariates than for blocking.

If none of the above

three methods suffice, the alternative is to increase replication number (sample

size) until the degree of precision desired is attained.
The blocking principle implies that extraneous, non-treatment variation in

the experimental material is controlled and that this variation can be removed

- 10 -

from estimates of treatment variation.

ment

error,

thereby

increasir~

the precision of estimated treatment effects.

Blocking can reduce the amount of

degree of precision.

Blocking can be used to reduce experi-

exper~e~tal

In general, blocking is not well understood by experimenters,

whether in agriculture, medicine, or some

types of statisticians.
non-existent.

material required for a specified

Theorems on and

othe~

field.

p~ocedures

The same holds for several

for blocking are relatively

One exception is Vithayasai and Robson

(1974) who have given a

blocking strategy for an experiment.

Orthogonality
The

p~inciple

of orthogonality is a desirable one, as orthogonal experiment

designs have the property that the various sets of effects are orthogonal to each
other, that simple statistical analyses result, that valid estimates of effects
and their variances are obtained, and the greatest precision usually results.

Simple arithmetic averages may be used to obtain the differences between estimated
effects, and the variances of these differences are easily computed when a linear
model holds.

Since there is no mixing up of effects (confounding) in orthogonal

experiment designs, there is no loss in information due to this source, and maximum precision results.
As Preece

(1977) has amply demonstrated, the term "orthogonal designs" has

been used in at least three distinct ways.

In addition, the adjective "orthogonal"

has several conflicting meanings as does the noun "design".

The reader is referred

to Preece's excellent paper for a complete and up-to-date discussion on orthogonality.

An early paper on the subject was written by Yates (1933).

To illustrate

some of the difficulties, suppose that we wish a combinatorial definition of

orthogonality.

Let r .. equal the number of times treatment i occurs in block j.

l.J

Now consider the relative proportions of times that treatments 1, 2, , v occur

in a given block j, i.e., r 1 j : r 2 j : r 3j :

: r vj

If these relative proportions

- ll -

are the same for every block j, we say that treatment effects are orthogonal to

block effects (Federer, 1973).

An additive linear model formulation for this

situation would be given the following

~efinition

of orthogonality (Federer, 1973).

If differences between arithmetic means for any and all pairs of means, contains
only treatment and random error terms, treatments are orthogonal to all other
effects in the linear model.

If one

would be altered as follows:

the yatios of pairs of any two geometric means,

ignoring random error terms,

ha~

contai~s

For example, the ratio of treatment

a multiplicative model, the definition

o~~y

the ratio of treatment parameters.

geo~etric

means for treatments one and two,

.
.
. J..l.Tl\( .IIr .A. . . \1/r;
( nr .....A . )1/r
1.gnor2ng
eryor t erms, lS
)
i ...:.1: 2 .
.
=
'J=l J.

J=l J.

1:

I
1 -r 2

Th e comb"2na t or1.c
.

iefinition is not model dependent as aye the other two.

Use of the orthogonality principle has several advantages.

However, the

user must first decide what type of orthogonality is being considered for his

situation.

other principles may

deter~~e

that a non-orthogonal design will be

the most precise in terms of cost and utilization of material.

One such example

is discussed under balance.

Confounding
Orthogonal experiment designs do not exist for all situations, and hence it
is necessary to use non-orthogonal designs.

Whenever non-orthogonality is present

there is a mixing up (confounding) of the effects in an experiment.

If two or

more effects are not separable, the effects are said to be completely confounded.
Otherwise, when the effects can be separately estimated and they are not orthogonal,
they are said to be partially confounded.
plans for v

=3

For

treatments, A, B, C, and for b

ex~ple,

=4

consider the two design

blocks:

- 12 -

Plan I

Plan II

Block

Block

c
c
c
c

c
c

In the first design plan treatment A effects a!:d blocks 1 and 2 are completely
confounded.

No separate estimates of tt.e A effects or of the blocks 1 a!:d 2

effects can be obtained.

a.~d

In design

pl~

they can be estimated separately.

II, tr.e effects are partially

I:~

co~foQ~dei,

-creatme!:t A had been in twice

block and if treatments B and C had bee!: ir.cluied once in each of the

i~

fo~

every
blocks,

the plan would have been an orthogonal design plar. with blocks and treatment

effects being completely unconfounded, i.e., orthogonal.

founding is useful for many design situations.

The principle of con-

If there are more treatments

th~~

can be included in a block, one must use an incomplete block design with partial
confounding of treatment and block effects.
on some effects than others, or when it is

When less information is required

necessa_~

by the nature of the treat-

ment, the principle of confounding may be invoked to obtain smaller blocks than
can accommodate all the treatments in each block.
When there is no confounding of various types of effects in a statistical
design, orthogonality results.
be estimated in the experiment.
statistical information.

When there is complete confounding, effects cannot

ConfoQ~ding

results in loss of precision and

In order to have confounding of block and treatments,

blocking must be present, illustrating the connection between the principles of

blocking and confounding.

If treatment effects are of equal importance, con-

founding should be avoided whenever possible and desirable .

- 13 -

Balancedness
Balancedness of effects is another useful principle in selecting and constructing design plans for an experiment.

Balancedness and orthogonality are

sometimes both obtainable, but in general different classes of experiment designs

are obtained.

Balancedness refers to a relationship between blocking and treat-

ment effects.

Hence, this illustrates the connection between the two principles.

There are many types of balance (see Heiayat and Federer, 1974, and Preece, 1982),
but attention here is confined mostly to variance balance; i.e., when the errors
in the experiment come from a distribution with a single ,variance parameter (homoscedastic), all differences between treatments will have the same variance.

When

orthogonality cannot be obtained, then balancedness is a desirable second choice.

It is relatively easy computationally to obtain solutions for the effects in most
cases.

There are situations where a balanced design has a lower error variance

for differences of means than a

and Federer, 1979).

orthogonal experiment design (see Shafiq

For example, consider the following three designs for treat-

ments A, B, and C.
I.

Complete Block

II.

Complete Block

III.

Complete Block

_g_

_3_

c
c

c
c

c
c

c
c

c
c

c
c
c

c
c

Suppose that groups of k

competL~

8 relatively homogeneous e.u. 's were available.

One

could form blocks of k = 8 as in plan I, blocks of k = 6 as in plan II, or blocks

of k

=3

as in plan III.

Plan I would require 3 groups, II 4 groups, and III 8

- 14 -

groups.

II would cost

1/3

more than I as one more group would be required, and

III would cost 8/3- l = 5/3 more than I and twice as much as II.

If each e. u.

costs the same regardless of which group it is in, then the three plans would
cost the same.

However, there are many situations where the e.u. 's do not cost

the same and some e.u. 's would need to be discaried to obtain plans II and III.
For example, suppose that there were 8 positions in an oven for baking pies.
There would only be 3 bakings for I but 8 for III, resulting in considerably
more expense.
Plan I is variance balanced, and II and III are botq variance balanced and
orthogonal.

This assumes a linear model for effects.

The three designs are also

pairwise balanced in that every pair of treatments occurs together in the blocks
an equal number of times A; note that A

= 21

for I, 16 for II, and 8 for III.

As stated above, there are many types of balance.

Although it took Preece

(1977) 23 pages to discuss the orthogonality muddle, it took him (1982) 102 pages
to discuss the balance tangle.

His papers should be read by users of these two

terms.
Efficiency
The principle of efficiency is useful in deciding which one or ones of a
class of experiment designs is optimal (best) with respect to a selected criterion.
Making use of this principle, designs can be ranked with respect to their relative
efficiencies using the selected criterion.

Orthogonal designs usually have 100%

relative efficiency with respect to variance (perhaps not if cost and variance
are both considered).

Partially confounded designs have efficiency less than

100%, as some information is lost due to the corounding.

The efficiency principle

says to utilize the most efficient design at one's disposal and suited for the

experiment.

In almost all situations discussed in statistical literature,

efficiency is related to variances of treatment contrasts from two different

- 15 -

iesigns.

For example, the three experiment designs given under

balance~~ess

average variances of a difference between two treatment means of 16cr2/63,

E

and cr2j4.

have

a2/4,
E

The relative intrablock efficiencies, when corrected for the differences

in degrees of freedom for the remainder sums of squares (i.e., 19, 18, and 14,
respectively), are 0.989, 1.000, and 0.975.

This example shows that even ignoring

cost of the additional groups required for designs II and III, a balanced design
is more efficient than a competing orthogonal design, III, and is almost as
efficient, 0.989, as the best competing orthogonal design.
additional cost, this balanced incomplete block design
either of the competing orthogonal designs.

If one considers the

more efficient than

i~

Hence, it is incorrect to say that

an orthogonal design is always more efficient than a balanced block iesign.

Another fallacious statement which frequently appears in statistical literature is that a balanced incomplete block design is variance optimal among all

incomplete block designs.

That this statement is false is illustrated in the

examples below for treatments A, B, C, and D:

Plan IV. Balanced
Incomplete Block

v.

Plan

Partially Balanced
Incomplete Block

Block

Block

_2_

_3_
A

c
c

_]_

_5_

c
c

c
c

c
c

D
D

The average variance of a difference for plan IV is cr2/2 and that for plan V is
E

7cr2/18.
E

Since plan V has a smaller variance than IV, it demonstrates that IV is

not variance optimal.

The statement is only true if occurrences of treatments

is limited to the case where treatment i occurs in block j either zero or once,
i.e., n ..
lJ

0, 1.

It is not true in general for n . . = m'""'

lJ

- 16 Another erroneous statement is that a

qua::era_~,

than a ternary design, a ternary than a

basic designs where n ..
~J

true for nij = m0,

n ..
~J

= 0,

~'

= 0,
~'

iesign is always more efficient

etc.

It is only true for

1, 2, (see Shafiq and Federer, 1979).

2, a binary design.

~ina_~

The above example demonstrates this.

Plan V is a

~ernayy

iesign, n ..
~J

= 0,

It is not
In plan IV,

1, 2, and has

a smaller variance.
Other criteria are often necessary

i~

experimentation.

For example, in a

nutrition experiment, a variance minimal design was selected for one-half of the
experiment because it was nutritionally

opt~~al

nutritionists believe the results of the

optimal, thus compromising nutritional

in the

eA~erime~t.

a~d

~ense

of having other

The other half was variance

statistical optimality.

Other criteria

could be cost, ability of techniciar.s to ccc.iuct the experiment, etc.

Optimality
The optimal principle states that the
utilized in conducting experiments.

~st

optimal experiment design be

When variance optimality is used, this en-

sures the greatest precision with minimum

eA~nditure

of experimental material.

There are many criteria which could be utilized in setting up optimality measures.

A great majority of the statistical literature on optimality deals with variance

optimality, with the adjective variance frequently being omitted.

One could set

up measures of combinatorial optimality or optimality criteria for a multiplicative response model.

Variance optimality is very much related to efficiency.

Connectedness
The connectedness principle is related to the blocking and confounding
principles.

Totally confounded designs are not connected.

(orthogonal) and partially confounded

desi~s

Unconfounded

are connected, i.e., all estimable

treatment contrasts can be estimated from the experiment design.

should be applied to all proposed

experimer.~al

This principle

procedures to determine confounding

- 17 -

structures and

con~ectedness.

The first plan

confounding is not

give~ ~:ier

connected but the remaining plans given are.

Some relatively

~ecent

papers on this

princi~le

~~d

on designs wnich have

or have not the property of being connected are by Vartak

Hedayat (1974 ), Eccleston an:i Russell (1975) and

(1953), Eccleston and

c:.- Raghavarao and Fe:ierer (1975).

Resolvability
The principle of resolvability is to have all treatments in a complete block
~egardless
~.-hen

of how

man~.- inco~plete

blocks are used..

Resolvable desig::s are useful

alternate analyses may be used, silch as, e. g., a

~andomized

con:plete blocks

.:iesign analysis for an ex:peri.ment actually laid o..:t as an incomplete block design.
Cccasionally it Tiay be desirable to eli,.,.,inate one

Dr

r:.ore complete blocks in the

analysis.

Sensitivity
The sensitivity of a statistical design first reached attention of researchers
~rhen

Fisher (1935) used the example of a lady tasting tea where the t-,.,..o kinds of

tea were milk first and then tea versus tea with milk infused.
on sensitivity since that time include papers by

3~adley

Some literature

and Schumanr. (1957),

Dar ( 1962, 1964), Lashof et al. ( 1956), Mandel and Stiehler ( 1954), and Schumann
and Bradley (1957).

In the first citation above, several examples are given

illustrating a need for comparing sensitivities of various experimental procedures

in several different fields.

Experimenters need to select procedures which will

be the most sensitive for attaining the goals of the experiment.

For example,

measurement of time to recurrence of cancer after treatment is a much more sensitive measurement than is measuring whether or not the disease recurred.
the effect of
~

applyir~

Measuring

nitrogen on the amount foU"-:i in the plant can be done much

more effectively in samples from the 8- 10 node ~ea than in the leaves.
node area samples were able to detect a 25 lb.

inc~ease

The 8- 10

in nitrogen fertilizer

- 18 -

with the same precision as leaf samples could detect a 150 lb. increase.

Thus

the 8-10 node samples were six times as sensitive as the leaf samples in detecting additional applications of nitrogen fertilizer.

Samples taken at the base of

the sugar cane stalk would have been even more sensitive in measuring changes in
nitrogen levels.
Sufficiency
The sufficiency principle states that an experiment design and a given response

~odel

should proviie statistics that contain all the information in the

Experiment designs yielding a minimal set of sufficient statistics

exper~nt.

which is complete with respect to a given response model would be desirable.

For

example, Graybill and Weeks (1959) show that Yates' combined interblock and intrablock estimator is based on a set of minimal sufficient statistics but the set is
not coreplete.

The sufficiency principle needs much more amplification in sta-

tistic~ literature with regard to experiment design.

Principles of Treatment Design
Treatment design (the selection of treatment for an experiment) can be categorized into the following types:
(i)

controls, standards, placebos, or other points of reference,

(ii)

discrete levels of one or more factors (factorial designs),

(iii)

continuous level of one or more factors (regression or response

surface designs),

(iv)

single level of two or more factors in mixtures (diallel crossing,

tournaments, matched pairs, intercropping, etc.), and

(v)

combinations of the above.

The principles of treatment design discussed below apply to each of the above

types.

Principles found useful in treatment design are efficiency, variance

- 19 opt~ality,

replication, orthogonality, balance, symmetry, unbiasedness, connected-

ness, saturation, rotatability, and mean squared error.

Replication
Replication of treatments (or combinations) in an experiment should be related to their importance.

Otherwise replication should be equal.

For fraction-

ally replicated experiments, a certain class of treatment effects needs to be zero

or at least small in comparison with other classes of treatment effects.

In

special cases, a fractional replicate is all that is possible, e.g., diallel cross,
tour~aments,
cult~es

etc.

in statistical analyses and in interpretation of effects.

pri~ciple

The replication

needs to be considered seriously for any fractional replicate of a

treatment design.
o~

In some cases, missing subclass combinations leads to diffi-

Likewise, the replication principle can be applied to repetitions

a fractional replicate, i.e., a fractional replicate should not be repeatea unless

one is certain that the assumed class of effects is actually zero.

fraction and different combinations should be selected.

Instead another

This procedure tends

towari completion of a full replicate of a treatment design.

After a full repli-

cate is achieved then replication of a combination should be considered.

Orthogonality
The orthogonality principle for a treatment design ensures ease of statistical
~~alysis

and variance optimality problems.

Use of this principle has led to con-

struction of such fractional replicates of factorial designs as orthogonal main

effect plans, orthogonal resolution V plans, etc.

Use of the principle to obtain

orthogonal plans leads to more precise interpretation of effects, i.e., they are
unentangled with other effects.

- 20 -

Con~ounding

When

(aliasing)

~ractions

o~

~ull

treatment design are less than one, some

with other

e~~ects

will necessarily be

con~ounded

con~ound e~~ects o~

most importance with those that are small or negligible.

would allow zero or small bias in the estimates

magnitude

o~

treatment

The less important

as aliases

o~

e~~ects

e~~ects

is unknown,

which are

the more important

shoul~

o~

e~~ects.

important

This

When the

e~~ects.

should be avoided.

co~oundir~ o~ e~~ects

totally or partially, are denoted

con~ounded,

could use the principle that the type

The principle used here is to

e~~ects.

Following Raktoe et al. (1981), one

o~ con~ounding ~or

be similar in nature, i.e., a balanced arrangement

all important
o~

e~~ects

aliases.

Synmetry
The symmetry principle is to have the same number and type
of the

under study when the

~actors

~actors

are

o~

o~

levels

equal importance.

~or

each

One may know

something about the response function for certain ~actors, one may be interested
in the response
less detailed
t~ue,

~or

one

~actor

in~ormation

over a smaller range than others, or one may wish

on one or more

o~

the

~actors.

If any

o~

these were

non-symmetrical or asymmetrical levels would be selected.

Saturation
Saturated designs are parsimonious designs in that the number
taken is equal to the number

o~

parameters to be estimated.

o~

observations

In many situations,

e.g., some quality control investigations, observations are very costly.

a minimal number

o~

observations are required to estimate a parameter set, say

The saturation principle leads to construction

Of course, in the class

Hence,

o~

o~

~1

saturated treatment designs.

saturated designs one would use the one(s) which have

variance optimality properties .

- 21 -

Efficiency
The efficiency principle for treatment design applies in various ways.
a treatment design should be selected to meet the goals of the experiment.

First,
Unless

this is done, more than one experiment may be necessary to meet the stated goals,
resulting in wasteful expenditure of resources.
or variables must be selected with care.

Secondly, the levels of factors

Otherwise, the goals of the experiment

may not be achieved or achieved with low precision, resulting in low efficiency.
This principle is closely tied to variance optimality, which follows, when
the goal is to optimize a procedure with respect to a stated criterion.
Balance
As noted for experiment designs and by Preece (1982), there are many types
of balance for treatment designs and many uses of the term.

Many phrases have

been utilized as is amply demonstrated in Preece's (1982) discussion of balance.

Others could be added from the literature on linear models (e.g., Searle, 1971).
In utilizing the balance principle, one can say that in general most uses of the
principle have some desirable properties, and it is essential that the writer
precisely defines the context in which the term balance is being used.

Otherwise,

considerable confusion can arise.

Yates

(1936, 1937) introduced the idea of balance in constructing confounded

arrangements for factorial treatment design.

A balanced arrangement is one for

which all interactions of a specified order are confounded equally.

Thus, suppose

one confounds all three-factor interactions once with blocks and all four-factor
interactions three times with blocks; this would be a balanced confounding arrangement.

The idea is that if interactions of a specified degree are of equal impor-

tance, equal intrablock information should be obtained on each one.

Unbalanced

confounding would be used when unequal amounts of information are desired.

degree of confounding is related to the degree of importance of effects.

The

- 22 -

The same concept can be carried over to regression designs.

All

t~ends

of

a specified degree can be confounded equally, assuming that they are of equal
importance to the experimenter.
Raktoe et al. (1981) set up measures of alias optimality and alias balance.
If A is the aliasing matrix, then when the absolute value of the
minimal, this is denoted as alias optimal.
row and j'th column of A.

Let

~j

deteriTi~ant

of s is

be the element in tee h'th

TheG a measure of alias balance for a desigr is given

below:

When Q8 is a minimum, zero, the design is alias balanced.

variance Optimality
In statistical literature considerable emphasis is placed upon variance
optimality or precision of estimating treatment effects.
criteria have been established.

Many variance optimality

Some of these are (e.g., see Kiefer, 1959, and

Raktoe et al., 1981):

(i)

D-optimal or determinant optimal - minimum value of determinant of

inverse of the information matrix.

(ii)

A-optimal or average variance optimal - average variance of linear

contrasts of treatment effects is minimal.

(iii)

E-optimal or eigenvalue optimal - smallest eigenvalue is a maximum

over the class of designs possible.

(iv)

G-optimal or global optimal - largest variance for any treatment

effect is minimal.

- 23 -

The variance optimality principle is closely tied to the efficiency principle.

Use of either principle often leads to the same treatment designs.

Unbiasedness
In the absence of knowledge about the size and nature of the bias, the unbiasedness principle would indicate that unbiased treatment designs would be
utilized.

These designs yield unbiased estimates of all treatment effects.

Suppose that one has a full or complete treatment design with parametric vector

with an observation for every treatment, say

~'

and with a design matrix X.

Suppose the partitioning is as follows:

!:l
E(!) = E !:2

= X~ =

!:3

xll

x12

xl3

x21

x22

x23

x3l

x32

x33

~3

~
If one takes a fraction !:1 , say, then

1
E(!l)

= [Xll

Xl2

Xl3]

= Xlll

+ Xl22 + Xl33 .

If one wishes to estimate 1 , say, then

If either, or both,

~2

biased estimate of l

small.

or 3 are not identically equal to zero, the above is a

If 2 and 3 are small compared to

~l'

then the bias is

- 24 -

(XilX 11 )-~ilx12 and let 3 = 0, then A is called the aliasing matrix

o~ 2 with respect to 1 . I~ 3 I 0, and i~ A* = (Xi 1x 11 )-~ilx13 , then the
Let A =

aliasing matrices would be

[A

with respect to 1

A*]

~or

the vector

Raktoe et al. (1981) discuss

[::]
~our

situations regarding

whether or not 2 and/or 3 are zero.

To simplify the discussion in later sections, let 3 - 0.

Connectedness
As in experiment design, it is desirable that all non-zero

mable.

For the situation in the previous section, one needs to have a !l and a ! 2

that allow

estimation

0~

l and 2

must exist in order to estimate 1

~rom

be esti-

e~~ects

In the event that

When all estimable

a design, the design is said to be connected.

tical consulting and in reading

scienti~ic

13-2

= o,

e~~ects

'
)-l
then (x11x11
can be estimated

During the course

o~

statis-

literature, experiment and/or treatment

designs that are not connected are utilized.

This is a

waste~

use

o~

resources

and should be discontinued.

Mean Squared Error

Accuracy includes precision (variance) plus bias.

The mean squared error

principle states that the mean squared error (MSE), which is the variance plus
the square

measurement

o~

the bias, should be minimal in order to obtain the most accurate

o~

a parametric vector, say 1 , i.e.,

- 25 where cov(~ 1 ) means the covariance matrix for ~ 1 .

Raktoe et al. (1981) ~efine a

design to be alias optimal if the absolute value of the determinant of A is minimal

and alias balanced if the sum of squares of square roots of the sums of squares of
coefficients of the rows of A is zero.

They discuss some of the problems in

implementing the mean squared error criterion.

Further discussion also appears

in a recent paper by Welch (1983) on response surface designs.

Rotatability
Rotatability is the principle that states that all points equidistant from
the center of the region in regression designs have equal variance (see 3ox and
Hunter, 1957).

If levels of factors from the center for all factors are of

equal importance, this is a desirable criterion to use.

If they are not, then

some sort of ellipsoidal criterion should be used in the definition of rotatability.

Discussion
It seems odd that Sir Ronald A. Fisher did not relate the principles of
statistical design given in his design book (1935), viz. replication, randomization, and local control with the principles of estimation given in his statistical methods book (1925), viz. efficiency, sufficiency, and consistency.

It

would appear that principles of statistical design and principles of statistical

estimation should be considered together rather than as separate entities.

Should

the estimation principle of consistency be used as a statistical design principle?

Ferhaps not.

Should only mathematical and statistical considerations be used in

establishing a principle, or should applications of the procedure in actual

practice dictate the principles to be used?
that should be used in statistical design?

Are there nonstatistical principles

What is the relative importance of

the above discussed principles in the real world where these designs are used?

r.'

- co -

An overview of the principles of design and estimation indicates that there are

many unanswered questions and that the philosophical aspects need fUrther study.
Literature Cited
Box, G. E. P. and J. S. Hunter (1957). Multi-factor experimental designs for
exploring response surfaces. Annals of Mathematical Statistics 28, 195-241.
Bradley, R. A. and D. E. W. Schumann (1957). The comparison of sensitivities of
similar experiments: Applications. Biometrics 13, 496-510.
Dar, S. N. (1962). On the comparison of the sensitivities of experiments.
of the Royal Statistical Society ~ 24, 447-453.

Journal

Dar, S. N. (1964). Comparison of sensitivities of dependent experiments.

Biometrics 20, 209-212.
Eccleston, J. A. and A. Hedayat (1974). On the theory of co~-~ected designs:
Characterization and optimality. Annals of Statistics 2, 1238-1255.
Eccleston, J. A. and K. Russell (1975). Connecte~~ess and orthogonality in multifactor designs. Biometrika 62, 341-345.

Federer,

w.

T. (1973).

Statistics and Society.

Marcel Dekker, Inc., New York.

Federer, W. T. and L. N. Balaam (1972). Bibliography on Experiment and Treatment

Design Pre 1968. Published for the International Statistical Institute by
Oliver and Boyd, Edinburgh.
Fisher, R. A. (1925). Statistical Methods for Research Workers (first of several
editions). Oliver and Boyd, Edinburgh-.Fisher, R. A. (1935). The Design of Experiments (first of several editions).
Oliver and Boyd, Edinburgh.
Graybill, F. and D. L. Weeks (1959). Combining inter-block and intra-block information in balanced incomplete blocks. Annals of Mathematical Statistics 30,

799-805.
Hedayat, A. and W. T. Federer (1974). Pairwise and variance balanced incomplete
block designs. Annals of the L~stitute of Statistical Mathematics 26, 331-

338.
Khare, M. and W. T. Federer (1981). A simple construction procedure for resolvable
incomplete block designs for any number of treatments. Biometrical Journal

23(2), 121-132.

Kiefer, J. (1959). Optimum experimental designs.

Society ~ 21, 272-304.

Journal of the Royal Statistical

- 27 -

..

Lashof, T. w., J, Mandel, and V. Worthington (1956). Use of the sensitivity criterion for the comparison of the Bekk and Sheffield smoothness testers .
Tappi 39, 532-54 3.
Mandel, J. and R. D. Stiehler (1954). Sensitivity- A criterion for the comparison
of methods of the test. Journal of Research, Na.tio:r:al Bureau of Standards 53,
155-159.
Patterson, H. D. and E. R. Williams (1976).
block designs. Biometrika 63, 83-92.

A new class of resolvable incomplete

Preece, D. A. (1977). Orthogonality and designs:

Utilitas Mathematica 12, 201-223.
Preece, D. A. (1982). Balance and designs:
Utilitas Mathematica 21C, 85-186.

A terminological muddle.

Another

ter~nological

tangle.

Raghavarao, D. and w. T. Federer (1975). On connecteiness in two-way elimination

of heterogeneity designs. Annals of Statistics 3, 730-735.
Raktoe, B. L., A. Hedayat, and W. T. Federer (1981).
Wiley and Sons, Inc., New York.

Factorial Designs.

John

Schumann, D. E. w. and R. A. Bradley (1957). A comparison of the sensitivities

of similar experiments: Theory. Annals of Matt-ematical Statistics 28, 902920
Searle, S. R. (1971).

Linear Models.

M. and w. T. Federer (l979).

Biometrika 66, 115-123.

Shafi~

John Wiley and Sons, Inc., New York.

Generalized N-a_ry balanced block designs.

Vartak, M. N. (1963 ). Connectedness of Kronecker product designs.

India Statistical Association 1, 2l5-218.
Vithayasai, C. and D. S. Robson (1974).
Statistics 3(11), lo4l-l052.

A blocking strategy.

Journal of

Communications in

Welch, W. J. (1983). A mean squared error criterion for the design of experiments.
Biometrika 70, 205-2l3.
Yates, F. (l933).
experiments.

The principles of orthogonality and confounding in replicated

Journal of Agricultural Science XXIII, part I, 108-145.

Yates, F. (1936). Complex experiments (with discussion).

the Royal Statistical Society 2, 181-247.

Supplement, Journal of

Yates, F. (1937). The design and analysis of factorial experiments. Technical

Communication No. ~ rmperial Bureau of Soil Science, Harpenden, England.