Paradigm Development in the Social Sciences: A Proposed Research Strategy

Author(s): Kenneth D. MacKenzie and Robert House

Source: The Academy of Management Review, Vol. 3, No. 1 (Jan., 1978), pp. 7-23
Published by: Academy of Management
Stable URL: http://www.jstor.org/stable/257572 .
Accessed: 16/06/2011 06:33

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .
http://www.jstor.org/action/showPublisher?publisherCode=aom. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact [email protected].

Academy of Management is collaborating with JSTOR to digitize, preserve and extend access to The Academy
of Management Review.

http://www.jstor.org
Paradigm Development in the Social Sciences:
A Proposed Research Strategy

KENNETH D. MACKENZIE
University of Kansas
ROBERT HOUSE
University of Toronto

A paradigm development strategy of research consists of: fitting data

into a theoretical framework with general laws to explain the data; de-
ducing hypotheses from the general laws; and subjecting these hy-
potheses to empirical test. Deductive-nomological reasoning, crucial
experiments, and typology of experimental control strategies are pre-
sented. Methods of strong inference are discussed, followed by a de-
scription of paradigm development research. Examples illustrate the
strategy of paradigm research.

The purpose of this article is to suggest a social sciences. Such a research strategy is
paradigm development strategy for the develop- needed in the social sciences because of its effi-
ment of cumulative growth of knowledge in the ciency in generating knowledge.
Kuhn (19) argues that science is a series of
peaceful periods interrupted by intellectually in-
Kenneth D. Mackenzie (Ph.D. - University of California, tense revolutions. During the peaceful inter-
Berkeley) is Edmund P. learned Distinguished Professor, ludes scientists are guided by a paradigm - a set
School of Business, University of Kansas, and President, Or-
ganizational Systems, Inc., lawrence, Kansas.
of theories, standards, methods, and beliefs
which are accepted by most scientists in a field.
Robert House (ph.D. - Ohio State University) is Shell Profes- Masterman points out that Kuhn "with that
sor of Organization Behavior, Faculty of Management Studies,
quasi-poetic style of his, ... uses 'paradigm' in
University of Toronto, Toronto, Ontario, Canada.
not less than twenty-one different senses in his
Received 10/21/75; Revi5ed 9/17/76; Accepted 12/3/76; (1962 book)" (25, p. 61). Paradigms help organize
Revi5ed 7/22/77. the processes of science. They provide direction
7
8 Paradigm Development in the Social Sciences: A Proposed Research Strategy

for its development and help sort out facts in subsumption under general laws. Discovery of
terms of their relevance. I n the absence of a para- such general laws and development of deduc-
digm, all facts are more or less relevant and this tive-law governed (nomological) theories are ma-
gives the appearance of randomness to those jor characteristics of paradigm research. Tradi-
gathering the facts. The cumulation of knowl- tionally deductive systems are viewed in terms of
edge requires an organizing framework upon three parts:
which the facts and ideas are organized.
1. The phenomenon to be accounted for,
Byrne (2) presented the results of an exten-
called the explanandum phenomenon,
sive program of research based on his interpreta-
2. The statements describing the phenom-
tion of the term paradigm. By paradigm he means
enon,called the explanandum sentences,
a specific body of research, accepted by a group
3. The statements specifying the explana-
of scientists and consisti ng of specific proce-
tory information, called the explanans
dures, measuring devices, empirical laws, and a
sentences.
specific set of theoretical superstructures (2).
Some of Byrne's research illustrates our ap- The set of explanans sentences forms the explan-
proach. ans.
Most social sciences are still in the pre-para- There are two subsets of the explanans: (a)
digm or paradigm development stage, and they the set of general laws, L1, L2, ... , Lp and (b) the
are likely to remain in this stage indefinitely un- special conditions, C1, C2, ... , Ck, that are as-
less more persons begin to think carefully about sertions about particular facts. The laws are sub-
their paradigm development strategy. Elms (11) divided into two types of principles, called inter-
points to the crisis of confidence in current para- nal principles and bridge principles (16). Bridge
digms in social psychology. pri nciples "indicate how the processes envisaged
This article supplements earlier work by by the theory are related to empirical phenom-
House (18) and Mackenzie (21,22) and attempts ena with which we are already acquainted, and
to articulate and reconcile their hunches about which the theory may then explain, predict, or
the nature of paradigm development. Paradigm retrodict" (16, pp. 72-73). They permit a theory to
research consists, in part, of integrating data into have explanatory power and permit it to be
a common theoretical framework, inducing gen- tested empirically. The special conditions, C1,
eral laws to explain the data, deducing hypoth- C2, ... , Ck, usually mean the set of assumptions
eses from the general laws, and subjecting these under which a theory is expected to be valid.
hypotheses to empirical test. These conditions are usually in the form of ante-
Our suggested paradigm development strat- cedent conditions for an experiment.
egy is the strong inference approach. This strate- A deductive nomological explanation, then,
gy is based upon: (a) deductive-nomological rea- weaves the internal and bridge principles (L1, L2,
soning, (b) crucial ex peri me nt, (c) experimenter ... , Lr) and the special conditions or assumptions
control strategies, and (d) strong inference. These (C1, C2, ... , Ck) by deductive logic in order to
four bases are discussed in sufficient detail to em- produce the explanandum sentence. Or, follow-
ploy them in the paradigm development strategy. ing Hempel and Oppenheim (17), a deductive
nomological explanation can be described by:
Deductive Nomological Reasoning L 1, L 2, ... , L r
Explanans sentence
Theoretical frameworks are integrated net- C1, C2, ... , Ck
works of law-like statements. The Greek origin of
the word "law" is "nomos". Deductive nomolog- E Explanandum sentence
ical explanations are explanations by deductive If E has been observed and the explanans sen-
Academy of Management Review - January 1978 9

tences are provided afterwards, we have an ex- tively valid test of an hypothesis. On the other
planation for E. But if E is deduced from the ex- hand, a rejection using modus tollens may not be
planans sentences, then we have a prediction. a genuine rejection because of auxiliary hypoth-
Clearly, the requirements for explanation are eses. In fact, it is possible to use a modus tollens
those appropriate for prediction. Deductive argument to reject a rejection. But the result is
nomological explanations are not "cut and not an acceptance of H. It is simply not a rejec-
dried". There are often unstated, tacitly assumed tion.
premises that do not enter the logical deduction One conclusion is that one can conditionally
explicitly. These auxiliary hypotheses are often reach deductively valid rejections of a hypothe-
hard to define. Some rest upon deeper laws that sis but not achieve a deductively valid affirmation
are yet unknown, some lie in hidden assump- of a hypothesis. Non-rejections and further de-
tions, and some are a part of the system of meas- velopment lead to a synthesis and growth of the
urement. For example, a phenomenon may be deductive-nomological network. We can draw a
unobserved because of imprecist: instruments or "tree" of these relationsh ips, called a strong in-
it may be obscured by hidden assumptions about ference tree. Mackenzie (22) provides extensive
the causality. Some assumptions are, for the pres- descriptions of such a tree for his research on
ent state of knowledge, counterfactual. The group structures.
problems of working with counterfactual condi-
tionals in economic theory are discussed by Cyert Crucial Experiments
and Grunberg in their critique of the views of A second part of our strong inference strat-
Milton Friedman (12) in Cyert and March (8). egy is conducting crucial experiments whenever
The possible existence of auxiliary assump- possible. Crucial experiments - defined in terms
tions creates difficulties when judging outcomes of decisions made by the experimenter in gather-
of experiments. Suppose that there is a hypoth- ing and transforming information - represent
esis, H. We argue that if H is true, then there is a an ideal method for developing a strong infer-
test implication, I, that is true. The purpose of the ence tree, but it is not always possible to run one.
experiment is to establish whether or not I is, in Arranging for a crucial experiment is not al-
fact, true. A modus tol/ens form of argument for ways easy because of the many decisions that
rejection of H has this general line of reasoning: must be made concerning bridging principles
(a) if H is true, then so is I. But (b) if the evidence and special conditions. Many auxiliary hypothe-
shows that I is not true, we conclude that (c) H is ses are built into the analysis because of the na-
not true given the explanans sentences and logic ture of any experiment. The description here fol-
of deduction. The argument for acceptance is less lows that of Mackenzie and Barron (23).
conclusive. If we argue that (a) if H is true, then Abstractly, letun refer to the universe of
so is I, obtaining evidence that (b) I is true does potential observations. Then let H2E } denote the
not imply, logically, (c) that H is true. If we con- class of potential observations under the chosen
clude that H is true on the basis that I is true, we experiment. nE is a special case ofW}. The selec-
are committing the fallacy of affirming the con- tion ofnE from{n}is a decision. Or, let us suppose
sequent, which is deductively invalid. A conclu- that there is some transformation T 0 such that nE
sion can be false even if its premises are true. Pre- = Ton. If we plan to use nE to obtain data, a
sumably, too, for a given H one can derive a long decision must be reached for what aspects of the
series of test implications 11, 12, ... , I n that are events in nE the experimenter wishes to record.
all true. This still does not imply that logically H is let{O}denote the class of recorded observations.
true. Thus, the process of obtaining many enum- The particular set chosen, say 0, is the result of a
erations of a hypothesis cannot lead to a deduc- decision. Abstractly, there is some transforma-
10 Paradigm Development in the Social Sciences: A Proposed Research Strategy

tion T ! that describes how we record D out of 12E, types of errors: working on the wrong problem;
or D ='I12E. For example, in a communications using the wrong code; picking the wrong model;
network experiment, D can be the set of all and combining errors such as selecting the
transmitted messages among the subjects. This D wrong experimental situation, recording the
is a subset because there are many other ways of wrong information, and using correct measures,
extracting or recording information from 12E. model, and statistical test. One is not limited to
These recorded observations, D, then are coded type I and II errors of elementary statistics. The
to convert them into symbols. Let DR be the class dependence of any results on the prior decisions
of all recorded observations arising from D. We is clear and unavoidable in principle (32).
can say that there is a transformation t 2, called To arrive at a decision about a result, a long
coding, that converts 0 into raw data, DR. Or, sequence of decisions has been made. The
D R = '20. Coded data then gets transformed result, E, does not stand by itself, but is
again into various measures. Let{M} denote the understood within the context of these decisions
possible classes of measures. The particular set of (the set TO, T " T 2, T 3, T 4, and T 5). These
measures, M, is used as a transformation of the transformations depend upon the theory and the
raw data, DR. Or, M = T3D R. Normally, there is purposes of the experimenter. The{Tj},j =0, ...
a class of models that transforms the measures , 5, include the explanans and unstated auxiliary
into data that is ready for hypothesis testing, hypotheses. To the extent that the{T j}, j =0, ... ,
D HT. The particular models define how one 5, are defined explicitly, the experimenter has
transforms M into data that are ready for a hy- reduced the number of auxiliary assumptions.
pothesis test called D HT. Thus, we can write But it is probably impossible not to include some
D HT ='4M. Finally, there are many methods and auxiliary hypotheses, no matter what precautions
techniques for making a hypothesis test. These are taken. For example, the choice of TO and T 1
hypotheses testing methods convert, D HT, into may be dictated by experimental conventions in
results, {E}. These results are seen to be a result a field. These conventions involve auxiliary
of the method of hypothesis testing, denoted by assumptions. Similarly, the choice of code,
the transformation T 5. I n short, a result E = measures, model, and hypothesis testing involves
15 D HT· other auxiliary assumptions. For example, most
Four points are immediately obvious from measures assume constructs, and these are not
the foregoing description of the decision always fully understood. Probably the
processes of the experiment. Firstly, the most transformation involving the most deliberate
important is that the results, {f} ,are dependent listing of assumptions is T 5' the method of
upon the set of transformations, including the hypothesis testing.
last one, T 5. Secondly, the"[ 5 are not limited to Let us suppose that there are two rival the-
statistical hypothesis testing. In particular, there ories T and T' for which theITj } , j =0, 1, ... , 5, are
is a procedure, called strong inference, for which consistent, and that under T we expect E and un-
some advocate that the existence of a single der T' we expect E' using this set of{T jJ. Then,a
counter example, despite mountains of crucial experiment is the event:
"confirming" results, is sufficient to reject a {T j},j = 0, 1, ... ,5, (T, E), (T', E')
theory. Thirdly, mistakes can be made at any where E and E' are mutually exclusive. This defi-
stage of the analysis. Messick (26) discusses nition is similar to the one given earlier except
problems with meaning and values in that it provides more detail about how one ob-
measurement. Focusing only on possible tains E and E' from the transformations.
misinterpretations of results after employing a This reformulation of a crucial experiment
"[ 5 is a mistake because it overlooks other major makes it clear that the results are dependent
sources of problems. Fourthly, there are many upon a series of prior decisions. These prior deci-
Academy of Management Review - January 1978 11

sions can be viewed as "controls". The choice of in applied research will often conduct such ex-
the code, T2, can control the results. According- periments.
ly, in paradigm development, we are concerned These three dimensions can be employed to
with all of theTj. We consider statistical hypothe- describe a typology of strategies for a scientist.
ses testing procedures as the end point of a long- For the sake of discussion, assume that each di-
er sequence of decisions. While they are impor- mension can be characterized in terms of low,
tant, they are only a part of the process of para- medium, and high degree of control. Any strat-
digm development. egy then is conceived of as a 3-tuple described
by the amount of control on each of the three di-
Experimenter Control Strategies mensions of control. It helps to visualize this in
terms of locations in a cube whose three dimen-
For the purpose of discussion, let us consider
sions correspond to the degree to which the
three types of "controls". The first is the degree
three types of control are applied, as seen in Fig-
to which the scientist manipulates antecedent
ure 1.
conditions. This type of control concentrates on
manipulating the environment to match the set Generally, if we think of the cube of Figure 1
of assertions or special conditions, Cl, C2, ... , in terms of sections rather than subcubes, it is
Ck. The second type of control manipulates the easier to understand the range of possible behav-
degree to which measurement requirements are iors for each research strategy. For example, lab-
imposed on a situation. The imposition of meas- oratory experiments refer to the section where
urement requirements refers to the types of {Tj} there is a high degree of control over antecedent
that are imposed. For example, an extremely high conditions. In this diagram, there are nine possi-
degree of imposition of measurement require- ble classifications of type of experiment, depend-
ments produces data that is already at 0 HT. Ob- ing upon the degree to which measurement re-
server observations, where only T1 is controlled, quirements are imposed and the degree to which
are an example of a low degree of impostion of the explanans is structured. Crucial experiments
measurement requirements. The greater the de- occur where there is a high degree of control on
gree of imposition of measurement require- all three dimensions. Many experiments in social
ments, the smaller the discretion of the analyst to psychology involve low controls on the structure
apply ex post facto transformation in order to ob- of the explanans and varying degrees of controls
tain results. on imposition of measuring units. Many experi-
A third type of control is the degree to which ments in decision making require a high degree
one has structured the explanans - that is, the of imposition of measurement requirements by
extent to which one can state the internal and limiting behavior to the choice of a strategy. Oth-
bridging principles and the special conditions in ers who attempt to gather protocols may have
such a way that one can deductively derive the only a low degree of imposition of measurement
explanandum. A high degree of structuring of requirements.
the explanans does not imply that the theory is Naturalistic observations, a strategy pre-
correct. Rather it refers to the extent to which ferred by many because it reduces the chance of
one can explicate the reasoning upon which one artificially "tyi ng" dimensions of behavior in a
bases the prediction. Many scientists conduct laboratory where they would not be tied in a
"experiments" in which there is almost no struc- more natural setting and vice versa, involve por-
ture to the explanans. Such exercises often em- tions of the front face of the cube. Willems and
ploy elaborate controls on antecedent condi- Rauch (39) describe naturalistic observations in
tions and imposition of units. The person who terms of a low degree of structure to the explan-
follows the procedures of experimental design ans and the exclusion of joint combinations of
12 Paradigm Development in the Social Sciences: A Proposed Research Strategy

Degree to which antecedent

conditions are controlled

Figure 1. A Representation of Typologies of Research Strategies by

Degree and Type of Control

medium and high degree of control on the other a business setting, that may not be present in a
two dimensions. Naturalistic observations are of- laboratory, include: the presence of external in-
ten explorations into a phenomenon. Gadlin and cidents, changes that are a function of time and
Ingle (13) discuss the use and misuse of the lab- not manipulation, the possibility that testing can
oratory experiment in psychology, particularly affect responses, the possibility that the subjects
when method seems to dominate and precede or the observer's perceptions can change. In ad-
content. dition, many variables may be changing together.
Field experiments can be characterized by Controlling selection of subjects often is difficult,
(a) medium degrees of control of the degree to and subjects can drop out of the experiments.
which antecedent conditions are controlled, and There are also the problems of (a) ethical treat-
(b) medium to high degree to which measure- ment of subjects, especially when not all are vol-
ment units are imposed. Field experiments, unteers, (b) high cost of obtaining information,
which can range widely over the degree to which and (c) unique situation being studied that may
the explanans is structured, are a compromise not be capable of replication. We think of field
between naturalistic observations and laboratory experiments as investigations rather than as ex-
experiments. This compromise often is necessary ploration or crucial experiments. But Lippitt (20)
because of real constraints on using pretest and used strong inference in her study of committee
post-test control groups. Confounding factors in formation by university administrators. Clearly,
Academy of Management Review - January 1978 13

a large range of scientists' behavior can be classi- A good theory is one that holds together long
fied as field experiments. enough to get you to a better theory (14, p.
21).
The three way classification of research strat-
egies by the degree and type of control places a These assumptions are further illustrated in
perspective on the range of scientific research modern physicists' challenge of the unrestricted
strategies. Not only do different strategies re- range of application of classical mechanics, or in
quire different degrees of different types of con- biochemists' dispute over what is to count as liv-
trol, but it often is argued that this diversity is a ing (7). A guiding principle of the strong infer-
good thing because it provides multi-method ence research strategy is the Popperian position
replication and thus minimizes reliance on any that pursuit of knowledge is more efficient when
single set of auxiliary hypotheses or bridging scientists deliberately set out to disprove theo-
rules. Examining this three way classification of ries, or seek rejections, than when they attempt
research strategies places our proposed paradigm to assemble proof for theories. Such attempts to
development strategy into a wider perspective disprove theories not only reflect the tradition
than just a laboratory setting. The choice of con- of scientific skepticism but also focus the inves-
trols, and hence the particular research strategy, tigator on a different set of variables than do at-
will vary with the theory and purpose of the tempts to prove theories. Dubin (10) makes the
study. The stages in the paradigm development distinction between research directed at proving
will likewise depend on and affect the choice of a theory and research directed at improving a
research strategies as the paradigm develops.
1 The authors have some disagreements about the use of
Strong Inference strong inference. House, whose main interests have been in
field experiments, where there are lower degrees of experi-
A fourth characteristic of paradigm research
menter control, argues that statistical significance is usually
is the frequent use of what Platt (33) calls strong sufficient to make a decision to accept tentatively or to reject
inference - an ideal approach for the develop- empirical hypothesis. Mackenzie, whose main interests have
ment of cumulative knowledge through theory been in laboratory experiments, where there are higher de-
building. We argue that, while the prerequisites grees of experimenter control, argues that statistical signifi-
cance is insufficient as a criterion for the decision of whether
for conducting strong inference research do not
to tentatively accept or reject an empirical hypothesis. Mac-
obtain in all social science disciplines, a strong in- kenzie seeks to conduct crucial experiments that are capable
ference "attitude" in the paradigm development of producing counter examples to the theory. He recognizes
stage is clearly appropriate. For this reason we the existence of measurement errors and uses two criteria for
shall describe our interpretation of the strong in- counting a datum as a counter example: (al after allowing for
measurement error, a result is a counter example if it still lies
ference approach.1
within the set of results that disconfirm the stated result; and
Strong inference research is based on the (bl it is possible to replicate the conditions producing the
assumptions that: (a) all theories, no matter how counter example to provide more of them.
good at explaining a set of phenomena, are ul- While both subscribe to the paradigm development strat-
timately incorrect and consequently will under- egy outlined in this article, they disagree on the need to struc-
ture the explanans. Mackenzie is an inveterate tree builder
go modification over time (19); and (b) the fate
and likes to do research whenever he can proceed as if there
of the better theories is to become explanations is a high degree of structuring the explanans. House is more
that hold for some phenomena in some limited willing to proceed with field experiments where there is a
conditions. These assumptions are expressed by lower degree of structure on the explanans. The goals are
two quotations: identical but they have tactical disagreements on details for
implementing paradigm development strategies. These differ-
A theory which cannot be mortally endan- ences are evident to any reader who studies House's work on
gered cannot be alive (33, from personal com- leadership and Mackenzie's work on a theory of group struc-
munication by W.A.H. Ruston). tures.
14 Paradigm Development in the Social Sciences: A Proposed Research Strategy

theory. He points out that research directed at evidence also discovered the appropriate rule in
improving a theory causes the researcher to fo- fewer trials than subjects who sought confirming
cus: evidence.
Wason's experiment simulates in the most
particularly on the deviant cases and noncon-
forming results that do not accord with the pure form a scientific problem in which the vari-
predictions made by the hypotheses of his the- ables are unknown and in which evidence has to
oretical model. It is from the evidence con- be used to refute a support hypothesis. Those
tained in the deviant cases that the insights subjects indicating a disposition to refute rather
come on the basis of which the extant theory than to vindicate assertions and to tolerate the
is improved by reformulating it to generate
predictions that will incompass all the data, in- distress of negative results made more rapid
cluding those initially considered deviant (10, progress. This disposition to refute and an orien-
p. 234, emphasis ours). tation toward focusing on deviant cases evidence
what we call the "strong inference attitude". The
Dubin argues that none of the advantages of
strong inference approach is a natural result of
the theory-proving orientation are lost by the
such an attitude.
theory-improving orientation. Opportunities are
enhanced for theory modification and, there-
The Strong Inference Approach
fore, for growth and improvement of the theory.
Churchman (5, 6) calls for construction of a The strong inference approach consists of
counterperspective to one's view in order to the following steps. A substantial amount of time
flush out into the open one's basic assumptions and effort is invested in conceptualizing a theory
and tacit restrictions. His ideas are useful in gen- of a given set of phenomena in such a way that
erating instances of deviant cases which are so the theory resolves previous anomalies and/or
important to theory construction. explains important issues in the area of inquiry.
If it is assumed that all theories will undergo The emphasis is on development of well defined
modification over time, successful attempts at concepts and deductive-nomological theory. In
disproving them are contributions to knowledge, the absence of an initial theory that appears to
in that such disproof facilitates identification of the investigator to be adequate to guide re-
the specific aspects of the theory that are errone- search, initial work frequently begins with ex-
ous, or the boundaries within which the theory ploratory investigation, literature review, or
holds. Thus, attempts to disprove a theory con- sheer speculation. After development of a tenta-
stitute one of the most efficient processes where- tive theory, strong inference research usually in-
by knowledge can be discovered. This process is volves development of research technology, i.e.,
illustrated in an experiment on the effects of experimental designs, measurement instruments,
seeking confirming evidence versus seeking dis- and criteria for the rejection of hypotheses. Here
confirming evidence as a means of discovering the emphasis is on development of bridging
valid hypotheses. principles and transformations required to link
Wason (38) instructed subjects to infer a empirical observations to the concepts expressed
mathematical rule from a set of numbers. He in the explanandum sentence. Research is then
then observed the process by which the sub- directed at identifying counter examples to the
jects arrived at their conclusions. A greater pro- theory's most fundamental predictions. Identifi-
portion of subjects who systematically eliminated cation of clear counter examples requires the
alternative rules by seeking disconfirming evi- theorist to re-examine the theory as it was orig-
dence (counter examples) discovered the appro- inally stated. Such identification requires scien-
priate rule than subjects who sought confirming tists to specify the transformation to be used and
evidence. Subjects who sought disconfirming the criteria of acceptance to be applied. If coun-
Academy of Management Review - January 1978 15

ter examples are found, either a fundamentally conducted, indicating the branch, or branches,
new theory is advanced, or the initial theory is of the logical tree to be followed, contingent on
reformulated in such a way that the reformula- the outcome of the initial round of studies. The
tion accounts for the counter examples. The potential outcomes of each study are compared.
more fundamental the counter examples, the The studies with potential to eliminate the largest
more fundamental the revision of the theory. number of predictions and answer the greatest
Failure to identify clear counter examples of number of questions are conducted first.
the obvious and fundamental predictions of the If the result of the first series of studies is not
theory establishes tentative credibility of the the- inconsistent with this theory, one has more spe-
ory in the eyes of scientists, but does not end the cific information about the theory. As a result,
process of theory building. Failures to reject the the theory can be made less crude or more fine.
theory are not considered confirmation. Rather, If the experimenter feels that she or he can still
the theory is considered "not yet" invalidated reject the theory using a different research de-
and is considered tentatively to be useful. A "not sign, it should be tried. If the results are incon-
no, yet" result stimulates new attempts to pro- sistent with the theory, the theory must be
duce a rejection. After attempts to reject the bas- changed. After the change, the researcher re-
ic propositions of the theory have failed, more peats the steps to construct the first branches out
subtle and less fundamental predictions of the of this new base. These branches on the same
theory may be stated and attempts may be made level are called outward growths. Outward
at identifying clear counter examples of these growths represent: (a) the domain of the theory,
predictions. By focusing on counter examples, i.e., the populations and range of variables for
the researcher concentrates energy on improve- which the theory makes accurate predictions;
ment of the theory. Extensions, improvements, and (b) the ki nds of predictions that can be de-
and refinements of the theory are made by iden- rived validly from the propositions of the theory
tification of such counter examples. Clear coun- at that level. Those branches leading from other
ter examples to the theory's predictions not only branches, but which apply under a more re-
call those predictions into question, but also stricted range, are upward growths. The process
may call into question the more fundamental of strong inference starts at the base and pro-
propositions. Thus, each counter example re- ceeds upwards and outwards by asking questions
quires the researcher to reconsider the credibil- and making predictions that lead to more specifi-
ity placed on even the earlier successful predic- city about the theory or its mechanisms. Rejec-
tions of the theory, since prior successful predic- tions are used to prune the strong inference tree
tions may be sample specific, or they may be by cutting back "dead" branches. In this process,
based on faulty methodology. the branches in question may have been allowed
The strong inference process can be linked to grow from a conceptually incorrect branch. A
to the construction of a logical tree whose trunk branch may be theoretically irrelevant, and such
represents the basic propositions of the theory. growth must be weeded out from the proper
The objective is to grow the tree by careful nur- branches in the main tree; thus, pruning and
turing and pruning. A major emphasis is placed weeding often result in a reorganization of the
on conceptualization of a cumulating series of tree.
interrelated studies and on careful formulation A failure to reject adds a twig or a leaf. Every
of competing hypotheses and the criteria of clear rejection raises questions, and consistent rejec-
counter examples. Predictions are derived from tions allow one to chop off branches. The goal
the propositions, and studies are designed in an of strong inference is to produce a theory. This is
attempt to disprove them. A series of contin- done best by pruning and weeding because it is
gency statements is made before the studies are wasteful to expend resources following clearly
16 Paradigm Development in the Social Sciences: A Proposed Research Strategy

false leads, no matter how attractive they may scientists are separated geographically. The
seem. strong inference strategy is considered by platt
The planning phase of a strong inference (33) as the most efficient means of developi ng
strategy provides the intellectual framework for reliable theory.
theory building. The strong inference plan is a
Paradigm Development Research
tentative deductive nomological explanation.
But this explanation need not be static or rigid. The major values of the strong inference re-
New information may suggest rerouting to a low- search strategy are its efficiency and the fact that
er level branching point, or it may suggest entire- it permits research to be cumulative. For a re-
ly new branches. For example, suppose that an search strategy to be truly a strong inference
early study indicated taking Branch A as an alter- strategy, a guiding paradigm must exist.
native explanation and excluding Branch B. Sup- Few areas of inquiry in the social sciences
pose also that a later study indicated findings have developed to the paradigm stage of devel-
contrary to the first one. These suggest a prob- opment. What then should be the strategy of the
lem with the methodology of one of the two researcher who is interested in theory building?
studies or the existence of new variables not The starting point is development of the para-
specified at the outset. Such a sequence would digm. Specifically, one begins with a general idea
require backtracking to the initial decision point or a rather crude conceptual framework, a set of
where Branches A and B parted, specifying new explanandum sentences, with full recognition of
laws or bridge principles to account for the find- the need for its refinement.
ings of the second study, and recycling the effort. This is followed by research concerned with
Strong inference planning is likely to cut refinement of the transformations necessary to
down the number of investigations required, by test the theory. These transformations concern
providing a framework for arraying all relevant, development of instruments or a research tech-
available information and evaluating the poten- nology with which to develop and test initial the-
tial payoff of alternatives that can be eliminated ories and specification of clear criteria for the ac-
by the next study. While failures to disprove the ceptance or rejection of predictions derived
theory are comforting, less is likely to be learned from their theories. Thus, we argue for develop-
from them than from successful disproofs be- ment of a conceptual framework, operational
cause they do not help identify where the the- definitions, measurement instruments, experi-
ory made erroneous predictions. Identification of mental apparatuses and rejection criteria as the
erroneous predictions helps identify invalid starting point of paradigm development re-
propositions within the theory, inadequate con- search. Once these are developed, we have what
ceptualization of the theoretical variables and Byrne (2) defines as a paradigm, and the neces-
their relationships, and boundary conditions be- sary prerequisite knowledge and technology for
yond which the theory does not make valid pre- strong inference research.
dictions. If multiple, mutually exclusive, hypoth-
Question Raising and Hypotheses Testing
eses are advanced, failure to disprove one hy-
pothesis results in elimination of one or more of One method frequently employed in both
the other hypotheses. Disproof of one hypothe- the paradigm building and strong inference
sis results in further support for counter hypoth- stages of a research program is that of question-
eses. Thus, attempts to disprove these hypotheses raising rather than hypothesis testing. While the
constitute advances in the state of knowledge, search for counter examples is usually cast in the
regardless of the outcome of the study. Strong in- form of tests of hypotheses, the intent of proving
ference planning also can be used to coordinate or disproving the hypotheses is actually second-
the efforts of a set of research efforts, even if the ary to the intent to generate descriptive data with
Academy of Management Review - January 1978 17

which to build an improved theory. Dubin states Successive failures to reject a theory, regard-
that: less of how satisfying to the researcher, seldom
provoke additional questions or insights. Thus,
a moment's reflection on some of the land-
the researcher who seeks to improve the theory
mark contributions to the social sciences
makes clear that among them will be found es- after failing to reject one or more hypotheses
sentially descriptive studies (studies not must rely on information or insights from sources
guided by hypotheses but rather by research other than his or her investigation. Such sources
questions). Such a list would include Myrdal's may include second thoughts developed on re-
An American Dilemma, Thomas and Znaniec-
flection, opinions of colleagues, or data pub-
ki's The Polish Peasant in Europe and America,
Stouffer and his associates' The American Sol- lished by other investigators. Because such
dier, Sherif's work on the autokinetic effect, sources soon run dry, their usefulness is both un-
Asch's research on social pressures in small predictable and limited. In contrast, failures to
groups, and Bales' categorization of interac- confirm the theory almost assuredly raise new
tions within small groups (10, p. 227).
questions, thereby opening avenues for subse-
In a recent paper concerning human infor- quent investigation.
mation processing, Simon (36) raises objections The strategy of seeking to refute hypotheses
to the traditional form of experimental design is essentially a strategy of raising and eliminating
and argues for experimental designs for the pur- questions. Its advantage over attempts to prove a
pose of parameter estimating rather than hypoth- theory through hypothesis testing can be illus-
esis testing. He points out that the more orthodox trated with a hypothetical problem. Suppose two
approach to experimentation-testing for signif- people are engaged in a game. The object for
icance of differences between experimental person A is to identify a specific number between
treatments-yields "a one-bit" message that 1 and 100 that person B has in mind, in the mini-
there is a relationship between the independent mum time possible. If person A asks "ls the num-
and the dependent variable. ber x?", he or she is testing a hypothesis. If the
In strong inference research, attempts are hypothesis is incorrect, only one number has
made to design studies that go beyond hypothe- been eliminated. If A continues by testing hy-
sis testing to shed additional light about the phe- potheses about other numbers, as many as 99
nomena under question. As Simon (36) observes, questions may have to be asked before identify-
one such question concerns parameter estima- ing the correct number. Each question yields
tion. Other research questions that are frequent- only a one-bit information message. But suppose
ly raised concern the effects of interacting varia- A asks "Is the number above 50?" and then pro-
bles, the range over which predictions can be ex- ceeds to narrow the range of possible numbers
tended, and the magnitude of forces operating by halving the remaining possible numbers with
on theoretica I variables. If the researcher sets each additional question. By this process of elim-
out with the assumption that the theory is incor- ination the correct number always can be identi-
rect, the question most likely to yield the most in- fied within seven questions. Each question dis-
formation is one which, when answered, will in- confirms a range of alternatives and thus adds
dicate exactly where the theory is weak. Such a much more information than conventional hy-
question necessarily eliminates one or more pothesis testing because its answer directs fur-
propositions of the theory or some part of the ther search. Although conventional hypothesis
domain of the theory. Since attempts to disprove testing tells one if a theory is or is not supported,
the theory focus the researcher's attention on it does not direct further search upon failure to
avenues of elimination, such attempts are more confirm. In short, one of the main advantages of
likely to yield data with which to reformulate the strong inference is that it increases the amount
theory, if it is indeed rejected. of information per step in the research process.
18 Paradigm Deve/opment in the Socia/Sciences: A Proposed Research Strategy

Two Examples of Paradigm pose of specifically testing the theory in ques-

Development Research tion. Observation of role and behavior in an hier-
archical organization led Miner to conclude that
In addition to our own work, several social
managerial roles in hierarchical organizations de-
science research programs include many of the
mand their incumbents to have a generally posi-
characteristics of paradigm development re-
tive orientation towards the use of power as a
search. These include studies by: McClelland
means of attaining success.
and Atkinson and their associates (15) concern-
To test the theory a special sentence com-
ing the achievement motive; Maier (24) concern-
pletion test was constructed. The test was as-
ing social processes involved in problem solving;
sumed to be a measure of the degree to which
Rokeach (35) concerning dogmatism; Deutsch
the respondent desires to compete for and exer-
and his associates (9) concerning conflict and
cise power over people. Several hypotheses
conflict resolution; and Hovland and his associ-
were postu lated: (a) that the measu re of power
ates concerning persuasive communication. Two
motivation, the Miner Sentence Completion
such paradigm development programs will be
Scale (MSCS), should yield both concurrent and
briefly described.
predictive validity when compared against such
A little known, but impressive series of stud-
success indexes as organization~1 level, promo-
ies was conducted by Miner (27). Based on his
tion rate, ratings of effectiveness in managerial
observation of the role requirements of man-
work, and rated potential for promotion; (b) that
agers in a large bureaucratic organization, plus
if motivational changes are induced by manage-
his prior training in psychology, he postulated
ment education, such changes will be enforced
that individuals whose motivations correspond to
by the role requirements and the reward system
the requirements of the roles they fill will be
of the organization and will thus be sustained;
more successful in those roles. He hypothesized
(c) that management education efforts should be
that there are "certain components (motives)
differentially effective depending on the person-
which contribute to what amounts to a common
ality characteristics of the participant in the ed-
variant ... , which operates across a great many
ucation effort; and (d) that different occupation-
managerial positions . . ." This constituted
al groupings tend to attract individuals having
Miner's explanandum sentence. An auxiliary hy-
different levels of interest in the exercise of pow-
pothesis was " ... are presumed to occur with
er and that these differences will be 'feflected in
relative high frequency, and across a consider-
the MSCS scores.
able range of positions and organizations (27, p. These hypotheses represent the test implica-
42)". The specific role requirements which were tions of the theory. Miner's research progressed
identified theoretically were presumed to be through a series of stages. He began with a test of
among those which occur with high frequency in the concurrent validity of the MSCS scale and
business firms organized in accordance with the tested the hypothesis that there would be a sig-
Scalar principle. Miner stated that: nificant association between the MSCS and vari-
It is entirely possible that the theory is applica- ous criteria of success: job grade level, perform-
ble to managerial, or administrative jobs in ed- ance rating, and potential rating. Having estab-
ucational, governmental, medical, military and
other types of organizations, but it was not de-
lished significant correlations between the scale
vised with these positions in mind. The initial and the criteria, he raised the possibility that suc-
tests, at least, should be carried out within the cess may serve to precipitate motivational
particular segment of society for which it was changes and that these in turn contribute to the
originally constructed (27, p. 42). concurrent validity which had been found, or
Here we see Miner's first transformation of the conceivably some external factor might be re-
theory, selection of the test sample for the pur- sponsible for both the motivation and the suc-
Academy of Management Review - January 1978 19

cess. He called for evidence that those who pos- the scale. Miner reported several additional tests
sess the various motives measured tend subse- of the theory. He measured the degree to which
quently to perform particularly well on the vari- a course developed to increase attitudes meas-
ous organizational criteria. Miner stated that: ured by the MSCS scale, did indeed effect a
change in responses to the scale in the predicted
certainly, if a positive association between mo- dimension as compared with the control group
tivational variables and behavior in the mana-
gerial role cannot be demonstrated under of subjects who did not participate in the educa-
these conditions, the theory is deficient (27, p. tional program. He then tested whether the
60). change produced by the experimental treatment
was retained long enough for managerial per-
Here he stated a criterion for refutation of the formance to be affected materially.
theory. Following these validation studies, which
A test of the predictive validity of the MSCS can be considered tests of the adequacy of the
revealed significant correlations between the theory to predict within the specified domain of
MSCS and subsequent criteria of managerial suc- the theory, several additional studies were con-
cess. Miner then tested the possibility that in a ducted to answer descriptive research questions.
somewhat different type of business organiza- These concern the effects of motivational educa-
tion the relationships might not appear. Shifting tion among college students and the retention of
from the initial sample of managers in an organi- change resulting from such education, the de-
zatior producing heavy equipment, he tested gree to which female students respond as well,
the concurrent validity of his scale with a popu- the relationship between change and the climate
lation of male and female managers in a large de- in the organization, a search for organizational
partment store. Results revealed that three sub- and personality predictors of change, and a test
scales of the MSCS held up across samples. of the hypothesis that occupations tend to attract
Miner then raised a series of questions con- individuals having different levels of power of
cerning the MSCS scale. motivation. Since the original publication of the
Does the MSCS really measure the motivation- research described above (27), additional studies
al variables that it is devised to measure? Is it have been conducted to identify the boundary
really motivation of this particular kind, moti- conditions of the theory (28, 29, 30, 34). These
vation that is congruent with managerial role results generally confirm the proposition that
requirements that produces the success? Or is
it possible that the MSCS yields the resu Its it
power motivations significantly predict success
does due to the influence of component var- in hierarchical situations and fail to predict suc-
iables which are not those the test was con- cess in nonhierarchical situations in experimen-
structed to measure. Or perhaps some external tal laboratories, consulting firms and non-bu-
factor, highly correlated with the MSCS ac- reaucratic schools.
counts for the findings which have been ob-
tained (27, p. 90). Another example of paradigm development
research is reported by Byrne (1, 2). His specific
A variety of analyses were conducted in a num- research focused on the effect of the expression
ber of different samples. Correlations between of attitude statement on subsequent affective re-
the MSCS and age, intelligence, concept mastery, sponses directed toward the source of such state-
attitudes toward various stimuli, grades in under- ments. He began with the tentative empirical
graduate school, and education achievement generalization that people with similar attitudes
were computed. are attracted to one another. At that time a sub-
From a strong inference viewpoint, these stantial amount of common sense observation
tests can be considered attempts to rule out com- and empirical data supported this generalization.
peting hypotheses concerning interpretation of Byrne states two assumptions:
20 Paradigm Development in the Social Sciences: A Proposed Research Strategy

A meaningful and positive increase in knowl- gations could be meaningfully combined even
edge is possible only if identical or equivalent though attitude scales of various length had
operations serve as connecting links across ex- been used (1, p. 53-54).
periments (2, p. 47).
He then proposed a theoretical formulation to
... in the initial stages of paradigm research, a
preliminary theory is useful and perhaps es- account for the relationship between similarity
sential in guiding and and attraction, stating that:
efforts (2, p. 49).
the empirical attraction is interpreted as a
special case of reward and punishment. Spe-
The theoretical underpinning of the research cifically, it is proposed that attitude statements
conducted by Byrne and his associates was that: are affect arousing; the motive involved is the
learned drive to be logical and to interpret
attraction toward X is a function of the relative correctly one's stimulus world (2, p. 70).
number of rewards and punishments associ-
ated with X (2, p. 49). Byrne then derived implications from the at-
traction literature rather than from the classical
The initial study concerned development of conditioning or other learning paradigms. The
an instrument with which to vary the degree of relationship between attraction and stimulus in-
similarity between an experimental subject and a formation was conceptualized as a law of interac-
hypothetical other subject. Upon finding that tion: attraction toward X is a positive linear func-
with the use of a newly developed instrument he tion of the sum of the weighted positive rein-
was able to replicate prior studies, Byrne raised a forcements (number times magnitude) associated
question about the possible interpretation of the with X divided by the total number of weighted
results. He stated that the lack of attraction be- positive and negative reinforcements associated
tween dissimilar others in a subject could be due with X. Byrne and Rhamey (4) conducted an ex-
to perceptions of dissimilarity, or due to the per- periment to determine the effect of differential
ception of deviancy of the other from cultural weights of the magnitude of positive and nega-
norms. To deal with this problem Byrne con- tive reinforcers. A subseq uent study (3) showed
structed a new scale by which bogus strangers that the Byrne-Rhamey coefficients were appli-
could be made to express attitudes similar or dis- cable in subsequent experiments. Further studies
similar to those of the subjects, without at the showed that si milar attitude statements by others
same time expressing either conforming or de- may be used as positive reinforcers and dissimilar
viant beliefs. He then found that attraction is a attitudes as negative reinforcers. Based on a re-
fu nction of si mi larity of attitudes, but raised a view of prior studies, the utility of the law of at-
further question regarding this function. Was it traction was shown with respect to studies of top-
the number of similar attitudes, the number of ic importance, personal evaluations, physical at-
dissimilar attitudes, or the ratio of similar to dis- tractiveness, emotional disturbance, and race.
similar attitudes? Here we see an example of Several hypotheses were derived from the
question raising rather than hypothesis testing model, asserting the interrelationship of the
and research directed at parameter estimating three elements of the model: independent vari-
similar to that advocated by Simon (36). ables (any stimulus with reinforcement proper-
Subsequent experimentation revealed that ties), intervening variabl~ (any implicit affective
attraction was clearly a function of the ratio of response), and dependent variable (any evalua-
dissimilar to similar attitudes. Byrne then stated tive response such as attraction). Thus Byrne's
that: initial model of interpersonal attraction was
with the stimulus identified as proportion of broadened to explain attraction not only to other
similar attitudes, data from a variety of investi- persons, but also to things and experiences.
Academy of Management Review - January 1978 21

Byrne then reported a series of studies con- and the complex problems of these organizations
ducted by himself and his associates, designed to has been a continuing source of stimulus to im-
test several of the major hypotheses derived from prove the theories. One result is alteration of the
the theory. Several hypotheses were supported, program for developing the theory. It is neces-
thus leading to increased confidence in the the- sary to move back and forth between the two
ory. In addition, Byrne (2) reviewed a wide range settings. Problems from the practical world and
of practical applications of the theory and sug- from pure theory interact and supplement each
gested several avenues for its extension. These other.
include using the theory to explain such phe- Paradigm development research necessarily
nomena as how attitudes are formed (or valence implies commitment to a program of research.
of an object is developed) with respect to tasks, Theories seldom are built from a single study or
job candidates, spending funds for facilities for even from a small number of studies. We contend
the disabled, aptitudes and performance of oth- that almost all major contributions to the social
ers, educational activities, and persons accused sciences have resulted from a series of studies
of crimes. which have culminated over time in a reformula-
Byrne's research reflects clearly the iterative tion of earlier concepts and research findings.
process of formulating theory, testing predic- In all these research programs the initial the-
tions, raising questions, pruning branches and ories were reformulated through subsequent re-
supplanting them. search. Improvements in the theories resulted
not only from conventional hypothesis testing
Discussion and Conclusions but also from inquiry into questions raised for the
purpose of adding descriptive information with
Two essential characteristics qualify a re- which to refine and extend theoretical positions
search strategy as a paradigm development strat- that had received prior support.
egy: (a) commitment to theory building and (b) In one sense, the programmatic approach
commitment to a program of research. A commit- advocated here is counter to the "success ethic"
ment to theory building has characterized most whereby researchers act if they are "piece work-
major contributors to the social sciences. Theory ers" grinding out a quota of "successes" per unit
building efforts, to be effective, require a long time, with bonuses for above average productiv-
term commitment, considerable ingenuity, and ity. In the approach recommended here, one at-
flexibility. Theory building which is restricted to tempts to produce rejections instead of "success-
a single setting can become sterile, ingrained, es". A virtue is made out of success in producing
and increasingly arcane. While crucial experi- rejections. The error of working on the wrong
ments are important in the social sciences, much problem (examining twigs on a dead branch) is
is to be learned from applying the knowledge considered more serious than accepting a false
gained in the laboratory to outside settings. Ap- hypothesis (calling a twig "alive" when it is not),
plication is not the sole justification for basic re- or rejecting a true hypothesis (calling a twig
search, but it is certainly helpful in generating "dead" when it is alive). Because the error of
counter examples to a theory. working on the wrong problem (a type 3 error) is
One of the authors recently formed a corpo- rarely mentioned, even in conversations, and be-
ration to apply and develop theories of group cause conventional statistics does not treat this
structures which originated in a laboratory. This important error, seeking rejections is likely to be
extension forced him to consider other related efficacious in generating better theory. It would
issues and to develop a technology to bridge the seem that "piece work" based upon producing
gap from laboratory groups to real world organ- rejections is potentially better paid than "piece
izations. The confrontation between his theories work" based upon producing successes, assum-
22 Paradigm Development in the Social Sciences: A Proposed Research Strategy

ing of course that the goal is to develop theories, many universities convert to teaching institutions
not merely a larger number of publications. and seek contract research whose goals are dic-
Our proposed research strategy for develop- tated by the funding agency rather than by the
ing paradigms, which requires commitment to requirements of the research problem. But seri-
build theory and commitment to a program of ous scholars always have had to overcome and
research, is a recipe for serious scholars. Recent rise above their institutions. They have had to be
trends in evaluation of faculty, which reward determined, flexible, and ingenious. Above all
short term exploitation - in the stop and go they have needed the dual commitment to de-
trends in research funding, the lack of resources veloping theory and to establishing a program of
to engage in paradigm development, and the de- research. Our strategy is not an easy one and is
clining fraction of nontenured research posi- not intended for the faint-hearted. We argue that
tions - create institutional barriers against our it is, over the long term, more efficient and more
strategy. This pressure probably will intensify as exciting.

REFERENCES
1. Byrne, D. "Attitudes and Attractions," in l. Berkowitz 15. Heckhausen, H. The Anatomy of Achievement Motiva-
(Ed.), Advances in Experimental Social Psychology (New tion (New York: Academic Press,l966).
York: Academic Press, 1969), Vol. 4, pp. 36-90. 16. Hempel, C. G. Philosophy of Natural Science (Englewood
2. Byrne, D. The AUraction Paradigm (New York: Academic Cliffs, N.J.: Prentice-Hall, 1966).
Press, 1971). 17. Hempel, C. G., and P. Oppenheim. "Studies in the Logic
3. Byrne, D., and W. Griffitt. "A Development Investigation of Explanation," Philosophy of Science, Vol. 15 (1948),
of the Law of Attraction," Journal of Personality and So- 135-178.
cial Psychology, Vol. 4 (1966)' 699-702. 18. House, R. J. "Strong Inference in the Social Sciences: An
4. Byrne, D., and R. Rhamey. "Magnitude of Positive and Explanation and an Illustration from Leadership Re-
Negative Reinforcement as a Determinate of Attraction," search," Studies in Managerial Process and Organiza-
Journal of Personality and Social Psychology, Vol. 4, tional Behavior (J. H. Turner, 1961), (Glenview, III.: Scott,
(1966), 699-702. Foresman 1972), 403-410.
5. Churchman, C. W. The Systems Approach (New York: 19. Kuhn, T. S. The Structure of Scientific Revolutions (Chi-
Delacorte Press, 1968). cago, III.: University of Chicago Press, 1970).
6. Churchman, C. W. The Design of Inquiring Systems (New 20. lippitt, M. E. Development of a Theory of Committee
York: Basic Books, 1971). Formation (Ph.D. dissertation, The University of Kansas,
7. Cunningham, F. Objectivity on Social Science (Toronto, 1975).
Ontario, Canada: University of Toronto Press, 1973). 21. Mackenzie, K. D. A Theory of Group Structures, Vol. I:
8. Cyert, R. M., and J. G. March. A Behavioral Theory of the Basic Theory (New York: Gordon and Breach Science
Firm (Englewood Cliffs, N.J.: Prentice-Hall, 1963), Pubs., 1976).
9. Deutsch, M. The Resolution of Conflict: Constructive 22. Mackenzie, K. D. A Theory of Group Structures, Vol. II:
and Destructive Processes (New Haven, Conn.: Yale Empirical Tests. (New York: Gordon and Breach Science
University Press, 1973). Pubs., 1976).
10. Dubin, R. Theory Building (New York: The Free Press, 23. Mackenzie, K. D., and F. H. Barron. "Analysis of a'Deci-
1969). sion Making Investigation," Management Science, Vol.
11. Elms, A. C. "The Crisis of Confidence in Social Psychol- 17, No.4 (1970), B-226-B-241.
ogy," American Psychologist, Vol. 30, No. 10 (1975)' 967- 24. Maier, N. R. F. Problem Solving and Creativity in Individ-
976. uals and Groups (Belmont, Calif.: Brooks/Cole, 1970).
12. Friedman, M. Essays in Positive Economics (Chicago, III.: 25. Masterman, M. "The Nature of a Paradigm," in I. Lakatos
University of Chicago Press, 1953). and A. Musgrave (Eds.), Criticisms and the Growth of
13. Gadlin, H., and G. Ingle. "Through the On-Way Mirror: Knowledge (Cambridge: Cambridge University Press,
The limits of Experimental Self-Reflection," American 1970).
Psychologist. Vol. 30, No. 10 (1975), 1003-1009. 26. Messick, S. "The Standard Problem: Meaning and Val-
14. Hebb, D. O. "Hebb on Hocus-Pocus: A Conversation ues in Measurement and Evaluation," American Psychol-
with Elizabeth Hall," Psychology Today, Vol. 6 (1969). ogist, Vol. 30, No. 10 (1975), 955-966.
Academy of Management Review - January 1978 23

27. Miner, J. B. Studies in Managerial Education (New York: 34. Rizzo, J. R., J. B. Miner, D. N. Harlow, and J. W. Hill.
Springer, 1965). "Role Motivation Theory of Managerial Effectiveness in
28. Miner, J. B. "The Managerial Motivation of School Ad- a Simulated Organization of Varying Degrees of Struc-
ministrators," University Council for Educational Admin- ture," Journal of Applied Psychology, Vol. 59 (1974), 31.
istration Quarterly (Winter, 1968)' 57-72. 35. Rokeach, M. The Open and Closed Mind. (New York:
29. Miner, J. B. "Personality Tests as Predictors of Consulting Basic Books, 196OJ.
Success," Personnel Psychology, Vol. 24 (1971), 191-204. 36. Simon, H. A. "How Big is a Chunck?" Science, Vol. 183
30. Miner, J. B. "Motivation to Manage Among Women: (1974),482-488.
Studies of Business Managers and Educational Adminis- 37. Towne, C. H. "Quantum Electronics and Surprise in De-
trators," Journal of Vocational Behavior, Vol. 5 (1974), velopment of Technology: The Problem of Research
197 -208. Planning," in D. Byrne, The Attraction Paradigm (New
31. Miner, J. B. "Motivation to Manage Among Women: York: Academic Press, 1971).
Studies of College Students," Journal of Vocational Be- 38. Wason, P. C. "On the Failure to Eliminate Hypotheses in
havior, Vol. 5 (1974),241-251. a Conceptual Task," The Quarterly Journal of Experi-
32. Pinkham, G. N. "Some Doubts About Scientific Data," mental Psychology (1960), 129-139.
Philosophy of Science, Vol. 42 (1975), 260-269. 39. Willems, E. P., and H. L. Raush. (Eds.). Naturalistic View-
33. Platt, J. R. "Strong Inference," Science, Vol. 146, No. 3642 points in Psychological Research (New York: Holt, Rine-
(1964)' 347-353. hart and Winston, 1969).