Coping with Uncertainty: A Naturalistic Decision-Making Analysis

ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES Vol. 69, No. 2, February, pp. 149–163, 1997 ARTICLE NO. OB972679 Coping with Uncertainty: A Naturalistic Decision-Making Analysis RAANAN LIPSHITZ AND ORNA STRAUSS University of Haifa, Haifa, Israel This paper is concerned with three questions: How do decision makers conceptualize uncertainty? How do decision makers cope with uncertainty? Are there systematic relationships between different conceptualizations of uncertainty and different methods of coping? To answer these questions we analyzed 102 selfreports of decision-making under uncertainty with an inclusive method of classifying conceptualizations of uncertainty and coping mechanisms developed from the decision-making literature. The results showed that decision makers distinguished among three types of uncertainty: inadequate understanding, incomplete information, and undifferentiated alternatives. To these they applied five strategies of coping: reducing uncertainty, assumption-based reasoning, weighing pros and cons of competing alternatives, suppressing uncertainty, and forestalling. Inadequate understanding was primarily managed by reduction, incomplete information was primarily managed by assumptionbased reasoning, and conflict among alternatives was primarily managed by weighing pros and cons. Based on these results and findings from previous studies of naturalistic decision-making we hypothesized a R.A.W.F.S. (Reduction, Assumption-based reasoning, Weighing pros and cons, Suppression, and Hedging) heuristic, which describes the strategies that decision makers apply to different types of uncertainty in naturalistic settings. q 1997 Academic Press Uncertainty and related concepts such as risk and ambiguity are prominent in the literature on decisionmaking (Kahneman, Slovic, & Tversky, 1982; March & Olsen, 1976). This prominence is well deserved. Ubiquitous in realistic settings, uncertainty constitutes a major obstacle to effective decision-making (Brunsson, 1985; Corbin, 1980; McCaskey, 1986; Orasanu & Connolly, 1993; Thompson, 1967). This study is an empirical investigation of three questions: (1) How do decision We thank Lee Beach, Ilan Fisher, Victor Friedman, Gary Klein, Henry Montgomery and Ramzi Suleiman for many helpful comments. Address correspondence and reprint requests to Dr. Raanan Lipshitz, Department of Psychology, University of Haifa, Haifa, Israel 31905. E-mail: [email protected]. makers (e.g., managers and military officers) conceptualize the uncertainty which they encounter in their work? (2) How do decision makers cope with their uncertainty? (3) Are there systematic relationships between different conceptualizations of uncertainty and different methods of coping? These questions are motivated by indications that decision makers and students of decision-making conceptualize uncertainty in different ways, thus reducing the propensity (or ability) of the former to use models and methods developed by the latter (Humphreys & Berkeley, 1985; Huber, Wider, & Huber, 1996; Lopes, 1987; March & Shapira, 1987). HOW DO DECISION MAKERS CONCEPTUALIZE UNCERTAINTY? Despite the centrality of uncertainty in the decisionmaking literature, only few studies (referenced above) addressed (indirectly) this question. These studies showed that people conceptualize uncertainty differently from the conceptualization of risk in Decision Theory. The literature, however, offers numerous conceptualizations of uncertainty: Argote (1982, p. 420) notes that “there are almost as many definitions of uncertainty as there are treatments of the subject”; Yates and Stone (1992, p. 1) suggest that “if we were to read 10 different articles or books about risk, we should not be surprised to see risk described in 10 different ways,” and Downey and Slocum (1975, p. 562, quoted in Milliken, 1987, p. 134), suggest that “the term ‘uncertainty’ is so commonly used that ‘it is all too easy to assume that one knows what he or she is talking about’ when using the term.” We now survey the answers that can be gleaned from the literature to the question at hand. Table 1 presents a sample of definitions of uncertainty and related terms that we culled from the decision-making literature between 1960 and 1990. The Table clearly illustrates the conceptual proliferation noted by Argote, Yates and Stone, and Downey and Slocum. Some of the distinctions in the Table reflect the manifold nature of uncertainty. For example, risk (as defined by item 6), ambiguity (item 9) and equivocality (item 11) are obviously distinct phenomena. Other 149 0749-5978/97 $25.00 Copyright q 1997 by Academic Press All rights of reproduction in any form reserved. 150 LIPSHITZ AND STRAUSS TABLE 1 Conceptualizations of Uncertainty Authors Term I. Behavioral decision theory 1. Anderson et al. (1981) Uncertainty 2. Anderson et al. (1981) Uncertainty 3. Humphreys & Berkeley (1985) Uncertainty 4. 5. 6. 7. Risk Risk Risk Risk Anderson et al. (1981) Anderson et al. (1981) MacCrimmon & Wehrung (1986) Arrow (1965) 8. Hogarth (1987) Ambiguity Conceptualization A situation in which one has no knowledge about which of several states of nature has occurred or will occur. A situation in which one knows only the probability of which of several possible states of nature has occurred or will occur. The inability to assert with certainty one or more of the following: (a) act-event sequences; (b) event-event sequences; (c) value of consequences; (d) appropriate decision process; (e) future preferences and actions; (f) one’s ability to affect future events. Same as (1) Same as (2) Exposure to the chance of loss in a choice situation. A positive function of the variance of the probability distribution of expected positive and negative outcomes. Lacking precise knowledge about the likelihood of events (second-order probability). II. Organization decision theory 10. Galbraith (1973) 11. March & Olsen (1976) Task uncertainty Task uncertainty Ambiguity 12. Terreberry (1968) 13. Weick (1979) 14. March & Simon (1958) Turbulence Equivocality Conflict 9. Thompson (1967) The inability to act deterministically owing to lack of cause-effect understanding; environmental dependencies and internal interdependencies The difference in the amount of information required to perform a task and the amount of information already possessed by the organization. Opaqueness in organizations owing to inconsistent or ill-defined goals; obscure causal relations in the environment unclear history, and interpersonal differences in focus of attention. Unpredictable changes in system-environment relations. The multiplicity of meanings which can be imposed on a situation. Absence of arguments which clearly favor a particular course of action. distinctions are probably idiosyncratic, as, for example, defining identical terms differently (e.g., risk, items 4–7) or defining different terms identically (e.g., risk, as in item 1 and uncertainty, as in item 4). Explaining “how decision makers conceptualize uncertainty” in a way that makes sense theoretically thus requires some clarification of the conceptual confusion illustrated in Table 1. To this end we developed three related conceptual propositions that allowed us to study our research questions empirically: PROPOSITION 1. Uncertainty in the context of action is a sense of doubt that blocks or delays action. Conceptualizing the uncertainty that impacts decision-making as a sense of doubt that blocks or delays action has three essential features: (1) it is subjective (different individuals may experience different doubts in identical situations), (2) it is inclusive (no particular form of doubt, e.g., ignorance of future outcomes, is specified), and (3) it conceptualizes uncertainty in terms of its effects on action (hesitancy, indecisiveness, and procrastination). Conceptualizing uncertainty as a subjective experience has a long tradition (Duncan, 1972; Smithson, 1989.) Though less conventional, conceptualizing it in terms of its effects on action is consistent with the English language (e.g., “hesitate” is defined as “to hold back in doubt or indecision” and “to pause,” Barnhart & Stein, 1964), and with several writers including Dewey (1933), who suggested that problemsolving is triggered by a sense of doubt that stops routine action; Peirce (Skagestad, 1981. p. 31), who defined inquiry as the struggle to end doubt and attain belief which, in turn, is “that upon which man is prepared to act”; Goldman (1986), who suggested that uncertainty is a state of indecision that results from continued competition among alternatives; and Yates and Stone (1992), who suggested that risk makes prospective options less appealing. Finally, March (1981) formulated the relationship between uncertainty and delayed action most explicitly in drawing a contrast between two generic decision-making models, consequential action and obligatory action. Consequential action (i.e., concurrent choice models such as EU, SEU, and Prospect Theory, Hogarth, 1987), requires the decision maker to answer the following questions: “What are my alternatives?” “What are my values?” and “What are the consequences of my alternatives for my values?” Having resolved these doubts, the decision maker can proceed to choose and implement the alternative that has the best consequences. In contrast, Obligatory action (i.e., sequential option evaluation models such as Klein’s, 1993, COPING WITH UNCERTAINTY RPD model), requires decision makers to answer a different set of questions: “What kind of a situation is this?” “What kind of a person am I?” and “What is appropriate for me in a situation like this?” Having resolved these doubts the decision maker can proceed to implement the action that is appropriate for his/her situation. Coping with uncertainty thus lies at the heart of making a decision. PROPOSITION 2. The uncertainty with which decision makers must cope depends on the decision-making model which they employ. Proposition 2 is basically a corollary of Proposition 1. Granted that uncertainty is a sense of doubt that blocks or delays action, enacting models that have different informational requirements (Grandori, 1984) will be blocked or delayed by different doubts. For example, consider March’s distinction between Consequential and Obligatory actions. Because Consequential action requires knowledge of one’s alternatives, their outcomes, and the relative attractiveness of these outcomes, employing this model is contingent on coping with doubts regarding these issues. In contrast, employing Obligatory action is contingent on coping with doubts regarding a different set of issues, as this model requires knowledge of one’s situation and role requirements in this situation. Using Proposition 2, we can partly clarify the conceptual confusion illustrated in Table 1 by dividing its elements into three clusters of basically similar conceptualizations. The first cluster (items 1–8 and item 14) consists of conceptualizations that specify blocks to Consequential action. (Note that item 14 qualifies as a form of uncertainty according to Proposition 1, though not in Behavioral Decision Theory.) The second cluster (items 9–13) consists of conceptualizations that specify blocks to Obligatory action. Finally, although Weick’s conceptualization (item 13) is a variant of Obligatory action, the emphasis of its underlying model on meaning-making (Weick, 1979, 1995), deserves a separate cluster. PROPOSITION 3. Different types of uncertainty can be classified according to their issue (i.e., what the decision maker is uncertain about) and source (i.e., what causes this uncertainty.) Three basic issues are outcomes, situation, and alternatives. Three basic sources are incomplete information, inadequate understanding, and undifferentiated alternatives. As noted above, several researchers found that decision makers conceptualize uncertainty differently from one particular conceptualization of uncertainty, i.e., risk (as defined in elements 4–7 of Table 1). To study how decision makers conceptualize uncertainty more inclusively, Proposition 3 presents a generic, two-dimensional classification scheme of types of doubts that block or delay action. To allow comparison with the 151 various elements of Table 1, the three categories of the first dimension (what are the doubts that block or delay action) are based on the three clusters identified in Proposition 2—the scheme assumes that decision makers are blocked or delayed by doubts about alternatives, the outcomes of these alternatives, and the nature of the situation. To check the generality of these issues note that both consequential action and obligatory action are blocked or delayed by doubts about alternatives, that consequential action is also blocked or delayed by doubts about outcomes, and that obligatory action is also blocked or delayed by doubts about the situation. Similar classifications of uncertainty can be found in Berkeley and Humphreys (1982) and Milliken (1987). A threefold rationale underlies the second dimension of the classification scheme. (a) Incomplete information is possibly the most frequently cited source of uncertainty (Conrath, 1967; Galbraith, 1973; Smithson, 1989). (b) Decision makers are sometimes unable to act not because they lack information but because they are overwhelmed by the abundance of conflicting meanings that it conveys (Weick, 1987, 1995). (c) Finally, incomplete information and inadequate understanding do not exhaust the sources of uncertainty because decision makers may be blocked from taking action if they have perfectly understood, but undifferentiated (i.e., equally attractive or unattractive), alternatives. We refer to this source of uncertainty as conflict, following March and Simon (1958) who pointed out the debilitating effect of such conflict on action (Table 1, item 14). More recently Svenson (1992) proposed that decision-making is essentially the process of differentiating one alternative sufficiently from its competitors to convince the decision maker that it is worth implementing. Similar to the first dimension of the classification scheme, the categories of its second dimension can also be found in Table 1. Conceptualizations 1–8 in the Table attribute uncertainty to imperfect knowledge, conceptualizations 9–13 attribute it to inadequate understanding and conceptualization 14 attributes uncertainty to the similarity among alternatives. In conclusion, the literature on decision-making offers many hypotheses and scant empirical evidence regarding how decision makers conceptualize uncertainty. The advantage of conceptualizing uncertainty inclusively as a sense of doubt that blocks or delays action is that it relates uncertainty directly to action and encompasses all these conceptualizations. Its disadvantage is that it replaces one unspecified concept— uncertainty—by another—doubt. To remedy this drawback we suggest specifying doubts that block or delay action in terms of three broad categories of issues and 152 LIPSHITZ AND STRAUSS sources which are derived from the literature. Proceeding this way we strike a middle ground between a purely bottom-up approach and purely top-down approaches, in an attempt to obtain results that are comparable to existing conceptualizations without imposing any one of them a priori. HOW DO DECISION MAKERS COPE WITH UNCERTAINTY? This question has received considerable attention by students of decision-making. Smithson (1989, p. 153) suggests that the prescription for coping with uncertainty in traditional and modern Western treatments of the subject is “First, reduce ignorance as much as possible by gaining full information and understanding . . . Secondly, attain as much control or predictability as possible by learning and responding appropriately to the environment . . . Finally, wherever ignorance is irreducible, treat uncertainty statistically,” Thompson (1967) suggested that organizations constrain the variability of their internal environments by instituting standard operating procedures and constrain the variability of external environments by incorporating critical elements into the organization (i.e., acquisition) or by negotiating long-term contractual arrangements. Similarly, Allaire and Firsirotu (1989) listed several “power responses” used by organizations to cope with environmental uncertainty including shaping and controlling external events, passing the risk on to others, and disciplining competition. Finally, the standard procedure for coping with uncertainty in formal and behavioral decision theories can be labeled the R.Q.P. heuristic: Reduce uncertainty by a thorough information search (Janis & Mann, 1977), Quantify the residue that cannot be reduced, and Plug the result into some formal scheme that incorporates uncertainty as a factor in the selection of a preferred course of action (Cohen, Schum, Freeling & Innis, 1985; Hogarth, 1987; Raiffa, 1968; Smithson, 1989.) The term “quantify and plug” should not be taken to imply mindless automaticity. Quite the contrary: expert application of the R.Q.P. heuristic requires considerable judgment, ingenuity and artistry in constructing an appropriate formal model of the decision problem, assessing decision makers’ uncertainties and interpreting the results of analysis (Brown, 1992; Humphreys & Berkeley, 1985). Thus, the R.Q.P. heuristic underlies a coherent, flexible and rigorous approach to studying and coping with uncertainty (Camerer & Weber, 1992; Dawes, 1989; Kahneman, Slovic & Tversky, 1982.) Notwithstanding the elegance of the R.Q.P. heuristic and its amenability to rigorous formal treatment it has several drawbacks as a guide for describing and prescribing for decision-making in naturalistic settings. To begin with, reducing uncertainty by collecting additional information is often problematic in the real world. On many occasions information is simply unavailable. On other occasions information is ambiguous or misleading to the point of being worthless (Feldman & March, 1981; Grandori, 1984; Wohlstetter, 1962.) Finally, collecting additional information does not help decision quality when environmental uncertainty is very high (Fredrickson & Mitchell, 1984). Quantification is possibly even more problematic than reduction from descriptive and prescriptive standpoints. Basically, the problem is that “there are many areas of both practical and theoretical inference in which nobody knows how to calculate a numerical probability value” (Meehl, 1978, p. 831.) More specifically, despite the sophistication of available methods for assessing subjective probabilities, the validity of these measurements is still open to question. Translations of verbal expressions of uncertainty into specific probabilities show large variations (Budescu & Wallsten, 1995); verbal, numerical and different numerical expressions of identical uncertainties are processed differently (Gigerenzer, 1991; Zimmer, 1983), and the use of quantitative estimates of uncertainty was shown to degrade the quality of decisions (Erev & Bornstein, 1993). The reluctance of managers to use quantified measures of uncertainty (March & Shapira, 1987), which handicaps the application of decision support systems that rely on quantification (Eden, 1988, Isenberg, 1985) should not, perhaps, be dismissed lightly. Assuming that decision makers do first try to reduce uncertainty by collecting additional information, the question then is what they do with uncertainty that cannot be reduced this way, assuming that they do not resort to quantification. Researchers in Behavioral Decision Theory have recently begun to explore this question. Shafir, Simonson and Tversky (1993) suggested that people make decisions under risk by constructing compelling qualitative arguments that justify their decisions. Similarly, Hogarth and Kunreuther (1995) suggested that people make decisions in ignorance (i.e., without information on the probabilities and utilities of potential outcomes) by following arguments that do not quantify risks. Integrating the treatments reviewed above, we distinguish among three basic strategies of coping with uncertainty: reducing uncertainty, acknowledging uncertainty, and suppressing uncertainty, each of which consists of more specific tactics of coping with uncertainty. COPING WITH UNCERTAINTY Reducing Uncertainty The obvious strategy of coping with uncertainty is to reduce it or remove it altogether. Tactics for reducing uncertainty include collecting additional information before making a decision (Dawes, 1988; Galbraith, 1973; Janis & Mann, 1977); or deferring decisions until additional information becomes available (Hirst & Schweitzer, 1990). When no additional information is available it is possible to reduce uncertainty by extrapolating from available information. One tactic of extrapolation is to use statistical methods to predict future events from information on present or past events (Allaire & Firsirotu, 1989; Bernstein & Silbert, 1984; Thompson, 1967; Wildavsky, 1988). Another tactic of extrapolation is assumption-based reasoning, filling gaps in firm knowledge by making assumptions that (1) go beyond (while being constrained by) what is more firmly known and (2) are subject to retraction when and if they conflict with new evidence or with lines of reasoning supported by other assumptions (Cohen, 1989). Using assumption-based reasoning, experienced decision makers can act quickly and efficiently within their domain of expertise with very little information (Lipshitz, & Ben Shaul, 1997). A tactic of reducing uncertainty that combines prediction and assumption-based reasoning is mental simulation (Klein & Crandall, 1995) or scenario building (Schoemaker, 1995), imagining possible future developments in a script-like fashion. Finally, uncertainty can be reduced by improving predictability through shortening time-horizons (preferring shortterm to long-term goals, and short-term feedback to longrange planning, Cyert & March, 1963), by selling risks to other parties (Hirst & Schweitzer, 1990), and by selecting one of the many possible interpretations of equivocal information (Weick, 1979). The tactics listed so far rely, one way or another, on information processing. An entirely different approach to reducing uncertainty is to control the sources of variability which reduce predictability. Thompson (1967) suggested that organizations constrain the variability of their internal environments by instituting standard operating procedures, and constrain the variability of external environments by incorporating critical elements into the organization (i.e., acquisition) or by negotiating long-term contractual arrangements. Allaire and Firsirotu (1989) refer to control tactics as “power responses” and list several such tactics, including shaping and controlling external events, passing the risk on to others, and disciplining competition. Acknowledging Uncertainty This strategy can be applied when reducing uncertainty is either unfeasible or too costly. Decision makers 153 can acknowledge uncertainty in two ways: by taking it into account in selecting a course of action and by preparing to avoid or confront potential risks. The Rational Choice model presents a sophisticated tactic of accounting for uncertainty by including it as a factor in concurrent option evaluation. According to this model, the attractiveness of an option is a compensatory function of the attractiveness of its outcomes, the probability that they will materialize, and the cost of collecting information to reduce uncertainty concerning the first two factors (Raiffa, 1968). Less sophisticated tactics of incorporating uncertainty as a factor in concurrent option evaluation are the minimax regret and maxmin strategies (Coombs, Dawes & Tversky, 1971). A still less sophisticated tactic is avoiding ambiguity by preferring options with clear outcome probabilities (Curley, Yates & Abrams, 1986). Thompson (1967) and Allaire and Firsirotu (1989) proposed several tactics of acknowledging uncertainty by preparing to avoid or confront potential risks. According to Thompson (1967), organizations cope with uncertainty this way by buffering (e.g., building slack to shield production from unstable supply of required input) and by rationing (rearranging priorities following unanticipated contingencies). Hirst and Schweitzer (1990) suggest that electric utility companies can confront potential risks by planning very carefully for all reasonable contingencies, and by adopting a flexible strategy that allows for easy and inexpensive change. Allaire and Firsirotu refer to this tactic of coping with uncertainty as “the structural response” which includes, among others, broadening the product and market scope of the firm, building a capability to respond quickly to market change, and (similar to Thompson) by building and hoarding strategic resources. Finally, Cohen, Tolcott, and McIntyre (1987) found that fighter pilots combine assumption-based reasoning with preparing for potential risks: If their sensors confirm the presence of the threat but are inconclusive regarding its classification, pilots adopt a worst case assumption, [under] . . . the rationale . . . that the failure to classify the threat is itself evidence that the threat is a new system, and therefore likely to be more dangerous than previously known threats. On the other hand, if available information is inadequate to confirm the existence of a threat, pilots tend to make a best case assumption until more definite information is obtained [under] . . . the rationale . . . that actions taken to avoid the threat would almost certainly expose the aircraft to risk from other known threats. Nevertheless, even in this situation, limited action, e.g., speeding up the plan, might be taken to reduce risk from the unconfirmed threat. (Cohen et al., 1987, p. 52)1 1 Note that under certain conditions reduction tactics (other than collecting additional information) can be classified as acknowledgment tactics. For example, assumption-based reasoning and mental 154 LIPSHITZ AND STRAUSS Suppressing Uncertainty This strategy includes tactics of denial (ignoring or distorting undesirable information) and tactics of rationalization (coping with uncertainty symbolically by going through the motions of reducing uncertainty or acknowledging it). A wealth of anecdotal and systematic descriptions of suppression tactics can be found in the literature. The quintessential suppression tactic of ignoring undesirable information was described as the Pollyanna effect, the acquisition of an (often false) sense of security through the belief that “this [unfortunate outcome] cannot happen to me” (Matlin & Stang, 1978). Janis & Mann (1977) and Montgomery (1988) described various suppression tactics that decision makers use to align their preferences and beliefs with their decisions. Finally, Devons (1961) reported a fascinating example of coping with uncertainty symbolically. Puzzled by the fact that the UK National Coal Board published voluminous annual statistical analyses supporting its plans, even though these analyses “hardly served to reduce uncertainty and risk to any great extent,” Devons conjectured that The Coal Board . . . dare not admit, either to themselves or to the public, complete ignorance of rational criteria on which to base such decisions . . . [T]he role of economic statistics in our society and the functions which magic and divination play in primitive society . . . [is to make] possible for decisions on important issues to be taken where there is apparently no alternative rational basis of decision. (Devons, 1961, pp. 125–135) Consistent with Devons’ conjecture, Bolman and Deal (1991) proposed that people use symbols to resolve confusion, increase predictability and provide direction when the latter cannot be achieved by rational analysis, and Feldman and March (1981), showed that much information processing in organizations serves symbolic functions. Brunsson (1985), Lipshitz (1995), and Montgomery (1988) also argued that seemingly irrational tactics of suppressing uncertainty help decision makers avoid paralysis when they cannot cope with their uncertainty by reduction or acknowledgment tactics. ARE THERE SYSTEMATIC RELATIONSHIPS BETWEEN DIFFERENT CONCEPTUALIZATIONS OF UNCERTAINTY AND DIFFERENT METHODS OF COPING? Distinguishing between different types of uncertainty and different strategies and tactics of coping is important because decision makers encountering different uncertainties respond differently (Milliken, simulation can be considered as tactics of acknowledgment when the decision maker is cognizant that he is filling gaps in factual knowledge, and when his reasoning is done critically and expertly. (Lesgold, et al., 1988; Voss; Greene, Post & Penner, 1983) 1987) or are advised to respond differently (Grandori, 1984). The existence of such contingent coping is a recurrent theme in the literature: Cyert and March (1963) proposed that “[organizations] achieve a reasonably manageable decision situation by avoiding planning where plans depend on prediction of uncertain future events and by emphasizing planning where the plans can be made self confirming through some control device” (Cyert & March, 1963, p. 119). Grandori specified which of five decision-making methods should be selected given the magnitude of uncertainty caused by lack of information and conflicting values. Thompson specified which of four decision-making methods should be selected given the magnitude of uncertainty caused by disagreements about what outcomes are desirable and the methods that will effect them. Finally, Argote (1982) and Fredrickson and Mitchell (1984) showed that comprehensive decision-making is suitable for stable, simple environments, while prompt and flexible response is suitable for complex, dynamic environments. Thus, students of organizational decision-making report the existence of various patterns of contingent coping. Our purpose was to test if such patterns can be found at the level of individual decision-making. This completes the conceptual analysis of our research questions. Next we turn to their empirical investigation. METHOD Subjects One hundred two students in a course in decisionmaking at the Israel Defense Forces (I.D.F.) Command & General Staff College participated in the study. Most of the students at the College are male officers from all branches of the military with ranks from Captain to Lt. Colonel. Procedure As part of the course requirements, students wrote a case of decision-making under uncertainty based on their personal experience. Cases were written prior to the beginning of the course so as to prevent the students from being influenced by it. Instructions encouraged students to write fully and frankly, without defining either decision-making or uncertainty: “Write a case of decision-making under uncertainty from your personal experience in the I.D.F. Later on you will be asked to analyze the case applying the concepts and models that you will learn in the course. To facilitate your analysis, write as detailed a factual description of the case as possible. The case will be read only by your instructor.” Students were also told that their cases would be used 155 COPING WITH UNCERTAINTY for research (thus requiring wider distribution), and that they may ask to exclude their cases from such use if they wished to, with no consequences to themselves. Case Analysis Instrument To identify conceptualizations of uncertainty and tactics of coping with uncertainty in narrative reports we developed an instrument consisting of 16 conceptualizations and 12 tactics of coping based on the conceptual schemes presented in the Introduction. Types of Uncertainty A preliminary analysis of 25 cases (which were not included in the final analysis) showed (a) that not all 9 issue 3 source classifications of uncertainty are required in case analysis and (b) that fine-grained analysis of particular cases is required to distinguish among three forms of incomplete information: partial lack of information (corresponding to risk in Behavioral Decision Theory), complete lack of information (corresponding to uncertainty in Behavioral Decision Theory), and unreliable information (which abounds in organizational life, March & Sevon, 1982). Similarly, finegrained analysis is required to distinguish between three forms of inadequate understanding—inadequate understanding owing to equivocal information (Weick, 1979), inadequate understanding owing to novelty of situations (Louis, 1980) and inadequate understanding owing to fast-changing or unstable situations (Lanir, 1989)—and two types of conflicted alternatives: conflict owing to equally attractive or unattractive outcomes (March & Simon, 1958) and conflict owing to incompatible role requirements (Kahn, Wolfe, Quinn, Snoek, & Rosental, 1964). The 16 conceptualizations which were included in the instrument are presented in Table 2. Tactics of coping. Based on the conceptual analysis TABLE 2 A Classification System of Conceptualizations of Uncertainty Topics Types Information Completely lacking Partly lacking Unreliable Inadequate understanding Owing to equivocality Owing to instability Owing to novelty Conflict Outcomes Situation Role u u u u u u u u u u u u u u u u Note. Check marks denote a type of uncertainty included in the study’s instrument. presented in the Introduction, literature review and preliminary analysis of 25 cases, we identified 12 tactics of coping which are presented in Table 3 with their operational definitions. Analyzing cases with the final instrument was fairly labor intensive. A trained analyst required 20–40 min to analyze a case, depending on the narrative’s length and detail and the degree to which the description matched the categories specified in the instrument. To obtain a preliminary estimate of the instrument’s reliability we asked five independent judges, who were trained for approximately 2 h by the second author, to classify the type of uncertainty and tactics of coping in five retrospective case reports (not included in the initial or final samples of cases). Interjudge agreement among the five judges was .89 # k # 1.00 for the conceptualizations of uncertainty and .87 # k # 1.00 for the tactics of coping. This shows that independent judges can be trained to use the instrument with satisfactory interjudge agreement. Since we use retrospective reports drawn from longterm memory, it is unlikely that our data provide accurate descriptions of the reported cases (Ericcson & Simon, 1984). However, since the cases were written prior to the course in response to minimal instructions, it is fair to assume that they present students’ naive conceptualizations of what uncertainty is and how to cope with it. RESULTS The 102 cases included a single instance of coping with uncertainty and 10 cases included two instances. In 17 cases decision makers used two, and in one case three coping tactics to deal with a single uncertainty, and in one case the decision maker used three tactics in a certain instance. When several tactics were used we analyzed only the first because of possible order effects that may affect the choice of second and third tactics. Thus, our data included 122 pairs of uncertainty and coping tactics. Interjudge agreement between the second author, who analyzed all the cases, and a second judge (who did not know the research questions) who analyzed independently a randomly selected sample of 40 cases, was k 5 .83 for the conceptualizations of uncertainty and k 5 .93 for the tactics of uncertainty. To answer our three research questions we analyzed the distributions of the various conceptualizations of uncertainty (Question 1) and coping tactics (Question 2) and their joint distribution (Question 3). The results were as follows: Conceptualization of Uncertainty The frequency distribution of the different conceptualizations of uncertainty is presented in Table 4. The 156 LIPSHITZ AND STRAUSS TABLE 3 Tactics of Coping with Uncertainty Tactic Definition 1. Collect additional information 2. Delay action 3. Solicit advice 4. Follow SOPs, norms, etc. 5. Assumption-based reasoning Tactics of reduction Conduct an active search for factual information. Postpone decision-making or action taking until additional information clarifies the decision problem. Solicit advice/opinion of experts, superiors, friends or colleagues. Act according to formal and informal rules of conduct. Construct a mental model of the situation based on beliefs that are (1) constrained by (though going beyond) what is more firmly known, and (2) subject to retraction when and if they conflict with new evidence or with lines of reasoning supported by other assumptions. 3. Avoid irreversible action 4. Weighing pros & cons Tactics of acknowledgment Generate specific responses to possible negative outcomes. Develop a general capability to respond to unanticipated negative developments (e.g., put forces on the alert, leave some resources unused). Prefer or develop reversible course of action, prepare contingencies. Choose among alternatives in terms of potential gains and losses. 1. Ignore uncertainty 2. Rely on “intuition” 3. Take a gamble Tactics of suppression Act as if under certainty. Use hunches, informed guesses, etc., without sufficient justification. “Take a chance,” throw a coin, etc. 1. Preempting 2. Improve readiness two most frequent conceptualizations are inadequate understanding of the situation owing to equivocal information (24.6%), and conflict among alternatives owing to equally attractive outcomes (24.6%). Since the former is consistent with matching mode decision making and the latter with consequential choice mode decisionmaking (Lipshitz, 1994; March, 1981), we decided to test which of the two modes was more characteristic in our sample of case reports. To do this we compared the combined frequency of conceptualizations that indicate decision-making by concurrent choice or by consideration of future outcomes (items 3, 5, and 12 in Table 4) with the combined frequency of conceptualizations that indicate decision-making by matching action to the requirements of the decision maker’s role or situation (items 1, 2, 4, 6–11). The last conceptualization, which is a hybrid, was excluded from the comparison. Consistent with studies of naturalistic decision-making (Lipshitz, 1995), matching was more frequent than consequential choice, which dominates the literature on Behavioral Decision-making (65.7% vs 32.8%, respectively). Coping with Uncertainty The frequency distribution of the different tactics of coping with uncertainty is presented in Table 5. Tactics of reduction are reported most frequently (46.8%), followed closely by tactics of acknowledgment (41.8%) and tactics of suppression (11.5%). The latter result may reflect low social desirability of such tactics. Three tactics are reported more frequently than all the others, TABLE 4 Frequency Distribution of Conceptualizations of Uncertainty Type of uncertainty Subject of uncertainty Frequency Percent 1. Complete lack of information 2. 3. 4. Partially lacking information 5. 6. Unreliable information 7. Inadequate understanding owing to equivocal information 8. 9. Inadequate understanding owing to novelty 10. 11. Inadequate understanding owing to instability 12. Conflict among alternatives owing to equally attractive outcomes 13. Conflict among alternatives owing to incompatible role demands Situation Role Outcomes Situation Outcomes Situation Situation Role Situation Role Situation Outcomes Role 9 1 6 8 4 8 30 3 6 4 11 30 2 7.4 .8 4.9 6.6 3.3 6.6 24.6 2.5 4.9 3.3 9 24.6 1.6 157 COPING WITH UNCERTAINTY TABLE 5 TABLE 6 Frequency Distribution of Coping Tactics Joint Distribution and Conditional Probabilities of Strategies of Coping (Rows) Given Conceptualizations of Uncertainty (Columns) Tactic Tactics of reduction Collecting additional information Seeking backing or advice Relying on doctrines & SOPs Assumption-based reasoning S Tactics of acknowledgment Improving readiness Preempting Avoiding irreversible action Weighing pros & cons S Tactics of suppression Ignoring uncertainty Acting on the basis of “intuition” Taking a gamble S Frequency Percent 14 10 6 27 57 11.5 8.2 5 22.1 46.8 1 26 1 23 51 .8 21.3 .8 18.9 41.8 9 3 2 14 7.4 2.5 1.6 11.5 assumption-based reasoning (22.1%), preempting (21.3%) and weighing pros and cons (18.9%). Contingent Coping Owing to our sample size, we had to group the data in Table 4 and 5 to perform this analysis. The 13 conceptualizations in Table 4 were grouped into three categories according to their sources of uncertainty (incomplete information, conceptualizations 1–8; inadequate understanding, conceptualizations 9–11; and undifferentiated alternatives, conceptualizations 14–15). The tactics in Table 5 were grouped into five categories, reduction (tactics 1–3), suppressing (tactics 9–11), and three spin-offs from the original acknowledgment strategy: assumption-based reasoning (tactic 4), forestalling (tactics 5–7) and weighing pros and cons (tactic 8). The rationale of the latter division was the ambiguous classification of assumption-based reasoning as a tactic of reduction or a tactic of acknowledgment (see Introduction above) and the rational vs single-option difference between weighing pros and cons and forestalling (Lipshitz, 1995). In the following discussion we refer to these categories as strategies of coping. The results in Table 6 confirm that conceptualizations of uncertainty and strategies of coping are related differentially (x2(8) 5 32.3, p , .001), and reveal the following pattern of contingent coping: If uncertainty is conceptualized as lack of information, decision makers use assumption-based reasoning (p 5 .33) and forestalling (p 5 .22); if uncertainty is conceptualized as inadequate understanding, decision makers use reduction (p 5 .37) and forestalling (p 5 .24); and if uncertainty is conceptualized as conflict among undifferentiated alternatives, decision makers use weighing pros and cons Lack of Inadequate information understanding Conflict Reduction Assumption-based reasoning Forestalling Weighing pros and cons Suppression S 7 20 3 S 30 .19 12 .33 8 .22 4 .11 5 .13 .37 13 .04 13 .24 4 .07 4 .07 .09 .22 15 .47 5 .15 36 54 32 2 .06 7 .25 27 .22 28 .23 23 .19 14 .11 122 (p 5 .47) and forestalling (p 5 22). Put differently, the similarities between their base rates and three conditional probabilities show that decision makers are equally inclined to use forestalling and suppression with all types of uncertainty. In contrast, examination of these parameters shows that decision makers tend to use the three remaining strategies to cope with particular types of uncertainty. The conditional probability of reduction given inadequate understanding is .37 compared with its unconditional probability of .25; the conditional probability of weighing pros and cons given conflict is .47, compared with its unconditional probability of .19, and the conditional probability of assumption-based reasoning given lack of information is .33 compared with its unconditional probability of .22. In addition, these conditional probabilities were considerably larger than the two remaining conditional probabilities in each category of uncertainty. The pattern of differential response in Table 6 indicates that the five strategies in the table, rather than the three originally posited strategies, are psychologically distinct. This conclusion should be treated cautiously owing to some exceedingly small cell sample sizes in Table 6. Finally, we analyzed the decision rules used by decision makers in the 30 instances in which they weighed the pros and cons of the outcomes of conflicting alternatives. To this end we counted the number of positive and negative attributes associated with every alternative, counting uncertainty as a negative attribute (Potter & Beach, 1994) if the decision maker referred to uncertainty as such explicitly. In 20 of the 25 instances decision makers chose one of the conflicted alternatives. In 15 of these instances the chosen alternative was 158 LIPSHITZ AND STRAUSS associated either with more positive or with less negative attributes (or both) than the rejected alternatives. In the five exceptions to this rule decision makers explicitly mentioned that they “decided to take a risk” or that they “decided to take this course of action no matter what.” In 5 instances decision makers developed and implemented a new alternative. In all these instances the original competing alternatives were either associated only with negative attributes, or associated with an equal mix of positive and negative attributes. This result is consistent with Lipshitz (1994, p. 60) who suggested that no-win choice problems “creates an avoidance–avoidance conflict, leading to ‘flight from the field’ (Lewin, 1948) in the form of reframing [i.e., development of a new alternative].” It is also consistent with Klein’s RPD model (Klein, 1993) which posits that decision makers develop a new alternative when they find that the alternative that they currently consider is unacceptable. DISCUSSION In this study we investigated three questions in regard to how decision makers cope with uncertainty in naturalistic settings: How do decision makers conceptualize the uncertainty which they encounter in naturalistic settings? How do they cope with this uncertainty? Are there systematic relationships between different types of uncertainty and different methods of coping? Defining uncertainty in the context of action broadly as a sense of doubt that blocks or delays action, we analyzed how uncertainty was conceptualized and handled in retrospective case reports of decision-making under uncertainty. The results can be summarized as follows: 1. Three conceptualizations of uncertainty were identifiable in the cases: inadequate understanding (approximately 44% of the cases), undifferentiated alternatives (approximately 25%), and lack of information (approximately 21%.) In most instances (approximately 67%) decision makers were uncertain about their role or situation. In the remaining 33% their uncertainty concerned the potential outcomes of their options. These results are consistent with emphases on the subjective nature of uncertainty (Howell & Burnett, 1978; Milliken, 1987; Weick, 1995), as decision makers attribute uncertainty more often to subjective sources (i.e., inadequate understanding and undifferentiated alternatives) than to objective source (i.e., incomplete information). In addition, these results are consistent with the assertion that naturalistic decision-making is characteristically driven by situation assessment (Lipshitz, 1993; March, 1981). 2. Five broad strategies of coping were identifiable in the cases: reduction (approximately 25%), forestalling (approximately 23%), assumption-based reasoning (approximately 22%), weighing pros and cons (approximately 19%) and suppression (approximately 11%). 3. Decision makers used different strategies to cope with different types of uncertainty. Inadequate understanding was primarily managed by reduction; incomplete information was primarily managed by assumption-based reasoning; and conflict among alternatives was primarily managed by weighing pros and cons. Forestalling was equally likely to be used as a back-up strategy with all forms of uncertainty, and suppression was least likely to be used with all of them. These results show that Behavioral Decision Theory’s R.Q.P. heuristic provides an incomplete description of how decision makers cope with uncertainty in naturalistic settings. While reduction is consistent with this heuristic, and weighing of pros and cons can be regarded as informal adaptations of two of its versions (SEU and MAU, respectively), these strategies account for less than half of the instances of coping in our sample. Furthermore, the R.Q.P. heuristic does not posit contingent coping, and none of our cases indicates the use of quantification. In the following discussion we first show how the pattern of contingent coping found in our sample is consistent with six models of decisionmaking in naturalistic settings, thus enabling us to develop it into a Naturalistic Decision-Making alternative to the R.Q.P. heuristic, the R.A.W.F.S. heuristic (designating its five components: Reduction, Assumption-based reasoning, Weighing pros and cons, Forestalling, and Suppression. We conclude with a discussion of the contributions and limitations of the study and the additional research that it suggests. The R.A.W.F.S. Heuristic The significance of the pattern of contingent coping presented in Table 6 is that it fits nicely—and ties together—several models of naturalistic decision-making, notably Beach (1990); Cohen, Freeman, and Wolf (in press); Janis and Mann (1977); Klein (1993); Montgomery (1989); Weick (1979, 1995). This compatibility, which helps to makes sense of the contingent pattern in Table 6 and validates it in a bootstrapping-like fashion, is captured by the hypothetical R.A.W.F.S. heuristic (Fig. 1). The heuristic describes how decision makers conceptualize and cope with uncertainty in naturalistic settings. The numerals in square brackets in the following discussion refer to corresponding elements in Fig. 1. Consistent with our findings and the Naturalistic Decision-making framework (Lipshitz, 1993), the COPING WITH UNCERTAINTY 159 FIG. 1. Coping with uncertainty: The R.A.W.F.S. heuristic hypothesis. R.A.W.F.S. heuristic presumes that decision makers use both situation assessment [1] coupled with serial option evaluation [2] and concurrent choice [6]. Based on Klein (1993), Pennington and Hastie (1993), and Weick (1979; 1995), the heuristic assumes that decision-making begins with an attempt to understand, recognize or make sense of the situation [1]. If this attempt is successful, decision makers initiate a process of serial option evaluation [1] → [2] which they complement, if time permits, by mentally simulating the selected option [2] → [3] (Beach, 1990; Klein, 1993; Klein & Crandall, 1995). When sensemaking fails, decision makers experience inadequate understanding to which they respond, consistent with our findings (as well as Klein’s, 1993), by using reduction or by forestalling [1] → [4a] → [4b]. If additional information is not available (as is often the case in the real world, e.g., Devons, 1961; Grandori, 1984; Lipshitz, 1995; Quinn, 1980), decision makers experience lack of information, to which they respond by assumption-based reasoning or by forestalling [4a] → [5]. This, again, is consistent with our findings, as well as with the Recognition/metacognition model (Cohen, Freeman & Wolf, 1996; Cohen, Adelman, Tolcott, Bresnick & Marvin, 1993). If decision makers generate 160 LIPSHITZ AND STRAUSS two or more good enough options they experience conflict [6], to which they respond by weighing pros and cons or by forestalling [6] → [7]. The three-parts sequence [2] → [6] → [7] is consistent both with our findings and with Image Theory (Beach, 1990), which includes a similar sequence of serial followed by concurrent option evaluation. Finally, if decision makers either fail to identify a single good enough option, or to differentiate among several good enough options they resort to suppression, forestalling, or the generation of a new alternative ([6] → [8] and [7] → [8], respectively). The links leading to suppression in the R.A.W.F.S. heuristic are consistent with Janis & Mann (1977) and Montgomery (1989). In addition, locating this strategy as a response of last resort to all types of uncertainty is consistent with its undifferentiated pattern of relationships in Table 6, as well as with its low observed frequency (which can also be attributed to low social desirability). Three of the multiple locations of forestalling, [5], [7] and [8] are consistent with its undifferentiated pattern of relationships in Table 6. The remaining location [9] is based on Froot, Scharfstein, and Stein (1994), who suggest that decision makers forestall when they understand the risk posed by the situation either owing to reduction or assumption-based reasoning [8] or without them [9]. Finally, generating new alternatives [8] is consistent with our secondary analysis of weighing pros and cons. The sequence depicted in Fig. 1 is not obligatory. For example, if a decision maker frames his or her uncertainty as undifferentiated alternatives to begin with, he or she will “enter” the process at [6]. The R.A.W.F.S. heuristic offers a naturalistic decision-making alternative to the R.Q.P. heuristic from which it differs in several respects. First, the R.Q.P. heuristic is driven by formal analytic models that conceptualize decision-making under uncertainty as a form of gambling. In contrast, the R.A.W.F.S. heuristic is empirically driven and recognizes gambling as one, but by no means the only, conceptualization of decisionmaking under uncertainty. Consequently, while the R.Q.P. heuristic emphasizes computation (Pennington & Hastie, 1988), the R.A.W.F.S. heuristic emphasizes other forms of reasoning, notably assumptionbased reasoning and forestalling. Secondly, while the R.Q.P. heuristic suggests that how decision makers cope, or ought to cope, with uncertainty is dictated by the magnitude or intensity of uncertainty, the R.A.W.F.S. heuristic suggests that how decision makers cope, or ought to cope, with uncertainty is principally determined by the nature or quality of uncertainty. Finally, the R.A.W.F.S. heuristic presents a more favorable picture of how decision makers cope with uncertainty than the picture that emerges from studies associated with the R.Q.P. heuristic: We often dread uncertainty. A common way of dealing with uncertainty in life is to ignore it completely, or to invent some “higher rationale” to explain it, often a rationale that makes it more apparent than real . . . In fact, we even tend to deny the random components in trivial events that we know to be the result of chance. (Dawes 1988, p. 256.) Contrary to Dawes’ pessimistic judgment, the R.A.W.F.S. heuristic suggests that decision makers cope with uncertainty adaptively, matching different types of uncertainty with different coping strategies that are suitable to human bounded rationality, resorting to suppression tactics only if all other strategies of coping fail. These differences have significant implications for decision-aiding and the design of decision training programs. Regarding the former, decision support systems should be expanded beyond the R.Q.P. heuristic to support elements of the R.A.W.F.S. heuristic such as sense making (Weick, 1995) and assumption-based reasoning (Cohen, 1989). Regarding the latter, training programs should aim at teaching novices or mediocre performers the strategies and tactics that are used by experienced decision makers in the same domain (Kelley, 1993) rather than the lessons of Judgment and Decision-making research (Beyth-Marom, Fischhoff, Quadrel, & Furby, 1991; Fischhoff, 1982). Our study adds to the (surprisingly) few studies focusing specifically on how decision makers cope with uncertainty within the Naturalistic Decision-making framework (Cohen, Freeman & Wolf, 1996; Cohen, Tolcott & McIntyre, 1987; Potter & Beach, 1994; Serfaty, Entin, & Riedel, 1991.) Although we did not observe decision-making in situ, we used naturalistic methodology in that we did not identify decision-making uniquely with concurrent choice (Lipshitz, 1993); we did not associate uncertainty uniquely with future consequences, and we analyzed decision makers’ self reports with minimal conceptual imposition (defining uncertainty inclusively as a sense of doubt that blocks or delays action). Although the R.A.W.F.S. heuristic is compatible with several existing models and (at least to us) seems reasonable, we advance it as a hypothesis for several good reasons. First, Fig. 1 is extrapolated from results that do not pertain to sequences of experienced uncertainty → coping tactic pairs. In addition, we used retrospective self reports obtained from a fairly small nonrepresentative sample drawn from a specific population. Thus, we suspect that when decision makers are at a total loss COPING WITH UNCERTAINTY 161 as to what to do (a type of uncertainty altogether missing from our data) suppression tactics (which are found fairly infrequently) are more ubiquitous than our findings indicate. Finally, retrospective self reports drawn from long-term memory cannot be regarded as veridical reports of external past events or internal cognitive processes (Ericcson & Simon, 1984). Such reconstructions are, however, valid sources of evidence for the schemas that people use to conceptualize their experiences and actions (Lipshitz & Bar Ilan, 1996; Mandler, 1984). In conclusion, the R.A.W.F.S. heuristic suggests several directions for future research. First, the heuristic clearly needs to be tested with decision-making in vivo. Given the difficulties associated with this type of research, it is possible to use simulators such as those that are used or training (Lipshitz & Ben Shaul, 1997; Serfaty et al., 1991), or experimental small world simulations (Brehmer & Dorner, 1993.) These studies can provide more informative data on the frequency of use of different conceptualizations and coping tactics than the data presented in Tables 4 and 5. The R.A.W.F.S. heuristic can also be tested and elaborated by way of applications. Cohen and his associates have already outlined how assumption-based reasoning can be used to support situation-assessment-based decision-making (Cohen et al., 1993, 1996.) This work can be extended to include other strategies and meta-strategies which specify conditions of optimal use for each strategy. Finally, since the tactics (and certainly the strategies) are fairly abstract, studying how practitioners in different domains operationalize them concretely can be used to design training programs. Our teaching experience shows that decision makers (e.g., managers and officers) think that the R.A.W.F.S. heuristic and the coping tactics presented in Table 3 are useful and evocative. Our results show that some decision makers devised sensible, and usable, strategies of coping with uncertainty that are well worth studying. Beach, L. R., (1990). Image theory: Decision-making in personal and organizational contexts. London: Wiley. REFERENCES Cohen, M. S., Freeman, J. T., & Wolf, S. (1996). Meta-recognition in time stressed decision-making: Recognizing, critiquing and correcting. Journal of the Human Factors and Ergonomics Society. Allaire, Y., & Firsirotu, M. E. (1989). Coping with strategic uncertainty. Sloan Management Journal, 3, 7–16. Beach, L. R. (1993). Broadening the definition of decision-making: The role of prechoice screening of options. Psychological Science, 4, 215–220. Beach, L. R., & Lipshitz, R. (1993). Why classical decision theory is an inappropriate standard for evaluating and aiding most human decision-making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision-making in action: models and methods (pp. 21–35). Norwood, NJ: Ablex Publishing. Berkeley, D., & Humphreys, P. (1982). Structuring decision problems and the ‘bias’ heuristic. Acta Psychologica, 50, 201–252. Bernstein, P. L., & Silbert, T. H. (1984). Keeping informed. Harvard Business Review, 62(5), 32–40. Beyth-Marom, R., Fischhoff, B., Quadrel, M. J., & Furby, L. (1991). Teaching decision-making to adolescents: A critical review. In J. Baron & R. Brown (Eds.), Teaching decision-making to adolescents (pp. 19–59). Hillsdale, NJ: Erlbaum. Bolman, L. G., & Deal, T. E. (1991). Reframing organizations. San Francisco: Jossey Bass. Brehmer, B., & Dorner, D. (1993). Experiments with computer-simulated microworlds: Escaping both the narrow straights of the laboratory and the deep blue sea of the field study. Computers in Human Behavior, 9, 171–184. Brown, R. V. (1992). The state of art of decision analysis: A personal view. Interfaces, 22, 5–14. Brunsson, N. (1985). The irrational organization. Chichester: Wiley. Budescu, D. V., & Wallsten, T. S. (1995). Processing linguistic probabilities. In J. Busemeyer, & D. L. Medin (Eds.), Decision-making from a cognitive perspective (pp. 275–318). Camerer, C., & Weber, M. (1992). Recent developments in modeling preferences: Uncertainty and ambiguity. Journal of Risk & Uncertainty, 5, 325–370. Clark, R. E., & Halford, L. (1980). Reducing uncertainty and building trust: The special use of auctions. In S. Fiddle (Ed.), Uncertainty: Behavioral and social dimensions (pp. 305–322). New York: Praeger. Cohen, M. S. (1989). A database tool to support probabilistic assumption-based reasoning in intelligence analysis. Proceedings of the 1989 Joint Director of the C2 Symposium, Ft. McNair, VA, June 27–29. Cohen, M. S., Adelman, L., Tolcott, M. A., Bresnick, T. A., & Marvin, F. M. (1993). A cognitive framework for battlefield commanders’ situation-assessment. TR 93-1, United States Army Research Institute, For Leavenworth Field Unit. Anderson, B. F., Deane, D. H., Hammond, K. R., & McClelland, G. H. (1981). Concepts in judgment and decision research. New York: Praeger. Cohen, M. S., Schum, D. A., Freeling, A. N. S., & Chinnis, J. O. (1985). On the art and science of hedging a conclusion: Alternative theories of uncertainty in intelligence analysis. Falls Church, VA: Decision Science Consortium. Argote, L. (1982). Input uncertainty and organizational coordination in hospital emergency units. Administrative Science Quarterly, 27, 420–434. Cohen, M. S., Tolcott, M. A., & McIntyre, J. (1987). Display techniques for pilot interactions with intelligent avionics: A cognitive approach. Falls Church, VA: Decision Science Consortium. Arrow, K. J. (1965). Aspects of the theory of risk bearing. Helsinki: Yrjo Johanssonis Saatio. Conrath, D. (1967). Organizational decision-making behavior under varying conditions of uncertainty. Management Science, 13, B487– B500. Barmhart, C. L. and Stein, J. (1964). The American college dictionary, New York: Random House. Coombs, C. H., Dawes, R. M., & Tversky, A. (1971). Mathematical 162 LIPSHITZ AND STRAUSS psychology: An elementary introduction. Englewood Cliffs, NJ: Prentice Hall. Corbin, R. M. (1980). Decisions that might not get made. In T. Wallsten (Ed.), Cognitive processes in choice and decision behavior. Hillsdale, NJ: Erlbaum. Curley, S. P., Yates, F., & Abrams, R. A. (1986). Psychological sources of ambiguity avoidance. Organizational Behavior and Human Decision Process, 38, 230–256. Cyert, R., & March, J. (1963). A behavioral theory of the firm. Englewood Cliffs, NJ: Prentice-Hall. Dawes, R. M. (1988). Rational choice in an uncertain world. New York: Harcourt Brace Jovanovich. Devons, E. (1961). Statistics a basis for policy. In E. Devons, Essays in economics, (pp. 122–137). London: Allen & Unwin. Howell, W. C., & Burnett, S. A. (1978). Uncertainty measurement. A cognitive taxonomy. Organizational Behavior and Human Performance, 22, 45–68. Huber, O., Wider, R., & Huber, O. W., (1996). Active information search and complete information presentation in naturalistic risky decision tasks. Acta Psychologica. Humphreys, P., & Berkeley, D. (1985). Handling uncertainty: Levels of analysis of decision problems. In G. Wright (Ed.), Behavioral decision-making. New York: Plenum Press. Isenberg, D. (1985). Some hows and whats of managerial thinking: Implications for future army leaders. In J. G. Hunt & J. Blair (Eds.), Military leadership in the future battlefield New York: Pergamon Press. Dewey, J. (1933). How we think. Boston: D. C. Heath. Janis, I. L., & Mann, L. (1977). Decision-making: A psychological analysis of conflict, choice and commitment. New York: Free Press. Downey, H. K., & Slocum, J. (1975). Uncertainty: Measures, research, and sources of variation. Academy of Management Journal, 18, 562–578. Kahn, R. L., Wolfe, D. M., Quinn, R. P., Snoek, J. D., & Rosental, R. A., (1964). Organizational stress: Studies in role conflict and ambiguity New York: Wiley. Duncan, R. (1972). Characteristics of organizational environments and perceived environmental uncertainty. Administrative Science Quarterly, 17, 313–327. Kahneman, D., Slovic, P., & Tversky, A. (Eds.) (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, MA: Cambridge Univ. Press. Eden, C., (1988). Cognitive mapping. European Journal of Management Research, 36, 1–13. Kelley, R. (1993). How Bell Labs create star performers. Harvard Business Review, 71(4), 128–139. Erev, I., Bornstein, G., & Wallsten, T. W. (1993). The negative effect of probability assessment on decision quality. Organizational Behavior and Human Decision Processes, 55, 78–94. Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision-making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision-making in action: Models and methods, (138–147). Norwood, CT: Ablex. Ericcson, K. A., & Simon, H. A. (1984). Protocol analysis. Cambridge, MA: MIT Press. Feldman, M. S., & March, J. G. (1981). Information in organizations as signal and symbol. Administrative Science Quarterly, 26, 171– 186. Fischhoff, B. (1982). Debiasing. In D. Kahneman, P. A. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. New York: Cambridge Univ. Press. Fredrickson, J. W., & Mitchell, T. R. (1984). Strategic decision processes: Comprehensiveness and performance in an industry with an unstable environment. Academy of Management Journal, 27, 399–423. Frisch, D., & Clemen, R. T. (1994). Beyond expected utility: Rethinking behavioral decision research. Psychological Bulletin, 116, 46–54. Froot, K. A., Scharfstein, D. S., & Stein, J. C. (1994). A framework for risk management. Harvard Business Review, 72(6), 91–102. Galbraith, J. (1973). Designing complex organizations. Reading, MA: Addison Wesley. Gigerenzer G. (1991). How to make cognitive illusions disappear: Beyond “heuristics and biases.” European Review of Social Psychology, 2, 83–115. Goldman, A. (1986). Cognition and epistemology. Cambridge, MA: Harvard Univ. Press. Grandori, A. (1984). A prescriptive contingency view of organizational decision-making. Administrative Science Quarterly, 29, 192–209. Hirst, E., & Schweitzer, M. (1990). Electric-utility resource planning and decision-making: The importance of uncertainty. Risk Analysis, 10, 137–146. Hogarth, R. M. (1987). The psychology of judgment and choice (2nd ed.). San Francisco, CA: Jossey Bass. Hogarth, R. M., & Kunreuther, H. (1995). Decision-making under ignorance: Arguing with yourself. Journal of Risk and Uncertainty, 10, 15–36. Klein, G. A., & Crandall, B. W. (1995). The role of mental simulation in naturalistic decision-making. In P. Hancock, J. Flach, J. Caird, & K. Vincente (Eds.), Local applications of the ecological approach to human-machine systems, (Vol. 2, pp. 324–358). Hillsdale, NJ: Erlbaum. Klein, G. A, Orasanu, J. R., Calderwood, R., & Zsambok, C. (Eds.), Decision-making in action: Models and methods, Norwood, CT: Ablex. Lanir, Z., (1989). The reasonable choice of disaster: The shooting down of the Libyan airliner on 21 February 1973. Journal of Strategic Studies, 12, 479–493. Lesgold, A., Robinson, H., Feltovich, P., Glaser, R., Klopfer, D., & Wang, Y. (1988). Expertise in a complex skill: Diagnosing X ray pictures. In M. T. H. Chi, R. Glaser, & M. Farr (Eds.), The nature of expertise. Hillsdale NJ: Erlbaum. Lewin, K. (1948). Resolving social conflict. New York: Harper. Lipshitz, R. (1993). Converging themes in the study of decision-making in realistic settings. In G. A. Klein, J. Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision-making in action: Models and methods, (103–137). Norwood, CT: Ablex. Lipshitz, R. (1994). Decision-making in three modes. Journal for the Theory of Social Behavior, 24, 47–66. Lipshitz, R. (1995). The road to “Desert Storm”: Escalation of commitment and the rational vs. single-option paradigms in the study of decision-making. Organization Studies, 16, 243–263. Lipshitz, R., & Bar Ilan, O. (1996). How problems are solved: Reconsidering the phase theorem. Organizational Behavior and Human Decision Processes, 65, 48–60. Lipshitz, R., & Ben Shaul, O. (1997). Schemata and mental models in recognition-primed decision-making. In C. Zsambok, & G. A. Klein (Eds.), Naturalistic decision-making (pp. 292–303). Hillsdale, NJ: Erlbaum. Lopes, L. L. (1987). Between hope and fear: The psychology of risk COPING WITH UNCERTAINTY taking. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 20, pp. 255–293). New York: Wiley. Louis, M. R. (1980). Surprise and sense making: What newcomers experience in entering new organizational settings. Administrative Science Quarterly, 25, 226–251. MacCrimmon, K. R., & Wehrung, D. A. (1986). Taking risks. New York: Free Press. Mandler, J. M., (1984). Stories, scripts, and scenes: Aspects of schema theory. Hillsdale, NJ: Erlbaum. March, J. G. (1981). Decisions in organizations and theories of choice. In A Van De Ven & W. F. Joyce (Eds.), Perspectives on organization design and behavior (pp. 205–244). New York: Wiley. March, J. G., & Olsen, J. P. (1976). Ambiguity and choice in organizations. Bergen: Universitetsforlaget. March, J. G., & Shapira, Z. (1987). Managerial perspectives on risk and risk taking. Management Science, 33, 1404–1418. March, J. G., & Sevon, G. (1982). Gossip, information, and decisionmaking. In L. S. Sproull & P. D. Larkey (Eds.), Advances in organizations (Vol. 1). Greenwich, CT: JAI Press. March, J. G., & Simon, H. A. (1958). Organizations. New York: Wiley. Matlin, M. W., & Stang, D. J. (1978). The Pollyanna principle: Selectivity in language, memory and thought. Cambridge, MA: Schenkman. McCaskey, M. B. (1986). The executive challenge: Managing change and ambiguity. Harper Business. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Counseling and Clinical Psychology, 46, 806–834. Milliken, F. C. (1987). Three types of perceived uncertainty about the environment: State, effect, and response uncertainty. Academy of Management Review, 12(1), 133–43. Montgomery, H. (1988). From cognition to action: The search for dominance in decision-making. In H. Montgomery & O. Svenson (Eds.), Process and structure in human decision-making (pp. 471– 483). New York: Wiley. Orasanu, J., & Connolly, T. (1993). The Reinvention of decision-making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision-making in action: Models and methods (pp. 3–20). Norwood, NJ: Ablex Publishing. Pennington, N., & Hastie, R. (1988). Explanation based decisionmaking: Effects of memory structure on judgment. Journal of Experimental Psychology: Learning, Memory & Cognition, 14, 521– 533. Potter, R. E., & Beach, L. R. (1994). Imperfect information in prechoice screening of options. Organizational Behavior and Human Decision Processes, 59, 313–329. Received: May 20, 1996 163 Quinn, J. B. (1980). Strategies for change: Logical incrementalism. Homewood IL. Irwin. Raiffa, H. (1968). Decision analysis: Introductory lectures on choices under uncertainty. Reading, MA: Addison Wesley. Schoemaker, P. J. H. (1995). Scenario planning: A tool for strategic thinking. Sloan Management Review, 36(2), 25–50. Serfaty, D., Entin, E. B., & Riedel, S. L. (1991). The role of uncertainty in the generation, exploration, and implementation of tactical options. Burlington, MA: Alphatech Inc., TR-325. Shafir, E., Simonson, I., & Tversky, A. (1993). Reason-based choice. Cognition, 49, 11–36. Skagestad, P. (1981). The road of inquiry: Charles Peirce’s pragmatic realism. New York: Columbia Univ. Press. Smith, G. F., Benson, G. P., & Curley, S. P. (1991). Belief, knowledge and uncertainty: A cognitive perspective on subjective probability. Organizational Behavior and Human Decision Processes, 48, 291– 321. Smithson, M. (1989). Ignorance and uncertainty: Emerging paradigms. New York: Springer Verlag. Svenson, O. (1992). Differentiation and consolidation theory of human decision-making: A frame of reference for the study of pre- and post-decision processes. Acta Psychologica, 80, 143–168. Terreberry, S. (1968). The evolution of organizational environments. Administrative Science Quarterly, 12, 589–613. Thompson, J. (1967). Organizations in action. New York: McGraw Hill. Voss, J. F., Greene, T. R., Post, T. A., & Penner, B. C. (1983). Problemsolving skill in the social sciences. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 17, pp. 261–285), Hillsdale, NJ: Erlbaum. Weick, K. E. (1979). The social psychology of organizing. Reading, MA: Addison Wesley. Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage. Wildavsky, A. (1988). Searching for safety. New Brunswick (USA): Transactions Publishers. Wohlstetter, R. (1962). Pearl Harbor: Warning and decision. Stanford CA: Stanford Univ. Press. Yates, J. F., & Stone, E. R. (1992). The risk construct. In J. F. Yates (Ed.), Risk-taking behavior (pp. 1–26). New York: Wiley. Zimmer, A. C. (1983). Verbal vs. numerical processing of subjective probabilities. In Scholz, R. W. (Ed.), Decision-making under uncertainty. Amsterdam: North Holland.