Why Expertise Matters: A Response to the Challenges

HUMAN-CENTERED COMPUTING Editor: Robert R. Hoffman, Institute for Human and Machine Cognition, [email protected] Why Expertise Matters: A Response to the Challenges Gary Klein, MacroCognition LLC Ben Shneiderman, University of Maryland Robert R. Hoffman and Kenneth M. Ford, Institute for Human and Machine Cognition We dedicate this article to our colleague Robert Wears, who tragically died in July 2017 just before we started to work on this article. O verwhelming scientific evidence demonstrates that experts’ judgments can be highly accurate and reliable. As defined in the scientific literature,1 experts • employ more effective strategies than others, and do so with less effort; • perceive meaning in patterns that others do not notice; • form rich mental models of situations to support sensemaking and anticipatory thinking; • have extensive and highly organized domain knowledge; and • are intrinsically motivated to work on hard problems that stretch their capabilities. Our society depends on experts for mission-critical, complex technical guidance for high-stakes decision making because they can make decisions despite incomplete, incorrect, and contradictory information when established routines no longer apply.2 Experts are the people the team turns to when faced with difficult tasks. Despite this empirical base, we witness a number of challenges to the concept of expertise. Tom Nichols’ The Death of Expertise presents a strong defense of expertise,3 a defense to which we are adding in this article. We address the attempts NOVEMBER/DECEMBER 2017 made by five communities to diminish the credibility and value of experts (see Figure 1). These challenges come from • decision researchers who show that simple linear models can outperform expert judgment; • heuristics and biases researchers who have claimed that experts are as biased as anyone else; • sociologists who see expertise as just a social attribution; • practice-oriented researchers seeking to replace professional judgments with data-based prescriptions and checklists; and • technophiles who believe that it is only a matter of time before artificial intelligence (AI) surpasses experts. Each of these communities has questioned the value of expertise, using different arguments, perspectives, and paradigms. Society needs experts, even though they are fallible. Although we are expert-advocates, eager to highlight the strengths of experts, we acknowledge that experts are not perfect and never will be. Our purpose is to correct the misleading claims and impressions being spread by the expertise-deniers. Then we hope to engage productively with each of these communities to improve human performance through better training, workflows, and technology. We begin with the challenge from the decision research community because this body of work can be traced back the furthest, to the mid-1960s, and echoes to this day. 1541-1672/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society 67 Challenges to expertise Evidence-based practices Heuristics and biases Computer science Expertise Sociology Decision research Figure 1. The five communities that have challenged the concept of expertise. The Challenge from the Decision Research Community Research by some experimental psychologists shows that in judgment tasks, simple linear models will be more consistent in their performance than human judges. Examples are faculty ratings of graduate students versus a model based on grades and test scores, or physicians’ ratings of cancer biopsy results versus a model based on survival statistics: 4,5 There are some, but only a few, truly remarkable judges, whereas there are many so-called experts who are no better than complete novices … the picture of the expert painted in broad brush strokes by this research is relatively unflattering … whenever possible, human judges should be replaced by linear models.6 A few aspects of this literature are noteworthy: • The linear models are derived in the first place from the advice of experts about what the key variables are—the variables that experts themselves use in making judgments. 68 • The decision research tends to reduce expertise to single measures, such as judgment hit rate, ignoring more qualitative contributions to performance. • For many of the studies, it is not obvious that the particular judgment task that is presented to the participants is actually the same task that the experts routinely perform, and therefore might not be the task at which they are proficient. • For many of the studies, there is scant evidence that the participants who are called experts actually qualify for that designation, apart from their having had a certain numbers of years of experience. • Advocates of this view go beyond their empirical base by generalizing from studies using college students as judges to argue for the fallibility of all judges. • Although linear models are consistent, when they fail, they fail miserably. One problem involves “broken leg cues.” A linear model might do a decent job of predicting whether a given person is likely to go to the movies this weekend, but will fail because it is blind to the fact that the person in question just broke a leg.7 Experts will perform better than the model if they have information to which the model is insensitive. 8 Who are the experts? In some studies, the linear models were compared to college students, sometimes called “judges.” And even in studies in which the judges were professionals, perhaps the linear models should be compared to the best of the professionals rather than to the average. Even when the linear models outperformed the experts, it is a mistake to infer that the linear models got it right and the experts failed miserably. www.computer.org/intelligent The linear models had their greatest edge in domains6 and prediction tasks involving human activity (that is, clinical psychologists, psychiatrists, counselors, admissions officers, parole officers, bank loan officers, and so on). However, the linear models weren’t very accurate—it was just that the experts were even worse. In one often-cited study of cancer diagnosis, the linear model performed better than oncologists at predicting patient longevity, but a closer look shows that the model only accounted for 18 percent of the variance in the judgment data.9 The clearest conclusion from this and other studies on linear modeling is that some things are of intrinsic low predictability. Next is the challenge from the heuristics and biases (HB) community, which can be traced to the early 1970s. The Challenge from the Heuristics and Biases Community Led by Daniel Kahneman and Amos Tversky,10 the HB community has called into question assumptions about rationality by demonstrating that people fall prey to a wide variety of biases in judgment and probabilistic reckoning, and that even experts sometimes show these biases. This finding helped create a mindset that experts are not to be trusted. The proliferation of HB research in academic psychology departments has strengthened the impression that expert judgments are not accurate. The HB paradigm typically uses participants who are not experts (college students) and gives them artificial laboratory tasks that require little training and have little or no ecological validity. The tasks, conveniently enough, can be performed in a college class period. Bias effects found using this “paradigm of convenience” IEEE INTELLIGENT SYSTEMS diminish or disappear when researchers add context11 or when researchers have genuine experts engage in their familiar environments rather than work on artificial puzzles and probability-juggling tasks. Variations in the materials, instructions, procedures, or experimental design can cause bias effects to diminish or disappear.12 Although some studies have shown that bias can occur in expert reasoning,13 several studies show that bias effects are much smaller than those of the college students.14 There is mixed evidence for the claim that experts tend to be overconfident, and what evidence there is stems from narrow methods for measuring confidence. Experts such as weather forecasters and firefighters are careful to keep their judgments within their core specialty and to use experience and accurate feedback to attain reasonable levels of confidence in their judgments. Weather forecasters search for evidence that confirms a hypothesis; it would be irrational not to. On the other hand, weather forecasters deliberately and deliberatively look for evidence that their hypotheses might be wrong. The HB researchers’ antipathy toward experts opened the way for the emergence of the naturalistic decision-making movement,15 which regards heuristics as strengths acquired through experience, rather than weaknesses. Tversky and Kahneman were careful to state that, “In general these heuristics are quite useful, but sometimes they lead to severe and systematic errors.”16 However, the HB field usually ignores this caveat and emphasizes the downside of heuristics. We now come to the challenge from sociology, which began in the 1970s and emerged most forcefully in the 1980s. NOVEMBER/DECEMBER 2017 The Challenge from Sociology Sociological analysis of the consequences of occupational specialization has considered the value of professions to society.17 Given its close association to the concept of professions, the concept of expertise was also assessed from the sociological perspective, referred to as “science and technology studies.” Ethnographers, sociologists, and philosophers of science researched expertise in domains including astronomy, physics, and endocrinology.18–20 Their resonant paradigms have been referred to as “situated cognition,” “distributed cognition,” and the “sociology of scientific knowledge.”21–24 Some individuals have defined their paradigm, in part, as a reaction to cognitive psychology: If one relegates all of cognition to internal mental processes, then one is required to pack all the explanatory machinery of cognition into the individual mind as well, leading to misidentification of the boundaries of the cognitive system, and the over-attribution to the individual mind alone all of the processes that give rise to intelligent behavior. 25 Proponents of the situated cognition approach offer many good examples of why one should define “the cognitive system” as persons acting in coordination with a social group to conduct activities using tools and practices that have evolved within a culture.25 The core claim is that expertise and cognition reside in the interaction among the individual and the team, community, or organization. The strongest view is that expertise is a social attribution or role, a matter of prestige and authority. A moderate view is that individual cognition is an enabling condition for expertise, which just happens to be a www.computer.org/intelligent condition that is not of particular interest in a sociological analysis. One of the valuable aspects of this perspective is to sensitize us to the importance of external resources and community relationships for the acquisition, expression, and valuation of expertise. Thus, we respect these researchers and their contributions. The importance of context has been recognized in cognitive psychology for decades, 26 and in computer science as well.27 We agree completely that resources for cognition are in the world. We agree that teamwork and organizational issues are an important part of naturalistic decision making. Indeed, the notion of “macrocognition”28 refers to coordinating and maintaining common ground as primary functions. However, cognitive scientists are disappointed by any approach that takes the strong stance that expertise is merely a social attribution, a stance that discounts the importance and value of individual cognition, knowledge, and expertise.27 There is overwhelming empirical evidence that individual knowledge is crucial in expert reasoning and problem solving. If you plug experts and nonexperts into the same work settings you will find huge differences in the quality of the outputs of the groups/teams. The claims derived from a dismissive reaction to cognitiveindividualist views move the pendulum too far. Sociologists including Harry Collins29 and Harald Mieg30 have taken the balanced view, which we advocate, describing the importance of individual expertise along with social and contextual factors that can be essential for developing and maintaining expertise. We remain hopeful that over time, this balanced view will predominate. We now come to challenges that have emerged most recently. 69 The Challenge from the Evidence-Based Practices Community The evidence-based practices community argues that professionals need to find the best scientific evidence, derive prescriptive procedures for decisions, and adhere to these procedures rather than rely on their own judgments.31 This approach has been advocated in healthcare, where it is referred to as evidence-based medicine. This community argues against trusting experts because they rely on anecdotal practices and out-ofdate and ineffective remedies. This takeaway message seeks to replace reliance on experts with faith in defined scientific studies. Clearly, empirical evaluation studies have great value, but we do not believe that such studies deserve uncritical acceptance. Witness how the evidence seems to change every two years. Witness also the difficulty of sorting out the evidence base for a patient with multiple medical problems. Clinicians must consider the individual patient, who may differ from the criteria on which the evidence is based. We seek a balance between scientific evidence and broad experience.32 One way that evidence is compiled is through checklists. These are valuable safety tools to prevent decision makers from omitting important steps in a process, but they are not decision-support tools. We believe that reducing complex judgments to simple checklists often misses essential aspects of decision making.33 Checklists work for stable, welldefined tasks, and have to be carefully crafted with a manageable number of steps. If the checklist is sequential, each step must lead to a clear outcome that serves as the trigger for the next step. However, in complex and ambiguous situations, the antecedent conditions for each step are likely 70 to be murky; expert decision makers must determine when to initiate the next step or whether to initiate it at all. Although checklists can be helpful, it is risky to have individuals use checklists for complex tasks that depend on considerable tacit knowledge to judge when a step is appropriate, how to modify a step, and how to decide whether the checklist is working.34 Experts must decide what to do when the best practices conflict with their own judgments. They must revise plans that do not seem to be working. It is one thing to hold physicians to task for relying on ineffective remedies and ignoring scientific evidence that the procedures that they were once taught that have since been shown ineffective, but it is another thing to compel physicians to rely on scientific evidence by proceduralizing clinical judgment in a checklist and penalize them for not following the steps. Guidelines, rules, and checklists raise the floor by preventing silly errors—mistakes that even a firstyear medical student might recognize as an error. But they also lower the ceiling, making it easy to shift to an unthinking, uncritical mode that misses subtle warning signs and does not serve the needs of patients. Finally, we come to the challenge from within computer science itself. The Challenge from Computer Science This challenge has been presented on three fronts: AI, big data, and automation. It is claimed that these technologies are smarter and more reliable than any human. Since experts are the gold standard of performance, demonstrations of smart technology win big when they beat out an expert. AI successes have been widely publicized. IBM’s Deep Blue beat Garry Kasparov, the reigning chess www.computer.org/intelligent champion at the time. IBM’s Watson beat a panel of experts at the game of Jeopardy. AlphaGo trounced one of the most highly regarded Go masters. These achievements have been interpreted as showing that AI can outperform humans at any cognitively challenging task. But the successes involve games that are well-structured, with unambiguous referents and definitive correct answers. In contrast, most decision makers face wicked problems with unclear goals in ambiguous and dynamic situations. Roger Schank, an AI pioneer, stated flatly that “Watson is a fraud.”35 He objected to IBM’s claims that Watson could outthink humans and find insights within large datasets. Although Watson excels at keyword searches, it does not consider the context of the passages it is searching, and as a result is insensitive to underlying messages in the material. Schank’s position is that counting words is not the same as inferring insightful conclusions. Our experience is that AI developers have much greater appreciation for human expertise than the AI popularizers. A good example of the challenge to expertise comes from the weather forecasting domain. Articles with titles such as “All Hail the Computer!”36 promulgate the myth that if more memory and faster processing speeds could be thrown at the problem, the need for humans would evaporate. Starting in the late 1980s, as more computer models were introduced into operational forecasting, prognostications were made that computer models would outperform humans within the next 10 years—for example, “[The] human’s advantage over the computer may eventually be swamped by the vastly increased number crunching ability of the computer ... as the computer driven models will simply get bigger IEEE INTELLIGENT SYSTEMS and better.”37 Articles in the scientific literature as well as the popular press continue to present the stance of human versus machine, asking whether “machines are taking over.”36 This stance conveys a counterproductive attitude of competition in which the experts cannot beat the computers. A more productive approach would be to design technologies that enhance human performance. The evidence clearly shows that the expert weather forecaster adds value to the outputs of the computer models. Furthermore, “numerical prediction models do not produce a weather forecast. They produce a form of guidance that can help a human being decide upon a forecast of the weather.”38,39 Next, we turn to the denigration of expertise that has been expressed by advocates of big data analytics. Despite their widely publicized successes, a closer look often tells a different story. For instance, Google’s FluTrends project initially seemed successful at predicting flu outbreaks, but over time it misled public health planners.40 Advocates of big data claim that the algorithms can detect trends, spot problems, and generate inferences and insights; no human, no matter how expert, could possibly sift through all of the available sensor data; and no human can hope to interpret even a fraction of these data sources. These statements are all true. But the big data community wants to reduce our trust in domain experts so decision makers become comfortable using automated big data analyses. Here is a typical and dangerous claim: “The big target here isn’t advertising, though. It’s science … faced with massive data, this approach to science—hypothesize, model, test—is becoming obsolete … Petabytes allow us to say: Correlation is enough. We can stop looking for models.”41 NOVEMBER/DECEMBER 2017 A balanced view recognizes that big data analytics can identify patterns where none exist. Big data algorithms can follow historical trends but might miss departures from these trends, as in the broken leg cues, cues that have implications that are clear to experts but aren’t part of the algorithms. Further, experts can use expectancies to spot missing events that may be highly significant. In contrast, big data approaches, which crunch the signals received from a variety of sources, are unaware of the absence of data and events. Finally, we consider the challenge offered by proponents of automation. Some researchers in the automation community have promulgated the myth that more automation can obviate the need for humans, including experts. The enthusiasm for technologies is often extreme.42 Too many technologists believe that automation can compensate for human limitations and substitute for humans. They also believe the myth that tasks can be cleanly allocated to either the human or the machine. These misleading beliefs have been questioned by cognitive systems engineers for more than 35 years, yet the debunking has to be periodically refreshed in the minds of researchers and program managers.43 The misleading beliefs persist because of the promissory note that more automation means fewer people, fewer people means fewer errors, and (especially) fewer people means reduced costs.44 Nearly every funding program that calls for more automation is premised with the claim that the introduction of automation will entail a need for fewer expert operators at potentially lower cost to the organization. But the facts are in plain view: The introduction often requires more experts. Case studies 45 show that automation creates new kinds of cognitive work www.computer.org/intelligent for the operator, often at the wrong times. Automation often requires people to do more, to do it faster, or to do it in more complex ways. The explosion of features, options, and modes often creates new demands, new types of errors, and new paths toward failure. Ironically, as these facts became apparent, decision makers seek additional automation to compensate for the problems triggered by the automation.44 We see technology—AI, big data, and automation—continuing to improve, which will make computers ever more valuable tools. But in the spirit of human-centered computing, we define intelligent systems as human-machine systems that amplify and extend human abilities.46 The technology in such work systems is designed to improve human performance and accelerate the achievement of expertise. We hope that expertise-deniers can get past the mindset of trying to build systems to replace the experts and instead seek to build useful technologies that empower experts. If the challenges to expertise hold sway, the result might be degradation of the decision making and resilience of organizations and agencies. Once such organizations accept the expertise-deniers’ arguments, they may sideline domain experts in favor of statistical analysts and ever more automation. They are likely to divert funding from training programs that might produce experts into technology that makes decisions without human intervention or responsible action. Shifting cognitive work over to automation may deskill workers, erode the expertise that is crucial for adaptability, and lead to a downward spiral of diminishing expertise. Experts are certainly not perfect, so the challenges can be useful for increasing our understanding of the 71 boundary conditions of expertise. We do not want to return to an era where medicine was governed by anecdote rather than data—we think it essential to draw from evidence and from expertise. We appreciate the discoveries of the heuristics and biases researchers—the heuristics they have uncovered can have great value for fostering speculative thinking. We respect the judgment and decision research community—we want to take advantage of their efforts to improve the way we handle evidence and deploy our intuitions. We want to productively move forward with improved information technology—we want these tools to be designed to help us gain and enhance expertise. We value the social aspects of work settings—we want to design work settings and team arrangements that magnify expertise. Our hope is to encourage a balance that respects expertise while designing new ways to strengthen it. We regard the design of cognitive work systems as the design of humanmachine interdependencies, guided by the desire to make the machines comprehensible, predictable, and controllable. This course of action seems best suited to promote human welfare and enable greater achievements.47,48 Acknowledgments We thank Bonnie Dorr, Hal Daume, Jonathan Lazar, Jim Hendler, Mark Smith, and Jenny Preece for their comments on a draft of this article; and Jan Maarten Schraagen and Paul Ward for their helpful comments and suggestions and for their patience and encouragement. This essay was adapted from a longer and more indepth account, “The War on Experts,” that will appear in the Oxford Handbook of Expertise. 49 References 1. K.A. Ericsson et al., Cambridge Handbook of Expertise and Expert 72 Performance, 2nd ed., Cambridge Univ. Press, 2017. 2. B. Shneiderman and G. Klein, “Tools That Aid Expert Decision Making: Supporting Frontier Thinking, Social Engagement and Responsibility,” blog, Psychology Today, Mar. 2017; www .psychologytoday.com/blog/seeing-what -others-dont/201703/tools-aid-expert -decision-making-rather-degrade-it. 3. T. Nichols, The Death of Expertise, Oxford Univ. Press, 2017. 4. R. Dawes, “The Robust Beauty of Improper Linear Models,” American Psychologist, vol. 34, no. 7, 1979, pp. 571–582. 5. P.E. Meehl, “Seer Over Sign: The First Good Example,” J. Experimental Research in Personality, vol. 1, no. 1, 1965, pp. 27–32. 6. R. Hastie and R. Dawes, Rational Choice in an Uncertain World, Sage Publications, 2001. 7. K. Salzinger, “Clinical, Statistical, and Broken-Leg Predictions,” Behavior and Philosophy, vol. 33, 2005, pp. 91–99. 8. R. Johnston, Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study, Center for the Study of Intelligence, Washington, DC, 2005. 9. H.J. Einhorn and R.M. Hogarth, “Judging Probable Cause,” Psychological Bull., vol. 99, no. 1, 1978, pp. 3–19. 10. D. Kahneman and A. Tversky, “Prospect Theory: An Analysis of Decision under Risk, Econometrica, vol. 47, no. 2, 1979, pp. 263–291. 11. D.W. Cheng et al., “Pragmatic Versus Syntactic Approaches to Training Deductive Reasoning,” Cognitive Psychology, vol. 18, no. 3, 1986, pp. 293–328. 12. R. Hertwig and G. Gigerenzer, “The ‘Conjunction Fallacy’ Revisited: How Intelligent Inferences Look like Reasoning Errors,” J. Behavioral Decision Making, vol. 12, no. 2, 1999, pp. 27–305. 13. B. Fischhoff, “Eliciting Knowledge for Analytical Representation,” IEEE Trans. Systems, Man, and Cybernetics, vol. 19, no. 3, 1989, pp. 448–461. www.computer.org/intelligent 14. M.D. Shields, I. Solomon, and W.S. Waller, “Effects of Alternative Sample Space Representations on the Accuracy of Auditors’ Uncertainty Judgments,” Accounting, Organizations, and Society, vol. 12, no. 4, 1987, pp. 375–385. 15. G. Klein, R. Calderwood, and A. ClintonCirocco, “Rapid Decision Making on the Fire Ground,” Proc. Human Factors and Ergonomics Soc. Ann. Meeting, vol. 30, no. 6, 1986, pp. 576–580. 16. A. Tversky and D. Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” Science, vol. 185, Sept. 1974, pp. 1124–1131. 17. J. Evetts, “Professionalism: Value and Ideology,” Current Sociology, vol. 61, no. 5–6, 2013, pp. 778–779. 18. H.M. Collins, Changing Order. Replication and Induction in Scientific Practice, 2nd ed., Univ. of Chicago Press, 1992. 19. B. Latour and S. Woolgar, Laboratory Life. The Social Construction of Scientific Facts, Sage Publications, 1979. 20. M. Lynch, Scientific Practice and Ordinary Action, Cambridge Univ. Press, 1993. 21. K.D. Knorr-Cetina, The Manufacture of Knowledge, Pergamon Press, 1981. 22. J. Lave, “Situating Learning in Communities of Practice,” Perspectives on Socially Shared Cognition, L.B. Resnick, J.M. Levine, and S.D. Teasley, eds., American Psychological Assoc., 1993, pp. 63–82. 23. L. Suchman, Plans and Situated Actions: The Problem of Human-Machine Communication, Cambridge Univ. Press, 1987. 24. E. Wenger, Communities of Practice: Learning, Meaning, & Identity, Cambridge Univ. Press, 1998. 25. M.S. Weldon, “Remembering as a Social Process,” The Psychology of Learning and Motivation, vol. 40, no. 1, 2000, pp. 67–120. 26. G.A. Miller, “Dismembering Cognition,” One Hundred Years of Psychological Research in America, IEEE INTELLIGENT SYSTEMS Johns Hopkins Univ. Press, 1986, pp. 277–298. 27. N.M. Agnew, K.M. Ford, and P.J. Hayes, “Expertise in Context: Personally Constructed, Socially Selected and Reality-Relevant?” Int’l J. Expert Systems, vol. 7, no. 1, 1994, pp. 65–88. 28. G. Klein et al., “Macrocognition,” IEEE Intelligent Systems, vol. 18, no. 3, 2003, pp. 81–85. 29. H.M. Collins, “A Sociological/ Philosophical Perspective on Expertise: The Acquisition of Expertise Through Socialization,” Cambridge Handbook of Expertise and Expert Performance, 2nd ed., K.A. Ericsson et al., Cambridge Univ. Press, 2017. 30. H.A. Mieg, “Social and Sociological Factors in the Development of Expertise,” Cambridge Handbook of Expertise and Expert Performance, K.A. Ericsson et al., Cambridge Univ. Press, 2006, pp. 743–760. 31. A.R. Roberts and K.R. Yeager, eds., Evidence-Based Practice Manual: Research and Outcome Measures in Health and Human Services, Oxford Univ. Press, 2004. 32. E. Barends, D.M. Rousseau, and R.B. Briner, Evidence-Based Management: The Basic Principles, Center for Evidence-Based Management, Amsterdam, 2014. 33. R.L. Wears and G. Klein, “The Rush from Judgment,” Annals of Emergency Medicine, forthcoming, 2017. 34. D.E. Klein et al., “Can We Trust Best Practices? Six Cognitive Challenges of Evidence-Based Approaches,” J. Cognitive Eng. and Decision Making, vol. 10, no. 3, 2016, pp. 244–254. 35. R. Schank, “The Fraudulent Claims Made by IBM about Watson and AI,” 2015; www.rogerschank.com /fraudulent-claims-made-by-IBM -about-Watson-and-AI. 36. R.A. Kerr, “Weather Forecasts Slowly Clearing Up,” Science, vol. 38, no. 388, 2012, pp. 734–737. NOVEMBER/DECEMBER 2017 37. P.S. Targett, “Predicting the Future of the Meteorologist: A Forecaster’s View,” Bull. Australian Meteorological and Oceanographic Soc., vol. 7, no. 1, 1994, pp. 46–52. 38. H.E. Brooks, C.A. Doswell, and R.A. Maddox, “On the Use of Mesoscale and Cloud-Scale Models in Operational Forecasting,” Weather and Forecasting, vol. 7, Mar. 1992, pp. 120–132. 39. R.R. Hoffman et al., Minding the Weather: How Expert Forecasters Think, MIT Press, 2017. 40. D. Lazer et al., “The Parable of Google Flu: Traps in the Big Data Analysis,” Science, vol. 343, 14 Mar. 2014, pp. 1203–1205. 41. C. Anderson, “The End of Theory: The Big Data Deluge Makes the Scientific Method Obsolete,” Wired, 23 June 2008; www.wired.com/2008/06 /pb-theory. 42. E. Brynjolfsson and A. McAfee, The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies, W.W. Norton & Co., 2014. 43. J.M. Bradshaw et al., “The Seven Deadly Myths of ‘Autonomous Systems’,” IEEE Intelligent Systems, vol. 28, no. 3, 2013, pp. 54–61. 44. “Technical Assessment: Autonomy,” report from the Office of the Assistant Secretary of Defense for Research and Engineering, Office of Technical Intelligence, US Department of Defense, 2015. 45. R.R. Hoffman, T.M. Cullen, and J.K. Hawley, “Rhetoric and Reality of Autonomous Weapons: Getting a Grip on the Myths and Costs of Automation,” Bull. Atomic Scientists, vol. 72, no. 4, 2016; doi:10.1080/00963402.2016.1194619. 46. J.M. Johnson et al., “Beyond Cooperative Robotics: The Central Role of Interdependence in Coactive Design,” IEEE Intelligent Systems, vol. 26, no. 3, 2011, pp. 81–88. 47. M. Johnson et al., “Seven Cardinal Virtues of Human-Machine Teamwork,” IEEE Intelligent Systems, vol. 29, no. 6, 2014, pp. 74–79. www.computer.org/intelligent 48. G. Klein et al., “Ten Challenges for Making Automation a ‘Team Player’ in Joint Human-Agent Activity,” IEEE Intelligent Systems, vol. 19, no. 6, 2004, pp. 91–95. 49. P. Ward et al., eds., Oxford Handbook of Expertise, Oxford Univ. Press, forthcoming. Gary Klein is senior scientist at MacroCognition LLC. His research interests include naturalistic decision making. Klein received his PhD in experimental psychology from the University of Pittsburgh. Contact him at [email protected]. Ben Shneiderman is distinguished university professor in the Department of Computer Science at the University of Maryland. His research interests include human-computer interaction, user experience design, and information visualization. Shneiderman has a PhD in computer science from SUNY-Stony Brook. Contact him at [email protected]. Robert R. Hoffman is a senior research scientist at the Institute for Human and Machine Cognition. His research interests include macrocognition and complex cognitive systems. Hoffman has PhD in experimental psychology from the University of Cincinnati. He is a fellow of the Association for Psychological Science and the Human Factors and Ergonomics Society and a senior member of IEEE. Contact him at rhoffman@ ihmc.us. Kenneth M. Ford is director of the Insti- tute for Human and Machine Cognition. His research interests include artificial intelligence and human-centered computing. Ford has a PhD in computer science from Tulane University. Contact him at kford@ ihmc.us Read your subscriptions through the myCS publications portal at http://mycs.computer.org 73