Methodologies-theories-praxis

Schneider, Carsten Q. and Claudius Wagemann. 2012. Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University Press. Methodologies—Theories—Praxis https://doi.org/10.5281/zenodo.910219 Joachim Blatter University of Lucerne, Switzerland [email protected] Markus Haverland Erasmus University, Rotterdam [email protected] We are grateful for the thoughtful discussion of our book by Goetz and Mahoney and the valuable remarks of other contributors. In our final statement we want to address three points. First, we would like to revisit our distinction between three approaches to explanatory case studies. This is triggered by Goertz and Mahoney’s statement that our congruence approach usually requires elements of one of our two other approaches, the covariational approach and causal-process tracing. Secondly, in a response to comments by Rohlfing we want to briefly clarify what we mean by a co-variational approach to explanatory small-N research. Finally, we would like to take up the notion of Goertz and Mahoney that we are witnessing a third wave of qualitative methodology. We will argue that work of this new wave is particularly promising when it comes to closing the gap between those interested in the methodological per se and those looking for concrete guidelines and advice for actually conducting qualitative research. Goertz and Mahoney address our distinction between three approaches to explanatory case study research. They argue that to achieve the goal of congruence analysis, which is the evaluation of the explanatory power of alternative theories, we usually need to draw either on the covariational approach or on causal process tracing. We would like to stress that we take a broader and more pluralistic epistemological perspective on theory development in the social sciences than the other books and this implies that congruence analysis is not just a mix between the other two approaches. When dealing with theories in the context of congruence analysis, we have theories in mind that operate on a higher level of abstractions than theories (or explanations) that are typically utilized in research adhering to the co-variational approach and or causal-process tracing. These theories are often explicitly linked to paradigms, they are not oriented towards a specified population, and they often embody propositions about constitutive concepts. Such theories are for instance prominent in International Relations (think of rationalism and social constructivism) and they often link empirical research to political philosophy. The co-variational approach has more affinities with what could be called “empiricist” theories. This understanding of theories fits the view presented in the books of Goertz and Mahoney, Rohlfing, and Schneider and Wagemann. Empiricist Qualitative & Multi-Method Research, Spring 2013 theories operate on a lower level of abstraction and apply to large or medium-sized, more or less clearly delineated populations of similar cases. These theories are for instance prominent in Comparative Politics. The theories are represented by variables that operate on one or a few levels of analysis. Compare the corresponding “data set observations” with the diverse and non-standardized observations yielded in congruence analysis. Moreover, while co-variational analysis heavily relies on comparisons across cases, congruence analysis is based on comparisons of bundles of empirical observations with predictions and propositions deduced from multiple theories. Causal-process tracing has affinities to mechanism-oriented approaches where theories operate on a low level of abstraction, focusing on a single case or small populations. This concept of theory is more akin to Beach and Pedersen’s view. Note also that crucial to our causal-process tracing approach is the very idea of processes. Although in congruence analysis propositions and predictions can concern processes, it is by no means necessary to do so. In short, we would argue that congruence analysis is sufficiently distinct from both covariational analysis and causal-process tracing to merit a separate treatment. Our second point relates to Rohlfing’s comment that we “misrepresent” the co-variational approach and that we do not discuss Bayes’ theorem. Perhaps we could have been more clear from the outset than when we talk about the co-variational approach, we do not talk about a general approach to the social sciences. Rather our co-variational approach is a very specific approach that is (a) tailored to a small-N setting; (b) informed by the experimental template; and (c) has the goal of causal inference. It draws on what Lijphart has described as “the comparative method” (1971). We believe that if we had called this approach “the comparative method,” it would have raised more eyebrows. Bayes theorem is indeed not part of this approach to explanatory case study research. But it is actually discussed in the context of our congruence analysis approach (Blatter and Haverland 2012: 176–177,194). Our final point moves somewhat beyond the confines of methodology and focuses on the extent to which the principles and guidelines we are discussing will trickle down to the practice of case study research. Goertz and Mahoney have subsumed the books of this symposium under what they call the “third wave of qualitative methodology.” This wave succeeds the work on (comparative) case studies in the 1960s and 70s, such as Lijphart’s article mentioned above, and the responses to the attempt by King, Keohane, and Verba’s Designing Social Inquiry (1994) to fit qualitative research into the statistical template, such as Brady and Collier’s edited volume Rethinking Social Inquiry (2004). We believe that generally speaking, the third wave of qualitative methodology as represented in this symposium does more than the first two waves to contribute to closing the gap between methodological discourse on the one hand, and concrete guidelines and advices for those primarily interested in conducting case studies on the other hand. For one, the third wave as represented here consists of 23 Qualitative & Multi-Method Research, Spring 2013 monographs. This very format helps to put into practice what we preach. Although many scholars as well as better graduate students might be able to synthesize the disparate and articlelength methodological pieces, a common format of the first two waves, into a coherent approach for their own work, many others will fare well by book-length treatments written by one or two scholars. Such a format allows for in-depth treatment, integrated accounts, and consistency in methodological language. Second, all authors make ample use of published real world research to illustrate their arguments. This is an important step forward. As users of methodological advice we were often discouraged by the artificial character of the hypothetical examples with which methodologists accompanied their ideas. Benefitting to some extent from the achievements of the first and second wave of qualitative methodological thinking, we and the other contributors were able to draw on an emerging set of best practices. Finally, some books take additional efforts to get their message across to practitioners of case study research. Schneider and Wagemann’s book on set-theoretic methods offers features like an “easy reading guide,” a “how-to section,” and a link to online learning material. Beach and Pedersen’s book on process tracing provide a lot of useful advice regarding the often neglected topic of data collection as well as a user-friendly checklist at the end of the book. In our own book we tried to mirror the research process as closely as possible, starting from research goal and questions and concluding with modes of generalization and the format of presentation. These features of the third wave should help to pass what should be the litmus test of any methodological debate: whether the principles and guidelines we are preaching will result in an improved practice of case study research. References Blatter, Joachim and Markus Haverland. 2012. Designing Case Studies: Explanatory Approaches in Small-N Research. Houndsmills, Basingstoke: Palgrave Macmillan. Brady, Henry and David Collier, eds. 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, MD: Rowman and Littlefield. King, Gary, Robert O. Keohane and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press. Lijphart, Arend. 1971. “Comparative Politics and the Comparative Method.” American Political Science Review 65:3, 682–693. 24 Principles and Practice of Social Science Methods Reloaded Ingo Rohlfing University of Cologne, Germany [email protected] The central theme of my first contribution to the symposium is the distinction between the practice and principles of social science methods (or, in the terminology of Two Cultures (chap. 1), typical practice vs. possible and best practice). The existing discussions of Two Cultures, including those in the recent symposium in Comparative Political Studies (Goertz and Mahoney 2013), emphasize the salience of this distinction for two reasons that I focus on in the following. First, I need to correct Goertz and Mahoney’s (GM) potentially misleading characterization of the way in which I discuss principles of case selection in qualitative research in Case Studies and Causal Inference (CSCI). Second, in light of GM’s contribution to this symposium, I should clarify and reiterate what I agree and disagree with regarding Two Cultures. Case Selection in Qualitative Research GM assert about CSCI that it “also follows a long tradition in discussing types of case studies in light of statistical results; for case selection, these results help to identify typical cases, deviate cases, extreme cases, and so. Hence, much of his discussion of multimethod research has quantitative analysis in the leading role and the case study cast as a supporting actor.” This statement is potentially misleading for multiple reasons. First, GM might appear to infer from a discussion of types such as the typical case or the deviant case that one is engaged with case selection on the basis of statistical results. This ignores that these types also figure in set-relational multimethod research (Schneider and Rohlfing, forthcoming), which is largely disregarded in Two Cultures and chapter 9 on multimethod research in particular. More importantly, GM ignore here the fact that typical cases, deviate cases, and most-likely and least-likely cases are established types of cases in the qualitative literature. They were devised by Eckstein (1975) and Lijphart (1971), decades before the advent of multi-method research. They thus have a much longer tradition in qualitative research than in multi-method studies. In fact, then, multimethod research of the correlational and set-relational type borrowed these types from the case study literature. The salience of these types of cases for qualitative research is additionally attested to by discussions in other publications, including George and Bennett (2005) and Beach and Pedersen (2013). Second, readers of this newsletter for whom “statistical analysis” specifically calls to mind regression analysis (as it does for me due to GM’s reference to Gerring’s [2007] largely regression-based discussion of case selection) would be misled by the claim that “quantitative analysis [is] in the leading