Case Study - Methodologies
Case Study - Methodologies
Case Study - Methodologies
net/publication/263620395
Using the Case Survey Method for Synthesizing Case Study Evidence in
Information Systems Research Completed Research Paper
CITATIONS READS
10 1,455
3 authors, including:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Marlen Jurisch on 04 July 2014.
ABSTRACT
A common characteristic of the IS discipline is that the bulk of the empirical evidence is embodied in case studies. However,
the ever-growing body of case based evidence also constitutes a major challenge to the IS discipline. Although each case
study may provide rich insights into specific phenomena, it is difficult to generalize on the basis of single-N or small-N case
studies. What IS research would benefit from is a method that allows for the quantitative inquiry of the vast amount of
primarily qualitative case studies. The purpose of this paper is to introduce the case survey method as new mode of inquiry to
supplement the rich repertoire of IS review methods. Therefore, we show how the case survey method is embedded in the
landscape of review methods used in IS research and what its principal stages, techniques, limitations and potentials are.
Keywords
INTRODUCTION
A common characteristic of the information systems (IS) discipline is that the bulk of the empirical evidence is embodied in
case studies (Alavi & Carlson, 1992). However, the ever-growing body of case based evidence constitutes a major challenge
to the IS discipline. Although each case study may provide rich insights into specific phenomena, it is difficult to generalize
on the basis of single-N or small-N case studies (Darke & Shanks, 1998; Newig & Fritsch, 2009). Large-N case studies
would be a better choice, but due to high complexity case studies are rarely framed as large-N studies (Larsson, 1993;
Piekkari & Welch, 2011). The main crux is not that IS research is lacking knowledge but rather that this knowledge lies
distributed over countless single-N and small-N case studies. IS research has progressed to the point that there is a need for
identifying the ‘cement that glues’ these unique case studies (Stall-Meadows & Hyle, 2010).
The case survey method presents a powerful approach for identifying and statistically testing patterns across case studies
(Larsson, 1993; Lucas, 1974). Case surveys draw on the richness of numerous case studies and therefore allow for wider
generalizations than single-N and small-N case studies (Larsson, 1993). The bulk of IS case studies presents a rich pool of
relevant empirical findings (Eisenhardt & Graebner, 2007; Myers & Avison, 1997) whose “individually limited scientific
contributions can be enhanced through systematic analysis of patterns across cases” (Larsson, 1993). The case survey method
holds the following four potentials for IS research: (1) allows IS researchers to tap the vast experiences enclosed in IS case
studies, (2) provides an approach for synthesizing qualitative data into quantitative results, (3) helps in answering some basic
questions in IS research and (4) supports to establish summative validity for some of the theories developed or extended in IS
case studies.
The paper is organized as follows. Section 2 provides a brief summary of the various review methods used in IS research for
synthesizing data across research studies and shows how the case survey method fits into this landscape. Section 3 specifies
the stages and techniques of the case survey method. In section 5, we discuss the limitations and in section 5 its potentials for
IS research.
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 1
Jurisch et al. Case Survey Method for Information Systems Research
METHODS USED IN INFORMATION SYSTEMS FOR SYNTHESIZING DATA ACROSS RESEARCH STUDIES
Researchers have a number of methods at their disposal for synthesizing data across research studies (King, 2005). These
review methods allow for the aggregation of existing research findings with respect to their historical content and the analysis
of contradictions that might exist between the primary studies (Rumrill & Fitzgerald, 2001). To provide an overview, we
compiled a profile of the various qualitative, quantitative and mixed methods used in IS research to synthesize data across
studies (see figure 1).
(Statistical)
Vote Counting Meta-Analysis
Quantitative
Other review
Stylized Facts
Data 2 3 methods
Sources 1 4 Focus of this
paper
Qualitative Content
Analysis
Case Survey
Qualitative
Descriptive
Narrative Analysis Content Analysis
Qualitative Quantitative
Type of
Results
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 2
Jurisch et al. Case Survey Method for Information Systems Research
What IS research currently misses is a method with which one could transform qualitative case based evidence into
quantifiable results (e.g., test hypothesis or extent existing theories). The case survey method could fill this gap. The case
survey method allows to convert qualitative case study data into statistically analyzable quantitative data, using a coding
scheme and expert judgments by multiple coders (Larsson, 1993; Lucas, 1974; Yin & Heald, 1975). In the past five decades,
the case survey method was successfully employed and developed further in political science (Lucas, 1974; Yin & Heald,
1975), management science (Larsson, 1993) and public administration (Jensen & Rodgers, 2001). Applications of the method
in these disciplines have proven that case surveys are powerful and rigorous.
Case study sourcing Criteria for case Scanning of Rejection Case study
selection literature criteria sample
Final output
Cumulation of case study
Theory development / extension
knowledge
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 3
Jurisch et al. Case Survey Method for Information Systems Research
capabilities matter for successful business process change?”. Notwithstanding the type of research question posed, it appears
essential to thoroughly reflect on the type of knowledge one seeks to gain from the analysis of patterns across case studies
(Newig & Fritsch, 2009).
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 4
Jurisch et al. Case Survey Method for Information Systems Research
One has to be aware, however, that more complex and detailed coding schemes may lead to lower interrater-reliability.
Larsson (1993) posits that the limit on how much complexity a case survey can capture is reached whenever interrater-
reliability becomes unacceptable. But the only way to identify this point is to start with a more comprehensive coding scheme
and to simplify it throughout several pretests until the level of reliability is acceptable (Larsson, 1993).
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 5
Jurisch et al. Case Survey Method for Information Systems Research
case survey had a sample size of 33 case studies (Bullock & Tubbs, 1990). Hence, further specification on the minimum
sample size is needed.
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 6
Jurisch et al. Case Survey Method for Information Systems Research
REFERENCES
1. Alavi, M., & Carlson, P. (1992). A review of MIS research and disciplinary development. Journal of Management
Information Systems, 8(4), 45–62.
2. Benbasat, B. I., Goldstein, D. K., & Mead, M. (1987). The Case Research Strategy in Studies of Information Systems
Case Research: Definition. MIS Quarterly, September, 369–387.
3. Benbasat, I., & Zmud, R. W. (2003). The identity crisis within the IS discipline: Defining and communicating the
discipline’s core properties. MIS Quarterly, 27(2), 183–194.
4. Bence, V. (1995). St James’s Hospital and Lucas Engineering Systems. Business Change and Re-engineering, 2(3), 30–
39.
5. Bullock, R. J. (1986). A Meta-Analysis Method for OD Case Studies. Group & Organization Management, 11(1-2), 33–
48.
6. Bullock, R. J., & Svyantek, D. J. (1985). Analyzing meta-analysis: Potential problems, an unsuccessful replication, and
evaluation criteria. Journal of Applied Psychology, 70(1), 108–115.
7. Bullock, R. J., & Tubbs, M. E. (1990). A case meta-analysis of gainsharing plans as organization development
interventions. The Journal of Applied Behavioral Science, 26(3), 383–404.
8. Carugati, A., & Rossignoli, C. (2011). Emerging Themes in Information Systems and Organization Studies (p. 372).
Heidelberg Dodrecht London: Springer Verlag.
9. Chen, W., & Hirschheim, R. (2004). A paradigmatic and methodological examination of information systems research
from 1991 to 2001. Information Systems Journal, 14, 197–235.
10. Darke, P., & Shanks, G. (1998). Successfully completing case study research: combining rigour, relevance and
pragmatism. Information Systems Journal, 8(4), 273–289.
11. Doolin, B. (1996). Alternative views of case research in information systems. Australasian Journal of Information
Systems, 3(1), 21–29.
12. Dubé, L., & Paré, G. (2003). Rigor in information systems positivist case research: current practices, trends, and
recommendations. MIS Quarterly, 27(4), 597–635.
13. Eisenhardt, K. M., & Graebner, M. E. (2007). Theory Building from Cases: Opportunities and Challenges. The Academy
of Management Journal, 50(1), 25.
14. Fichman, R. G. (2004). Real Options and IT Platform Adoption: Implications for Theory and Practice. Information
Systems Research, 15(2), 132–154.
15. Houy, C., Fettke, P., & Loos, P. (2011). Stilisierte Fakten in der gestaltungsorientierten Wirtschaftsinformatik –
Allgemeine Potentiale und erste Erfahrungen. Wirtschaftsinformatik (pp. 1157–1166). Zürich.
16. Hughes, M., & Golden, W. (2001). Intranet Technology as as Enabler of BPR: An Exploratory Study in Public
Healthcare. ECIS 2001 Proceedings (pp. 1149–1158). Bled, Slovenia.
17. Jauch, L. R., Osborn, R. N., & Martin, T. N. (1980). Structured Content Analysis of Cases: A Complementary Method
for Organizational Research. The Academy of Management Review, 5(4), 517.
18. Jensen, J., & Rodgers, R. (2001). Cumulating the intellectual gold of case study research. Public Administration Review,
61(2), 235–246.
19. Jurisch, M. C., Palka, W., Wolf, P., & Krcmar, H. (2013). Which capabilities matter for successful busines process
change? Business Process Management Journal, to appear.
20. King, W. R. (2005). Understanding the role and methods of meta-analysis in IS research. Communications of the
Association for Information Systems, 16, 665–686.
21. Kohler Riessman, C. (2005). Narrative Analysis. Narrative, Memory & Everyday Life (pp. 1–7). Huddersfield:
University of Huddersfield.
22. Krippendorf, K. (1980). Content Analysis. An Introduction to its Methodology. Beverly Hills: Sage Publications.
23. Lacity, M. C., Solomon, S., Aihua, Y., & Willcocks, L. P. (2011). Business process outsourcing studies: a critical review
and research directions. Journal of Information Technology, 26(4), 221–258.
24. Laitenberger, O., El Emam, K., & Habrich, T. G. (2001). An Internally Replicated Quasi-Experimental Comparison of
Checklist and Perspective- Based Reading of Code Documents. IEEE Transactions on Software Engineering, 27(5),
387–421.
25. Larsson, R. (1993). Case Survey Methodology: Quantitative Analysis of Patterns across Case Studies. The Academy of
Management Journal, 36(6), 1515–1546.
26. Larsson, R., & Finkelstein, S. (1999). Integrating strategic, organizational, and human resource perspectives on mergers
and acquisitions: A case survey of synergy realization. Organization Science, 10(1).
27. Lee, A. S., & Hubona, G. S. (2009). A scientific basis for rigor in information systems research. MIS Quarterly, 33(2),
237–262.
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 7
Jurisch et al. Case Survey Method for Information Systems Research
28. Levina, N., & Ross, J. W. (2003). From the vendor’s perspective: exploring the value proposition in information
technology outsourcing. MIS Quarterly, 27(3), 331–364.
29. Loos, P., Fettke, P., Weißenberger, B. E., Zelewski, S., Heinzl, A., Frank, U., & Iivari, J. (2011). What in Fact is the
Role of Stylized Facts in Fundamental Research of Business and Information Systems Engineering? Business &
Information Systems Engineering, 3(2), 107–125.
30. Lucas, W. A. (1974). The case survey method: Aggregating case experience. Santa Monica, CA: Rand Corporation.
31. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: an expanded sourcebook (p. 337). London: Sage.
32. Myers, M. D., & Avison, D. (1997). Qualitative research in information systems. Management Information Systems
Quarterly, 21(June), 241–242.
33. Newig, J., & Fritsch, O. (2009). The case survey method and applications in political science. Political Science
Association [APSA) 2009, 49(September), 3–6.
34. Piekkari, R., & Welch, C. (2011). Rethinking the Case Study in International Business and Management Research (p.
557). Northampton, MA: Edward Elgar.
35. Provan, K. G., & Milward, H. B. (1995). A Preliminary Theory of Interorganizational Network Effectiveness : A
Comparative Study of Four Connmunity Mental Health Systems. Administrative Science Quarterly, 40(1), 1–33.
36. Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: recent developments in quantitative methods for literature
reviews. Annual review of psychology, 52, 59–82.
37. Rumrill Jr, P. D., & Fitzgerald, S. M. (2001). Using narrative literature reviews to build a scientific knowledge base.
Work: A Journal of Prevention, Assessment and Rehabilitation, 16(2), 165–170.
38. Smith, C. P. (2000). Content Analysis and Narrative Analysis. In H. Reis & P. Shaver (Eds.), Handbook of research
methods in social and personality psychology (pp. 313–335). New York: Academic Press.
39. Srnka, K., & Koeszegi, S. (2007). From words to numbers: how to transform qualitative data into meaningful
quantitative results. SBR, 59(January), 29–58.
40. Stahl, G., & Kremershof, I. (2004). Trust dynamics in mergers and acquisitions: A case survey. Academy of
Management. Singapore.
41. Stall-Meadows, C., & Hyle, A. (2010). Procedural methodology for a grounded meta-analysis of qualitative case studies.
International Journal of Consumer Studies, 34(4), 412–418.
42. Stanley, T. (2001). Wheat from chaff: Meta-analysis as quantitative literature review. The Journal of Economic
Perspectives, 15(3), 131–150.
43. Weber, R. P. (1990). Basic Content Analysis (p. 97). Newbury Park, CA: Sage Publications.
44. Whiston, S. C., & Li, P. (2011). Meta-Analysis: A Systematic Method for Synthesizing Counseling Research. Journal of
Counseling & Development, 89(3), 273–281.
45. Yin, R.K., & Heald, K. . (1975). Using the case survey method to analyze policy studies. Administrative Science
Quarterly, 20, 371–381.
46. Yin, R. K, & Heald, K. A. (1975). Using the Case Survey Method to Analyze Policy Studies. Administrative Science
Quarterly, 20(September), 371–382.
Proceedings of the Nineteenth Americas Conference on Information Systems, Chicago, Illinois, August 15-17, 2013. 8