Enhancing Computing Education in India: A Design Story

UEF//eRepository DSpace Artikkelit https://erepo.uef.fi Luonnontieteiden ja metsätieteiden tiedekunta 2017 Enhancing Computing Education in India: A Design Story Kode Sandhya info:eu-repo/semantics/article info:eu-repo/semantics/acceptedVersion © Authors All rights reserved http://dx.doi.org/10.1109/LaTiCE.2017.21 https://erepo.uef.fi/handle/123456789/5182 Downloaded from University of Eastern Finland's eRepository Enhancing Computing Education in India: a design story Sandhya Kode EnhanceEdu, IIIT Hyderabad Hyderabad, India email: [email protected] Erkki Sutinen Department of Future Technologies, University of Turku, Turku, Finland email: [email protected] Abstract: In the states of Andhra Pradesh and Telangana also teachers and their teaching. While there are several national teacher training programs in India, like the large scale virtual classroom training T10kT [2], National Institute for teacher training and research NITTTR [3] and UGC Academic Staff colleges [4], we are not aware of any reports of Level 3, [Felder, 5] i.e. evaluating effects on students’ learning as a result of teacher training, of these approaches. Felder and Brent say it is very difficult to measure at Level 3, even in their successful NETI implementation in the US [5]. Gibbs and Coffey [6], Ho et al. [7] came close as they measured students’ approaches to studying indirectly, and came up with the results that deep learning approaches helped students to display improved learning outcomes compared with using surface learning methods. In our research study, we have addressed Level 3 evaluation of our teacher training program (TTP) by design. The design includes teacher training followed by student training implementation by these trained teachers, including progress monitoring with formative and summative assessments. But, can trained teachers effect changes? Are they empowered? Here, we expanded our approach to include management of colleges, as they would need to take ownership and support all the stakeholders [15,17]. To track and monitor the progress at all levels we created a new center, EnhanceEdu, at IIIT, Hyderabad. We present a novel pragmatic research method that helps develop radically new empowering educational interventions based on the lessons learned and analyzed at each step of the design story of EnhanceEdu. The Design Story Research (DeStoRe) approach we describe here is based on a retrospective and purposeful review of our work in EnhanceEdu. in India, the number of engineering institutions grew from under 10 to over 500 from 1978 to 2008. This resulted in a severe shortage of good quality teachers and therefore poor quality of graduating students. In India, the employability of engineering graduates is 25%. Considering the scale and wickedness of the problem, the lead author and her team worked on designing innovative and radical solutions to this problem during the last eight years. We restrict our research focus to those colleges who responded to our call for partnership in Andhra Pradesh and Telangana and to the center, EnhanceEdu, we created at IIIT (International Institute of Information Technology) at Hyderabad. We present the EnhanceEdu design story using a new pragmatic approach called Design Story Research (DeStoRe) which has its basis in Design Science Research. An overarching design story frames past, present and future work of an entity, seeing possibilities for radical innovation. Keywords: Learning by Doing, engineering education, information technology education, employability, wicked problem, Design Science Research, teacher professional development, Design Story Research I. INTRODUCTION According to The National Association of Software and Services Companies (NASSCOM) report in 2005, 25% of university graduates in engineering in India were employable [1]. The key areas in which graduates needed to improve were technical domain skills, soft skills like communication skills and learning to learn. Recent NASSCOM reports show that employability of graduating engineers is a continuing problem [1]. The state of higher education in the 90s changed with the onset of IT globalization resulting in a large and growing demand for software engineers. The number of engineering colleges grew 50 fold by 2008 in the states of Andhra Pradesh and Telangana. We took up the project1 of imparting the necessary skills to engineering students to increase their technical domain skills using learning by doing. We wanted to make training complete and applicable and to ensure that what is learned is transferred to the field. We soon saw that concentrating only on training students meant that, once the students graduate, we need to train new students repeatedly. This insight increased our worldview to include not just students and their learning, but 1 Project EQITEEC funded by MeitY (Government of India -Ministry of Electronics and IT) II. LITERATURE A. Design Science Research and Design Stories The design science research (DSR) paradigm has its roots in engineering and sciences of the artificial [8]. It is fundamentally a problem-solving paradigm. The main outputs of DSR are artifacts, which are constructs, models, methods and instantiations designed to meet desired goals [9]. DSR in Information Systems addresses wicked problems [10]. Characteristics of a wicked problem are: i) requirements and constraints are unstable, based on ill-defined environmental contexts, with complex interactions among subcomponents. ii) a serious dependence upon human cognitive and social abilities to produce effective solutions [10]. The problem of student employability appears to qualify as a wicked problem. We use the DSR by Hevner [11] with its three cycles of relevance, rigor and design, providing a sound methodological process for designing and building artifacts for addressing various problems. While storytelling has a very rich and long heritage in human culture, storytelling research and design have received more attention in the last few decades [8, 12, 13, 14]. FODEM (FOrmative DEvelopment Method) captures threads and dependencies for developing learning environments for sparse learning communities [14]. Similar to FODEM, design stories have parallel independent threads which we call frames. Frames can be design stories themselves. A design story can proceed to a defined goal and/or evolves with each frame inspired by its predecessor. A design story may evolve in time, or may evolve in learning accumulated from iterations due to continuous improvement. RESEARCH DESIGN B. Research questions 1) What are the characteristics of a designoriented research method that support the development process of new empowering educational interventions to address student employability? 2) How can we validate the new research method by evidence of its usefulness in a real-life case? C. Research context The lead author and her team have worked on enhancing quality of IT education and employability with a large number of interventions spanning from 2008 to 2016, training over 500 teachers in 70+ engineering colleges, and through trained teachers, over 6000 students [15, 16, 17] in 9 iterations. D. Research method In order to answer our research questions, we design a novel pragmatic research method that helps to develop radically new, empowering educational interventions, based on lessons learned and analyzed at each step of the design story. We conduct a retrospective analysis of EnhanceEdu as an individual design story taking various interventions (design processes) and applying Design Science Research (DSR) to each intervention and proceeding to the next step in the sequence of unfolding. DSR follows seven guidelines, but Hevner advises against rote following [11]. The first two are below and others are discussed in DSR example in Section IV A. Guideline 1. Design as an artifact. There are a large number of artifacts that are the output of the design cycle for addressing the problem under study [15, 16, 17]. In our case, the artifact will also be a new research method, called Design Story Research, also called DeStoRe (Research Contributions - Guideline 4). While DSR is used to craft an individual artifact using the seven guidelines of DSR [11], DeStoRe constructs a sequence of design processes influencing its successors. Guideline 2. Problem relevance. The core problems of student employability and teacher quality are both relevant to the economy of any country, not just India. III. RESULTS: DESIGN STORY RESEARCH Our concept design for a design story consists of a sequence of design processes (which we call frames), each inspiring the next, to work towards a goal. Any frame can operate independently and concurrently like in a story. An arc from one frame to another is a dependency. In a design story, we could have a new frame inserted in the next iteration of an instantiation, carrying its own DSR process. Thus any arc can be broken and a new intervention (frame) inserted. Fig. 1 depicts a design story frame. This frame includes the three cycles in DSR by Hevner [11]. Ri refers to the Rigor cycle. Re refers to the Relevance cycle, and D refers to the Design cycle with its Build and Evaluate components. Re Ri D Figure 1. Design Story frame with 3 cycles of DSR A. CIT Course Content Development frame: In Fig. 2, a frame of the Content Development process is shown per Hevner’s 3 cycle DSR. This represents the building of Certificate of IT (CIT) course content (Computational Thinking, Java and Data Structures). The problem was one of designing a set of IT courses usable by teachers and students in engineering colleges dispersed over a wide geographical area, with poor internet connections, and different levels of knowledge. The artifacts included IT courses using Learning by Doing, on a portal with a learning management system, and rubrics for evaluating each task in each module of each course. Teachers could evaluate student submissions using the rubrics, and an EnhanceEdu coordinator at another location, could monitor and calibrate. These design artifacts are published in earlier work we have done [15, 16, 17] (Research Contributions – Guideline 4 and Communication of research – Guideline 7). In this paper, we use DSR and DeStoRe lenses to view and analyze the design story. This design is informed by (the Rigor cycle) Learning by Doing [18], ideas on what builds expertise [19], Bigg’s Constructive Alignment and related Rubrics [20], and Capability Maturity Model- CMM Level 5 continuous improvement process [21] and from the author’s experience and expertise building a SEI CMM Level 5 organization (Research Rigor – Guideline 5). The Relevance cycle helps identify the business needs of the stakeholders, namely, teachers, students, colleges and management, with their compute equipment and internet constraints. improves employability of students helps conceptualize custom designs (artifacts) for each stakeholder group. In Fig. 3, in the EnhanceEdu frame, the team is built, its culture, tasks and goals established, and content development, teacher training and student training goals set. The content development frame in Fig. 2 builds CIT course content. Principal’s meeting frame is used for introducing the TTP and CIT seeking management commitment and teacher nominations through MoU (Memorandum of Understanding) and other artifacts, using Roger’s theory of innovation of diffusion [22] in the rigor cycle. TTP frame built methods to train teachers readying them for training students. Students interested in CIT course and its advantages, sign up for it in Student Orientation. The CIT frame built methods for teachers to conduct training for the signed up students in their colleges, with formative and summative assessments [17]. Students completing the CIT course had improved confidence and technical skills [16, 17]. Each frame in the design story analyzed with DSR, results in building artifacts like methods and instantiations and evaluated at each step of the way. a. Figure 2. CIT Content Development frame – 3 cycle DSR view The Design Cycle includes the Build and Evaluate components where the course content are built using the theories in the Rigor cycle and evaluated (Design Evaluation - Guideline 3) by a method called Batting Practice by EnhanceEdu team emulating teachers and working through the content like real users and capturing time taken, issues in content etc. [16]. Further evaluation is done by an independent review team against the principles of constructive alignment and rubrics, and in the TTP instantiation. Each instantiation enables the artifacts to be more robust with continuous improvement applied (Design as a search process – Guideline 6). B. Design Story itself as a frame and evolving: Our first design story was one frame, EnhanceEdu. This expanded to three frames – content development, TTP and student training (CIT) [15]. However, we knew that these interventions (shown as frames) alone would not work, as having good e-content and training teachers did not imply automatic use or application [22, 23, 24]. We expanded the design story with more frames in Fig. 3. Past, present and future design processes can be represented as frames in the design story. EnhanceEdu story weaves in design interventions for various groups of stakeholders to empower them. The thinking of what b. Figure 3. Evolving generic model of the design story The generic model (meta-story) in Fig. 3, abstracts the design story and problem. Fig. 3a has feedback loops at every step (frame), with inputs for continuous improvement. However, since continous improvement was an implicit goal in our design story, we remove the explicit feedback loops and depict the meta-story as in Fig. 3b. The artifacts created in this design story may be used for enhancing engineering education in another context. The instantiation of a part of EnhanceEdu design story is shown in Fig. 4. We view the design story of addressing the larger theme of graduating engineer employability over multiple iterations of the teacher training and student training programs [16]. The far right in Fig. 4 shows CIT being conducted for students by trained teachers in their colleges. Each of these instantiations provides feedback to the preceding frames of content development, teacher training etc. This iterative improvement is taken as new business needs from the relevance cycle and helps improve the design of the artifacts. A historically earlier frame can learn from later (in time) frames as the feedback goes to enhance the frames in the generic model (metastory) of EnhanceEdu, which is instantiated with the new learning for the next iteration. Thus the top horizontal design story in the Fig. 4 is evolving in learning as a meta-story. The next several design stories horizontally below indicate iterations. These are evolving in time. The design story starting from a generic frame (frame of a generic model or meta-story) like TTP indicates the design story of the various iterations of TTP as TTP1, TTP2 etc. also over time. Our problem solving approach uses design, design as a search process (Guideline 6), design of the many artifacts and a sequence of design processes to engage various stakeholders for gaining their commitment and support for the interventions to be introduced and used. Figure 4. Instantiation of a part of EnhanceEdu design story C. Learning by Doing perspective of the design story: The theories used to inform the design of CIT course content and guide the design story, are shown in Fig. 2. Learning by doing ripples through all the frames of our instantiated story (Fig. 4) having an implication for each frame for each stakeholder. This is particularly important in the CIT student training where the students also learn by doing the tasks in the content, and the trained teachers act as mentors [16, 17]. Learning by Doing perspective is carried through the design story with each stakeholder understanding and supporting the new methodology. This resulted in many artifacts like Computing course content, scaffolding for content, dashboards for monitoring implementations [16, 17] and CS and ECE Courses [25]. D. Story of Change: Teachers made their plans for implementation at their respective colleges during their TTP. They worked on removing potential barriers that could arise for training students when they returned to their colleges. When they have clarity of what to do and how to do it, then they can do it [23]. TTP gave them the confidence [16], and the process of making detailed plans gave them the necessary clarity [23] to effect change. E. Benefits and Validation of Design Story Research: How do we know that DeStoRe makes a difference? Here is a first cut validation of DeStoRe by examining its usefulness through real-life cases. The benefits of DeStoRe are many, and a full discussion of these is outside the scope of this paper. Briefly the benefits include: 1) Ability to take a design story and frame each of its interventions, instantiations etc. using DSR. 2) Framing past, present and/or future work of an entity for tracking and improvement. 3) Tracing productivity and effectiveness of organizations.Example: TTP and Content development. 4) As a strategic planning tool; Planning strategy over a period of a few years is like building the future story of an organization. This maps well into DeStoRe as one can have frames in the story calling for a solution to a problem, or meeting an objective etc, viewed through a DSR lens. 5) Work partitioning among teams (having clear artifacts as outputs of one instantiation, feeding back to the knowledge base informing another intervention) 6) Seeing possibilities for radical innovation: the design story offers a unique view to an expert with rich experience, even without an explicit relevance cycle. He/she can find opportunities for new models, constructs etc. for an unnamed need - opportunities for radical innovation. We use the sense making and technology use ideas for radical innovations [26]. New empowering innovations emerged when the systems view and EnhanceEdu design story intersected as shown in Fig. 5. These are the Butterfly model, Art of Teaching (AoT) and Wikiday workshops [25, 27] shown as frames starting new design stories in Fig. 5. The Butterfly model [25] is the design story of an instructional design model building and publications, with two grants from MHRD (Govt. of India - Ministry of Human Resource Development) for developing 2 pilot and 17 courses in Computer Science and Electronics & Communication. Another empowering innovation is the design story of Art of Teaching, for teachers of humanities, arts, sciences or engineering [27]. A few hundred teachers have benefited from the 3-day Art of Teaching workshop. Limitations: It is possible to miss frames (by missing to create/ review with DSR) in the design story. On the flip side, one can go to an extreme by following everything through the DSR lens making slow progress. V. CONCLUSIONS AND FUTURE WORK We have introduced a new pragmatic method, Design Story Research (DeStoRe), with its basis in Design Science Research, giving the rigor, relevance and design required to address the wicked problems we are tackling of student employability and teacher quality. We have exemplified DeStoRe with examples from the EnhanceEdu design story. Only process steps without a DSR lens would miss the rigorous practical solutions. We further show how to move from a generic model in the research design to instantiations with multiple iterations of the design story evolving both in time and learning, with iterative improvement of artifacts. The benefits of DeStoRe were discussed, including framing past, present and future work of an entity and seeing possibilities of radical innovation with real-life examples. DeStoRe limitations were also presented. [8] [9] [10] [11] [12] [13] [14] [15] [16] Figure 5. Design story view of new, empowering innovations DeStoRe can be explored further for bringing rigor to design stories. Future work includes further elaborating DeStoRe, Its features, methodology and benefits. The wider future implications of our design story include using the artifacts built along the way, in methodologies for improved teacher training and computing engineering education in India and beyond. ACKNOWLEDGMENTS We acknowledge Prof. Kesav Vithal Nori for his belief, encouragement and guidance, the many colleagues who worked with EnhanceEdu, MeitY and MHRD for funding EQITEEC and LbD Content Development Projects, And Prof. K. Viswanath for his help in the writing of this paper. REFERENCES [1] [2] [3] [4] [5] [6] [7] NASSCOM Annual Reports 2007, 2008, 2013-2014 Kannan, K. and Narayanan, K., 2012. ICT-enabled scalable workshops for engineering college teachers in India. In PostSecondary Education and Technology (pp. 127-146). Palgrave Macmillan US. NITTTR www.nitttr.ac.in UGC Academic Staff Colleges: http://www.ugc.ac.in/page/Academic-Staff-Colleges.aspx Felder, R.M. and Brent, R., 2010. The National Effective Teaching Institute: Assessment of impact and implications for faculty development. Journal of Engineering Education, 99(2), pp.121-134. Gibbs, G., & Coffey, M. 2004. The impact of training of university teachers on their teaching skills, their approach to teaching and the approach to learning of their students. Active learning in higher education, 5(1), 87-100. Ho, A., Watkins, D. and Kelly, M., 2001. The conceptual change approach to improving teaching and learning: An [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] evaluation of a Hong Kong staff development programme. Higher Education, 42(2), pp.143-169. Simon, H.A., 1996. The sciences of the artificial. MIT press. March, S. T. and Smith G. F. 1995. Design and natural science research on information technology. Decision Support Systems 15(4): 251–266. Rittel, H., Webber. M. 1984. Planning problems are wicked problems. Developments in Design Methodology. New York: John Wiley & Sons, pp.135-144. Hevner, A.R., 2007. A three cycle view of design science research. Scandinavian journal of information systems, 19(2), p.4. Gerdt, P., Kommers, P., Looi, C.K. and Sutinen, E., 2001. Woven stories as a cognitive tool. In Cognitive Technology: Instruments of Mind (pp. 233-247). Springer Berlin Heidelberg. Quesenbery, W. and Brooks, K., 2010. Storytelling for user experience: Crafting stories for better design. Rosenfeld Media. Suhonen, J. and Sutinen, E., 2006. FODEM: developing digital learning environments in widely dispersed learning communities.Educational Technology & Society, 9(3),pp.43-55. Reddy, K.S.K. and Kode, S., 2010, July. Certificate in Information Technology (CIT). In Technology for Education (T4E), 2010 International Conference on (pp. 138-144). IEEE. Kode, S., and Nori, K.V., 2016, March. Enhancing IT Education: Education Technology for Teacher Training. In Learning and Teaching in Computing and Engineering (LaTICE), 2016 International Conference on (pp. 94-98). IEEE. Kode, S., Thadasina, S.R., Reddy, S.K., Nagaraju, K. and Gollapudi, L., 2012, July. Adapting to Learning by Doing (LBD): Challenges faced in implementing the Student Enhancement Program (STEP). In Technology for Education (T4E), 2012 IEEE Fourth International Conference on (pp. 16). IEEE. Schank, R.C., Berman, T.R. and Macpherson, K.A., 1999. Learning by doing. Instructional-design theories and models: A new paradigm of instructional theory, 2, pp.161-181. Litzinger, T., Lattuca, L.R., Hadgraft, R. and Newstetter, W., 2011. Engineering education and the development of expertise. Journal of Engineering Education, 100(1), pp.123-150. Biggs, J.B., 2011. Teaching for quality learning at university: What the student does. McGraw-Hill Education (UK). Paulk, M.C., Curtis, B., Chrissis, M.B. and Weber, C.V., 1993. Capability maturity model, version 1.1. IEEE software, 10(4), pp.18-27. Rogers, E.M., 2010. Diffusion of innovations. Simon and Schuster. Heath, C. and Heath, D., 2010. Switch: How to change when change is hard. Zemsky, R. and Massy, W.F., 2004. Thwarted innovation. What happened to e-learning and why, A final report for the Weather station Project of the Learning Alliance at the University of Pennsylvania in cooperation with the Thomson Corporation, Pennsylvania. Karri, S.K.R. and Kode, S., 2011, July. Butterfly Model: An Innovative Way to Deliver and Personalize Content in the" Learning by Doing" Methodology. In Technology for Education (T4E), 2011 IEEE International Conference on (pp. 103-108). IEEE. Norman, D. A., & Verganti, R. (2014). Incremental and radical innovation: Design research vs. technology and meaning change. Design Issues, 30(1), 78-96. Kode, S., Nagaraju, K., Gollapudi, L. and Reddy, S.K., 2013, December. Using MediaWiki to increase teaching expertise in engineering colleges. In Technology for Education (T4E), 2013 IEEE Fifth International Conference on (pp. 204-205). IEEE.