Forward-oriented design for learning: illustrating the approach

Research in Learning Technology Vol. 21, 2013 Forward-oriented design for learning: illustrating the approach Yannis Dimitriadisa* and Peter Goodyearb a GSIC-EMIC, School of Telecommunications Engineering, Universidad de Valladolid, Valladolid, Spain; bCoCo, Faculty of Education and Social Work, University of Sydney, Sydney, Australia (Received 17 December 2012; final version received 10 June 2013) This paper concerns sustainable approaches to design for learning, emphasising the need for designs to be able to thrive outside of the protective niches of project-based innovation. It builds on the ‘‘in medias res’’ framework and more specifically on a forward-oriented approach to design for learning: one that takes a pro-active design stance with respect to each of the phases of an extended lifecycle. We draw on fieldwork notes and interview data to describe two cases that illustrate some of the key features of the approach. Recommendations for further R&D in the area of design for learning are provided, derived from the theoretical framework, and illustrated in this paper. Keywords: design for learning; indirection; sustainable design 1. Introduction to forward-oriented design for learning Many tools, approaches and representations have been developed and studied during the last few years in the field of design for learning.1 However, many of the innovative pedagogies pioneered in technology-enhanced learning (TEL) environments are still a long way from becoming mainstream educational practice. Orthodox thinking blames teachers for being insufficiently adept at integrating pedagogy and technology. We think it is time to change tack
to approach design with the strengths and limitations of real teachers in mind, and at the same time to consider how design methodologies could be improved by (a) dispensing with the assumption that design is usually for a ‘‘green field’’ site, and (b) designing proactively for each of the main phases through which TEL-based innovations pass in their lifecycle. In short, we want to open up new perspectives on the conception of design for learning that take seriously the uncertainties and complexities of real-world learning environments. TEL developers, researchers and even professional learning designers often adopt an optimistic vision of educational reality, in which all students do what is expected of them, and make progress as predicted. For example, in group-based learning situations, it is rare to design-in provision for groups affected by students who simply do not turn up. In online discussion forums, the norm is to assume that students will contribute (even if they have to be bribed with marks). Similarly, it is often assumed that all the tools in the environment will provide the exact data needed so that proper scaffolding or evaluation can be performed by any of the agents involved (students, *Corresponding author. Email: [email protected] Research in Learning Technology 2013. # 2013 Y. Dimitriadis and P. Goodyear. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) Licence (http://creativecommons.org/licenses/by/3.0/) permitting use, reuse, distribution and transmission, and reproduction in any medium, provided the original work is properly cited. Citation: Research in Learning Technology Supplement 2013, 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 1 (page number not for citation purpose) Y. Dimitriadis and P. Goodyear teachers, software). Designing for TEL environments is inherently complex and so it is not surprising that designers make simplifying assumptions that wish away many of the practical constraints of real classes. Therefore, even a pedagogically sound design may result in an ineffective enactment or in inefficient performance. Latecomers, students who do not show up, rooms without enough power supply sockets or teachers without enough time are not the exception, they are the rule. Recent research in orchestration (Roschelle, Dimitriadis, and Hoppe 2013) has pointed out several of these shortcomings, which need to be considered if we want to move beyond the perfect ‘‘green field’’ landscape. Attention has to shift to the necessary conditions that must be satisfied if there is to be any significant chance of enacting designs in ways that really integrate good pedagogy and innovative technology [e.g. in areas like Computer Supported Collaborative Learning (CSCL)]. Classroom ecologies are complex, dynamic and somewhat unpredictable (Luckin 2010). They are constituted by interactions between the epistemic (valued tasks and activities), the physical (interleaving material and digital realities), and the social (how individuals, groups and communities work together) (Goodyear and Carvalho 2013). The companion paper (Goodyear and Dimitriadis 2013) has outlined some elements of a new theoretical framework, which considers design for learning ‘‘in medias res,’’ i.e. as an on-going procedure for improving mainstream educational settings and processes rather than as a methodology that only works well in the protected niches of project-based innovation (Ellis and Goodyear 2010). The framework stresses the importance of these elements: (1) Design needs to be understood as having an indirect effect on learning. Learning itself cannot be designed; things can be designed which can have a beneficial effect on learning. (2) Teachers are often essential actors at learntime, since they may intervene with respect to the real-time coordination of classroom events (orchestration, in its strict interpretation). Design methodologies need to be able to take into account the various times and ways in which teachers, as actors with bounded capabilities, can enhance what occurs at learntime. Design methodologies need to be robust and general enough to cope with face-to-face, online and blended contexts, with synchronous and asynchronous interactions, as well as situations where teachers’ time, skills or attention are limited
and even with situations in which there is no teacher (e.g. in self-study courses). (3) Design for learning needs to find ways of working with the dialectical relationships between structure and agency. Providing structures such as scripts and scaffolding is not antithetical to student autonomy. Design is, par excellence, a discipline for resolving competing forces, including balancing structure and freedom, at various scale levels (whole course, learning episode; infrastructure, tool, etc). (4) All design is future-oriented, of course. But when we talk about forwardoriented design, we mean something much stronger. It is partly about designing with a sensitivity to the complexities and unpredictability of what happens after a design ‘‘goes live.’’ But in addition to designing with contingencies in mind, forward-oriented design embraces the following sense. Once a design goes live, it is affected by processes that are active in different 2 Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 (page number not for citation purpose) Research in Learning Technology ways, and to different degrees, during successive phases of the lifecycle. Different things happen during configuration, orchestration and evaluation/ reflection. Note that we are saying something different here than that design is part of a bigger lifecycle. That is well represented in classic approaches to design and development such as ISD and ADDIE (Tennyson 1994; Molenda 2003). Rather, we are saying that at design time, careful thought needs to be given to design features that will be required at and for orchestration, at and for evaluation, etc. Contingency during the various phases of the extended lifecycle is probably the most important, but certainly not the only, feature to be taken into account in a forward-oriented design process. This paper aims to illustrate the proposed ‘‘in medias res’’ framework presented in the companion paper in this special issue, and especially the forward-oriented approach that is based on the aforementioned extended lifecycle. Recommendations and lessons learnt from these illustrative examples and cases are presented towards the end of the paper. 2. Motivating design for orchestration and reflection In this section, we describe two facets of the forward-oriented design approach, i.e. design for orchestration and design for reflection. We draw examples from the recent literature to make the key points. 2.1. Orchestrating face-to-face learning in a technology-enhanced classroom Orchestration is typically concerned with how a teacher manages, in real-time, multilayered activities in a classroom context with multiple constraints (Dillenbourg and Jermann 2010). However, a broader view of orchestration (Prieto et al. 2011) considers multiple contexts (Sharples 2013) beyond the physical classroom. In such a ubiquitous learning context, home, field, or online activities constitute a continuum, which should be considered when designing for orchestration. Several design principles can be devised so that orchestration can be realised more effectively. As an illustration, we mention the TinkerLamp technologies, devised for use in a vocational learning classroom environment (Do-Lenh et al. 2012). Researchers and designers involved in the creation of TinkerLamp recognised, through their field observations, that the students involved in the vocational course concerned were rather easily distracted, spent excessive time running all possible variants of simulations, found various ways of playing with what was an ‘‘entertaining’’ physical device, and could not readily take paper-based versions of their work to share with their employers (who therefore under-valued what the students were able to do). Also, even though the teacher was able to detect moments when it would be valuable to introduce a new idea, or synchronise the work of the student groups, it was pragmatically very difficult to regain the attention of the whole class to do so. With the introduction of TinkerLamp in a version more geared to a teacher’s classroom orchestration, typical orchestration moves come to include: the teacher ‘‘freezing’’ the devices for all groups, gaining the attention of all the students, explaining an important concept, and then ‘‘un-freezing’’ the devices so that students can resume their work. To prepare for such situations, a designer should consider the Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 3 (page number not for citation purpose) Y. Dimitriadis and P. Goodyear creation of orchestration tools, such as the ‘‘freezing and un-freezing’’ feature. These synchronising actions form part of the orchestration repertoire and they do not preclude other orchestration moves, which may be applied by a teacher at learntime. In fact, one of the most challenging issues in design for orchestration is to allow for choosing several moves in a flexible and effective way (see also Section 3.1). Additionally, design should make available a device or mechanism so that the teacher can detect these problematic situations easily and unobtrusively. Neither device is typically provided in learning environments, since it is normally assumed that the task flow will be performed as designed. However, it is necessary to design so that these orchestration tools can be seamlessly integrated in the constrained learning environment. For example, the teacher needs to use the ‘‘freezing and unfreezing’’ function, while they move around the classroom, or when they need to show a new concept to the whole classroom using a shared device. In addition, it is desirable that they can have a quick overview of the whole classroom situation so that they can take decisions to interrupt the activities (or not) at various times. In the example above, we can see that orchestration requires flexible coordination mechanisms that can be embedded in a classroom that is subject to several constraints. Careful design is then required in order to address this complex, albeit frequently occurring, problem. Thus, we need to have (and therefore design for) appropriate orchestration technologies that may act on technologies that can themselves be easily orchestrated (Tchounikine 2013). For example, a tablet with a minimalist dashboard may allow the teacher to perform the aforementioned orchestration moves. However, this orchestration tool is not effective if it cannot act on ‘‘orchestratable’’ technologies. For example, the overall script to be followed by the students has to be flexible enough so that it can be modified on-the-fly. If the script is completely ‘‘hard-wired’’ then the orchestration action cannot be easily introduced and therefore it will not be possible for it to have the desired effect. In addition, the learning environment should be able to collect all the key data and present them on the tablet to support the teacher’s awareness of unfolding classroom activities (see also ‘‘design for monitoring’’). In this case, instead of the tablet-based solution, an ambient lamp (Alavi and Dillenbourg 2012) could also be placed in the physical environment so that the teacher could assess the time spent by each group on each task, at a single glance. Whether it be a minimalistic intervention in the physical environment (e.g. a lamp) or a more sophisticated ICT-based solution (e.g. automatic data collection and collation), orchestrating technologies should match the technologies to be orchestrated. Design needs to be attuned to the settings of the environment. (See also configuration and reflection phases, below). 2.2. Designing tools for reflection based on Interaction Analysis One of the most important phases of the extended lifecycle deals with the reflection that is triggered by data monitored during the learning process. Such reflection (performed either manually by the student/teacher or automatically by a computer agent) may lead to scaffolding, evaluation, orchestration or even redesign. Let’s use an example from the Interaction Analysis (IA) field, and more specifically the tools and techniques of Social Network Analysis (SNA) in order to 4 Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 (page number not for citation purpose) Research in Learning Technology illustrate the need to ‘‘design for reflection’’ or, even more particularly, ‘‘design for monitoring.’’ Recent research evidence has shown that techniques based on IA and SNA are effective at detecting the roles played by group work participants and in supporting regulation of complex learning processes, especially in the CSCL domain (Soller et al. 2005; de Laat et al. 2007). Also, there are a variety of SNA-based tools, reported in the literature, that implement the aforementioned techniques. However, in spite of powerful arguments in the research literature, adoption of tools and techniques derived from research projects has been scarce in practice (Martı́nez, Dimitriadis, and Harrer 2011). Where are the problems that cause this phenomenon of nonsustainable innovation? Lack of (appropriate) data that can be processed easily is one of the most common causes for limited adoption. SNA-based and other IA tools need to be fed with data produced by the tools available in the learning environment. However, few tools are designed with the express goal of producing data for IA, even in cases where they are explicitly designed for learning purposes. For example, when we used the highly flexible CSCL tool Group-Scribbles2 in a learning situation, it was impossible to perform any useful IA for awareness and evaluation purposes. The only source of data came from the log files of the underlying TupleSpaces infrastructure, developed by IBM for debugging purposes. In this case, the excessively fine grain of the data and the fact that it was intended for debugging meant that it was impossible to carry out a semantically significant IA. The intention of the learning tool designer did not include the objective of supporting IA. We can see that the data available were abundant but not appropriate for use for such IA purposes. This example also shows that even if data exist, it is necessary to provide interoperable formats and mechanisms that connect the two types of tools (learning and IA), in the same way that orchestration tools should match the orchestrable learning environment. This issue is especially topical, given that the Learning Analytics movement aims to use ‘‘big data’’ to understand and optimise learning. We should also mention that the issue, previously described in a networked classroom setting, can be also found in blended and ubiquitous learning spaces, in which computer-mediated interaction data are abundant. Therefore, one may conclude that providers of learning tools should design with an eye on the need for monitoring appropriate, actionable data, ideally in partnership with the providers of IA tools. The same holds for other stakeholders, like learning system integrators, who have to provide such monitoring opportunities using general-purpose tools that have not been explicitly designed for learning. The key actors in our study correspond to the direct designers of learning situations, such as teachers or professional learning designers. In order to avoid the aforementioned problems they should ‘‘design for monitoring,’’ i.e. follow a ‘‘monitoring-aware’’ design process (Rodrı́guez-Triana et al. 2011). The designer needs to consider this facet ahead of time and choose the learning tools or define the learning tasks in compliance with the need for appropriate monitoring. The next section provides a more extensive illustration of the main ideas of the ‘‘in medias res’’ framework, through a pair of case studies. A complete account of all design facets for all phases of the extended lifecycle is outside the scope of this paper, although the examples and case studies are meant to provide some valuable insight in this complex field. Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 5 (page number not for citation purpose) Y. Dimitriadis and P. Goodyear 3. Two illustrative case studies In this section, we draw on observational and interview notes from two cases studies, conducted while the first author was engaged in fieldwork in Australia. 3.1. The case of an innovative higher-education interactive tabletop classroom This case study draws on triangulated data from semi-structured interviews with the teacher and the research team, non-participant observations of the tutorial sessions, documentation of the participatory design process as well as detailed follow-up. This case involves a regular university course in which the teacher is a significant actor in all phases of the lifecycle. We need to emphasise the physicality of the learning space, reinforced by the fact that interactive tabletops are introduced as a new part of the classroom ecology. We should also mention that this case refers to a single session of activities, i.e. a significant innovation (a multi-touch tabletop classroom) is introduced into a limited part of the course. Finally, the case reflects a special situation in which teachers and researchers are involved in a participatory design process, aiming to respect as much as possible the authentic conditions of the environment. So this case provides some insights into the eventual transition from project-based innovation and design on a ‘‘green field site’’ to a real-world context in which design and redesign form part of the on-going activities. The course was in an introductory area of management. In 2012, 250 students were enrolled. The innovative element on which we are focussing took place in a 50minute tutorial session and involved 15
20 students. These students worked in small groups on a topic related to the distribution of power in a company. The topic was regarded by the teacher as open-ended, and capable of benefiting from visual representations, provided by a concept mapping tool and the multi-touch tabletops. The teacher in the tutorial session, and the research team that has been developing the multi-touch interactive tabletops for educational use, are the main actors involved in this case. They are engaged in a participatory design process in which they jointly define all the design components (tasks for the students to carry out), social organisation (divisions of labour, roles within groups) and physical-digital environment (classroom, tabletops, etc). It is important to mention that, according to the teacher, the design is only marginally influenced by the objectives of the research team. In other words, the driving force is the educational ideas of the teaching team; good social relations between the teachers and researchers simply enabled better construction of a shared understanding and a more effective participatory design process. Thus, we can characterise the case as a design intervention involving action-research in an authentic environment. This initial cycle of innovation and design is only part of the typical on-going activities of the teaching team. Since the full description of the case is beyond the scope of this paper, we refer the interested reader to Martı́nez-Maldonado et al. (2012) and draw attention to the following aspects of the forward-oriented approach: 3.1.1. Design for configuration The specific tutorial session is chosen since it matches several affordances of the tabletop-based classroom (Dillenbourg and Evans 2011). The digital teaching and learning environment includes a concept mapping tool, instrumentation to capture 6 Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 (page number not for citation purpose) Research in Learning Technology voice and touch log data, software for the post-processing and mining of educational data, as well as a dashboard for real-time classroom coordination. The teacher adapted the set of tasks for this tutorial, set parameters for group formation and agreed with the research team on the basic physical setting of the classroom, including the use of four interactive tabletops and a non-interactive whiteboard. In this case, we can observe the process of designing for configuration (tasks, environment, organisation) with respect to the specific context. Two significant configuration aspects of the environment emerged. Firstly, the teacher was concerned (for ethical reasons) to protect the anonymity of the students
so all personal identification data were stripped, though the system was nevertheless able to discriminate between the participating students. Second, the overall noise level in the classroom was so high that the tabletops’ voice-based functionalities (e.g. the indicators for participation based on voice detection) were deactivated. Both configuration aspects affect significantly the capacity to obtain important monitoring data, and therefore reduce the capacity for awareness, orchestration and reflection, since the data mining techniques cannot be used to their full potential. Thus, we can see that there is a strong interdependence among different phases of the lifecycle, which affects and is affected by important decisions regarding design for configuration. This latter observation is crucial, since it shows the complexity of design decisions, when moving away from ‘‘green field sites’’ to real-world contexts. 3.1.2. Design for orchestration Enactment of such an innovative setting in terms of tasks and environment poses, as one might expect, a number of problems for the real-time coordination of the classroom. For example, there were two distinct phases in the student tasks, followed by whole class discussions mediated by the teacher (see Figure 1). Note that only 15 minutes could be allocated to each phase. Failure to meet the time restrictions was considered to be a critical issue by the teacher and so there needs to be some method available for managing the work of the class. Figure 1. Design plan for the activity using the multi-tabletop classroom. This is scripted in 6 phases. Two are performed at the interactive tabletops (blue squares). The actors are students in small groups (lower) and their teacher (upper). The right half shows the phases, tools and atomic actions [taken from (Martı́nez-Maldonado et al., 2012)]. Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 7 (page number not for citation purpose) Y. Dimitriadis and P. Goodyear Additionally, the teacher wants to control the transition between phases, in order to synchronise, gain students’ attention and manage the rhythm of the class (see also, Section 2.1). Having real-time information on equality of participation or contributions is useful for the teacher, as a complement to her own personal real-time ‘‘reading’’ of the evolving classroom situation. These considerations motivated the adoption of a minimalist dashboard for classroom orchestration. However, several ‘‘practical’’ constraints turn out to limit its use during enactment. For example, the dashboard is implemented on a laptop, which cannot easily be carried by the teacher, without impinging on the fluency and timeliness of her interventions. She makes it clear that she does not want to be stuck in a ‘‘classroom corner’’ managing the classroom ‘‘remotely.’’ This element (dashboard on a laptop) reduces her direct social contact and empathy with the students, and she is not able to collect direct ‘‘physical’’ information (movement of students, level of discussion, status of concept maps, etc.) on group progress or intervene appropriately. Therefore, the dashboard is used much less than expected. The limitation of the dashboard implementation in a device which cannot easily be carried is shown to be important, since it interferes with the social component and the important ‘‘physically oriented’’ configuration of the classroom. Such an observation informs us about the importance of the delicate balance between digital and non-digital devices in physical classrooms. What turn out to be inappropriate design decisions regarding configuration reduce significantly the impact of the orchestration device, which otherwise has a high potential. Therefore, we can observe again the close interdependence between lifecycle phases, and the need to accumulate and adequately employ design knowledge for configuration and orchestration. The above discussion shows that the technology-enhanced classroom ecology is especially complex and therefore the design tensions are even harder to resolve. However, we do not suggest that the dashboard is always less effective than the conventional method. We rather suggest that a better design of the dashboard is necessary, taking into account the special characteristics of such a multi-tabletop networked classroom. In fact, a later study (Martı́nez-Maldonado et al. 2013), which took place as an effect of the ‘‘design for redesign’’ phase, shows that performance was much better with respect to the use of the dashboard. 3.1.3. Design for reflection and redesign One of the most important elements of the supporting environment consists of educational log data processing, mining and reporting. While some of the data are employed during enactment through the orchestration device, the core reports (mainly in graph form) are produced after the session. Graphs show the final state and the progress related to timing, equity of participation and contributions to the final concept maps, as well as quality of the maps with respect to the master map generated by the teacher. These reports are shown to, and assessed by, the teacher. Evaluation data indicate that the teacher easily interprets the graphs, appreciates several of them highly and, most importantly, employs them in redesign of a similar tutorial in another course that is held in the following semester. For example, the information on progress of equity of participation and contribution makes her reflect on the reasons for non-equal participation (e.g. existence of a leader who dominates group work). Thus, she reflects on new methods of group formation, and on interventions at the level of regulation during group work. 8 Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 (page number not for citation purpose) Research in Learning Technology However, such reflection shows also the deficiencies of the data monitoring system. The lack of voice-based information (see design for configuration, above) means she cannot find out how many student contributions are made through oral discussion. Thus, the information on equity is not complete, since it is based exclusively on data coming from the touch interaction with the tabletops. The analysis of the quality of contribution makes her also reflect on the information that ideally should be available to her during the session. While she is moving around the class, observing the work of the different groups, she can have a global view of the concept map state but she cannot assess, in real-time, the progress made in terms of the fundamental concepts and links. Thus, she shows interest in getting information in real-time on the progress status of each group, through a comparison between the master concept map and the maps being produced by the students. Such a reflection reveals the need to provide information through the dashboard that cannot be easily captured through the ‘‘third eye of the experienced teacher.’’ Accordingly, the dashboard (orchestration design element) had to be redesigned. Such a redesign turns out to have a major effect in the next course (Martı́nez-Maldonado et al. 2013), since the real-time information on the progress status allows the teacher to attend to the weakest groups in a timely manner, leading to more balanced performance across the class as a whole. Finally, the teacher’s analysis of the deviations in timing between the tasks as designed and the students’ actual activity has an effect on redesign. The teacher decides to change the distribution of time between the two phases, since she considers the second phase to be more important, due to the expected learning gains. We may observe that reflection has produced new design knowledge that is readily transferred to a similar tutorial in another course. Such redesign has an effect on the teachers’ design decisions and on the provider of the supporting environment, for example in the design of the orchestration tool or decisions about the data to be monitored. Although ‘‘design for redesign’’ may be considered a natural element of any design process, it turns out to be missing in both the case studies presented in this paper. Thus, we stress the importance of this issue: designers should plan with redesign in mind. 3.2. A case of professional learning on a web-based distance program The second case comes from a large-scale professional learning (PL) program in which almost all activities are carried out online. Similar to other projects in online professional development, its lifecycle is rather long, i.e. it corresponds to a coarse grain size. The case of this project is especially interesting due to the high levels of experience of the core project team, the explicit use of patterns as a source of design guidance. Finally, this case study is useful, since it illustrates the need to design for redesign. The main objective of the program is to ‘‘connect Australian teachers with each other and with the latest knowledge in curriculum change’’ and to ‘‘identify and plan for opportunities for school improvement through curriculum change’’ within the context of the new Australian curriculum. Teachers are expected to plan and implement a curriculum project for their school, thinking about how to manage and lead change, towards sustainable innovation. The program’s delivery platform is a Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 9 (page number not for citation purpose) Y. Dimitriadis and P. Goodyear website built using a proprietary toolkit, developed and enhanced through time by the core design team. The study of this case is based on close inspection of the website, several documents provided by the project team, as well as informal interviews with two members of the core team. In this paper we focus on a few issues that illustrate the ‘‘design for redesign’’ facet of the forward-oriented approach. The project team is small (5 to 8 persons), though its core members have experience of at least 10 years dedicated to designing and carrying out online programs of professional development. Their considerable experience and success in this field has been consolidated in a pattern language (Robinson et al. 2011). Note, that design patterns have been shown to be effective, especially if they are carefully elicited and integrated in the project lifecycle, since they encapsulate critical knowledge (Goodyear and Retalis 2010). They support design-based research (DBR) principles as a basis for design innovation and consequently of design improvement, including the need for rigorous documentation. Also, they establish an important list of ‘‘key design factors for successful online groups’’ that provides their roadmap during the different cycles of DBR innovation. On its first iteration, the teachers participating in the program did not behave as the design team expected. According to the team’s ‘‘Redesign Analysis’’ document ‘‘the challenge of finding time to engage in the program accounts for 62% of all comments on problems/issues from a survey of all participants.’’ The design team’s diagnosis of the roots of this problem led them back to an earlier stakeholder workshop, in which what turns out to be an excessively demanding set of learning tasks has its origins
too much is packed into the program; no-one says loudly and clearly that this will demand too much time from the intended participants. A second discrepancy between the design and participants’ subsequent behaviour at learntime is related to the way different elements are linked in an integrated online learning space. The layout of the web interface suggests a linear structure for the modules that provide the backbone of the program. Learners then follow what they assume to be the required linear order, tackling almost all the tasks on each module, although not all of them are designed as obligatory. This surprises the design team, who are used to participants taking much more control over their own paths through online learning materials and ‘‘dipping in’’ to the materials as time permits. The higher than expected attrition rates on the program are partly attributable to this unexpected adherence to an apparently prescribed linear flow. The design team also noted a disconnection between the ‘‘structured learning modules’’ (mentioned above), and the spaces in which the participants were to develop their school-based projects (i.e. the task associated with the main learning objective) and communicate with one another as peers (i.e. the social organisation or community level). Since ‘‘participants further relate that the concept of planning and executing a project is quite foreign and seems like a big hurdle’’ not enough learners reach the point of working on and producing a valid project. The analysis of the lack of integration between the different spaces suggests to the core team a global redesign of the website dashboard, providing a clear, non-linear and modular connection of structured learning modules, learning topics, project planning and progress tracking. The chief design strategist acknowledges that the core team did not employ, in this case, a design pattern, which they had found useful elsewhere
creating a single space for the discussions and the learning materials of each module. 10 Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 (page number not for citation purpose) Research in Learning Technology The aforementioned design issues are especially important due to the nature of the specific learning process. Participants follow their own learning path in the unguided web-based environment, i.e. interpret the tasks and their flow (see indirect character of design in the ‘‘in medias res’’ framework). Also, teachers are not present during learntime, except the online tutors who react on demand, and therefore they cannot remedy emergent problems of design for orchestration (a typical and problematic assumption in many approaches to design for learning). However, flexible and effective on-the-fly redesign and orchestration were also observed. In this case, the core team was able to intervene on time, when they detected that the quality of the first set of projects was not high enough. Then, the original peer review process was disabled, and an expert reviewer (a tutor) stepped in to evaluate the projects. Later, the option of peer review was activated again. After this intervention, the quality of the new projects was significantly higher. Thus, we can see that orchestration is possible, due to a timely assessment procedure, which enabled this type of scaffolding (on-the-fly change of the learning flow). The general approach of the core team favoured a flexible redesign phase, since it was based on a set of clear, ‘‘internalised and embodied’’ design principles or patterns. These patterns can be flexibly applied to the different phases of the lifecycle. On the other hand, there is a clear decoupling of phases, so that e.g. implementation can be ‘‘independent of the medium.’’ Also, the coherent nature of the core team and the complete control of the proprietary delivery system allow short cycles of redesign, redeployment, etc. 4. Concluding remarks Design for learning is a highly ambitious field, which deals with non-trivial problems. Once we acknowledge that learning cannot be designed, we see more clearly that the relations between what can be designed and how learning activities unfold is at the heart of the field. A comprehensive account of the field of design for learning needs also to acknowledge that design rarely takes place on a green field site, and that designs need to be robust enough to survive when they emerge from the protected niche of a funded R&D project. The companion paper (Goodyear and Dimitriadis 2013) proposes a theoretical view of the design for learning field, stepping back and reflecting on the essential elements that should be considered. The ‘‘in medias res’’ framework stresses several elements, such as the indirect character of design and the locus of control in learning, the main components that may be designed and the actors involved, as well as the multi-objective nature of design for learning that makes it worthwhile. Also, it proposes an extended view of the design lifecycle and presents the ‘‘forwardoriented design approach,’’ which foregrounds design for configuration, orchestration, reflection and redesign. The main objective of the present paper has been to shed some light on the important problems and eventual solutions that arise through the analytical lens of the ‘‘in medias res’’ framework. The proposal is timely, since there exist several tools, methods and approaches that have been produced and published in recent years (see other papers in this issue for an overview of some important contributions in the field). While the ‘‘in medias res’’ framework provides a revised theoretical view, its focus is highly practical, since it focusses on sustainable innovation. Adopting a constraints-oriented approach, Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 11 (page number not for citation purpose) Y. Dimitriadis and P. Goodyear it considers the critical elements that should be designed in terms of tasks, physical/ digital learning environment and social architecture. More specifically in the technology-enhanced classroom ecology, technological and pedagogic innovations may be combined in practice. For example, in the CSCL physical classroom presented in the first case study, collaborative and case-based pedagogies were enhanced and supported by interactive tabletops and data mining techniques. Designing a dashboard fit for classroom orchestration was shown to be an important challenge. What information should be shown on the dashboard, or whether it should be implemented on a portable device, were issues that influenced the success of the orchestration moves. On the other hand, contextual elements, such as the noise level or the time duration of the learning session, were taken into account in the design phase in which the tasks or the learning environment were configured. Finally, reflection was directly considered in the design phase. Appropriate information was shown to the teacher after the first round of tutorials and triggered reflection on important design issues and produced significant changes in a second round of innovation. The second case study of an online PL setting showed the importance of locus of control and the indirect character of design. Design is crucial and difficult
learners interpret the tasks and act according to their own decisions and the opportunities and constraints they experience. Failing to consider configuration aspects, such as the integration of online work spaces, may turn out to be a critical factor. Also, the case underscores the value of designing in ways that support timely interventions and onthe-fly changes. Acknowledgements Yannis Dimitriadis’ work during his research stay at the University of Sydney in 2012 was supported the Spanish Ministry of Education, Culture and Sports (PR2011-0137). The Spanish Ministry of Economy and Competitiveness (TIN2008-03-23, TIN2011-28308-C03-02) and the Autonomous Government of Castilla and León, Spain (VA293A11-2) have provided additional funding for this work. Peter Goodyear’s work is supported by an Australian Research Council Laureate Fellowship (Grant FL100100203). Notes 1. 2. See the Learning Design Grid website: http://www.ld-grid.org/ http://groupscribbles.sri.com/ References Alavi, H. S. & Dillenbourg, P. (2012) ‘An ambient awareness tool for supporting supervised problem solving’, IEEE Transactions on Learning Technologies, vol. 5, no. 3, pp. 264
274. de Laat, M., et al., (2007) ‘Investigating patterns of interaction in networked learning and computer-supported collaborative learning: a role for social network analysis’, International Journal of Computer-Supported Collaborative Learning, vol. 2, no. 1, pp. 87
103. Dillenbourg, P. & Evans, M. (2011) ‘Interactive tabletops in education’, International Journal of Computer-Supported Collaborative Learning, vol. 6, no. 4, pp. 491
514. Dillenbourg, P. & Jermann, P. (2010) ‘Technology for classroom orchestration’, in New Science of Learning: Cognition, Computers and Collaboration in Education, eds M. S. Khine & I. M. Saleh, Springer, Dordrecht, pp. 525
552. Do-Lenh, S., et al., (2012) ‘TinkerLamp 2.0: designing and evaluating orchestration technologies for the classroom’, In Proceedings of the European Conference on Technology Enhanced Learning (ECTEL 2012), Saarbrücken, Germany, September 18
21, 2012, Springer-Verlag, Berlin, Heidelberg, Volume 7563/2012, pp. 65
78. 12 Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 (page number not for citation purpose) Research in Learning Technology Ellis, R. & Goodyear, P. (2010) Students’ Experiences of e-Learning in Higher Education: The Ecology of Sustainable Innovation, Routledge Falmer, New York. Goodyear, P. & Carvalho, L. (2013) ‘The analysis of complex learning environments’, in Rethinking Pedagogy for a Digital Age: Designing and Delivering e-Learning, eds H. Beetham, & R. Sharpe, Routledge Falmer, London, pp. 49
63. Goodyear, P. & Dimitriadis, Y. (2013) ‘In medias res: reframing design for learning’, Research in Learning Technology, vol. 21. DOI: 10.3402/rlt.v21i3.19909. Goodyear, P. & Retalis, S., (eds) (2010) Technology-Enhanced Learning: Design Patterns and Pattern Languages, Sense Publishers, Rotterdam. Luckin, R. (2010) Redesigning Learning Contexts: Technology-Rich, Learner-Centred Ecologies, Routledge, New York. Martı́nez-Monés, A., Dimitriadis, Y. & Harrer, A. (2011) ‘An interaction aware design process for the integration of interaction analysis in mainstream CSCL practices’, in Analyzing Collaborative Interactions in CSCL: Methods, Approaches and Issues, eds S. Puntambekar, G. Erkens, & C. Hmelo-Silver, Springer, Science+Business Media, New York, pp. 269
291. Martinez-Maldonado, R., et al., (2012) ‘Orchestrating a multi-tabletop classroom: from activity design to enactment and reflection’, In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (ITS’12), ACM, New York, pp. 119
128. Martinez-Maldonado, R., et al., (2013) ‘Orchestrating a multi-tabletop classroom: from activity design to enactment and reflection’, In Proceedings of the International ComputerSupported Collaborative Learning (CSCL2013), International Society of the Learning Sciences, vol. 1, pp. 320
327. Molenda, M. (2003) ‘In search of the elusive ADDIE model’, Performance Improvement, vol. 42, no. 5, pp. 34
37. Prieto, L. P., et al., (2011) ‘Orchestrating technology enhanced learning: a literature review and a conceptual framework’, International Journal of Technology-Enhanced Learning (IJTEL), vol. 3, no. 6, pp. 583
598. Robinson, L. et al., (2011) ‘A design pattern language for effective professional development programs for clinicians: a decade of design-based research’, Design Principles and Practices: An International Journal, vol. 5, no. 4, pp. 553
570. Rodrı́guez, M., et al., (2011) ‘Monitoring pattern-based CSCL scripts: a case study’, In Proceedings of the European Conference on Technology Enhanced Learning (ECTEL 2011), Springer Berlin, Heidelberg, pp. 313
326. Roschelle, J., Dimitriadis, Y. & Hoppe, U. (2013) ‘Classroom orchestration: synthesis’, Computers & Education, DOI: 10.1016/j.compedu.2013.04.010. Sharples, M. (2013) ‘Shared orchestration within and beyond the classroom’, Computers & Education, DOI: 10.1016/j.compedu.2013.04.014. Soller, A., et al., (2005) ‘From mirroring to guiding: a review of state of the art technology for supporting collaborative learning’, International Journal of Artificial Intelligence in Education, vol. 15, no. 4, pp. 261
290. Tchounikine, P. (2013) ‘Clarifying design for orchestration: orchestration and orchestrable technology, scripting and conducting’, Computers & Education, DOI: 10.1016/j.compedu. 2013.04.006. Tennyson, R., (ed) (1994) Automating Instructional Design, Development and Delivery, Springer Verlag, Berlin. Citation: Research in Learning Technology Supplement 2013; 21: 20290 - http://dx.doi.org/10.3402/rlt.v21i0.20290 13 (page number not for citation purpose)