Assessing Laptop Use in Higher Education: The Laptop Use Scale

J Comput High Educ (2016) 28:18–44 DOI 10.1007/s12528-015-9106-5 Assessing laptop use in higher education: The Laptop Use Scale Robin Kay1 • Sharon Lauricella2 Published online: 30 December 2015  Springer Science+Business Media New York 2015 Abstract The laptop computer is considered one of the most used and important technological devices in higher education, yet limited systematic research has been conducted to develop a measure of laptop use in college and university. The purpose of the following study was to develop a research-based, theoretically grounded scale to assess student use of laptops inside and outside higher education classrooms. The Laptop Use Scale addressed four key areas: in-class academic use, in-class nonacademic use, outside of class academic use, and outside of class non-academic use. Tested on 156 higher education students using laptops computers, the Laptop Use Scale showed acceptable internal reliability and good validity (face, content, construct, and convergent validity). It is argued that this scale can help assess and calibrate pedagogical strategies used to integrate laptops into higher education classrooms. Suggestions for future research on assessing student use of laptops are offered including a focus on multi-tasking behavior. Keywords Evaluate
Assess
Use
Scale
Higher education
University
Laptop & Robin Kay [email protected] Sharon Lauricella [email protected] 1 Faculty of Education, University of Ontario Institute of Technology, 11 Simcoe St N, PO Box 385, Oshawa, ON L1H 7L7, Canada 2 Faculty of Social Science and Humanities, University of Ontario Institute of Technology, 55 Bond Street East, Bordessa Hall, Oshawa, ON L1H 7L7, Canada 123 Assessing laptop use in higher education: The Laptop Use… 19 Overview According to the most recent study by the Educause Center for Analysis and Research (ECAR) on the use of information technology by over 100,000 undergraduate students from 14 countries, laptop computers (hereafter referred to as laptops) are cited as the most used and most important device for academic purposes (Dahlstrom et al. 2013). However, research on the efficacy of using laptop computers in higher education classrooms has produced mixed results. On the one hand, a number of studies have identified clear benefits to using laptops during class such as keeping students on-task and engaged (Hyden 2005), increased capability for following lectures via PowerPoint or multimedia (Debevec et al. 2006), note taking, academic use of software, collaboration among students, and improved organization (Lauricella and Kay 2010). On the other hand, researchers have observed non-academic, off-task use of laptops during class including surfing the web for personal reasons, sending instant messages and emails to friends, playing games, and watching movies (Barak et al. 2006; Barkhuus 2005; Lauricella and Kay 2010). There are numerous factors that can influence the use and effectiveness of laptop use in the classroom including teaching and learning strategies (for example Awwad and Ayesh 2013; Dalsgaard and Godsk 2007; Enfield 2014; Kay and Lauricella 2011), classroom management (e.g. Aguilar-Roca et al. 2012; McCreary 2009), student motivation, cognitive engagement (for example Barak et al. 2006; Fried 2008; Skolnik and Puzo 2008) and the nature of the course content (for example Kay and Lauricella 2014). Regardless of the influential factor, though, it is critical to have a theoretically grounded metric sensitive enough to assess a sufficiently wide range of potential benefits and challenges of laptop use inside and outside of the classroom. Without a scale that can provide reliable and valid feedback on laptop use, it is very difficult to assess the relative import of variables such as pedagogy, student characteristics, subject area, and engagement. It is also challenging to build a solid, evidence-based understanding of laptop use to guide educators without a consistent, comprehensive scale to compare results. Confusion and contradictions over the impact of laptops reported in previous studies is partially confounded by limitations in the metrics used to assess student use of laptops in higher education classrooms (Lauricella and Kay 2010; Kay and Lauricella 2011). Some key problem areas include scales that focus on general, non-specific behaviors (for example Awwad and Ayesh 2013; Barak et al. 2006; Kraushaar and Novak 2010), a bias toward focusing on distractions or negative behaviors (for example Barak et al. 2006; Fried 2008), limited reliability and validity (for example DiGangi et al. 2007; Lindroth and Bergquist 2010; Wurst et al. 2008), and the absence of a guiding framework. The purpose of the current paper, then, is to develop a comprehensive, research-based measure of student use of laptops, grounded in instructional theory. 123 20 R. Kay, S. Lauricella Assessing laptop use in higher education: key parameters Key parameters used to assess laptop use in higher education can be organized into two main categories: behaviors that help students learn (academic use) and behaviors that distract students from the learning process (non-academic use). Each of these categories will be discussed in turn. Academic use A review of the research from 2006 to 2014 uncovered nine peer-reviewed articles on assessing the academic use of laptops in higher education classrooms. Six academic uses of laptops were identified including note taking, using the web to research concepts and ideas when required, communicating with peers about concepts presented in class, organizing files or information, using software, and engaging with web-based interactive tools such as online surveys, case-studies or instructional podcasts. These academic uses are summarized in Table 1 with the associated research studies. Note-taking (n = 8) and web-based research (n = 7) were the laptop activities assessed most often, whereas as using software (n = 2) and web-based interactive tools were examined least often. Non-academic use A review of the literature from 2006 to 2014 uncovered nine articles examining nonacademic student use of laptops observed in higher education classrooms. Six nonacademic behaviors were noted: sending emails, surfing the web, instant messaging, playing games, watching videos, and social networking. These non-academic activities are displayed in Table 1 with their respective research studies. The most frequent non-academic activities observed were sending emails (n = 9), surfing the web (n = 7) and instant messaging (n = 6). The least frequent non-academic use of laptops studied was social networking (n = 2). Methodological issues A thorough examination of ten previous metrics (see Tables 1, 2) used to examine student use of laptops in higher education revealed at least six limitations including a non-specific or general focus, a narrow range of behaviors assessed, an imbalance in the number of academic and non-academic behaviors, the absence of reliability and validity estimates, not examining student use of laptops outside of class, and testing scales only in traditional, lecture-style classrooms. Each of these limitations will be discussed in turn. Non-specific or general focus A number of studies used metrics that had a general focus (Awwad and Ayesh 2013; Barak et al. 2006; Kraushaar and Novak 2010; Skolnik and Puzo 2008). Instead of asking about a range of specific activities, a single, non-specific question about 123 Assessing laptop use in higher education: The Laptop Use… 21 Table 1 Research summary of academic student use of laptops exhibited during class (n = 9 studies) Student use of laptops examined na Research Theoretical perspective Note-taking 8 Annan-Coultas (2012), Awwad and Ayesh (2013), DiGangi et al. (2007), Gaudreau et al. (2014), Lauricella and Kay (2010), Lindroth and Bergquist (2010), McCreary (2009) and Skolnik and Puzo (2008) Associative Annan-Coultas (2012), Awwad and Ayesh (2013), Gaudreau et al. (2014), Kay and Lauricella (2011), Lindroth and Bergquist (2010), McCreary (2009) and Skolnik and Puzo (2008) Constructive (Individual) Annan-Coultas (2012), Lauricella and Kay (2010), Kraushaar and Novak (2010), Lindroth and Bergquist (2010) and McCreary (2009) Constructive (Social) Annan-Coultas (2012), Annan-Coultas (2012), Kay and Lauricella (2011) and McCreary (2009) Constructive (Individual) Web-based research Communication Organizing 7 5 4 (Gagné 1985; Wilson and Myers 2000) (Bruner 1960; Newell 1980; Piaget 1970) (Laurillard 2002; Vygotsky 1978) (Bruner 1960; Newell 1980; Piaget 1970) Use software 2 Kraushaar and Novak (2010) and Skolnik and Puzo (2008) Constructive (Individual) (Bruner 1960; Newell 1980; Piaget 1970) Web-based interactive tools 1 Lauricella and Kay (2010) Constructive (Individual) (Bruner 1960; Newell 1980; Piaget 1970) Situative (Lave and Wenger 1991) a Number of studies to examine this student use of laptops academic or non-academic use was listed, with examples provided in brackets (for example Barak et al. 2006; Kraushaar and Novak 2010; Skolnik and Puzo 2008). Asking non-specific questions makes it difficult to assess the relative frequencies of specific behaviors and how these behaviors might link to pedagogical practices of the instructor. Lack of specificity also creates confusion when comparing studies because different authors list different examples when referring to academic or nonacademic use of laptops. Narrow range of behaviors assessed As a whole, the ten studies assessing the use of laptops in higher education identified a wide variety of behaviors. However, individually, a number of studies 123 22 R. Kay, S. Lauricella focused on a narrow range of behaviors. For example, when targeting academic use of laptops, Kraushaar and Novak (2010) noted MS-Office and browsing skills, Annan-Coultas (2012) employed taking PowerPoint notes and ‘‘googling’’ concepts, and Fried (2008) listed note taking exclusively. All of these criteria are reasonable, but would arguably work better at building overall understanding of student use of laptops if they were combined. Academic versus non-academic imbalance In a majority of studies that assessed the use of laptops, there is an imbalance between academic and non-academic use of laptops in favor of the latter category (Awwad and Ayesh 2013; Barak et al. 2006; Fried 2008; Gaudreau et al. 2014; Kraushaar and Novak 2010; Lauricella and Kay 2010). It is common practice to list one or two academic behaviors (for example note taking, browsing the web) and Table 2 Research summary of non-academic student use of laptops exhibited during class (n = 9 studies) Student use of laptops examined na Research Theoretical underpinnings Personal emails 9 Annan-Coultas (2012), Awwad and Ayesh (2013), Barak et al. (2006), Fried (2008), Gaudreau et al. (2014), Kraushaar and Novak (2010), Kay and Lauricella (2011), McCreary (2009) and Skolnik and Puzo (2008) Collaboration Barak et al. (2006), Fried (2008), Gaudreau et al. (2014), Kraushaar and Novak (2010), Lauricella and Kay (2010); McCreary (2009) and Skolnik and Puzo (2008) Freedom Annan-Coultas (2012), Fried (2008), Awwad and Ayesh (2013), Kraushaar and Novak (2010), Lauricella and Kay (2010) and McCreary (2009) Collaboration Barak et al. (2006), Fried (2008), Kraushaar and Novak (2010), Lauricella and Kay (2010), McCreary (2009) and Skolnik and Puzo (2008) Entertainment Awwad and Ayesh (2013), Barak et al. (2006), Gaudreau et al. (2014) and Lauricella and Kay (2010) Entertainment Annan-Coultas (2012) and Kraushaar and Novak (2010) Collaboration Surfing the web 7 Instant messaging 6 Playing games 5 Watching webbased media 4 Social networking 2 a Number of studies to examine this student use of laptops 123 (Prensky 2010; Small and Vorgan 2009; Tapscott 2009) (Prensky 2010; Small and Vorgan 2009; Tapscott 2009) (Prensky 2010; Small and Vorgan 2009; Tapscott 2009) (Prensky 2010; Small and Vorgan 2009; Tapscott 2009) (Prensky 2010; Small and Vorgan 2009; Tapscott 2009) (Prensky 2010; Small and Vorgan 2009; Tapscott 2009) Assessing laptop use in higher education: The Laptop Use… 23 three to six non-academic behaviors (for example email, instant messaging, browsing the web, playing games, watching videos). While it is possible that more non-academic behaviors are pursued by students, the built-in imbalance in the presentation of possible options may bias the results and conclusions. A balanced number of academic and non-academic behaviors permits a more balanced analysis. Limited reliability and validity Out of the ten studies that measured student use of laptops in higher education classrooms, only two provided estimates of reliability (Awwad and Ayesh 2013; Lauricella and Kay 2010) and one examined validity (Lauricella and Kay 2010). While critical information can be gleaned from preliminary Likert questions, case studies, and open-ended questions (for example DiGangi et al. 2007; Lindroth and Bergquist 2010; Wurst et al. 2008), reliability and validity of measures helps to increase confidence in assessment tools and the possibility for extending the results to a larger population. Missing outside class behaviors No peer-reviewed studies have examined student use of laptops outside the classroom. While it may seem more critical to determine the impact of laptops in the classroom, understanding how laptops are used outside the classroom is also important, particularly with respect to distractions and potential barriers to students completing academic work. Furthermore, there may be a connection between how a student behaves in class and how they behave outside of class. Limited context of testing scales Eight out of the ten studies evaluating laptop use in higher education have been conducted in classrooms where a traditional lecture-style teaching approach is used (Awwad and Ayesh 2013; Barak et al. 2006; DiGangi et al. 2007; Fried 2008; Gaudreau et al. 2014; Kraushaar and Novak 2010; McCreary 2009; Skolnik and Puzo 2008). This kind of restricted context may bias the results and account for the more limited and general parameters used to assess in class academic behaviors. Some might argue that a lecture format is representative of a majority of higher education classrooms, however, a number of alternative approaches are being used (for example Dalsgaard and Godsk 2007; Enfield 2014; Kay and Lauricella 2011; Kolb and Kolb 2005; Lasry et al. 2008; Lewis and Lewis 2005). A comprehensive scale for assessing laptop use in higher education classrooms needs to extend further than lecture-driven activities, such as note taking or browsing the web, in order to capture the full range of possible behaviors experienced. Guiding framework for academic and non-academic behaviors Previous research on evaluating laptop use in higher education has yet to define a guiding framework or set of principles to organize behaviors displayed. As stated 123 24 R. Kay, S. Lauricella earlier, two general categories of behaviors have emerged: academic and nonacademic behaviors. Potential theoretical underpinnings of each of these categories will be discussed. Academic behaviors Mayes and de Freitas (2007) reviewed four learning theories that support pedagogy for a digital age including associative, constructive-individual, constructive-social, and situative. Associative learning (Gagné 1985; Wilson and Myers 2000), or building understanding and competencies step-by step, is closely aligned with the lecture and note-taking use of laptops observed in previous studies (Table 1). Constructive learning by an individual (Bruner 1960; Newell 1980; Piaget 1970), or achieving personal understanding through active, independent discovery is indicative of laptop use such as web searches by students for concepts, organization of digital materials, and using software or web-based interactive tools to explore new ideas (Table 1). Social construction of knowledge (Laurillard 2002; Vygotsky 1978), or acquiring understanding through dialogue and collaboration, is represented by the use of laptop communication tools with peers (Table 1). Finally, situated learning (Lave and Wenger 1991), or developing practice in a particular community with authentic tasks is aligned with the use of specific, real-world webbased learning tools (Table 1). These four theories provide a framework that supports the full range of academic-based student use of laptops reported in the literature and included in the assessment tool developed in the current study. Non-academic behaviors Small and Vorgan (2009) suggested that the brains of digital natives or the net generation (Tapscott 2009) are evolving differently from those of previous generations of students. Specifically, this new breed of student has a shorter attention span when faced with more traditional forms of teaching. In addition, students are in a state of ‘‘contiguous partial attention’’ (Tapscott 2009, p. 18)—they keep tabs on everything and never truly focus on anything. Prensky (2010) has also noted unique characteristics of the digital generation including twitch speed, multitasking, random-access, graphics-first, connectedness, and a desire to have fun. Tapscott (2009) took the analysis of the Net Generation one step further when he and his colleagues studied almost 10,000 individuals born between 1977 and 1997 from 12 countries. The results revealed a set of key behaviors indigenous to the Net Generation including freedom to do whatever they want, whenever they want, a propensity for collaboration, and a desire for entertainment in work and play. These characteristics line up well with those identified in previous studies assessing laptop use (Table 2). Surfing the web for a wide range of personal needs and information is representative of the net generation’s expectation of freedom without contextual boundaries. Sending personal emails, instant messaging, and social networking are consistent with the Net Generation’s desire to keep in contact and to collaborate. Playing games and watching video-based materials is aligned well with the ‘‘Net Generation’s’’ proclivity for being entertained. Tapscott (2009) study provides a 123 Assessing laptop use in higher education: The Laptop Use… 25 lens in which to organize non-academic use of laptops observed previously and employed in the scale developed for the current study. Purpose The purpose of this study was to develop and test a comprehensive, research-based instrument for assessing student use of laptops inside and outside of higher education classrooms. Method Participants One hundred fifty-six university students (54 males, 102 females) in their first (n = 40), second (n = 63), third (n = 3) or fourth year (n = 50) of university participated in the study. They were enrolled in either communication (n = 107 out of a possible 120 students, 89 % response rate) or teacher education (n = 49 out of a possible 58 students, 84 % response rate) courses while using their laptops. The average self-reported grade for the course in which they used the laptop was 81.4 % (SD = 6.3, range 45–90). A majority of students reported that they were either very interested (n = 62, 40 %) or interested (n = 75, 48 %) in the course they were taking when using their laptop. Over 90 % of the students reported being very comfortable (n = 93, 60 %) or comfortable (n = 52, 33 %) with using computer technology. All students leased an IBM laptop from the university and had wireless access to the web throughout the campus. Teaching context The scale was tested in communication and teacher education university classes where the four primary learning frameworks, outlined by Mayes and de Freitas (2007) were used (see Table 1). First, an associative approach was followed when the primary learning goal was to present and discuss key concepts, with a step-bystep, traditional PowerPoint lecture with discussion. Second, a constructive (individual) approach was employed when students were achieving understanding through active, independent discovery in the form of web-based searches for concepts, reviewing published articles, participating in online voting and surveys, working through interactive web-based learning objects, creating presentations and learning materials with tool-based software, and engaging with subject-specific software. Third, social construction of knowledge was pursued with team activities such as online case studies, concept-map creation, assessment of video podcasts, and sharing and coordinating ideas with discussion boards. Finally, situative learning was used by deliberate efforts to weave real-world examples, problems, and situations into the associative and constructive-based activities listed above. 123 26 R. Kay, S. Lauricella Instrument development Three steps were followed to develop and select items for the Laptop Use Scale in this study. First, the results from Lauricella and Kay’s (2010) original study were carefully reviewed and used to ultimately expand the range of content assessed for academic and non-academic use of laptops. Based on student qualitative feedback, several items were added to the academic use construct including expanded forms of note taking, interacting with the web in a variety of ways, communicating with peers, and using subject-specific software. Regarding non-academic use of laptops, more current technology-based activities were added including watching video podcasts and using Facebook. Second, after creating a list of new items based on student feedback (Lauricella and Kay 2010), items were carefully checked by two experts in the field of education and technology who taught in laptop-based classrooms. The experts agreed that the items suggested by students were reasonable and representative of student interaction with laptops. The experts added the use of Twitter because they perceived this new activity as one that students in Lauricella and Kay’s (2010) study may not have used. A similar process was followed to select items for academic and non-academic use of laptops outside of the classroom. The non-academic behaviors largely mirrored those items selected for the in-class scale. However, based on student comments from Lauricella and Kay’s (2010) study and the analysis of the two experts, academic use was expanded to include organizing notes, sharing notes, and collaboration with peers. Finally, an extensive review of laptop scales used in the previous studies was conducted (see Table 1) to determine the full range of possible items that could be included in the Laptop Use Scale. These items were merged with those identified in steps one and two above to create the final version of the Laptop Use Scale (‘‘Appendix’’). Procedure In the final class of the semester, students were invited to participate in an anonymous, online survey. Participation was voluntary, and the instructor, who left the class while the survey was being completed, was unable to determine who chose to participate. As a further precaution, the data was not accessed until all marks for the courses were submitted. The survey took 10–15 min to complete. Data collected Descriptive variables Students were asked their gender, year of study, average grade in the previous year of study, course taken while using their laptop computer, estimated average grade in the course in which they were enrolled, comfort level with computer technology and how many hours per day they used their laptop computer (‘‘Background information’’ section—Items 1 to 7 in Appendix). 123 Assessing laptop use in higher education: The Laptop Use… 27 Laptop Use Scale The Laptop Use Scale focused on four key areas of use: academic use of laptops inside the classroom, non-academic use of laptops inside the classroom, academic use of laptops outside the classroom, non-academic use of laptops outside the classroom. Academic use of laptops inside the classroom (‘‘Academic use DURING class’’ section—8 items in Appendix) focused on note taking, searching the web, interactive tools, and communication with peers. Non-academic use of laptops of laptops inside the classroom (‘‘Non-academic use DURING class’’ section—6 items in Appendix) looked at game playing, watching podcasts, and communication with social media and email. Academic use of laptops outside the classroom (‘‘Academic OUTSIDE if class’’ section—9 items in Appendix) focused on organizational tasks, searching the web, production tools, sharing resources and collaborating with peers. Finally, non-academic use of laptops outside of the classroom (‘‘Academic OUTSIDE if class’’ section C—7 items in Appendix) included game playing, watching videos, communication with social media, and email. All questions used a five-point Likert scale with the following options: never, rarely, sometimes, frequently, or very frequently. Descriptive statistics for the Laptop Use Scale are presented in Table 3. Student comments Students were asked four open-ended questions about academic and non-academic use of laptops inside and outside of the classroom: 1. 2. 3. 4. Overall what are the biggest benefits to having a laptop IN class for this course? Why? Overall what are the biggest distractions in having a laptop IN class for this course? Why? Overall what are the biggest benefits to having a laptop OUTSIDE class? Why? Overall what are the biggest distractions to having a laptop OUTSIDE class? Why? Table 3 Description of Laptop Use Scale Scale No. items Possible range Internal reliability 8 8–40 r = 0.80 LES In-class (academic use) In-class (non-academic use) 6 6–30 r = 0.87 Outside class (academic use) 8 8–40 r = 0.85 Outside class (non-academic use) 7 7–35 r = 0.78 123 28 R. Kay, S. Lauricella Data analysis A series of analyses and procedures were conducted to assess the reliability and validity of the Laptop Use Scale. These included conducting: 1. 2. 3. 4. 5. 6. 7. internal reliability estimates for the Laptop Use Scale constructs (reliability); student and expert analysis of items (face validity); frequency of laptop use assessed by the Laptop Use Scale constructs (content validity); student comments (content validity); a principal component factor analysis for the Laptop Use Scale (construct validity); correlations among constructs within the Laptop Use Scale (construct validity); and correlations among Laptop Use Scale constructs and descriptive variables— self-reported average grade (current course and previous year), interest in course, year of study, hours per day on laptop (convergent validity). It should be noted parametric statistics were used to assess the Likert data obtained from the Laptop Use Scale. Some researchers have suggested that nonparametric statistics are required because Likert data is ordinal, not continuous (Jamieson 2004; Kuzon et al. 1996). However, Norman (2010) after a detailed analysis of arguments for non-parametric tests, argues that parametric tests are valid for Likert data. Norman’s (2010) conclusions are supported by a number of other statistical analysts (Carfio and Perla 2008; Murray 2013). Student comments were categorized using an inductive approach outlined by Miles and Hubrman (1994) and based on a scoring scheme developed by Lauricella and Kay (2010). Once the categories were identified, inter-rater reliability estimates from 96 to 98 % were achieved. Results Internal reliability The internal reliability estimates for the Laptop Use Scale constructs based on Cronbach’s a were 0.80 (in-class academic use of laptops), 0.87 (in-class non academic use of laptops), 0.87 (outside class—academic use of laptops), and 0.77 (outside class—non-academic Use of laptops) (Table 3). According to Kline (1999) and Nunnally (1978), these moderate to high values are considered acceptable internal reliability levels for measures in the social sciences. Face validity Face validity for the Laptop Use Scale was established by comparing, contrasting and adding items based on the results and feedback from a previous scale assessing 123 Assessing laptop use in higher education: The Laptop Use… 29 laptop- related behaviours in higher education settings (Lauricella and Kay 2010). Next, two experts in the field of education and technology agreed that the items proposed were reasonable and representative of student laptop use in the classroom. Finally, all items were cross-checked with a composite list created from a comprehensive literature review of laptop use scales (Table 1). This triangulation of data sources helped to establish face-validity for the Laptop Use Scale in this study. Content validity Frequency analysis: behaviors in the classroom A frequency analysis was run to determine the extent to which students reported inclass academic and non-academic behaviors chosen for the Laptop Use Scale (Table 4). At least 20 % of the students engaged in most items sometimes, frequently, or very frequently. However, only 6–8 % of the students watched movies or used Twitter. We decided to drop these two items from the in-class, non-academic construct because they were not representative of what was actually taking place in the classroom. Frequency analysis: behaviors outside of the classroom A second frequency analysis was conducted for out of class academic and nonacademic behaviors chosen for the Laptop Use Scale (Table 5). At least 30 % of the Table 4 Frequency analysis for academic and non-academic behaviors inside the classroom (n = 156) Variable M (SD) Never/rarely (%) Sometimes (%) Freq/very Freq (%) Academic use Use software program for academics 4.3 (1.0) 6 10 84 Use notes posted by professor 4.0 (0.9) 6 20 74 Follow a PowerPoint presentation 3.8 (1.2) 15 14 71 Search web for academic purposes 3.7 (1.0) 9 28 63 Communicate with peers for academics 3.5 (1.3) 23 22 54 Take notes 3.2 (1.3) 31 24 44 Use online interactive tools 3.3 (1.0) 16 42 42 Participate in online surveys 2.9 (1.1) 39 32 29 Personal instant messaging 3.1 (1.4) 36 16 48 Search web for personal reasons 3.1 (1.2) 27 33 40 Facebook 2.9 (1.4) 40 21 40 Personal email 2.9 (1.3) 40 24 35 Play games 1.8 (1.1) 80 11 10 Watch podcasts 1.8 (1.1) 80 10 10 Watch moviesa 1.3 (0.8) 92 5 3 Use twittera 1.2 (0.7) 94 3 3 Non-academic use a Variable was not used in the final scale because it was not exhibited frequently enough 123 30 R. Kay, S. Lauricella Table 5 Frequency analysis for academic and non-academic behaviors outside of the classroom (n = 156) Variable M (SD) Never/rarely (%) Sometimes (%) Freq/very Freq (%) Academic use Use software program for academics 4.3 (0.8) 4 7 89 Search the web for academics 3.9 (0.8) 4 22 74 Communicate with peers for academics 3.8 (1.1) 12 22 65 Working with peers on assigned group work 3.7 (1.0) 13 24 63 Sharing notes and course resources 3.6 (1.0) 17 25 58 57 Organizing course notes and materials 3.6 (0.9) 9 34 Online interactive activities 3.2 (1.0) 26 37 37 Searching the university library databases 3.0 (1.0) 29 39 32 Non-academic use a Personal web search 4.4 (0.8) 3 11 87 Personal email 4.2 (0.9) 4 19 77 Facebook 4.0 (1.2) 12 15 73 Personal instant messaging 4.0 (1.3) 14 15 71 Watch podcasts 3.6 (1.2) 18 25 58 Watch movies 3.2 (1.3) 30 27 43 Play games 2.8 (1.3) 39 27 34 Use Twittera 1.6 (1.2) 84 4 12 Variable was not used in the final scale because it was not exhibited frequently enough students engaged sometimes, frequently, or very frequently in all but one of the behaviors listed. Only 16 % of the students used Twitter, therefore we decided to remove this item from the scale, because it was not representative of what was actually taking place outside of the classroom. Student comments: behaviors in the classroom Students offered 175 comments about academic behaviors inside the classroom. Reports of taking notes (n = 53 comments), searching the web (n = 51 comments), following PowerPoint presentations (n = 31 comments), using academic software (n = 29 comments), and collaboration with peers (n = 16 comments) were consistent with the proposed scale items for in-class academic behaviors. The only items not commented on were the two items involving online interactive tools. This finding is consistent with the relatively infrequent use on interactive tools noted in the frequency analysis presented in Table 4. Students made 163 comments about non-academic use of laptops inside the classroom. Seventy-nine comments (64 %) focused on the use of Facebook, instant messaging, and sending personal emails, a result that is consistent with the first three items chosen for the in-class non-academic scale in Table 4. Comments about playing games and entertainment, in general (n = 15 comments), were also consistent with the items suggested for the in-class non-academic scale. Students did 123 Assessing laptop use in higher education: The Laptop Use… 31 not report watching video podcasts, movies, or using Twitter which is consistent with the relatively low frequencies reported for these items in Table 4. Student comments: behavior outside of the classroom Students commented 155 times on the academic use of laptops outside of the classroom. Use of software for academic purposes (n = 17 comments), searching the web and library databases to conduct research (n = 66), collaborating with peers (n = 40 comments), and organization (n = 10 comments) were consistent with the frequency of student laptop use reported Table 5. Engaging in online interactive activities was the only item on the outside-class, academic construct that was not commented on by students, a result that is consistent with the low frequency of use reported in Table 5. Students offered 111 comments on non-academic use of laptops outside of classroom. Reports of social interaction (for example Facebook, email, instant messaging) with peers (n = 54 comments), watching videos and podcasts, and playing games (n = 42 comments) were consistent with the frequency of laptops behaviors reported in Table 5. Personal web-search and use of Twitter were not commented on by students which was consistent with the low frequency score of these behaviors in noted Table 5. Construct validity Principal component analysis The first principal components analysis was conducted to determine whether in-class academic construct was distinct from in-class non-academic construct. The results from the varimax rotation (using Kaiser normalization) are presented because they simplify the interpretation of the data (Field 2005). The Kaiser–Meyer–Olkin measure of sampling adequacy (0.846) and Bartlett’s test of sphericity (p \ .001) indicated that the sample size was acceptable. The analysis confirmed that the two proposed constructs, in-class academic and in-class non-academic use of laptops, were distinct (Table 6). The only item to overlap was ‘‘Communication with Peers’’. A second principal components analysis was run to determine whether the outside class academic construct was distinct from outside class non-academic construct. The results from the varimax rotation (using Kaiser normalization) are presented because they simplify the interpretation of the data (Field 2005). The Kaiser–Meyer– Olkin measure of sampling adequacy (0.815) and Bartlett’s test of sphericity (p \ .001) indicated that the sample size was acceptable. The analysis confirmed that the two proposed constructs, in-class academic and in-class non-academic use, were distinct (Table 7). The only item to overlap was ‘‘Emailing for Personal Reasons’’. Correlations among Laptop Use Scale constructs Correlations among all but one of the Laptop Use Scale constructs were modest but significant ranging from 0.20 to 0.57 (Table 8). Shared variances, ranging from 4 to 123 32 R. Kay, S. Lauricella Table 6 Varimax rotated factor loadings on inside class behaviors (Laptop Use Scale) Construct Factor 1 Factor 2 In class use (academic) Search the web for academic purposes .776 Use the notes posted by the instructor .717 Follow a PowerPoint presentation on your laptop computer .713 Use software program for academic purposes .694 Take notes on my laptop .650 Use online interactive tools (for example learning objects, applets) .566 Participate in online surveys .515 Communicate with peers for academic reasons .450 .568 In-class use (non-academic) Use instant messaging for personal reasons (for example MSN, Skype) .852 Search the web for personal reasons .851 Go on Facebook .838 Watch short video clips for personal use(for example YouTube) .717 Use email for personal reasons .679 Play games .667 Factor Eigenvalue PCT of VAR CUM PCT 1 4.10 29.3 29.3 2 3.46 24.7 55.0 32 %, were small enough to support the assumption that each construct measured was distinct. Correlations were generally higher within academic activities (inside and outside the classroom) and within non-academic activities (inside and outside of the classroom) than between academic and non-academic activities. In other words, (a) students who engaged academic activities during class appeared more likely to engage in academic activities outside of class and (b) students who engaged nonacademic activities in class seemed more likely to engage in non-academic activities outside of class. There was one exception—the correlation between the in-class and outside-class academic constructs was the same as the correlation between the inclass academic and outside-class non-academic construct. Convergent validity In-class academic use The only significant correlation for the in-class academic use construct was a positive correlation with interest in the course (r = 0.23, p \ .01). Students who were more interested in the course engaged more in academic activities (Table 9). 123 Assessing laptop use in higher education: The Laptop Use… 33 Table 7 Varimax rotated factor loadings on outside of class behaviors (Laptop Use Scale) Construct Factor 1 Factor 2 Outside class use (academic) Search the web for academic purposes .795 Working with peers on assigned group work .783 Communicate with peers for academic purposes .756 Organizing course notes and materials .726 Sharing notes and course resources .717 Use software program for academic purposes .614 Online interactive activities (for example learning objects. Applets) .613 Searching the university library databases for articles/books .578 Outside class use (non-academic) Watch short video clips for personal use (for example YouTube) .793 Watch movies .761 Search the web for personal reasons .692 Use Instant Messaging for personal reasons (for example MSN, Skype) .664 Go on Facebook .619 play games .581 Use email for personal reasons .382 .398 Factor Eigenvalue PCT of VAR CUM PCT 1 4.14 27.6 27.6 2 3.22 21.5 49.1 Table 8 Correlations among Laptop Use Scale constructs (n = 155) Scale In-class (academic) In-class (nonacademic) Outside-class (academic) Outside-class (non-academic) In-class (academic) 1.00 0.38** 0.38** 0.24** 1.00 0.08 0.57** 1.00 0.20* In-class (non-academic) Outside-class (academic) Outside-class (non-academic) 1.00 * p \ .05; ** p \ .01 In-class non-academic use The correlation between the non-academic construct was significant and negative for average grade reported for the previous year (r = -0.31, p \ .01), current grade reported (r = -0.26, p \ .01), and year of study (r = -0.29, p \ .01). In other words, students who reported lower grades and being new to the university were 123 34 R. Kay, S. Lauricella Table 9 Correlations among Laptop Use Scale constructs and external variables Scale Previous grade average Current grade in course Interest in course Year of study Hours on laptop per day In-class (academic) 0.06 0.12 0.23** -0.13 0.10 0.10 In-class (nonacademic) -0.31** -0.26** 0.01 -0.29** 0.10 0.18* Outside-class (academic) 0.24* 0.40** -0.06 Outside-class (nonacademic) -0.06 -0.16 0.04 0.15 0.32** -0.26** 0.23** Computer comfort level -0.04 0.34** * p \ .05; ** p \ .01 somewhat more likely to engage in more non-academic use of laptops during class (Table 9). Outside-class academic use Correlations between the outside-class, academic construct were significant and positive for average grade reported for the previous year (r = 0.24, p \ .05), current grade reported (r = 0.40, p \ .01), and hours per day on the laptop (r = 0.32, p \ .01). Students who reported higher grades and used laptops more regularly were more engaged in more academic laptop-related behaviors outside of class (Table 9). Outside-class non-academic use Correlations between the outside-class, non-academic construct were significant and negative for year of study (r = -0.26, p \ .01) and significant and positive for hours per day on the laptop (r = 0.23, p \ .01) and computer comfort level (r = 0.34, p \ .01). In other words, students who were new to the university, as well as students who were on the laptops more during the day and who felt more comfortable with computers were more likely to participate in non-academic activities outside of class (Table 9). Discussion Overview The purpose of the current study was to develop a comprehensive, research-based metric for assessing student use of laptops inside and outside of higher education classrooms. Particular attention was directed toward developing a broad scale, 123 Assessing laptop use in higher education: The Laptop Use… 35 addressing a wide and balanced range of academic and non-academic use of laptops, providing reliability and validity estimates, and testing in a context where a variety of teaching methods was used. A scale was developed to assess academic and nonacademic use of laptops inside and outside of higher education classrooms. Addressing methodological concerns Six methodological concerns were observed in previous research and addressed in the current study. First, each of the academic and non-academic constructs had a wide range of clear, specific behaviors. Second, a comprehensive list of items was used based on a conglomerate of previous laptop scale items. Third, a roughly equal balance of academic and non-academic uses was included. Fourth, reliability and validity of the current scale were assessed. Fifth, an assessment of use of laptops outside the classroom was conducted. Finally, the scale was tested in a teaching environment that incorporated a wide range of teaching methods and not strictly limited to a lecture-based format. Reliability The internal reliability estimates (0.77–0.87) for the Laptop Use Scale were good (Kline 1999; Nunnally 1978), as was the inter-rater reliability of the categories (96–98 %) used to assess student comments. Only two previous studies provided estimates of reliability (Awwad and Ayesh 2013; Lauricella and Kay 2010), yet it is argued that this metric is a standard and fundamental element of any evaluation tool and should be calculated for future research studies, if the sample size permits. Validity Aside from face validity, which was determined by a review of Laptop Use Scale items by two experts in field of education and technology, three types of validity were assessed for the Laptop Use Scale—content, construct and convergent. Content validity was examined to determine whether the items truly matched the types of use that students display inside and outside higher education classrooms. Construct validity was calculated to determine if there were clear academic and non-academic categories of laptop use. Convergent validity was examined to determine if the academic and non-academic constructs of the Laptop Use Scale were consistent and associated with other variables such as interest in the course, grades, year of study, and hours per day on the laptop. Each of these forms of validity will be discussed in turn. Content validity For the Laptop Use Scale to have sufficient content validity, we needed to determine the extent to which the measure represents the use of laptops by higher education students. A frequency analysis of academic and non-academic use of laptops, both inside and outside of the classroom, were consistent with the list of activities chosen 123 36 R. Kay, S. Lauricella for the Laptop Use Scale. Frequency of student comments also matched the selection of items for the Laptop Use Scale. It is reasonable to conclude that content of the Laptop Use Scale is representative of how students behave with laptops inside and outside of higher education classroom. Outlier behaviors that were removed from the final scale included watching fulllength movies and using Twitter. Previous research noting multitasking, twitchspeed, and limited focus of Net Generation students (Prensky 2010; Small and Vorgan 2009) is consistent with the higher education students in this study not watching full length movies. The limited use of Twitter is predicted from a recent report indicating that only 18 % of college students use this social media tool compared to 68 % who use Facebook (Pew Research Center 2013). It is important to recognize that use of tools can change from year to year, so specific scale items may need to be modified. For example, according the Pew Research Centre, Pinterest and LinkedIn are popular social media tools that might deserve consideration in future assessments of laptop use (Pew Research Center 2013). Another option would be to include a general item labelled ‘‘social media tools’’ because the specific tools used may not be as critical as the level of distraction observed. Construct validity The first principal components analysis revealed two distinct constructs related to the use of laptops in higher education classrooms (academic use, non-academic use) with only one item overlapping, ‘‘communication with peers.’’ It is conceivable that communication with peers through email, instant messaging, and social media is a continuous task for students of the Net Generation. As Small and Vorgan (2009) suggested, today’s generation of students want to be in contact with everything, all the time so whether they are engaging in academic or non-academic behavior may be irrelevant. Nonetheless, academic and non-academic constructs are consistent with previous research on student use of laptops (Awwad and Ayesh 2013; Barak et al. 2006; DiGangi et al. 2007; Fried 2008; Gaudreau et al. 2014; Kraushaar and Novak 2010; McCreary 2009; Skolnik and Puzo 2008). The second principal components analysis revealed two distinct constructs related to the use of laptops outside higher education classrooms (academic use, non-academic use) with only one item overlapping, ‘‘emailing for personal reasons.’’ While previous research has not examined laptop use outside the classroom, the academic versus non-academic categories are consistent with those observed inside the classroom. A positive, significant but relatively small correlation (r = 0.38) between academic use of laptops inside and outside of the classroom supports the assumption that these two constructs are related, but still distinct. A similar conclusion can be drawn with respect to the relatively small (r = 0.57), but significant correlation between non-academic behaviors inside and outside the classroom. 123 Assessing laptop use in higher education: The Laptop Use… 37 Convergent validity Six variables were used to explore convergent validity and included: previous average grade, current grade average in the course, interest in the course, year of study, hours per day on the laptop, and computer comfort level. With respect to non-academic use of laptops during class, the expected negative correlations with grades and year of study were observed. However, for academic use of laptops in the class, the only significant correlation was with higher interest in the course. One would predict that students with better grades or more experience at the university would be more inclined to engage in academic use of laptops. Further research, perhaps in the form of interviews to focus groups, is needed to understand this anomaly. With respect to correlations among academic and non-academic laptop use outside of the classroom and the six convergent variables assessed, significant correlations with grades, interest in the course, and hours per day on the laptop indicated that the Laptop Use Scale had some degree of convergent validity. One notable exception was the significant and positive correlation between nonacademic use of laptops outside the classroom and greater comfort level with computers. This same correlation was not significant with respect to academic use of laptops outside the classroom. Perhaps students who are very comfortable with computers use them within a much broader spectrum than students who are less comfortable, and extend their use to gaming and entertainment. Again, more indepth research needs to be conducted to understand the complexities of why students behave the way they do with laptops. Summary The purpose of this study was to develop a research-based, comprehensive scale to assess student use of laptops inside and outside of the classroom. The Laptop Use Scale (Laptop Use Scale) was comprised of four constructs: academic use (inside class), non- academic use (inside class), academic use (outside class), non-academic use (outside class). All scale constructs showed acceptable internally reliability. A principalcomponents factor analysis confirmed good construct validity. Correlations among the Laptop Use Scale constructs were significant but small enough to support the existence of four distinct constructs. Content validity was reinforced by frequencies of laptop use reported and student comments. Finally, the four Laptop Use Scale constructs were correlated some but not with all six of the variables used to establish convergent validity. Further research is recommended with respect to understanding the dynamics of academic and non-academic laptop use in in higher education classrooms, perhaps employing interviews or focus groups. Implications for education The ubiquity of student laptop ownership and use in higher educational institutions (Dahlstrom et al. 2013) requires a clear understanding of how these devices are used 123 38 R. Kay, S. Lauricella inside and outside the classroom. One starting point for assessing student use of laptops is to develop an evaluation scale. Previous studies have used an assortment of metrics that have produced mixed results (Awwad and Ayesh 2013; Barak et al. 2006; DiGangi et al. 2007; Fried 2008; Gaudreau et al. 2014; Kraushaar and Novak 2010; McCreary 2009; Skolnik and Puzo 2008). In this study, we developed a scale that was based on a composite of previous laptop us items, and grounded in current theory on technology-based learning (Mayes and de Freitas 2007) as well as typical behaviors observed in the Net Generation (Tapscott 2009). Reliability and validity estimates indicated that the Laptop Use Scale could be used to assess academic benefits and challenges experienced by higher education students. There are several possible scenarios for using the Laptop Use Scale in educational settings. First, this scale could be helpful for exploring the impact of specific teaching strategies on academic versus non-academic laptop behaviors in the classroom. Second, the use of laptops in variety of disciplines could be compared to determine the extent to which subject area and context influence the use of laptops in the classroom. Third, scale feedback could help assess and address non-academic use of laptops outside of the classroom. While instructors could and would not control behaviors beyond the realm of their personal classrooms, describing and disseminating patterns of use might be helpful to higher education students, particularly if they were in their first year of study. Fourth, having a reliable and valid tool for assessing laptop use is essential to test the effectiveness of various in-class, laptop management strategies such as having laptop-free zones, requiring students to close laptops at certain points in a class, and restricting access to particularly distracting sites. Caveats and future research Data from the Laptop Use Scale appeared to be reliable, valid, and grounded in previous research. However, there are several caveats that should be articulated for future research on student use of laptops. First, the sample size, while reasonably large, consisted of communication and education students. The Laptop Use Scale needs to be tested on students in a wider range of subject areas such as medicine, arts, law, engineering and science where use of laptops could vary substantially. Second, even though the participation response rate was over 80 %, there is risk of bias based on students whose chose to participate in the study and those who did not. However, it is not clear what the precise nature of this bias would be. To address potential biases in participation, future researchers might provide a small incentive to encourage all students to participate. Third, while qualitative data were collected in the form of written comments, it would be prudent to collect interview or focus group data to understand why students use certain tools or engage in particular activities inside and outside the classroom. Fourth, the Laptop Use Scale does not address multitasking or the switching between academic and non-academic laptop use. This behavior has been reported in 123 Assessing laptop use in higher education: The Laptop Use… 39 two recent laptop studies (Kraushaar and Novak 2010; Sana et al. 2013) and should be considered in future scale development. Fifth, the specific uses of laptops observed are partially linked to technological developments. Five years ago, social media tools would not have been relevant, but now they occupy considerable academic and non-academic attention from many higher education students. Consequently, specific academic and non-academic uses of laptops may need to be added or subtracted depending on how the technology changes. Sixth, to maximize the accuracy of assessing academic and non-academic in class laptop activities, it would particularly valuable and informative to install tracking software, like RescueTime, to record time spent specific laptops programs. Matching detailed tracking information with scale response data is one way to further establish validity of the Laptop Use Scale. Seventh, it would be a judicious next step to assess the predictive validity of the Laptop Use Scale constructs with student performance in the class where the laptop was being used. Finally, the Laptop Use Scale could be used to investigate individual differences in academic and non-academic laptop activities inside and outside of the classroom based on gender, grade level, computer experience, academic ability, and special learning needs. Appendix: Laptop Use Scale Background information 1. 2. 3. 4. 5. 6. 7. What is your gender? (Male, Female) What year of university are you in? (1, 2, 3 or 4) What was your average grade in all your courses last year? (\50, 50–59, 60–69, 70–70, 80–89, 90?) What course are you taking? _____________ What is your average in the course right now? (\50, 50–59, 60–69, 70–70, 80–89, 90?) How comfortable are you with using computer technology? (Not at all Comfortable, Somewhat Comfortable, Comfortable, Very Comfortable) About how many hours per day do you spend using your laptop computer? _____ 123 40 R. Kay, S. Lauricella Academic use DURING class How often did you do the following activities DURING class in this course? Never Rarely Sometimes Freq Very Freq 1. Take notes on my laptop 1 2 3 4 5 2. Use the notes posted by the instructor 1 2 3 4 5 3. Search the web for academic purposes 1 2 3 4 5 4. Use online interactive tools (for example learning objects, applets) 1 2 3 4 5 5. Participate in online surveys 1 2 3 4 5 6. Follow a PowerPoint presentation on your laptop computer 1 2 3 4 5 7. Communicate with peers for academic reasons (for example instant messaging, email) 1 2 3 4 5 8. Use a software program for academic purposes (e.g. Word, Excel, Access) 1 2 3 4 5 9. Overall what (if any) do you see are the biggest benefits to having a laptop IN class for this course? Why? Non-academic use DURING class How often did you do the following activities DURING class in this course? Never Rarely Sometimes Freq Very Freq 1. Play games 1 2 3 4 5 2. Watch movies 1 2 3 4 5 3. Watch short video clips for personal use (for example YouTube) 1 2 3 4 5 4. Search the web for personal reasons 1 2 3 4 5 5. Go on Facebook 1 2 3 4 5 6. Use Twitter 1 2 3 4 5 7. Use instant messaging for personal reasons (for example MSN, Skype) 1 2 3 4 5 8. Use email for personal reasons 1 2 3 4 5 9. Overall what (if any) do you see are the biggest distractions to having a laptop IN class for this course? Why? 123 Assessing laptop use in higher education: The Laptop Use… 41 Academic OUTSIDE if class How often did you do the following activities OUTSIDE of class in this course? Never Rarely Sometimes Freq Very Freq 1. Organizing course notes and materials 1 2 3 4 5 2. Search the web for academic purposes 1 2 3 4 5 3. Online interactive activities (for example learning objects. Applets) 1 2 3 4 5 4. Using a software program for academic purposes (for example Word, Excel) 1 2 3 4 5 5. Sharing notes and course resources 1 2 3 4 5 6. Communicate with peers for academic purposes (for example instant messaging, email) 1 2 3 4 5 7. Working with peers on assigned group work 1 2 3 4 5 8. Getting help from peers on computer related tasks 1 2 3 4 5 9. Searching the university library databases for articles/books 1 2 3 4 5 10. Overall what (if any) do you see are the biggest benefits to having a laptop OUTSIDE class? Why? Non-academic use DURING class How often did you do the following activities DURING class in this course? Never Rarely Sometimes Freq Very Freq 1. Play games 1 2 3 4 5 2. Watch movies 1 2 3 4 5 3. Watch short video clips for personal use (for example You Tube) 1 2 3 4 5 4. Search the web for personal reasons 1 2 3 4 5 5. Go on Facebook 1 2 3 4 5 6. Use Twitter 1 2 3 4 5 7. Use instant messaging for personal reasons (for example MSN, Skype) 1 2 3 4 5 8. Use email for personal reasons 1 2 3 4 5 123 42 9. 10. R. Kay, S. Lauricella Overall what (if any) do you see are the biggest distractions to having a laptop IN class for this course? Why? Overall what (if any) do you see are the biggest distractions to having a laptop OUTSIDE class? Why? References Aguilar-Roca, N. M., Williams, A. E., & O’Dowd, D. K. (2012). The impact of laptop-free zones on student performance and attitudes in large lectures. Computers and Education, 59(4), 1300–1308. doi:10.1016/j.compedu.2012.05.002. Annan-Coultas, D. L. (2012). Laptops as instructional tools: Student perceptions. TechTrends, 56(5), 34–41. Awwad, F., & Ayesh, A. (2013). Effectiveness of laptop usage in UAS university undergraduate teaching. The Turkish Online Journal of Educational Technology, 12(2), 77–88. Barak, M., Lipson, A., & Lerman, S. (2006). Wireless laptops as means for promoting active learning in large lecture halls. Journal of Research on Technology in Education, 38(3), 245–263. Barkhuus, L. (2005). Bring your own laptop unless you want to follow the lecture: Alternative communication in the classroom. In Proceedings of the 2005 International ACM SIGGROUP Conference on Supporting Group Work (pp. 140–143). New York, ACM. doi: 10.1145/1099203. 1099230. Bruner, J. (1960). The process of education. Cambridge, MA: Harvard University Press. Carfio, J., & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42(12), 1150–1152. doi:10.1111/j.1365-2923.2008.03172.x. Dahlstrom, E., Walker, J. D., & Dziuban, C, M. (2013). The ECAR study of undergraduate students and information technology (Research Report). Louiseville, CO: EDUCAUSE Center for Analysis and Research. Retrieved from https://net.educause.edu/ir/library/pdf/ERS1302/ERS1302.pdf Dalsgaard, C., & Godsk, M. (2007). Transforming traditional lectures into problem-based blended learning: Challenges and experiences. Open Learning: The Journal of Open, Distance and e-Learning, 22(1), 29–42. doi:10.1080/02680510601100143. Debevec, K., Shih, M., & Kashyap, V. (2006). Learning strategies and performance in a technology integrated classroom. Journal of Research on Technology in Education, 38(3), 293–307. DiGangi, S., Kilic, Z., Yu, C. H., Jannasch-Pennell, A., Long, L., Kim, C., et al. (2007). One to one computing in higher education: A survey of technology practices and needs. AACE Journal, 15(4), 367–387. Enfield, J. (2014). Looking at the impact of the flipped classroom model of instruction on undergraduate multimedia students at CSUN. TechTrends, 57(6), 14–27. doi:10.1007/s11528-013-0698-1. Field, A. (2005). Discovering statistics using SPSS (2nd ed.). Thousand Oaks, CA: Sage Publications. Fried, C. B. (2008). In-class laptop use and its effects on student learning. Computers and Education, 50(3), 906–914. Gagné, R. (1985). The conditions of learning. New York: Holt, Rinehart and Winston. Gaudreau, P., Miranda, D., & Gareau, A. (2014). Canadian university students in wireless classrooms: What do they do on their laptops and does it really matter? Computers and Education, 70(3), 245–255. doi:10.1016/j.compedu.2013.08.019. Hyden, P. (2005). Teaching statistics by taking advantage of the laptop’s ubiquity. New Directions for Teaching and Learning, 101, 37–42. Jamieson, S. (2004). Likert scales: how to (ab)use them. Medical Education, 38(12), 1217–1218. doi:10. 1111/j.1365-2929.2004.02012.x. Kay, R. H., & Lauricella, S. (2011). Exploring the benefits and challenges of using laptop computers in higher education classrooms: A formative analysis. Canadian Journal of Learning and Technology, 37(1). Retrieved from http://www.cjlt.ca/index.php/cjlt/article/view/565/299 Kay, R. H. & Lauricella, S. (2014). Investigating the benefits and challenges of using laptop computers in higher education. Canadian Journal of Learning and Technology, 40(2), 1–25. Retrieved from http://www.cjlt.ca/index.php/cjlt/article/view/831/393 123 Assessing laptop use in higher education: The Laptop Use… 43 Kline, P. (1999). The handbook of psychological testing (2nd ed.). London: Routledge. Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning and Education, 4(2), 193–212. doi:10.5465/ AMLE.2005.17268566. Kraushaar, J. M., & Novak, D. C. (2010). Examining the effects of student multitasking with laptops during the lecture. Journal of Information Systems Education, 21(2), 241–251. Kuzon, W. M., Urbanchek, M. G., & McCabe, S. (1996). The seven deadly sins of statistical analysis. Annals of Plastic Surgery, 37, 265–272. Lasry, N., Mazur, E., & Watkins, J. (2008). Peer instruction: From Harvard to the two-year college. American Association of Physics Teachers, 76(11), 1066–1069. Retrieved from http://mazur. harvard.edu/sentFiles/Mazur_61464.pdf Lauricella, S. & Kay, R. H. (2010). Assessing laptop use in higher education classrooms: The laptop effectiveness scale (LES). Australian Journal of Educational Technology, 26(2), 151–163. Retrieved from http://ajet.org.au/index.php/AJET/article/view/1087/342 Laurillard, D. (2002). Rethinking university teaching. A conversational framework for the effective use of learning technologies. London: Routledge. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press. Lewis, S. E., & Lewis, J. E. (2005). Departing from lectures: An evaluation of a peer-led guided inquiry alternative. Journal of Chemical Education, 82(1), 135. doi:10.1021/ed082p135. Lindroth, T., & Bergquist, M. (2010). Laptopers in an educational practice: Promoting the personal learning situation. Computers and Education, 54(2), 311–320. doi:10.1016/j.compedu.2009.07.014. Mayes, T., & de Freitas, S. (2007). Technology-enhanced learning: The role of theory. In H. Beetham & R. Sharpe (Eds.), Rethinking pedagogy for a digital age (pp. 17–30). New York: Routledge. McCreary, J. R. (2009). The laptop-free zone. Valparaiso University Law Review, 43, 1–87. Miles, M. B., & Hubrman, M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publication. Murray, J. (2013). Likert data: What to use, parametric or non-parametric? International Journal of Business and Social Science, 4(11), 258–264. Newell, A. (1980). One final word. In D. T. Tuma & F. Reif (Eds.), Problem solving and education: Issues in teaching and research (pp. 175–189). Hillsdale, New Jersey: Lawrence Erlbaum Associates. Norman, G. (2010). Likert scales, levels of measurement and the ‘‘laws’’ of statistics. Advances in Health Sciences Education—Theory and Practice, 15(5), 625–632. doi:10.1007/s10459-010-9222-y. Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill. Pew Research Center. (2013). Social media update 2013. Washington: DC. Retrieved from http://www. pewinternet.org/files/2013/12/PIP_Social-Networking-2013.pdf Piaget, J. (1970). Science of education and the psychology of the child. New York: Orion Press. Prensky, M. (2010). Teaching digital natives. Thousand Oaks, CA: Corwin. Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers and Education, 62(1), 24–31. doi:10.1016/j.compedu.2012.10.003. Skolnik, R., & Puzo, M. (2008). Utilization of laptop computers in the school of business classroom. Academy of Educational Leadership Journal, 12(2), 1–10. Small, G., & Vorgan, G. (2009). iBrain: Surviving the technological alteration of the modern mind. New York: Harper. Tapscott, D. (2009). Grown up digital: How the Net generation is changing your world. New York: McGraw-Hill. Vygotsky, L. S. (1978). Mind and society. Cambridge, MA: Harvard University Press. Wilson, B. G., & Myers, K. M. (2000). Situated cognition is theoretical and practical context. In D. H. Jonassen & S. M. Land (Eds.), Theoretical foundations of learning environments (pp. 57–86). New Jersey: Lawrence Erlbaum. Wurst, C., Smarkola, C., & Gaffney, M. A. (2008). Ubiquitous laptop usage in higher education: Effects on student achievement, student satisfaction, and constructivist measures in honors and traditional classrooms. Computers and Education, 51(4), 1766–1783. doi:10.1016/j.compedu.2008.05.006. 123 44 R. Kay, S. Lauricella Robin Kay is currently a Full Professor and the Director of Graduate Studies in the Faculty of Education at the University of Ontario Institute Of Technology in Oshawa, Canada. He has published over 120 articles, chapters and conference papers in the area of computers in education, is a reviewer for five prominent computer education journals, and has taught computer science, mathematics, and educational technology for over 20 years at the high school, college, undergraduate and graduate level. Current projects include research on laptop use in higher education, BYOD in K-12 education, web-based learning tools, classroom response systems, e-learning in secondary and higher education, video podcasts, scale development with respect to computer attitude, use, and behaviour, gender differences in computer related behaviour, emotions and the use of computers, the impact of social media tools in education, and factors that influence how students learn with technology. Sharon Lauricella is an award-winning Associate Professor. She is a two-time recipient of the UOIT Teaching Award, the CJPS Faculty Teaching Award, and has been nominated for provincial and national teaching recognition. Sharon holds a doctoral degree from Cambridge University in England. Her undergraduate work was completed in Boston, Massachusetts, and Edinburgh, Scotland. Sharon instructs courses including Nonviolent Communication, Professional Writing, Advanced Writing, Communication Ethics, and Public Speaking. Sharon is keenly interested in the interplay between spirituality and communication and student experiences with technology in learning. 123