As student enrollments in computer science increase, there is a growing need for pedagogies that ... more As student enrollments in computer science increase, there is a growing need for pedagogies that scale. Recent evidence has shown Peer Instruction (PI) to be an effective in-class pedagogy that reports high student satisfaction even with large classes. Yet, the question of the scalability of traditional lecture versus PI is largely unexplored. To explore this question, this work examines publicly available student evaluations of computer science courses across a wide range of class sizes (50--374 students) over a four year period. It first compares evaluations regardless of size and confirms prior work that PI classes are better appreciated by students than traditional lecture. It then examines how course evaluations change with class size and provides evidence that PI achieves a smaller decline in evaluations as class size increases.
Peer Instruction (PI) is a teaching method that supports studentcentric classrooms, where student... more Peer Instruction (PI) is a teaching method that supports studentcentric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success-the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to Standard Instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of widespread adoption of PI in computing programs.
A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of... more A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of a particular topic. This work presents a CI for Basic Data Structures (BDSI) and the process by which the CI was designed and validated. We discuss: 1) the collection of faculty opinions from diverse institutions on what belongs on the instrument, 2) a series of interviews with students to identify their conceptions and misconceptions of the content, 3) an iterative design process of developing draft questions, conducting interviews with students to ensure the questions on the instrument are interpreted properly, and collecting faculty feedback on the questions themselves, and 4) a statistical evaluation of final versions of the instrument to ensure its internal validity. We also provide initial results from pilot runs of the CI.
With growing enrolment and ongoing research into best practices for team work, many instructors a... more With growing enrolment and ongoing research into best practices for team work, many instructors are rethinking how to form, evaluate, and manage teams. In this BoF, instructors will discuss the strategies they have employed, the effectiveness of those approaches, and the tools that support administering teams. Discussion topics may include self- and peer-evaluation, grading strategies, software support, and conflict resolution.
Although Massive Open Online Courses have the potential to reach a much broader audience and offe... more Although Massive Open Online Courses have the potential to reach a much broader audience and offer a lower cost education than traditional in-person classes, they have struggled with low completion rates and low diversity amongst those enrolled and completing the courses. In 2015, we built a series of online courses in computing with the specific goal of attracting and retaining students from groups underrepresented in computing. In our design, we incorporated a number of features aimed at improving the inclusive nature of the courses including: a project-centered course design; an online version of Peer Instruction ConceptTests; videos where students, faculty, and professionals report their struggles when they first learned computing concepts; videos by professional software engineers explaining how computing concepts from the course are used in industry; and videos aimed at providing additional support on the project to students who might be struggling. In this work, we report on the design of the courses and examine how successful our courses were at attracting and retaining women students. We find that compared to other computing courses offered by our institution on the same platform, our courses have: a higher percentage of women enrollment, higher rates of course completion for both men and women, and a slightly smaller gap between completion rates for men and women. CCS CONCEPTS • Social and professional topics → Computing Education.
Our prior work found differences in study habits between high- and low-performers in a small-scal... more Our prior work found differences in study habits between high- and low-performers in a small-scale qualitative study, and this work seeks to verify and extend these findings by examining the study habits of a larger population of CS1 students. To do this, we devised a survey based on the findings of our prior qualitative study. The responses of CS1 students reveals that some study habits are more frequently practiced by higher-performers then lower-performers or vice versa. One concern with these findings is that the differences in study habits might simply be explained by prior experience. As such, we compare study habits between students with and without prior experience as well. We find that although prior experience translates to better class performance, it is not associated with the same study habits as lower- and higher-performers, suggesting that prior experience and study habits are separately associated with better student performance. These findings encourage further inquiry into the role of study habits in student success and whether explicit instruction on better study habits might be the basis for successful future interventions.
Computer science students struggle in early computing courses as evinced by high failure rates an... more Computer science students struggle in early computing courses as evinced by high failure rates and poor retention. As such, studies have attempted to characterize the root of student struggles from many perspectives, including cognitive, meta-cognitive, and social emotional. Typically, studies have limited their inquiry to a specific perspective or a single course. This paper reports the results of a broad student experience survey conducted across several computer science courses. Through a periodic survey, students rated various cognitive, socio-emotional, external, personal, and structural barriers in terms of how much each impacted their learning throughout the term. An exploratory factor analysis of these questions revealed four factors-personal obligations, lack of sense of belonging, in-class confusion, and lack of confidence-that capture a range of possible struggles students may face. We analyzed the prevalence of these factors across courses, performance quartiles, and demographic groups broken down by gender, race/ethnicity, and matriculation status. Students in lower performance quartiles report higher stress levels on multiple factors, with statistically significant differences found between all quartiles and courses, for most factors. Moreover, students from traditionally underrepresented groups report struggling more across all four factors, suggesting that they may be facing more challenges than classmates from represented populations. Overall, these findings indicate that student struggles are associated with stresses from many areas of their lives, suggesting that future interventions should target multiple areas of stress. CCS CONCEPTS • Social and professional topics → Computing Education.
Students' sense of belonging has been found to be connected to student retention in higher educat... more Students' sense of belonging has been found to be connected to student retention in higher education. In computing education, prior studies suggest that a hostile culture and a feeling of nonbelonging can lead women, Black, Latinx, Native American, and Pacific Islander students to drop out of the computing field at a disproportionately high rate. Yet, we know relatively little about how computing students' sense of belonging presents and evolves (if at all) through their college courses, particularly in courses beyond the introductory level, and little is known about how sense of belonging impacts student outcomes in computing. In an extension of a previous study, we examined students' sense of belonging in six early undergraduate computer science courses across three consecutive quarters at a large research-intensive institution in North America. We found that women and first generation students have a lower incoming sense of belonging across all courses. When exploring sense of belonging's tie to student outcomes we found that lower sense of belonging was correlated with negative course outcomes in terms of pass rates and course performance. We also found that it is less tied to student performance as students get further into the CS curriculum. Surprisingly, there was no indication that sense of belonging is predictive of retention in terms of persistence to the next CS course outside of the first course in our two-course CS1 sequence. CCS CONCEPTS • Social and professional topics → Computing Education.
Although recent work in computing has discovered multiple techniques to identify low-performing s... more Although recent work in computing has discovered multiple techniques to identify low-performing students in a course, it is unclear what factors contribute to those students' difficulties. If we were able to better understand the characteristics of such students, we may be better able to help those students. This work examines the characteristics of low-and high-performing students through interviews with students from an introductory computing class. We identify a number of relevant areas of student behavior including how they approach their exam studies, how they approach completing programming assignments, whether they sought help after identifying misunderstandings, how and from whom they sought help, and how they reflected on assignments after submitting them. Particular behaviors within each area are coded and differences between groups of students are identified.
Look at What's to Come I would like to take this opportunity to welcome back all UAAD members and... more Look at What's to Come I would like to take this opportunity to welcome back all UAAD members and invite any prospective members to join us this year. Our first meeting will be held September 19, 2007 in the City Union from 11:45 a.m.-1:00 p.m. It is a brown bag luncheon so bring your favorite food for lunch. Bring your favorite coworker to this meeting as well! Our speaker is Diane Sullivan from the Organization Station. Her topic is "Organizing Your Life and Having Your Weekends Free." I don't know if there is a correlation here, but this topic was added right after our program chairmen saw my desk! Please join us and see how YOU can get some organization in YOUR life!
Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1
Live coding-a pedagogical technique in which an instructor plans, writes, and executes code in fr... more Live coding-a pedagogical technique in which an instructor plans, writes, and executes code in front of a class-is generally considered a best practice when teaching programming. However, only a few studies have evaluated the effect of live coding on student learning in a controlled experiment and most of the literature relating to live coding identifies students' perceived benefits of live-coding examples. In order to empirically evaluate the impact of live coding, we designed a controlled experiment in a CS1 course taught in Python at a large public university. In the two remote lecture sections for the course, one was taught using live-coding examples and the other was taught using static-code examples. Throughout the term, we collected code snapshots from students' programming assignments, students' grades, and the questions that they asked during the remote lectures. We then applied a set of process-oriented programming metrics to students' programming data to compare students' adherence to effective programming processes in the two learning groups and categorized each question asked in lectures following an open-coding approach. Our results revealed a general lack of difference between the two groups across programming processes, grades, and lecture questions asked. However, our experiment uncovered minimal effects in favor of the live-coding group indicating improved programming processes but lower performance on assignments and grades. Our results suggest an overall insignificant impact of the style of presenting code examples, though we reflect on the threats to validity in our study that should be addressed in future work.
Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, 2018
Jigsaw learning is a cooperative learning technique enabling students to teach and learn from the... more Jigsaw learning is a cooperative learning technique enabling students to teach and learn from their peers. Although prior studies investigated the efficacy of Jigsaw learning in computing education by measuring student performance after Jigsaw activities, this work reports on student and instructor experiences with Jigsaw learning. Jigsaw activities were performed in lectures throughout the term and student experience data was collected through student surveys. The survey results reveal that 72% of survey respondents thought the Jigsaw activities helped their learning. Furthermore, 75% of respondents acknowledged their role in class as a more engaged learner and 44% of those students identified themselves as taking on the role of a teacher. The instructor found Jigsaw learning to be labor intensive but observed higher levels of student engagement.
Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, 2021
This panel explores experiences and insights from three successful propagators into how others ca... more This panel explores experiences and insights from three successful propagators into how others can successfully encourage more wide-spread use of their innovations. Issues covered include designing for dissemination, techniques for recruiting potential adopters, suggestions for convincing faculty to try an innovation and continue using it, and identifying points of friction and overcoming resistance from administrators, students, and/or peers. These topics are discussed and illustrated with personal experiences and anecdotes from our illustrious panelists.
United Kingdom and Ireland Computing Education Research conference., 2021
As instructors seek to design their courses to improve student experiences or outcomes, they can ... more As instructors seek to design their courses to improve student experiences or outcomes, they can benefit from the rich literature on education, STEM education, and computing education. In turn, instructors have an opportunity to contribute to that literature by carefully designing and evaluating their courses. In this talk, we will examine a few examples of teaching informing my research or vice versa. These examples include using the educational literature to improve our introductory computing course, addressing our need to compare student learning between course offerings by creating a validated assessment, and leveraging our background working with students to better understand how students experience computing courses. Such efforts offer both the benefit of improving our own teaching experiences and contributing to the broader computing education community.
In this feature of the Bulletin, we highlight members of the SIGCSE community. In this issue, Bul... more In this feature of the Bulletin, we highlight members of the SIGCSE community. In this issue, Bulletin co-editor Leo Porter interviewed Bob Aiken. In 1995 Bob received the SIGCSE Outstanding Contribution Computer Science Education Award and in 1999 received the ACM SIGCSE Lifetime Service to Computer Science Education for his advocacy for computer science education both in the US and abroad. Bob was elected an ACM Fellow in 2002. His background is in artificial intelligence, educational uses of computers, and online collaboration tools. He is working on an article about the history of SIGCSE which will appear in the December issue of Inroads for the 50th year celebration of SIGCSE.
Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, 2016
Ten selected questions have been included in 13 introductory programming exams at seven instituti... more Ten selected questions have been included in 13 introductory programming exams at seven institutions in five countries. The students' results on these questions, and on the exams as a whole, lead to the development of a benchmark against which the exams in other introductory programming courses can be assessed. We illustrate some potential benefits of comparing exam performance against this benchmark, and show other uses to which it can be put, for example to assess the size and the overall difficulty of an exam. We invite others to apply the benchmark to their own courses and to share the results with us.
Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, 2017
As class-sizes grow in computer science, the personal attention received by students tends to dim... more As class-sizes grow in computer science, the personal attention received by students tends to diminish. This work aims to replicate small-class community effects within a large class by creating "micro-classes"-small groups within the large class. These micro-classes consist of 20-30 students led by graduate teaching assistants and undergraduate tutors who are specifically trained in small-classroom instructional techniques. This paper studies the outcomes of the micro-classes framework in an upper-division data structures course and compares them to outcomes from the same class taught in a large lecture, active-learning format. Students report increased satisfaction and a higher perception of community in the micro-classes section, though there was no discernible difference in student academic performance.
Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
Incremental development is the process of writing a small snippet of code and testing it before m... more Incremental development is the process of writing a small snippet of code and testing it before moving on. For students in introductory programming courses, the value of incremental development is especially higher as they may suffer from more syntax errors, lack the proficiency to address complicated bugs, and may be more prone to frustration when struggling to correct code. However, to evaluate the effectiveness of interventions that aim to teach programming processes such as incremental development, we need to develop measures to assess such processes. In this paper, we present a way to measure incremental development. By qualitatively analyzing 15 student coding interviews, we identified common behaviors in the programming process that relate to incremental development. We then leveraged a dataset of over 1000 development sessions-about 52,000 code snapshots at compilation time-to automatically detect the common behaviors identified in our qualitative analysis. Finally, we crafted a formal metric, called the "Measure of Incremental Development" (MID), to quantify how effectively a student used incremental development during a programming session. The MID detects common non-incremental development patterns such as excessive debugging after large additions of code to automatically assess a sequence of snapshots. The MID aligns with human evaluations of incrementality with over 80% accuracy. Our metric enables new research directions and interventions focused on improving students' development practices. CCS CONCEPTS • Software and its engineering → Software development techniques; • Social and professional topics → Student assessment.
Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
Many institutions use undergraduate teaching assistants (tutors) in their computing courses to he... more Many institutions use undergraduate teaching assistants (tutors) in their computing courses to help provide more resources to students. Because of the role tutors play in students' learning experiences, recent work in computing education has begun to explore studenttutor interactions through the tutor's perspective and through direct observation of the interactions. The results suggest that these interactions are cognitively challenging for tutors and may not be as beneficial for students' learning as one might hope. Given that many of these interactions may be unproductive, this work seeks to understand how student expectations of these sessions might be impacting the interactions' effectiveness. We interviewed 15 students in a CS2 course to learn about the expectations and desires that students have when they attend tutoring sessions. Our findings indicate that there is variation in what students consider a desired result from the interaction, that assignment deadlines affect students' expectations and desires for interactions, and that students do not always want what they believe is beneficial for their learning. We discuss implications for instructors and potential guidance for students and tutors to make tutoring sessions more effective.
Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
Recent research in computing has shown that student performance on prerequisite course content va... more Recent research in computing has shown that student performance on prerequisite course content varies widely, even when students continue to progress further through the computing curriculum. Our work investigates instructors' perspectives on the purpose of prerequisite courses and whether that purpose is being fulfilled. In order to identify the range of instructor views, we interviewed twenty-one computer science instructors, at two institutions, that teach a variety of courses in their respective departments. We conducted a phenomenographic analysis on the interview transcripts, which revealed a wide variety of views on prerequisite courses. The responses shed light on various issues with prerequisite course knowledge, as well as issues around responsibility and conflicting pressures on instructors. These issues arise at the department level, as well as with individual course offerings.
As student enrollments in computer science increase, there is a growing need for pedagogies that ... more As student enrollments in computer science increase, there is a growing need for pedagogies that scale. Recent evidence has shown Peer Instruction (PI) to be an effective in-class pedagogy that reports high student satisfaction even with large classes. Yet, the question of the scalability of traditional lecture versus PI is largely unexplored. To explore this question, this work examines publicly available student evaluations of computer science courses across a wide range of class sizes (50--374 students) over a four year period. It first compares evaluations regardless of size and confirms prior work that PI classes are better appreciated by students than traditional lecture. It then examines how course evaluations change with class size and provides evidence that PI achieves a smaller decline in evaluations as class size increases.
Peer Instruction (PI) is a teaching method that supports studentcentric classrooms, where student... more Peer Instruction (PI) is a teaching method that supports studentcentric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success-the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to Standard Instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of widespread adoption of PI in computing programs.
A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of... more A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of a particular topic. This work presents a CI for Basic Data Structures (BDSI) and the process by which the CI was designed and validated. We discuss: 1) the collection of faculty opinions from diverse institutions on what belongs on the instrument, 2) a series of interviews with students to identify their conceptions and misconceptions of the content, 3) an iterative design process of developing draft questions, conducting interviews with students to ensure the questions on the instrument are interpreted properly, and collecting faculty feedback on the questions themselves, and 4) a statistical evaluation of final versions of the instrument to ensure its internal validity. We also provide initial results from pilot runs of the CI.
With growing enrolment and ongoing research into best practices for team work, many instructors a... more With growing enrolment and ongoing research into best practices for team work, many instructors are rethinking how to form, evaluate, and manage teams. In this BoF, instructors will discuss the strategies they have employed, the effectiveness of those approaches, and the tools that support administering teams. Discussion topics may include self- and peer-evaluation, grading strategies, software support, and conflict resolution.
Although Massive Open Online Courses have the potential to reach a much broader audience and offe... more Although Massive Open Online Courses have the potential to reach a much broader audience and offer a lower cost education than traditional in-person classes, they have struggled with low completion rates and low diversity amongst those enrolled and completing the courses. In 2015, we built a series of online courses in computing with the specific goal of attracting and retaining students from groups underrepresented in computing. In our design, we incorporated a number of features aimed at improving the inclusive nature of the courses including: a project-centered course design; an online version of Peer Instruction ConceptTests; videos where students, faculty, and professionals report their struggles when they first learned computing concepts; videos by professional software engineers explaining how computing concepts from the course are used in industry; and videos aimed at providing additional support on the project to students who might be struggling. In this work, we report on the design of the courses and examine how successful our courses were at attracting and retaining women students. We find that compared to other computing courses offered by our institution on the same platform, our courses have: a higher percentage of women enrollment, higher rates of course completion for both men and women, and a slightly smaller gap between completion rates for men and women. CCS CONCEPTS • Social and professional topics → Computing Education.
Our prior work found differences in study habits between high- and low-performers in a small-scal... more Our prior work found differences in study habits between high- and low-performers in a small-scale qualitative study, and this work seeks to verify and extend these findings by examining the study habits of a larger population of CS1 students. To do this, we devised a survey based on the findings of our prior qualitative study. The responses of CS1 students reveals that some study habits are more frequently practiced by higher-performers then lower-performers or vice versa. One concern with these findings is that the differences in study habits might simply be explained by prior experience. As such, we compare study habits between students with and without prior experience as well. We find that although prior experience translates to better class performance, it is not associated with the same study habits as lower- and higher-performers, suggesting that prior experience and study habits are separately associated with better student performance. These findings encourage further inquiry into the role of study habits in student success and whether explicit instruction on better study habits might be the basis for successful future interventions.
Computer science students struggle in early computing courses as evinced by high failure rates an... more Computer science students struggle in early computing courses as evinced by high failure rates and poor retention. As such, studies have attempted to characterize the root of student struggles from many perspectives, including cognitive, meta-cognitive, and social emotional. Typically, studies have limited their inquiry to a specific perspective or a single course. This paper reports the results of a broad student experience survey conducted across several computer science courses. Through a periodic survey, students rated various cognitive, socio-emotional, external, personal, and structural barriers in terms of how much each impacted their learning throughout the term. An exploratory factor analysis of these questions revealed four factors-personal obligations, lack of sense of belonging, in-class confusion, and lack of confidence-that capture a range of possible struggles students may face. We analyzed the prevalence of these factors across courses, performance quartiles, and demographic groups broken down by gender, race/ethnicity, and matriculation status. Students in lower performance quartiles report higher stress levels on multiple factors, with statistically significant differences found between all quartiles and courses, for most factors. Moreover, students from traditionally underrepresented groups report struggling more across all four factors, suggesting that they may be facing more challenges than classmates from represented populations. Overall, these findings indicate that student struggles are associated with stresses from many areas of their lives, suggesting that future interventions should target multiple areas of stress. CCS CONCEPTS • Social and professional topics → Computing Education.
Students' sense of belonging has been found to be connected to student retention in higher educat... more Students' sense of belonging has been found to be connected to student retention in higher education. In computing education, prior studies suggest that a hostile culture and a feeling of nonbelonging can lead women, Black, Latinx, Native American, and Pacific Islander students to drop out of the computing field at a disproportionately high rate. Yet, we know relatively little about how computing students' sense of belonging presents and evolves (if at all) through their college courses, particularly in courses beyond the introductory level, and little is known about how sense of belonging impacts student outcomes in computing. In an extension of a previous study, we examined students' sense of belonging in six early undergraduate computer science courses across three consecutive quarters at a large research-intensive institution in North America. We found that women and first generation students have a lower incoming sense of belonging across all courses. When exploring sense of belonging's tie to student outcomes we found that lower sense of belonging was correlated with negative course outcomes in terms of pass rates and course performance. We also found that it is less tied to student performance as students get further into the CS curriculum. Surprisingly, there was no indication that sense of belonging is predictive of retention in terms of persistence to the next CS course outside of the first course in our two-course CS1 sequence. CCS CONCEPTS • Social and professional topics → Computing Education.
Although recent work in computing has discovered multiple techniques to identify low-performing s... more Although recent work in computing has discovered multiple techniques to identify low-performing students in a course, it is unclear what factors contribute to those students' difficulties. If we were able to better understand the characteristics of such students, we may be better able to help those students. This work examines the characteristics of low-and high-performing students through interviews with students from an introductory computing class. We identify a number of relevant areas of student behavior including how they approach their exam studies, how they approach completing programming assignments, whether they sought help after identifying misunderstandings, how and from whom they sought help, and how they reflected on assignments after submitting them. Particular behaviors within each area are coded and differences between groups of students are identified.
Look at What's to Come I would like to take this opportunity to welcome back all UAAD members and... more Look at What's to Come I would like to take this opportunity to welcome back all UAAD members and invite any prospective members to join us this year. Our first meeting will be held September 19, 2007 in the City Union from 11:45 a.m.-1:00 p.m. It is a brown bag luncheon so bring your favorite food for lunch. Bring your favorite coworker to this meeting as well! Our speaker is Diane Sullivan from the Organization Station. Her topic is "Organizing Your Life and Having Your Weekends Free." I don't know if there is a correlation here, but this topic was added right after our program chairmen saw my desk! Please join us and see how YOU can get some organization in YOUR life!
Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1
Live coding-a pedagogical technique in which an instructor plans, writes, and executes code in fr... more Live coding-a pedagogical technique in which an instructor plans, writes, and executes code in front of a class-is generally considered a best practice when teaching programming. However, only a few studies have evaluated the effect of live coding on student learning in a controlled experiment and most of the literature relating to live coding identifies students' perceived benefits of live-coding examples. In order to empirically evaluate the impact of live coding, we designed a controlled experiment in a CS1 course taught in Python at a large public university. In the two remote lecture sections for the course, one was taught using live-coding examples and the other was taught using static-code examples. Throughout the term, we collected code snapshots from students' programming assignments, students' grades, and the questions that they asked during the remote lectures. We then applied a set of process-oriented programming metrics to students' programming data to compare students' adherence to effective programming processes in the two learning groups and categorized each question asked in lectures following an open-coding approach. Our results revealed a general lack of difference between the two groups across programming processes, grades, and lecture questions asked. However, our experiment uncovered minimal effects in favor of the live-coding group indicating improved programming processes but lower performance on assignments and grades. Our results suggest an overall insignificant impact of the style of presenting code examples, though we reflect on the threats to validity in our study that should be addressed in future work.
Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, 2018
Jigsaw learning is a cooperative learning technique enabling students to teach and learn from the... more Jigsaw learning is a cooperative learning technique enabling students to teach and learn from their peers. Although prior studies investigated the efficacy of Jigsaw learning in computing education by measuring student performance after Jigsaw activities, this work reports on student and instructor experiences with Jigsaw learning. Jigsaw activities were performed in lectures throughout the term and student experience data was collected through student surveys. The survey results reveal that 72% of survey respondents thought the Jigsaw activities helped their learning. Furthermore, 75% of respondents acknowledged their role in class as a more engaged learner and 44% of those students identified themselves as taking on the role of a teacher. The instructor found Jigsaw learning to be labor intensive but observed higher levels of student engagement.
Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, 2021
This panel explores experiences and insights from three successful propagators into how others ca... more This panel explores experiences and insights from three successful propagators into how others can successfully encourage more wide-spread use of their innovations. Issues covered include designing for dissemination, techniques for recruiting potential adopters, suggestions for convincing faculty to try an innovation and continue using it, and identifying points of friction and overcoming resistance from administrators, students, and/or peers. These topics are discussed and illustrated with personal experiences and anecdotes from our illustrious panelists.
United Kingdom and Ireland Computing Education Research conference., 2021
As instructors seek to design their courses to improve student experiences or outcomes, they can ... more As instructors seek to design their courses to improve student experiences or outcomes, they can benefit from the rich literature on education, STEM education, and computing education. In turn, instructors have an opportunity to contribute to that literature by carefully designing and evaluating their courses. In this talk, we will examine a few examples of teaching informing my research or vice versa. These examples include using the educational literature to improve our introductory computing course, addressing our need to compare student learning between course offerings by creating a validated assessment, and leveraging our background working with students to better understand how students experience computing courses. Such efforts offer both the benefit of improving our own teaching experiences and contributing to the broader computing education community.
In this feature of the Bulletin, we highlight members of the SIGCSE community. In this issue, Bul... more In this feature of the Bulletin, we highlight members of the SIGCSE community. In this issue, Bulletin co-editor Leo Porter interviewed Bob Aiken. In 1995 Bob received the SIGCSE Outstanding Contribution Computer Science Education Award and in 1999 received the ACM SIGCSE Lifetime Service to Computer Science Education for his advocacy for computer science education both in the US and abroad. Bob was elected an ACM Fellow in 2002. His background is in artificial intelligence, educational uses of computers, and online collaboration tools. He is working on an article about the history of SIGCSE which will appear in the December issue of Inroads for the 50th year celebration of SIGCSE.
Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, 2016
Ten selected questions have been included in 13 introductory programming exams at seven instituti... more Ten selected questions have been included in 13 introductory programming exams at seven institutions in five countries. The students' results on these questions, and on the exams as a whole, lead to the development of a benchmark against which the exams in other introductory programming courses can be assessed. We illustrate some potential benefits of comparing exam performance against this benchmark, and show other uses to which it can be put, for example to assess the size and the overall difficulty of an exam. We invite others to apply the benchmark to their own courses and to share the results with us.
Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, 2017
As class-sizes grow in computer science, the personal attention received by students tends to dim... more As class-sizes grow in computer science, the personal attention received by students tends to diminish. This work aims to replicate small-class community effects within a large class by creating "micro-classes"-small groups within the large class. These micro-classes consist of 20-30 students led by graduate teaching assistants and undergraduate tutors who are specifically trained in small-classroom instructional techniques. This paper studies the outcomes of the micro-classes framework in an upper-division data structures course and compares them to outcomes from the same class taught in a large lecture, active-learning format. Students report increased satisfaction and a higher perception of community in the micro-classes section, though there was no discernible difference in student academic performance.
Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
Incremental development is the process of writing a small snippet of code and testing it before m... more Incremental development is the process of writing a small snippet of code and testing it before moving on. For students in introductory programming courses, the value of incremental development is especially higher as they may suffer from more syntax errors, lack the proficiency to address complicated bugs, and may be more prone to frustration when struggling to correct code. However, to evaluate the effectiveness of interventions that aim to teach programming processes such as incremental development, we need to develop measures to assess such processes. In this paper, we present a way to measure incremental development. By qualitatively analyzing 15 student coding interviews, we identified common behaviors in the programming process that relate to incremental development. We then leveraged a dataset of over 1000 development sessions-about 52,000 code snapshots at compilation time-to automatically detect the common behaviors identified in our qualitative analysis. Finally, we crafted a formal metric, called the "Measure of Incremental Development" (MID), to quantify how effectively a student used incremental development during a programming session. The MID detects common non-incremental development patterns such as excessive debugging after large additions of code to automatically assess a sequence of snapshots. The MID aligns with human evaluations of incrementality with over 80% accuracy. Our metric enables new research directions and interventions focused on improving students' development practices. CCS CONCEPTS • Software and its engineering → Software development techniques; • Social and professional topics → Student assessment.
Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
Many institutions use undergraduate teaching assistants (tutors) in their computing courses to he... more Many institutions use undergraduate teaching assistants (tutors) in their computing courses to help provide more resources to students. Because of the role tutors play in students' learning experiences, recent work in computing education has begun to explore studenttutor interactions through the tutor's perspective and through direct observation of the interactions. The results suggest that these interactions are cognitively challenging for tutors and may not be as beneficial for students' learning as one might hope. Given that many of these interactions may be unproductive, this work seeks to understand how student expectations of these sessions might be impacting the interactions' effectiveness. We interviewed 15 students in a CS2 course to learn about the expectations and desires that students have when they attend tutoring sessions. Our findings indicate that there is variation in what students consider a desired result from the interaction, that assignment deadlines affect students' expectations and desires for interactions, and that students do not always want what they believe is beneficial for their learning. We discuss implications for instructors and potential guidance for students and tutors to make tutoring sessions more effective.
Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
Recent research in computing has shown that student performance on prerequisite course content va... more Recent research in computing has shown that student performance on prerequisite course content varies widely, even when students continue to progress further through the computing curriculum. Our work investigates instructors' perspectives on the purpose of prerequisite courses and whether that purpose is being fulfilled. In order to identify the range of instructor views, we interviewed twenty-one computer science instructors, at two institutions, that teach a variety of courses in their respective departments. We conducted a phenomenographic analysis on the interview transcripts, which revealed a wide variety of views on prerequisite courses. The responses shed light on various issues with prerequisite course knowledge, as well as issues around responsibility and conflicting pressures on instructors. These issues arise at the department level, as well as with individual course offerings.
Uploads
Papers by Leo Porter