ASEE 2015 12476 Final
ASEE 2015 12476 Final
ASEE 2015 12476 Final
American
c Society for Engineering Education, 2015
Correlation between engineering students’ performance in
mathematics and academic success
Abstract
It is quite popular among engineering educators to suppose that the academic performance of
undergraduate engineering students depends on their mathematics skills, but there is only a
little data available that substantiates this assumption. This study aims at shedding some light
on this perceived dependency by comparing examination results in mathematics with those of
the core engineering subjects over a period of more than ten years.
To identify the relationship between the students’ individual mathematical proficiency and
their performance in applied engineering subjects, the examination performance of students in
first, second and third semester mathematics courses has been correlated with their
corresponding performances in mechanics and other mechanical engineering subjects. The
results show that there is a significant positive correlation between the mathematics and
mechanics grades; a correlation between mathematics and other engineering core subjects also
exists but is, in general, less distinct.
The study has been supplemented with a comparison of the dropout rate of students enrolled
in the classes from 2002 until 2009 with their performance on an anonymous pre-course
diagnostic test of mathematical skills, which took place every year in the first week of study.
Prior studies on the relationships between students’ university entry scores and their
performance in terms of grade point averages showed that the expected correlations either do
not exist or are too weak to base educational interventions on. A comparison of pre-college
mathematics skills with degree program drop-out rates, on the other hand, shows a clear trend
that a below-average high school mathematics education entails an elevated risk of drop-out.
Introduction
Retention of students at colleges and universities has long been a concern for educators, and
several univariate and multiple-variable analysis models have been developed for the
prediction of a student’s probability to drop out from university (for an overview see e.g.
Murtaugh, Burns, and Schuster1). Precollege characteristics - like high school grade point
averages - as well as university entrance exams have, in general, turned out to be useful
predictors of student retention.
Due to the trend that the popularity of engineering degrees among undergraduates temporarily
declined in recent years, we are confronted with an increased number of ‘at risk’ students
attending our engineering department. This situation led to comparably high attrition rates of
over 50 % in the degree program under consideration, which by many faculty members are
mainly attributed to difficulties encountered in the mathematical content of the program.
These difficulties are thought to arise from a lack of understanding as to what engineering
involves and an insufficient mathematical preparedness.
The competence of engineers rests to a large extent on their mathematical training, since
mathematics is not only a set of tools to model and analyze systems; it also provides training
in logical reasoning. Within an online survey involving more than 5700 registered engineers,
Goold and Devitt6 found out that, while almost two thirds of engineers use high level
curriculum mathematics in engineering practice, mathematical thinking has an even greater
relevance to engineers’ work. Nevertheless, there are problems associated with the role of
mathematics in engineering education, in particular related to attracting and retaining students
in engineering degree programmes.
In the present work, the examination performance of students in first and second year
mathematics courses are compared with their corresponding performances in mechanics and
other mechanical engineering subjects. This study was initiated by the finding of an
astonishing conformity of the dropout rates and students’ performances on a pre-course
mathematics diagnostics test over a period of eight years.
Comparison of the dropout rate and the students’ performance on a pre-course
mathematics diagnostic test
Due to the fact that our students come from different schools or even educational systems, we
evaluate the precollege mathematics skills of our freshmen by a written 45-minutes
mathematics diagnostics test, which is anonymous and takes place within the first lecture of
an introductory mathematics course. The main purpose of this test is to give the instructors
information about the level of high school mathematics skills in the class. The instructors can
thus structure their lectures to accommodate to the students’ mathematics preparedness that
might be varying from year to year.
The test comprises about a dozen problems that have to be solved without the help of pocket
calculators. They cover essentially standard high school mathematics problems, supplemented
by a few questions that go beyond the average high school mathematics curricula.
We started with these tests in 2002 and have meanwhile gained some insight and statistical
data for an evaluation. The test as well as the pre-test conditions remained the same until 2009
and therefore we now have the test results of eight classes available. Table 1 gives an
overview of the percentage of correct answers to the above mentioned tasks (n = 368).
These results lead to the conclusion that many enrolees either did not master the content or
forgot much of the content they once mastered. Obviously, a considerable number of high
school graduates enrol in engineering programs under-prepared for the academic rigor of
university education. This, however, is not an isolated case; McCormick and Lucas’ study
revealed a serious disconnect between a high school diploma and the student’s preparedness
for post-secondary education5. The authors underlined that secondary and post-secondary
institutions have historically functioned as distinct entities with only modest interaction, and
offered a compilation of recommendations from experts who made notable advances in
mathematics preparedness in the education of high school students.
The 2007 class was asked to sit the precollege maths skills evaluation another two times, both
at the end of the first semester and at the end of the second semester. It came to a marked
improvement of the test results (see Figure 1), although the topics of the exam questions were
just a small part of the curriculum of the introductory mathematics course, and only implicitly
involved in the second semester mathematics course. The number of students in the class
decreased in the meantime from 76 to 66 after the first semester (although no exams took
place in between these two evaluations), and to 55 after one year of study. Obviously, more
than 13 % of the class decided to discontinue their study without failing an exam. This
phenomenon was investigated in a previous paper2 and attributed to weaknesses in the
students’ approach to work and dedication to their study, which would have been necessary to
make up in relatively short time for the missing skills. Another 15 % of the class dropped out
after the second semester because of failing a first semester course. In the remaining years
required to complete their degree another 12 % dropped out (mainly in the second year of
study), which eventually led to a 4-year graduation rate of 60 %.
Figure 1: Test results of the 2007 enrollees at the beginning of the first semester (left, blank,
n = 76), at the end of the 1st semester (middle, gray n = 66) and at the end of the 2nd semester
(right, dark gray, n = 45).
Experience showed that the attrition rates vary appreciably from year to year, even though the
curriculum, the faculty, and the overall study conditions remained essentially the same. The
circumstance that all the students of the classes 2002 to 2009 who persisted must have
graduated in the meantime (at our university, there is a maximum of three extra semesters
allowed for the completion of a degree program) provoked us to compare the percentages of
the anonymous precollege maths skills evaluation with the graduation rates. Due to the small
percentages accompanied by variances of the same order (see Table 1), only a few results
came into question for such a comparison. In Figure 2 the percentages of correct answers to
the compound fraction problem are compared with graduation rates, while in Figure 3 a
similar comparison with the beginners’ ability to correctly draw a sine function is depicted.
Although, of course, no causal relationship can be inferred from these figures, the conformity
of the trends is nonetheless astonishing. The product-moment correlation coefficient between
the graduation rates and the percentage of correctly solved compound fractions (Figure 2) is
0.844 at a confidence level of 99.5 %, and 0.815 at a confidence level of 99.0 % for the sine
functions (Figure 3). These results inspired us to investigate the degree of correlation between
the mathematics performances of our undergraduate students with their performances in other
subjects, which could shed some more light on the fluctuations in graduation rates.
Figure 2: Percentage of graduates per year of matriculation (left, black) and of correctly
simplified compound fractions in the anonymous mathematics diagnostics test (right, gray).
Figure 3: Percentage of graduates per year of matriculation (left, black) and of correctly
plotted sine functions in the anonymous mathematics diagnostics test (right, gray).
Figure 4: Decline in student number across the first four semesters, and the number of
students graduating after eight semesters of study (classes 2007 and 2008).
The courses that are considered by our students as the most difficult in the early semesters are
Engineering Mechanics and Engineering Mathematics. Failing one of these courses appears to
be particularly dispiriting and is most likely the main reason for the early student dropout. The
aim of this study was to find out to which extent the students’ mathematics skills affect their
performances in mechanics, which seems to be the crucial point in student retention.
In the degree program under discussion, the undergraduate mathematics education is covered
by a series of lectures taking place in the first three semesters of study. These Engineering
Mathematics lectures are comprehensive courses combining Algebra and Calculus as well as
Numerical Mathematics with an emphasis placed on the methods used in engineering (see
Tables A1 and A2 in the Appendix). There is, in general, not much creative leeway in the
design of the mathematics curriculum; it has to provide a logically organized sequence of
topics, but still permits a certain degree of freedom. Similar deliberations apply to
Engineering Mechanics. The classical sequence of topics in Engineering Mechanics starts
with statics and continues with kinematics and kinetics. However, statics can be regarded as a
special case of dynamics. The resulting, rather uncommon, arrangement of topics allows an
integrative and more deductive approach, which was implemented in our department
beginning with class 2001. Thus Engineering Mechanics 1 in the 1st semester referred to
mechanics of particles, whereas Engineering Mechanics 2 in the 2nd semester dealt with
mechanics of rigid bodies. Engineering Mechanics 3 in the 3rd semester was devoted to
analytical mechanics (in the European sense). In 2008 all Engineering Mechanics courses
were shifted one semester later.
For the investigation of the degree of correspondence between the Engineering Mathematics
and Engineering Mechanics results, the Mechanics scores have been plotted against the
Mathematics scores for each student per semester. In Figure 5 the correlation of the scores for
the first two semesters of the classes 2003, 2004, and 2007 are depicted, which are
representative for the mechanics and mathematics curricula prior to 2008 (see Table A1).
Scores of the classes 2005 and 2006 were not available.
The increasing number of dropouts (an exponential decay of the graduation rates with a decay
rate of λ = −0.04 year−1 is fitted to the data, see Figure 6) led to a curriculum reform in the
year 2008. The Engineering Mechanics lecture series, which, due to its characteristic
difficulty and level of abstraction, was regarded to be the main cause for early student drop-
out, should start only in the second semester to give the students the opportunity to improve
their mathematics skills beforehand. In addition, a better conformity of the mathematics
taught and the mathematics needed should be achieved (Table A2).
Figure 6: Percentage of graduates per year of matriculation (circles), and the percentages of
the correctly simplified compound fractions (squares) and correctly plotted sine functions
(triangles) in the corresponding anonymous mathematics diagnostics test. An exponential
function is fitted to each set of data.
Another subsequent consequence of the decreasing graduation rates was the introduction of a
university-wide mathematics bridge course for first-year students in the first two weeks before
the beginning of classes. This course can be attended voluntarily by students who do not feel
sufficiently prepared in mathematics for their future studies. As so often in education, the
question arises how students should catch up on the necessary mathematics skills and
understanding in only two weeks and how many of future students are reflective enough to
realize their need for additional preparation.
A comparison of the Engineering Mechanics scores and the Engineering Mathematics scores
in the second semester of the classes 2008 and 2010 (no scores were available of the class
2009) is illustrated in Figure 7. The plots resemble the first semester correlations in Figure 5
with similar correlation coefficients.
Figure 7: 2nd semester Engineering Mathematics scores versus Engineering Mechanics scores
of the classes 2008 and 2010.
In 2010, the degree program underwent a significant strategic review resulting in major
revisions in order to fulfil the requirements of the Bologna Accord (see e.g. Sanders and
Dunn9). The four-year undergraduate degree program has been transformed into a three-year
Bachelor’s program complemented by a two-year Master’s program. This reform had little
effect on the first year Mathematics and Mechanics courses, except that Analytical Mechanics
(formerly taught in Engineering Mechanics 3) was moved to the 1st semester of the Master’s
program and complemented with the application of variational principles to deformable
bodies (i.e. strength of materials). The correlation of the scores in these subjects of the classes
2011 and 2012 in the 2nd semester is depicted in Figure 8.
Figure 8: 2nd semester Engineering Mathematics scores versus Engineering Mechanics scores
of the classes 2011 and 2012.
Much more data are available if the students’ grades instead of scores are taken into account.
These data could be drawn from the final student grades database of the Registrar’s office.
The disadvantage of grades, compared with scores, is that comparability between different
grading systems is not automatically achieved. There are no letter grades in use in our
country; instead of that, grades from 1 to 5 are used (1 is the best grade, 5 means ‘failed’). A
conversion table is given in Table 2 with typical (but not universal) percentage intervals for
the corresponding grades.
Table 2: Grading system used
Grade Percentage Descriptor US equivalent
1 90 – 100 Excellent 'A'
2 77 – 89 Good 'B'
3 64 – 76 Satisfactory 'C'
4 51 – 63 Sufficient 'D'
5 0 – 50 Insufficient Failing grade 'E/F'
The correlations between Engineering Mathematics grades and the final grades in Engineering
Mechanics and other mathematically-oriented courses are illustrated in Figures 9 to 11. These
data were obtained from more than ten classes of the four-year degree program.
Figure 9: Engineering Mathematics grades versus Engineering Mechanics grades for the
respective semesters
The highest correlation coefficient was obtained for the case when both Engineering
Mechanics 1 and Engineering Mathematics 1 were taught in the first semester. Engineering
Mechanics 1 in the second semester (top right in Figure 9) is slightly less correlated with
Engineering Mathematics 2 but leads to a higher number of better final grades. The relative
number of better grades in both Mechanics and Mathematics increases in higher semesters,
which may partially be caused by the fact that the least motivated students in the class have
then already dropped out.
In Figures 10 and 11 the final grades in Engineering Mathematics are related to the final
grades in Strength of Materials and in Thermodynamics. Here we found similar correlation
coefficients as between Mathematics and Mechanics and, again, an increase in the relative
number of better grades in higher semesters can be observed.
Figure 11: Engineering Mathematics grades versus Thermodynamics grades in the 2nd (left)
and 3rd (right) semester
All the correlations presented above have in common that virtually no student with an
excellent grade (“1”) in the respective engineering courses has failed (“5”) in Mathematics.
However, an excellent final grade in Mathematics can be accompanied by lower grades in the
engineering courses, but hardly ever by a fail grade (“5”).
In the Figures below the correlations between Engineering Mathematics and Engineering
Mechanics final grades (Figure 12), and between Electrical Engineering as well as
Thermodynamics with Engineering Mathematics grades (Figure 13) are shown. These
correlations are obtained from the student grades database of the three-year Bachelor’s degree
program, which started in 2011. Due to the relatively brief existence of this program, the
database contains a comparatively smaller number of students, leading to more statistical
uncertainty. Figure 12 relates Mechanics and Mathematics grades in the 2nd and 3rd semester.
Compared with the data in the four-year degree program, the correlation is comparatively
higher, but the relative number of better grades does not increase with higher semesters. A
similar picture emerges from the correlations between Electrical Engineering and
Thermodynamics with Engineering Mathematics. Lower grades are dominating even in the
third semester. However, the number of students seems to be too small to draw conclusions at
this point.
Figure 12: Engineering Mathematics grades versus Engineering Mechanics grades for the 2nd
(left) and 3rd (right) semester
Figure 13: Engineering Mathematics grades versus Electrical Engineering (left) and
Thermodynamics (right) grades in the 2nd and 3rd semester respectively
In order to assess the conclusiveness of the correlation factors R and ρ of the grade
correlations, all courses offered in the first three semesters were correlated with each other,
and the respective rank correlation factors ρ are illustrated in Figures 14 and 15.
Figure 14: Spearman’s rank correlation for the grade distribution of all courses offered in the
first three semesters of the four-year degree program (classes 1996 to 2010). ρ increases from
magenta (ρ = 0) to red (ρ = 1).
Figure 15: Spearman’s rank correlation for the grade distribution of all courses offered in the
first three semesters of the three-year degree program (classes 2011 to 2013). ρ increases from
magenta (ρ = 0) to red (ρ = 1).
Figures 14 and 15 reveal that the correlation factor on its own does not suffice to make
credible inferences.
Nevertheless, there is information comprised in the bivariate data displays that can provide
better insight into the interrelation between the performances in Mathematics and other
courses. All the correlations of the final grades in Engineering Mathematics with the final
grades in engineering courses have in common that low mathematics grades seem to bear a
higher risk of failing mathematically-oriented engineering courses. This finding suggests to
study in more detail the students’ risk of failure in Engineering Mechanics in the context of
their Engineering Mathematics performance.
The concept of risk factor has already been used by Thomas, Henderson, and Goldfinch3 for
the investigation of the influence of university entrance scores on students’ performance in
Engineering Mechanics. Risk factors are extensively used in medical statistics. The
Framingham study, a heart study of a population cohort from a town outside Boston starting
in the late 1940s, first proposed the risk factor concept in which physiological and behavioral
characteristics such as blood pressure, blood lipids, and cigarette smoking predicted
subsequent disease events10. The risk factor concept can be adapted to the present study by
assessing the risk of failing the Engineering Mechanics course by relating the number of
students with a specific grade in Mathematics who failed the course to the total number of
students with the same grade in Mathematics.
In Figure 16 the Engineering Mechanics scores are cumulatively plotted against the
Engineering Mathematics scores of the classes 2003, 2004, 2007, 2008, 2010, 2011, and
2012. The scatter plot clearly shows that when the scores are dichotomized at the midpoint to
“passed” (> 50 %) and “failed” (≤ 50 %), almost none of the students who passed Mechanics
failed in Mathematics. On the other hand, there are a number of students with high
Mathematics scores who failed in Mechanics. Obviously, the application of mathematical
methods to real-world problems brings an extra dimension of difficulty.
Figure 16: Engineering Mathematics scores versus Engineering Mechanics scores
The risk factors for the students in their first year of study were determined from the score
correlation depicted in Figure 16 and are presented in Figure 17. This plot shows the
percentage of students who had to take board exams in the Engineering Mechanics course for
those intervals of Mathematics scores represented by the corresponding letter grades.
Figure 17: Risk factor of failing Engineering Mechanics based on Mathematics grades
Students with a Mathematics score of less than 50 % (i.e., who had to take board exams in
Mathematics) have a risk factor of more than 70 % of failing the written examinations in the
Mechanics course. This risk factor reduces almost linearly as the Mathematics score
increases, with a risk factor of only 4 % for those students with a Mathematics score between
90 % and 100 %.
Having identified the decline in the students’ basic mathematical skills and level of
preparation on entry into higher education and its impact on the dropout rate, the question is
what to do about it.
In the last decades a multitude of measures has been developed by universities worldwide.
These include supplementary lectures, computer assisted learning, mathematics support
centers, additional diagnostic tests and streaming. Each has its merits, and brief details are
given below. A comprehensive overview is given, for instance, by Hawkes and Savage in
their report11 emerging from the 1999 workshop “Measuring the Mathematics Problem” at the
Møller Centre Cambridge.
Virtually always on top of the list of recommendations is a diagnostics assessment that allows
new students to reflect on their level of mathematics learning on entry to the university and
thereby provides motivation for improvement in skills shown to be weak. Diagnostic tests
play an important part in identifying students at risk of failing because of their mathematical
deficiencies, in targeting remedial help and, last but not least, in removing unrealistic faculty
expectations. A prompt and effective follow-up support of students whose mathematical
background is found to be poor by the tests is regarded as essential.
Once the results of a diagnostic test are known the question arises as to how provide effective
support for those students with deficiencies in basic knowledge and skills. The follow-up
strategies offered by Hawkes and Savage11 include supplementary lectures, additional
modules, computer assisted learning, mathematics support centers, additional diagnostic tests
and streaming.
Many universities have introduced additional units of study (bridging courses) running prior
to or in parallel with the first year courses. The problem with it is the danger of overloading
students with too much additional work when they are probably struggling with other
modules. And, in addition, it often turns out that bridging courses are attended mainly by the
well prepared and most hard working students, who never miss a chance to learn something
extra (see e.g. Sazhin12).
Some universities attempt to cope with the different levels of mathematical preparedness by
streaming their students and teaching the weaker group separately, sometimes using the
services of experienced school teachers rather than university lecturers. In order to cover the
same syllabus as the stronger group it may be necessary to increase contact hours for the
weaker group13.
Amongst the range of strategies for coping with a serious decline in students’ mastery of basic
mathematical skills, a further option is to reduce syllabus content or to replace some of the
harder material with more revision of lower level work. This will definitely help the weaker
students. However, if more advanced material is removed to make space for this revision, it
disadvantages the more able making them less well prepared for the mathematical demands of
the more advanced parts of their studies. In addition, it has a longer-term effect of making
these students less able to compete for jobs13.
In some institutions, the mathematics diagnostic test forms only a part of the prognosis of the
student. Once the students have obtained their results, further informal assessment in the form
of informal pre-study support talks with mathematics tutors are offered. In their review of
eight years of mathematics study support14, Patel and Little concluded that individualized
learning programs and face-to-face mathematics study support, targeted through diagnostic
assessment tests, do really offer a solution to the problem of retaining and progressing
undergraduate engineering students who struggle with the mathematics content of their
courses. They suggest that a quiet, relaxed, supported study area together with peer-learning
study groups will help to overcome mathematics anxiety and result in tangible student
benefits.
Several universities have established such mathematics support centers that are tailored to
individual needs and where the students can attend at any time. Such centers provide one-to-
one support for all students with any type of mathematical or statistical problem they may
come across in their course. One of the central features of mathematics support centers seems
to be the provision of non-judgmental support of students outside their teaching departments.
It is a place where students can ask questions without being told ‘you should know that’14.
Such centers can be very effective in dealing with students having specific isolated
difficulties. However, for the less well-prepared students a drop-in consultation is hardly
sufficient. If they need to gain a coherent body of mathematical knowledge there is no
substitute for an extended and systematic study course11.
As a first measure against high dropout rates we are considering to “deanonymize” the
mathematics diagnostic tests. These tests will then not only allow identifying overall
deficiencies in the cohort but also the students’ individual strengths and weaknesses. This will
help to identify students who are significantly weaker than the rest of the group and thus be
offered individual help and attention. The establishment of a mathematics support center as an
additional provision to support students who are struggling with the mathematical
components of their undergraduate studies is also envisaged.
Conclusions
In order to gain some insight into the degree of relation between the students’ performances in
Engineering Mathematics and their success or failure in other courses, especially in
Engineering Mechanics, the students’ final grades and, if available, scores in both courses
were correlated. All comparisons resulted in significant positive correlations, which
corroborate the idea that the engineering students’ mathematics skills are closely connected to
their overall academic success.
However, the introduction of a risk factor, which relates the number of students with a
specific grade in mathematics who failed the mechanics course to the total number of students
with the same grade in mathematics, provides clear information and allows the prediction of
failure probabilities.
Appendix
Table A1: Mathematics and Mechanics course contents prior to Fall 2008
1. Paul A. Murtaugh, Leslie D. Burns, and Jill Schuster, “Predicting the retention of university students”,
Research in Higher Education, Vol. 40, No. 3, 1999
2. Emilia Andreeva-Moschen, “The five main reasons behind student enrollment and later drop-out”, ASEE
Annual Conference and Exposition, San Antonio, TX, June 10-13, 2012
3. Giles Thomas, Alan Henderson, and Thomas Goldfinch, “The Influence of University Entry Scores on
Performance in Engineering Mechanics”, 20th Australasian Association for Engineering Education
Conference, Adelaide, SA, Dec 6-9, 2009
4. Sarah Tumen, Boaz Shulruf, and John Hattie, “Student pathways at the university: patterns and predictors of
completion”, Studies in Higher Education, 33(3), 233-252, 2008
5. Nancy J. McCormick and Marva S. Lucas, “Exploring mathematics college readiness in the United States”,
Current Issues in Education, 14(1), 2011
6. Eileen Goold and Frank Devitt, “The role of mathematics in engineering practice and in formation of
engineers”, SEFI 40th annual conference: Mathematics and Engineering Education, Thessaloniki, 2012
7. Günter Bischof, Emilia Bratschitsch, Annette Casey, and Domagoj Rubeša, “Facilitating engineering
mathematics education by multidisciplinary projects”, American Society for Engineering Education, ASEE
Annual Conference and Exposition, Honolulu, HI, June 24-27, 2007
8. Ralph Stinebrickner and Todd Stinebrickner, “Academic Performance and College Dropout: Using
Longitudinal Expectations Data to Estimate a Learning Model”, The National Bureau of Economic Research
NBER, Working Paper No. 18945, Issued in April 2013
9. Ross H. Sanders and John M. Dunn, “The Bologna Accord: A Model of Cooperation and Coordination”,
Quest, Volume 62, Issue 1, 2010
10. Thomas R. Dawber, “The Framingham Study: the Epidemiology of Atherosclerotic Disease”, Cambridge,
Mass., Harvard University Press, 1980
11. Trevor Hawkes and Mike Savage, (eds.), “Measuring the Mathematics Problem”, London: Engineering
Council, 2000. Accessible via www.engc.org.uk/about-us/publications.aspx (12 March 2015)
12. Sergei Sazhin, “Teaching Mathematics to Engineering Students”, Int. J. Engng Ed. Vol. 14, No. 2, p. 145-
152, 1998
13. Duncan Lawson, Tony Croft, and Margaret Halpin, "Good practice in the provision of mathematics support
centres", LTSN Maths, Stats & OR Network, 2003
14. Chetna Patel and John Little, “Measuring maths study support”, Teaching Mathematics and its Applications
25 (3), 131-138, 2006