Computer-aided assessment in mathematical sciences

Symposium Presentation Computer-aided assessment in mathematical sciences Grant Keady, Mathematics and Statistics, The University of Western Australia, Australia Gary Fitz-Gerald, Mathematics and Statistics, RMIT University, Australia Greg Gamble, Mathematics and Statistics, Curtin University of Technology, Australia Chris Sangwin, School of Mathematics, University of Birmingham, England [email protected] [email protected] [email protected] [email protected] Abstract: In this paper Computer Aided Assessment (CAA) systems involving the delivery of questions across the web that are underpinned by Computer Algebra (CA) packages are discussed. This underpinning allows students to enter answers, have them parsed by the CA system, have them type-checked by the CA system, which are then passed through a marking procedure which can recognize any correct form of the answer. The underpinning CA also allows model solutions to be generated (immediately for formative assessment but after due-dates for summative ones) and to provide many other useful forms of feedback. With the increasing demand from students for focused feedback on their work in an environment of increasing class sizes and decreasing funding such tested alternative mechanisms to the more traditional labour intensive ones provided by the marking of hand written assessment tasks is becoming essential. In this way, these systems are as much about `Computer Aided Learning’ as they are for `Assessment’. Such CAA systems have been successfully used in a moderate number of Mathematics Departments. This paper publicizes their availability, and how they are being used, to a wider community of disciplines such as physics, engineering and mathematical economics. The authors believe such disciplines would greatly benefit from using such systems to support their teaching and learning strategies. Since the underpinning CA packages are widely used by physicists (and, in some cases, written by physicists) their discipline is especially well placed to exploit such CAA systems. Introduction Material supporting this paper is available at http://www.maths.uwa.edu.au/~keady/CAA/UniServeSep06/index.html CAA, without the CA support, has been widely used in science subjects such as physics and chemistry for several decades. Here the question types have been mostly restricted to MultipleChoice, Multiple-Response, and to some simple numeric answers. As well, CAA, in which the questions are delivered via the web, dates back about a decade. The genre treated in this paper is CAA underpinned by Computer Algebra (CA) packages. In fact, systems like AiM and CalMaeth pre-date the Web. Their transition to the web-delivered forms, for AiM described in Klai, Koloknikov and Van den Bergh (2000), led to a dramatic increase in usage, with students using the systems from home and not merely from on-campus computer labs. The major advantage with this genre is that students can enter symbolic answers, which, it can be argued, requires a deeper level of understanding of the material than choosing the correct alternative from a provided list. The best selling of the currently available commercial CA packages are Maple and Mathematica. Of the free, open-source CA packages, Maxima is probably the most appropriate for use by a majority of physicists (and physics students) given its developmental history. Four existing CAA systems of particular interest to the authors range from totally free and opensource (e.g. stack, underpinned by Maxima); free and open-source, except for the CA (e.g. AiM, underpinned by Maple); commercial (e.g. mapleTA and WebLearn—both underpinned by Maple— CalMaeth and MyMathLab/mathxl—both underpinned by Mathematica). Although there are other CAA systems underpinned by other CA packages, This paper confines its discussion to (some of) those underpinned by these Ma*-packages. The authors of this paper have each used at least two of them. The principal author (GK) has experience with both CalMaeth and AiM, and, for many of his second-year engineering-mathematics questions, has them coded in both systems. CS has coded 69 UniServe Science Assessment Symposium Proceedings Symposium Presentation questions both for AiM and stack, and also developed stack. GF has looked at AiM, but coded for WebLearn, and developed the Maple-mathematics add-in to WebLearn. GG has mostly worked on AiM for CAA, both contributing to its packages, documentation as well as coding questions: GG has also looked at CalMaeth in use. All four authors have looked at, though not had real experience with, mapleTA. Links associated with all these CAA systems, and more, are provided at the above ~keady URL. There is a huge literature, especially if one includes the on-line documentation for the systems, CD-ROM demonstrations of, and webex seminars on, the commercial ones. We will not duplicate this information in this paper. The message contained in this paper is simple: the genre is established, and it is opportune for other scientists, especially physicists, to consider the systems to support the teaching and learning of their disciplines. Which systems that should be looked at will depend on the CA systems the reader and their colleagues are most familiar with. However, there are cautions. It is not easy to predict which systems will survive into the more distant future, and it is important to future-proof the investment of time and effort put into them. Also, the sizes of the classes taught affect the economics: more about this is discussed at the above ~keady URL. Physicists have always been prominent amongst the users, and sometimes the developers, of CA packages. The Nobel Prize laureate Martin Veltman, designed Schoonship a program for symbolic mathematics, especially High Energy Physics, in 1963. Tony Hearn, an expatriate Australian physicist, produced REDUCE in 1968 and, again, there were many physicist users, from High Energy Physics to General Relativity and more. There were many physicists among the early users of Macsyma, and Maple. Steven Wolfram introduced SMP, in 1982, which was the precursor of today's Mathematica. There continue to be many physicist users of CA. For this reason this paper advocates that physicists in particular would benefit from exploring the CAA systems. The world is big enough for people to have different enthusiasms, and the authors of this paper admit to really enjoying coding in CA packages. Along with the recommendation that physicists look at the CAA systems, this paper advocates that CA-using physicists consider using their CA skills to author questions in them. At the very least, such CA-using physicists would be likely to enjoy looking at the code for some CAA questions so that they can see how some of it works. The authors are sure that the primary reason that mathematicians are ahead of the physicists in the uptake of the genre is that mathematics departments have larger service classes. However now, with several systems available and well tested, (repeating our main point) we are sure that the time is right for more physicists to check them out for possible use in their larger-enrolment classes. The plan for the paper is: To list briefly some of the features common to (or, at least potentially common to) the CAA systems in this genre—together with a few words of caution. • To elaborate on the CAA systems mentioned above. • A few examples of questions will be briefly described. • The genre CA underpinnings of CAA allow students to enter (symbolic) answers, have them parsed by the CA system, type-checked by the CA system, and then passed through a marking procedure which can recognize any correct form of the answer. Sometimes care is needed to achieve this recognition. There is an extensive battery of testing routines available for all sorts of areas of mathematics. Sometimes the testing routines are in the underlying CA package, e.g. odetest in Maple. Sometimes the testing routines are written in the CA’s code, but supplied with the CAA system. UniServe Science Assessment Symposium Proceedings 70 Symposium Presentation A note on authoring AiM’s authoring system is beautifully simple: see, http://aim.maths.uwa.edu.au/doc/format.html. GG has provided a site at Curtin with guest logins where both the questions can be done and the code which generated them is readable; see the above ~keady URL for this. At present there is no public URL giving the information on authoring in the commercial systems. However mapleTA provides the information via webex courses, instructional material on CD-ROM, etc. The URLs associated with each of the individual systems contain details. Brief description of some existing CAA systems Existing CAA underpinned by Maple mapleTA is described at the vendor’s information pages at http://www.maplesoft.com. mapleTA is, we think, a bit expensive, but it is used at quite a few universities—probably more than for any other CAA system in the genre. In Australia, the distributor is CEANET. mapleTA is used at the University of Adelaide. A review of this system is available at mathstore.ac.uk/newsletter/nov2004/pdf/mapleta.pdf. Various publishers, in particular Wiley, Pearson-Prentice Hall, WH Freeman, and McGraw-Hill all have a number of titles—including topics in Physics and Chemistry—where comprehensive question banks are available. AiM is free and open source, except for the Maple licence. Our estimate is that the software costs for AiM are typically about half those for mapleTA. It is worth noting that a single Maple ‘user’ can serve, via AiM, hundreds of students. The apparent inconsistency—a hundred or so AiM users all logged into AiM at the same time, but one Maple user—is because ‘AiM’ is THE one Maple user, and the individual students only get a ‘look in’, one at a time, to Maple only when they click on a page, e.g. when, after having entered an answer, they click on ‘Validate’ or on ‘Mark’. It needs a millisecond or so for the marking! Reviews are available at http://mathstore.ac.uk/reviews/software.shtml The following Maple-underpinned products are currently in use only at the universities that developed them (and their associated institutions). WebLearn is from RMIT, and, strictly speaking, is a MLE (Managed Learning Environment): contact GF for details—on both technical and pricing issues—both of the MLE and his mathematics CAA add-on. Physicists, chemists and others at RMIT have been using WebLearn for both formative and summative assessment for many years. However they have been using muliple-choice questions without the Maple back-end. GF did ‘mock up’ a number of the Physics questions that incorporated the Maple back-end for their experimentation but so far this hasn’t been taken any further. wallis is from the University of Edinburgh. Existing CAA underpinned by Mathematica There are, of course, variations on what is easy to achieve with the different CA packages. For example, Mathematica’s pattern matching may be used on Hold[studentAnswer] to analyze the answer. If the marking code has marked the student’s answer wrong, it may be appropriate to look at the answer the student entered, and to see if `parts’ of it are right, and then to give appropriate feedback. We now list a couple of the systems that use Mathematica as the underlying CA. CalMaeth is the mathematics development of a UWA-developed MLE, JellyFish. At present, CalMaeth appears to be in use only at UWA (and an associate institution in Singapore). There is a review of CalMaeth in Dec02; and, in Jan 03, a comparison between authoring in AiM and in CalMaeth at http://ltsn.mathstore.ac.uk/articles/maths-caa-series/index.shtml. For more details, contact Kevin Judd at [email protected]. 71 UniServe Science Assessment Symposium Proceedings Symposium Presentation mathxl/MyMathLab is a product from Pearson/Addison-Wesley and uses webMathematica. It is, currently, only run off servers in the USA. Currently there are no Physics titles using mathxl/MyMathLab probably because they also use http://www.masteringphysics.com/ Existing CAA underpinned by Maxima stack, produced by CS, is a completely open-source free system built on Maxima. stack can be run under the free open-source MLE, moodle. A few typical CAA questions Our examples will consist of questions GK and GG have delivered with AiM. In the first example, the question asks students, given the formula for a periodic function, to enter formulae for the coefficients of its Fourier series. We chose to work with examples where the form of the n-th Fourier coefficient, for the n=0, then, for the m>0, even n=2m and odd n=2m+1 cases separately, where each is a rational function of m. The type testing requires the students to enter a rational function of m. In the second example, the question shows the solution of a lightly-damped harmonic oscillator, forced with a given periodic function with slowly decaying Fourier coefficients. The forcing function, its Fourier coefficients, and the long-term periodic response are given, the last via a (AiM/Maple-generated) plot. The question then gives some information about the form of the natural frequency (and in the example the students are given, it is close to resonance with one of the early terms in the Fourier series). E.g. they might be told that the natural frequency squared is close to an integer (or some similar specification): and then they are asked, from the plot, to give its value. The type testing requires the students to enter a positive integer. Most of the thousand or so questions in GK and GG’s database of AiM questions concern first and second year mathematics topics. The Fourier series questions just mentioned have been used for the last 3 years in a second year engineering maths unit at UWA, where the class size is around 250 to 300 students per annum. GK and GG are currently enjoying writing a few AiM questions for a 3rd year unit `Mathematical Applications in Environmental Engineering’, and some of the pde-related topics would be equally suitable in a Physics setting: again there is re-use, in connection with solutions of partial differential equations by separation of variables, of parts of the Fourier series code. Though it may be economic in the longer term, with an annual enrolment of only about 25 students, it isn’t really economically viable in the short term. The class, however, is an appropriate one for investigating the AiM-moodle MLE combination. Conclusion: economics, sustainability and the future CAA systems will come and go. Some will have more users than others. Sharing systems, question databases and so on, is crucial for progress to be made. The book publishers’ involvement with mapleTA probably makes it the easiest choice for physicists and chemists to use at sites where it is available. It would be marvelous if were possible to convince the publishers supporting mapleTA to pick up the software costs for a mapleTA server, for courses using their books, in each of the Australian state capitals. No matter which CAA system is used, the more students who use it (and questions from its database of questions), and learn from them, the more economically worthwhile it becomes. The cost in time is the same, for developing the system, for setting up the system, for questions selected or authored, for downloads of the final results, whether the system is used by 1 student or by 1000 UniServe Science Assessment Symposium Proceedings 72 Symposium Presentation students. (More hassles from students asking questions will come from the larger enrolment units, but this merely means more attention to automated methods of delivering help to them come to the fore.) On the software side, at least for the academic systems, the main design goal for sustainability is to keep the systems simple. Part of this involves exploiting the CA system for what it is good at, and other appropriate software components where applicable. As subjects other than Mathematical sciences use MLEs and these provide facilities for students and staff to check marks, etc., it makes a lot of sense to build CAA modules that link in with these. A project in England built a Remote Question Protocol (RQP) to facilitate linking several CAAs (including AiM and stack) with (ultimately several) MLEs. (This is the AiM-moodle combination being explored at UWA Environmental Engineering, as the latter School uses moodle for all their units.) For more on this, and other issues relating to sustainability, see the ~keady URL given before. The integration of the sort of CAA discussed in this paper with widely used MLE’s, (e.g. WebCT, moodle, etc.) is developing. This will permit the students' learning experience across different subjects to be suitably harmonized. Further information Information, and often guest logins, to some of the systems described in this paper are available at: • http://aim.maths.uwa.edu.au • https://calmaeth.maths.uwa.edu.au • http://stack.bham.ac.uk • http://weblearn.rmit.edu.au References Klai, S., Kolokolnikov, T. and Van den Bergh, N. (2000) Using Maple and the Web to grade mathematics tests. In Proceedings of the International Workshop on Advanced Learning Technologies IWALT2000 (Palmerston North, New Zealand, Dec. 2000), IEEE Computer Society Press (ISBN 0-7695-0653-4/00), 89–92. © 2006 G.Keady, G. Fitz-Gerald, G. Gamble, C. Sangwin The authors assign to UniServe Science and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is re-produced. The authors also grant a non-exclusive licence to UniServe Science to publish this document on the Web (prime sites and mirrors) and in printed form within the UniServe 2006 Conference proceedings. Any other usage is prohibited without the express permission of the authors. UniServe Science resered the right to undertake editorial changes in regard to formatting, length of paper, and consistency. 73 UniServe Science Assessment Symposium Proceedings