Available online at www.sciencedirect.

com

Procedia Social and Behavioral Sciences 2 (2010) 2000–2005

WCES-2010

Developing disciplinary tasks to improve mathematics assessment

and pedagogy:
An exploratory study in Singapore schools
Fan Lianghuoa * , Zhao Dongshenga, Cheang Wai Kwonga, Teo Kok Menga,
Ling Peng Yapa
a
National Institute of Education, Nanyang Technological University, 1 Nanyang Walk, Singapore 637 616, Singapore

Received October 16, 2009; revised December 25, 2009; accepted January 8, 2010

Abstract

The Singapore Mathematics Assessment and Pedagogy Project (SMAPP) is a major research project undertaken by a group of
mathematicians and mathematics educators to develop and investigate an innovative and systematic approach to assessment
suited to the needs of mathematics teaching and learning in Singapore schools. A key research effort of the project is to develop
the so-called disciplinary tasks for classroom instruction. This paper introduces the task development model established for the
project and discusses the initial results of the implementation of the tasks in participating schools. The results showed that both
teachers and students found the tasks innovative, challenging, and they can facilitate mathematics teaching and learning.
© 2010 Elsevier Ltd. Open access under CC BY-NC-ND license.

Keywords: Mathematics assessment; mathematics pedagogy; mathemtics task development; Singapore mathematics education.

1. Introduction

During the recent two decades, the importance of assessment in mathematics education has received mounting
attention from mathematics educators, school practitioners, and policy makers in many countries (e.g., see Black &
Wiliam, 1998; Even, 2005; National Council of Teachers of Mathematics, 1995; Niss, 1993; Webb & Coxford;
1993). In particular, relatively new assessment (or the so-called alternative assessment) concepts and strategies have
been widely advocated by educational reformers and increasingly used by classroom teachers in their practices (e.g.,
see Clarke, 1997; Fan, 2002; Fan, Quek, Koay, Ng, Pereira-Mendoza, Yeo, et al., 2008; Hargreaves, Earl, &
Schmidt, 2002; Kulm, 1994; Williams, 1998). In Singapore, the Ministry of Education (MOE) issued two official
documents, Assessment Guides to Primary School Mathematics and Assessment Guides to Lower Secondary
Mathematics in 2004 (MOE, 2004a, 2004b), which highlighted that the main goal of mathematics assessment is to
improve the teaching and learning of mathematics. In this connection, a series of researches in the area of
mathematics assessment have been conducted in Singapore schools (for a review, see Quek & Fan, 2009)

* Fan Lianghuo. Tel.: +65-6790-3911; fax: + 6896-9417

E-mail address: [email protected]

1877-0428 © 2010 Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
doi:10.1016/j.sbspro.2010.03.271
 Fan Lianghuo et al. / Procedia Social and Behavioral Sciences 2 (2010) 2000–2005 2001

The Singapore Mathematics Assessment and Pedagogy Project (SMAPP) is a major new development in the
study of mathematics assessment in Singapore schools. It has three focus areas. First, it focuses on the integration of
assessment into, and hence supporting, the teaching and learning of mathematics. Second, it emphasizes the
integration of ICT in the process of assessment. Finally, it focuses on the teacher professional development and
support needed for implementing the new assessment approaches being investigated in the study.
A key research effort of the project is to develop the suitable mathematics problems, or the so-called
“disciplinary tasks”, for classroom assessment, as mathematics problems are the “heart of mathematics” (Halmos,
1980), and in terms of mathematics assessment, we believe that good mathematics tasks play an essential role in
good mathematics assessment practices. This paper is to present the task development model established for the
project and some initial results of the implementation of the tasks in participating schools.

2. Task Development Model

One underlying concept to the project’s approach and task development is “disciplinarity.” According to Hogan
(2007), disciplinarity in mathematics implies “immersion in authentic … practices [such as] generating, validating
(justifying), communicating and debating knowledge claims; deep (conceptual) understanding of key concepts and
principles and their relationships (knowledge); organizing the classroom as an epistemic community [that
encourages] collective reflection and discussion; and cultivating appropriate epistemic dispositions.” In fact, many
of these ideas have been reflected in the Singapore’s national mathematics curriculum since the early 1990s. The
challenge is to further improve our instructional and assessment practices to realize these aims in classrooms. For
this purpose, a “Task Development Model” is established for the project to guide the development of SMAPP tasks.
The task development model mainly contains three inter-related components, i.e., the core values of the project,
the task designing framework, and the task development process (see Figure 1).

Enhancing Disciplinarity
Mathematical Content
x Based on Singapore’s mathematics syllabus,
Improving Pedagogy
x Focuses on core mathematics knowledge and skills,
x Tailored to school’s scheme of work.
Increasing Effectiveness
Desired Learning Outcomes
Enriching Learning
x Creative thinking, x Communication skills,
x Critical thinking, x Logical reasoning,
x Application skills, x Modelling skills,
x Meta-cognition and reflection skills,
P ii i d i M h i
Pedagogical Features
x Contextualized,
x Developmental/formative in nature,
x Multiple-staged,
x Reasonably extended in terms of duration,
x Open-ended, Investigative / problem solving,
x ICT-based/digitalized.

Crafting Reviewing Implementation and

Feedback Digitalizing
(by task data collection
(from school (by IT sub-
development team) teachers) team) (school based)

Added to task Data analysis and

bank final revision
(deliverables) (by researchers)
2002 Fan Lianghuo et al. / Procedia Social and Behavioral Sciences 2 (2010) 2000–2005

In Figure 1, the core values represent the key objectives of the project: (1) Enhancing disciplinarity/quality in
mathematics teaching and learning in terms of mathematical concepts, skills, processes, ideas, knowledge
construction, and values, through carefully designed mathematics tasks and supporting materials, (2) Improving
pedagogy through integrating assessment into teaching and learning to, in particular, inform teachers’ instructional
decision and hence improve their pedagogical practices; (3) Increasing effectiveness (and efficiency) of assessment
through harnessing the power of ICT, i.e., ICT is fully embedded in the process of assessment, including
digitalization and online presentation, delivery, response capturing, and data analysis of assessment tasks, and (4)
Enriching learning by providing worthwhile mathematical tasks and rich learning experiences for students; the
process of their working on assessment tasks is also a process of their learning.
The core values of the project moulds and determines the framework for task designing. The designing
framework reveals the main characteristics of the SMAPP tasks in three areas, mathematics contents, desired
learning outcomes, and pedagogical features. Each task will reflect a number, if not all, of these characteristics (see
Table 1).

Table 1. Framework for Task Designing

Mathematical Content Based on Singapore’s mathematics syllabus, focus on core mathematics knowledge and
skills, tailored to the schools’ scheme of work.
Desired Learning Targeted to develop creative thinking, critical thinking, application skills, modelling skills,
Outcomes meta-cognition and reflection skills, logical reasoning, communication skills, and positive
attitudes in mathematics.
Pedagogical Features Often contextualized, developmental/formative in nature, open-ended,
investigative/problem solving, reasonably extended in terms of duration, ICT-
based/digitalized, and multiple-staged.

The task development process starts with deciding on the mathematical content involved and conceptualizing the
task scenario. After the context and mathematical content is decided, the researchers in the task development team
will then craft the initial version of the task. In each of the meetings, the researchers will examine the questions
proposed, discuss the pedagogical purpose of the questions, and make improvements and revisions along the way.
Once the initial draft is completed, the researchers will write up pedagogical supporting materials to illustrate how
the task can be used during classroom implementation. The supporting materials include the objectives and purpose
of the questions, and the pre-requisite knowledge required, and the pedagogical thinking and rationale behind the
design of these tasks. In addition, researchers also provide suggestions in the supporting materials regarding follow-
up actions that teachers may take, to help students encountering difficulties or to challenge them further.
This draft will be reviewed by a team of independent and experienced reviewers in the project team, who are
specialist in areas of curriculum design and mathematics education, and are not involved in the task development
process. The researchers in the task development team will discuss and revise the task based on the reviewers’
comments where necessary and produce a second draft. This second draft will be distributed to the participating
teachers for their feedback, based on which the task will be revised again by the researchers. The tasks will then be
uploaded into the project’s IT platform. In the process of digitalization, the software developers will also plan the
layout, the structure, and create the necessary IT features required for task administration.
During the task implementation, relevant data are collected from students and teachers through interviews and
surveys, etc. and analyzed to help teachers make informed decisions and researchers seek possible improvements.
Based on the data collected during the implementation, the researchers refine the task again if necessary. It is then
added to the task bank (Interested readers may contact the authors for a sample copy of the task).

3. Implementaion and Data Collection

In total, more than 20 classes at Secondary One (Grade 7) in five schools including both high-performing and
regular schools were selected to participate in the project, which officially started from Sept. 2008.
 Fan Lianghuo et al. / Procedia Social and Behavioral Sciences 2 (2010) 2000–2005 2003

In the first stage of the study conducted to understand students’ responses to the SMAPP task and the IT
platform, the task focusing on speed and algebra was selected. Data were mainly collected in May 2009 through
questionnaire surveys, classroom observations, and interviews with students and teachers. In all, data were collected
from 156 students (one class from each school) and 9 teachers.
The questionnaire survey for students consists of 15 questions related to the content of the task (7 questions) and
the IT platform (8 questions). The responses were grouped in a 4-point Likert type scale: 4: Strongly Agree, 3:
Agree, 2: Disagree, and 1: Strong Disagree, plus an extra option: Not Applicable. The questionnaire survey for
teachers consists of 20 questions related to mathematical content (4 questions), desired learning outcomes (8
questions), and pedagogical features (8 questions). The responses were also grouped in a 4-point Likert type scale:
4: Strongly Agree, 3: Agree, 2: Disagree, 1: Strongly Disagree, and finally, Not Applicable. Interviews were
conducted with 25 students (5 from each school) and 9 teachers for more in-depth feedback based on the task
implementation.

4. Findings

Table 2 summarizes the results of the data collected from questionnaire surveys with students. The results reveal
that overall the students offered quite positive views about the tasks and the IT platform. However, the results also
suggest that they found the tasks challenging (Items 1a and 1b), they were not used to online work (Item 2d), and
they hope IT can offer them more in the learning of mathematics. The interview results are consistent with the
findings from questionnaire surveys.
Table 2. Students’ questionnaire survey

Question N Mean SD Median

1a. I have difficulty understanding the questions. 156 3.06 0.76 3.00
1b. I can solve most of the questions easily. 154 1.99 0.74 2.00
1c. I find the problem scenario realistic. 156 2.83 0.79 3.00
1d. I feel that doing the problem is meaningful. 151 2.69 0.78 3.00
1e. Working on the task is useful in helping me 151 2.79 0.79 3.00
understand the topics covered in the task
1f. My mathematics knowledge /skills were reinforced 153 2.81 0.79 3.00
by doing the task.
1g. I gain some new knowledge by working on the task. 153 2.82 0.75 3.00
2a. The website (IT system) is user-friendly. 153 3.06 0.81 3.00

2b. The IT system is useful in helping me learn 151 2.77 0.80 3.00
mathematics.

2c. I prefer doing mathematical tasks on paper 153 3.09 1.03 3.00
compared to doing it online.

2d. I would like to have more online work in the 153 2.52 0.93 2.00
learning of mathematics.

2e. I would like the IT system to provide immediate 150 3.06 0.80 3.00
feedback on my answer.

2f. I would like the IT system to allow me to explore 151 2.99 0.80 3.00
with different answers.

2g. I would like IT system to show me more 153 3.12 0.80 3.00
simulations and animations.

2h. Overall, I like the IT system for this task. 153 2.84 0.78 3.00

Table 3 summarizes the results of the data collected from questionnaire surveys with teachers. The results reveal
that overall the teachers had positive views about the tasks and the IT platform, and they appreciated the
pedagogical value of the SMAPP tasks. However, teachers also felt the task challenging to students.
2004 Fan Lianghuo et al. / Procedia Social and Behavioral Sciences 2 (2010) 2000–2005

The results obtained from the questionnaire surveys were also consistent with what we found from the data
collected from the interviews. For example, one teacher said: “The task seems difficult for most students as they
have problem understanding/interpreting the questions.” Another teacher explained: “Some [sub-questions] were
easy and some were challenging as should be. But to string the questions together at one go might be overwhelming
for students. Students were particularly challenged by questions involving forming of equations”. These feedbacks
are very helpful for the refinement of the task and for the further development of other tasks.

Table 3. Teachers’ questionnaire survey

Question N Mean SD Median

1a. The task can be well integrated into my schools’ 8 3.38 0.52 3.00
scheme of work.
1b. The task deals with fundamental mathematics 9 3.00 0.50 3.00
knowledge.
1c. The task are pegged at an appropriate level of 9 2.56 0.73 2.00
difficulty.
1d. Overall the task is well designed in terms of 9 2.89 0.60 3.00
mathematics content.
2a. The task can help develop students’ creative thinking 9 3.11 0.60 3.00
skills.
2b. The task can help develop students’ critical thinking 9 3.33 0.50 3.00
skills
2c. The task can help develop students’ mathematics 9 3.22 0.44 3.00
application skills
2d. The task can help develop students’ mathematics 9 3.00 0.50 3.00
modeling Skills
2e. The task can help develop students’ metacognition 9 2.78 0.44 3.00
and self-reflection skills
2f. The task can help develop students’ logical reasoning 9 3.33 0.50 3.00
2g. The task can help develop students’ appreciation of 9 2.67 0.71 3.00
mathematics
2h. Overall the task is well designed in terms of desired 9 3.11 0.33 3.00
learning outcomes
3a. The task is well contextualized 8 3.13 0.64 3.00

3b. The task is sufficiently developmental in nature 8 3.25 0.46 3.00

3c. The task is sufficiently open-ended in nature 8 3.00 0.00 3.00

3d. The task is sufficiently investigative in nature 8 2.88 0.64 3.00

3e. The task is sufficiently extended in duration 8 3.25 0.46 3.00

3f. The task is sufficiently ICT-based/digitalized in 8 3.00 0.93 3.00

nature
3g. The task is sufficiently multiple-staged in nature 8 3.13 0.35 3.00
3h. Overall the task is well designed in terms of 9 3.00 0.50 3.00
pedagogical features

5. Summary

This article presents a task development model established in a major research project for the purpose of
improving mathematics assessment and pedagogy in Singapore educational context. The task development model
mainly consists of three inter-related components, i.e., the core values of the project, the task designing framework,
and the task development process. The initial results showed that both teachers and students found the tasks
developed under this development model innovative, challenging, and these tasks can facilitate mathematics
teaching and learning. Needless to say, there is still much work to do further in this direction.
 Fan Lianghuo et al. / Procedia Social and Behavioral Sciences 2 (2010) 2000–2005 2005

Acknowledgement

We wish to record our appreciation to participating school teachers and our fellow team members for their
various support an inputs for the SMAPP study, and in particular, for the task development. Relevant information
about the project can be found at the project website: http://math.nie.edu.sg/smapp/teamsite/

References

Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2): 139-148.
Clarke, D. (1997). Constructive assessment in mathematics: Practical steps for classroom teachers. Berkeley, CA: Key Curriculum Press.
Even, R. (2005). Using assessment to inform instructional decisions: How hard can it be? Mathematics Educational Research Journal, 17(3), 45-
61.
Fan, L. (2002). In-service training in alternative assessment with Singapore mathematics teachers. The Mathematics Educator, 6(2), 77-94.
Fan, L., Quek, K. S., Koay, P. L., Ng, C. H. J. D., Pereira-Mendoza, L., Yeo, S. M., et al. (2008). Integrating new assessment strategies into
mathematics classrooms: An exploratory study in Singapore primary and secondary schools (Final Research Report). Singapore: National
Institute of Education, Centre for Research in Pedagogy and Practice (http://crpp.nie.edu.sg/~pubs/CRP24_03FLH_FinalResRpt.pdf).
Halmos, P.R. (1980). The heart of mathematics. The American Mathematical Monthly. 87: 519-524.
Hargreaves, A., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment reform. Educational Researcher, 39(1), 69-95.
Hogan, D. (2007). Towards “invisible colleges”: Conversation, disciplinarity, and pedagogy in Singapore. Slide presentation available from
Office of Education Research, National Institute of Education, Nanyang Technological University.
Kulm, G. (1994). Mathematics assessment: What works in the classroom? San Francisco, CA: Jossey-Bass.
Ministry of Education. (2004a). Assessment guides to primary mathematics. Singapore: Author.
Ministry of Education. (2004b). Assessment guides to lower secondary mathematics. Singapore: Author.
National Council of Teachers of Mathematics. (1995). Assessment standards for school mathematics. Reston, VA: Author.
Niss, M. (1993). (Ed.). Cases of assessment in mathematics education: An ICMI study. Dordrecht, The Netherlands: Kluwer.
Quek, K. S., & Fan, L. (2009). Rethinking and researching mathematics assessment in Singapore: The quest for a new paradigm. In K. Y. Wong,
P. Y. Lee, B. Kaur, P. Y. Foong, & S. F. Ng (Eds.), Mathematics education: The Singapore journey (pp. 411-434). Singapore: World
Scientific.
Webb, N. L. & Coxford, A. F. (1993) (Eds.). Assessment in mathematics classroom. Reston, VA: NCTM.
Williams, C. (1998). Using concept maps to assess conceptual knowledge of function. Journal for Research in Mathematics Education, 29(4),
414-421.