Rethinking Assessment 2021

02 CSE
LEADING
EDUCATION
SERIES
APRIL 2021
Rethinking assessment
in education: The case
for change
B IL L LUC A S
Acknowledgements Contents
All my colleagues at Rethinking Assessment, especially Rosie 2 Introduction
Clayton, Peter Hyman, Rachel Macfarlane and Al McConville;
to the Edge Foundation for its support; and to the team at UCL 3 The wrong kind of nets for catching
School of Management for their research. young people’s strengths
Many along the way whose work has influenced my thinking
about assessment, including: Michelle Anderson, Ken Baker,
5 An education system fit for purpose?
Geoff Barton, Ron Berger, Sarah-Jayne Blakemore, Roy 12 The problem with educational
Blatchford, Esther Care, Christine Cawsey, Guy Claxton,
Paul Collard, Art Costa, Angela Duckworth, Carol Dweck,
assessment today
Charles Fadel, Sharon Foster, Michael Fullan, Valerie Hannon, 19 Revisiting the purposes of assessment
John Hattie, Lois Hetland, Rosemary Hipkins, David Howes,
Tony Mackay, Geoff Masters, Jonnie Noakes, James Pellegrino, 23 Promising practices from across
David Perkins, Mario Piacentini, Sandra Milligan, Andreas the world
Schleicher, Ellen Spencer, Michael Stevenson, Louise Stoll,
Stéphan Vincent-Lancrin, Dylan Wiliam. 33 Visible progress
38 References
ISSN 1838-8566 ISBN 978-1-925654-57-8

© 2021 Centre for Strategic Education, Victoria.
The Centre for Strategic Education welcomes usage of this
publication within the restraints imposed by the Copyright Act.
Where the material is to be sold for profit then written authority
must be obtained first. Detailed requests for usage not specifically
permitted by the Copyright Act should be submitted in writing to:
The Centre for Strategic Education
Mercer House, 82 Jolimont Street,
East Melbourne VIC 3002
Produced in Australia by Centre for Strategic Education
Mercer House, 82 Jolimont Street, East Melbourne VIC 3002
Editorial Team: Tony Mackay, Keith Redman,
Murray Cropley, Andrew Miller
Rethinking assessment
in education: The case
for change
B ILL L U C A S
Introduction
This paper is the second in the CSE evidence-based in teaching and learning, we
Leading Education Series and a are failing to keep up with the science of
collaboration between CSE in Australia assessment, preferring to rely on outdated,
and Rethinking Assessment in England. outmoded and unsubtle methods.
Across the world assessment is not Our young people require all of us working
working. We are not evidencing the kinds in education to establish greater clarity
of dispositions and capabilities that about the uses of assessment in education,
society increasingly wants. Educational linked to a greater understanding of the
jurisdictions are placing too much reliance science of assessment.
on high-stakes, standardised testing. They
We need nothing less than a paradigm shift
are testing the wrong things in the wrong
in our understanding about how best to
ways. High-stakes assessment is having
create assessment systems that use more
a damaging impact on the health and
effective ways of evidencing the full range
wellbeing of students and it is not giving
of student progress.
universities, colleges or employers the kind
of information they want. Assessment is In addition, we want to move rapidly from
out of sync with curriculum and pedagogy. theoretical debate to practical prototyping
Where we have become increasingly and implementation.
Rethinking assessment in education: The case for change / 2

The wrong kind of nets for
catching young people’s
strengths
To solely use standardised achievement tests is like casting a net into the sea – a
net that is intentionally designed to let the most interesting fish get away. Then, to
describe the ones that are caught strictly in terms of their weight and length is to
radically reduce what we know about them. To further conclude that all the contents
of the sea consist of fish like those in the net compounds the error further. We need
more kinds of fish. We need to know more about those we catch. We need new nets.
(William T Randolph, Commissioner of Education, Colorado1)
Metaphors abound in education. From In the UK, for example, GCSE exams
the Greeks via the Romans we took the routinely fail 33 per cent of all sixteen
idea that a child’s mind was a tabula rasa year olds. The students who do not make
or blank slate. Children, the comparison the grade have become known as the
suggests, know nothing and bring nothing; ‘forgotten third’ (Association of School
all is dependent on the experiences that and College Leaders, 2019). The system
adults offer them. They are empty vessels has sifted ‘sheep’ from ‘goats’, but the
waiting for those more knowledgeable than public, the shepherd in this analogy, has
them to fill up their minds. little understanding of what it all means
and the goats, the third who ‘fail’, are
There are many other
left with nothing much to show for their
metaphors we might draw
ATAR is a ladder in an on that are more cheerfully
compulsory schooling.
educational game of expansive: a search for hidden In Australia, the Australian Tertiary
snakes and ladders, treasure; an odyssey; discovery; Admission Rank (ATAR) is a kind of
whose higher rungs challenge. Randolph net, too. The score out of 100
hold out a promise of When it comes to assessment,
gives Australian youth a certain kind of
success, which turns out weight and length and then produces
William Randolph’s
a rank order. ATAR is a ladder in an
to equate to abstract thoughtlessly designed net
educational game of snakes and ladders,
rather than to real- seems an apt image for our
whose higher rungs hold out a promise
world intelligence. times. For, in different ways,
of success, which turns out to equate
educational assessment
to abstract rather than to real-world
systems across the world
intelligence.
have become very good at weighing and
measuring students, without reflecting on The Randolph net metaphor originated
whether the assessments they are making in the USA, where, notwithstanding
are relevant, meaningful or useful, and considerable varieties in provision between
without considering the consequences of states, it is reasonable to assume that it
the assessment process. has currency as a provocation beyond
Colorado.
3 / CSE Leading Education Series #02 April 2021

Words matter too. The nouns and verbs we Each of these words says something
use in connection with assessment come about the kind of learning imagined, the
freighted with semantic and educational method by which it might be assessed, the
baggage. perspective from which such assessment is
made and the validity or value that might
achievement, attainment, assessment,
be placed on the approach being described.
baccalaureate, badge, (balanced) score-
card, competition, curriculum vitae, As we unpack the practices of assessment,
demonstration, diploma, evidence, it may be helpful to stay close to the
exam, exhibition, expedition, feedback, words which have least baggage, such
illustration, interview, observation, as ‘evidence’ or ‘record’ (noun and verb)
passport, performance, portfolio, and phrases like ‘track the progress of’.
presentation, profile, project, publication, Too often we invest the scores and grades
qualification, record, score, score-card, used in end of school qualifications with a
task, test, transcript, viva … scientific validity they do not possess.
achieve, attain, assess, curate, We need new educational nets to catch
demonstrate, display, evidence, more of the capabilities young people need
examine, exhibit, illustrate, measure, to thrive today.
present, qualify, record, score, test,
track (the progress of) …

An education system
fit for purpose?
The pervasive obsession with academic grades and degrees, and corresponding elite
rewards at the expense of other people … results in narrow learning that severely
distorts what people learn and need in the 21st century.
Michael Fullan, 2021, p 8
Across the world there has been growing The beginnings of a global
discontent with the content of school
curricula, ever since the arrival of the curriculum
twenty-first century with its attendant Today there are a dozen or so well regarded
millennial symbolism. In addition to models of what contemporary curricula
traditional subjects such as literacy, maths should look like. The World Economic
and science, it is widely argued that Forum (2015) is widely cited (see Figure 1).
schools need to focus on what students can
Whether framed as foundational literacies,
do and who they are becoming.
competencies or character qualities, it is
increasingly recognised that, in our digital
age, there are more core literacies than
Figure 1. 16 skills for the twenty-first century (World Economic Forum, 2015)
21st century skills
Foundation literacies Competencies Character qualities

How students apply core How students approach How students approach
skills to everyday tasks complex challenges their changing environment
1. Literacy 7. Critical thinking/ 11. Curiosity

problem solving
2. Numeracy 8. Creativity 12. Initiative
3. Scientific 9. Communication 13. Persistence/grit

literacy
4. ICT literacy 10. Collaboration 14. Adaptability
5. Financial
literacy 15. Leadership
6. Cultural and
civic literacy 16. Social and cultural
awareness
LIFELONG LEARNING

Figure 2. Center for Curriculum is the one developed by the Center for
Redesign 4D Framework 1.02 Curriculum Redesign (see Figure 2).
There are many variants of such
contemporary curricula – of which the one
Knowledge Guy Claxton and I developed, Educating
‘What we know and understand’
Interdisciplinarity Ruby: What Our Children Really Need
Traditional (ie, Mathematics)
Modern (ie, Entrepreneurship) to Learn (2015), focusing on Confidence,
Themes (ie, Global Literacy)
Curiosity, Collaboration, Communication,
Creativity, Commitment and Craftmanship
is one. The approach adopted in New
21st
Century Pedagogies for Deeper Learning3 (Character,
Skills Learner Character Citizenship, Collaboration, Communication,
‘How we use ‘How we behave
what we know’ and engage in Creativity and Critical Thinking) is another.
the world’
We are building on the 3Rs of old to develop
Creativity Mindfulness
Critical Thinking Curiosity the 6 or 7Cs of today.
Communication Courage
Collaboration Resilience
Ethics Whether people like or do not like
Leadership
the framing of these dispositions as
Meta-Learning twenty-first century skills, or students as
‘How we reflect and adapt’
Metacognition twenty-first century learners (I do not,
Growth Mindset
finding the phrases vague, misleading
© Centre for Curriculum Redesign
and somewhat evangelical), there is now
substantial common ground as to what
we once thought. Whether we use words these dispositions or wider skills are.
like ‘competencies’ or ‘character ‘(or both) Importantly this consensus includes
there are certain important dispositions or perspectives from educationalists, parents,
capabilities for living a good life, and for psychologists and researchers, as well as
being a good learner, which schools have employers.
a role in cultivating.
Table 1 is an overview of different
Another model with a broadly similar evidence-based lists of such dispositions
framing of a contemporary curriculum (Lucas, 2019).
Table 1. Dispositions for a lifetime of learning (Lucas, 2019)
European Key Pellegrino and Gutman and Heckman and Lamb et al, 2017
Competences for Hilton, 2012 Schoon, 2013 Kautz, 2013
Lifelong Learning,
2007
• Communication • Critical thinking • Motivation • Perseverance • Critical thinking
in mother tongue • Information literacy • Perseverance • Self-control • Creativity
• Communication in • Reasoning • Self-control • Trust • Metacognition
foreign languages
• Innovation • Metacognitive • Attentiveness • Problem-solving
• Digital strategies
competence • Intellectual • Self-esteem and • Collaboration
openness • Social self-efficacy • Motivation
• Learning to learn competencies
• Work ethic • Resilience • Self-efficacy
• Social and civic • Resilience to adversity
competences • Conscientiousness • Conscientiousness
and coping • Openness
• Sense of • Positivity • Perseverance
• Creativity to experience
initiative and • Communication
entrepreneurship • Empathy
• Collaboration
• Cultural • Humility
• Responsibility
awareness and • Tolerance of
expression • Conflict resolution diverse opinions
• Engaging
productively
in society

A generic term, expansive education, Of course curriculum is only one of
has been developed by the author and the three core elements of education
colleagues at the University of Winchester, systems, the other two being pedagogy or
(Lucas, Claxton and Spencer, 2013) to instruction (how the curriculum is taught)
describe the kinds of dispositions that are and assessment (how performance and
desirable for success at school and in life, progress is evidenced). Of significance for
and how these can be interleaved into the this report is the chronic disjoint between
subject disciplines of the school timetable. curriculum, pedagogy and assessment
with, as yet, no focus or guidance on
Research by the Brookings Institution
the teaching or assessment of these
(Care et al, 2016) has shown that,
competencies/dispositions.
across the world, such dispositions
are gradually beginning to filter their Interestingly, the same Brookings research
way into schools, with 36 countries shows a kind of league table of progress in
mentioning them explicitly, 76 countries developing more expansive curricula in
identifying skills related to them, 51 countries and states across the world (see
locating them within the curriculum Table 3).
and 11 mapping their progression over
While the research does not cover every
the lifetime of formal schooling. The
educational jurisdiction in the world, it is
scope and sequence documents of the
worth noting that the countries and states
Australian capabilities are an example of
making most progress in implementing
the last of these categories. Recently the
new thinking about contemporary
Brookings Institution, using the Center
curricula according to this report are
for Curriculum Redesign model, has
Australia, British Columbia (Canada),
produced an overview of the prevalence
Singapore, Finland, Hong Kong, Victoria
of dispositions/competencies in different
(Australia) and New Zealand.
educational jurisdictions (see Table 2).
Table 2. The prevalence of Center for Curriculum Redesign competencies,

(Taylor et al, 2020)
Competency Inclusion Identification Progression Pedagogy Assessment

Creativity 21 12 5 0 0
Critical thinking 21 11 6 0 0
Skills
Communication 22 11 5 0 0
Collaboration 21 10 6 0 0
Mindfulness 17 10 5 0 0
Curiosity 17 7 3 0 0
Character
Courage 9 5 5 0 0
Resilience 15 8 6 0 0
Ethics 18 10 4 0 0
Leadership 10 7 4 0 0
learning
Metacognition 14 7 5 0 0
Meta-
Growth mindset 14 6 5 0 0

Table 3. The frequency of competencies/dispositions across jurisdictions
(Taylor et al, 2020)
Jurisdiction CRE CRI COM COL MIN CUR COU RES ETH LEA MET GRO Total
Australia (Federal) 3 3 3 3 3 3 3 3 3 3 3 3 36
British Columbia (Canada) 3 3 3 3 3 3 3 3 3 3 3 3 36
Singapore 3 3 3 3 2 3 3 3 3 2 2 3 33
Finland 3 3 3 3 3 3 3 2 3 3 3 32
Hong Kong (China SAR) 2 3 3 3 3 1 3 3 2 2 3 3 31
Victoria (Australia) 3 3 1 3 3 1 3 3 3 3 26
New Zealand 2 2 2 2 2 2 2 2 2 2 2 22
Portugal 2 2 2 2 2 2 2 2 16
Chinese Taipei (aka Taiwan) 2 2 2 2 2 2 2 14
Denmark 2 1 1 1 1 1 1 1 1 1 11
England (UK) 2 1 1 1 2 1 1 1 1 11
Scotland (UK) 1 1 1 1 1 1 1 1 1 1 1 11
South Korea 2 2 2 1 2 1 1 11
Alberta (Canada) 1 1 1 1 1 1 1 1 1 1 10
New Brunswick (Canada) 1 2 2 1 1 1 1 1 10
New South Wales (Australia) 1 1 1 1 1 1 1 1 1 1 10
Massachusetts (USA) 1 1 1 1 1 1 1 1 1 9
Ontario (Canada) 1 1 1 1 1 1 1 1 1 9
China 1 2 2 1 2 1 9
USA (Federal) 1 1 1 1 1 1 9
Japan 1 1 1 1 1 5
Russia 1 1 1 1 4
Total 38 38 38 37 32 27 19 29 32 21 26 25
Researchers looked at the frequency of mentions in curriculum documents across 5 categories - Competency inclusion, Competency
identification, Competency progressions, Competency pedagogies and Competency assessments. A “3” (blue) indicates that the jurisdiction
has identified that competency in 3 categories, a “2” (orange) indicates that the competency was identified in two categories and so on.
Differing approaches to that there is a middle way; that these are

false binary positions.
teaching and learning
In such a mid-position we might be asking
In terms of the kinds of pedagogy/ questions such as:
instruction needed today to develop both
the foundational literacies and the kinds of What kind of knowledge is it important
desirable dispositions listed in Table 1, the for all young people to have?
educational world has become unhelpfully What kind of dispositions is it important
polarised. for all young people to acquire?
One group, broadly those who might see How can we ensure that young people
themselves as traditional, tends to argue acquire and apply useful knowledge in
for the teacher’s role in transmitting a range of settings?
knowledge and to favour didactic methods.
The other, typically seen as progressive, How can we teach young people to
argues for student-led approaches, such as work across subject disciplines, as
problem-based learning. With Guy Claxton happens in the real world, ensuring that
(Claxton and Lucas, 2015) I have suggested they have the necessary building blocks
in place?

How can we ensure that important A misunderstanding of the role
dispositions for learning and for life are
best cultivated in a range of disciplinary of skills in learning
contexts? There is much nonsense talked about skills
How can we develop strength, breadth today.
and depth in learning to facilitate its By those who see the acquisition of
transfer across contexts? knowledge as the main purpose of
Which pedagogies work best for education, an emphasis on skills is
promoting deep learning? often portrayed as an attempt to dumb
down or distract schools from their core
How best to assess knowledge and purpose.
evidence dispositions?
By those who see dispositions and
As argued so far, there is emerging capabilities as being centrally important,
agreement as to the answers to the first two there is a temptation to hold fast to
questions, with continuing discussions bigger concepts, such as creativity or
about the other six. collaboration, without recognising that
There are many other questions we they are in reality made up of aspects of
could pose with regard to pedagogy or knowledge and clusters of skills.
instruction, and many sources of evidence Knowledge and dispositions are not
on which educators can draw (Hattie, polar opposites, just different ways
2008; Committee on Developments in the of categorising what we can learn.
Science of Learning, 2000; Coe et al, 2020). The ‘currency’ of both is skills. Skills
Importantly, any teacher reaching for such are what matter in life. Skills are the
guidance will need to consider not just ‘connective tissue’ between knowledge
which teaching methods promote success and dispositions. As we practise a skill
in terms of typical examinations, but in different contexts we become more
which methods also cultivate competent, confident and capable, until
the kinds of dispositions or it becomes a disposition, something we
Skills are what matter capabilities young people need. are disposed to do. Some examples, from
in life. Skills are the Ideally, methods that effectively simple to more complex, might include
‘connective tissue’ promote both outcomes will be
planning an essay;
between knowledge chosen.
and dispositions. delivering a speech;
critiquing an argument;
having a good idea when you need one;
tying your shoelace;

trying different approaches when faced Learning 2:
with a tricky problem; Learning is the ability to consciously
using your common sense when your modify understandings, beliefs and actions
satnav takes you to a cul-de-sac not yet in response to evidence, experience, and
updated in its software; reflection.
reading the mood of those with whom Schooling is one of many environments in
you are working; which humans develop the capability to
exercise judgement and control over what
facilitating a workshop where you are a
they learn, how they learn, and what they
content expert; or
intend to do with what they have learned.
facilitating a workshop when you have
Assessment is the means by which
only a basic knowledge of the context
individuals receive useful information
but can transfer facilitation skills learned
about the development of their capabilities
in other contexts to the task at hand.
as learners over time.
And, yes, recalling (Elmore, 2019, p 333)
decontextualised information
Assessment influences In this paper it is the second of these two
in a pencil and paper
not just what gets taught examination is a skill, but not conceptions that we shall be exploring.
but how it gets taught one that adults need to use
much in a digital age.
The tail that wags the dog
The deeper your knowledge and the more
you practise your skills in a variety of So, to assessment. Almost anyone who has
contexts, the more capable you become. worked in education knows that what gets
Dispositions are clusters of skills which assessed by and large gets taught. You can
have been practised so well that they have a bold and expansive curriculum, but
have become habitual; you are routinely as the time of examinations draws close,
disposed to deploy them; and skills are the the focus shifts to those aspects of the
mechanism by which knowledge is applied curriculum which will be assessed. This is
and dispositions are lived out. especially true in upper secondary schools
as students reach the age when they move
on to university, vocational training or
Learning 2.0 employment. The decisions are complex
for young people as they navigate their
Richard Elmore (2019) helpfully next steps, and the means by which such
summarises the consequences for school of decisions are made are often by ‘high-
two very different conceptions of learning, stakes assessment’.
as follows.
Assessment influences not just what gets
Learning 1: taught but how it gets taught. If Teacher
Learning is the ability to recall and deploy X uses a particular teaching method for
information and algorithms accurately and science with her class and students do
appropriately. well in their assessments, while Teacher Y
uses a different method and her class does
Schooling is the mechanism by which we
less well, then, assuming the classes share
organise social and status consistent with
similar enough characteristics, schools and
this definition of learning.
school systems will begin to draw lessons
Assessment is the means by which we from this. Reasonably enough they will
define, measure, evaluate, and confer ‘merit’, suggest that when teaching science the
consistent with this definition of learning. methods chosen by Teacher X are the ones
to use.

Most education systems
are seriously in need of
attention if they are to
be fit for purpose.
At first sight this is an intelligent system’s Most education systems are seriously in
response, but what if assessments in need of attention if they are to be fit for
science privilege decontextualised purpose. Curricula are changing, debates
recall of scientific theory and simplistic are at least being had about pedagogy
memorisation of scientific facts, which (Griffin, McGaw and Care, 2012; Vincent-
neither encourage students to think and Lancrin, et al, 2019) but, despite some
work like scientists nor equip them to promising initiatives, assessment needs
go on to deeper study of science and some serious rethinking.
its uses in society? What if thinking
Eight years ago Geoff Masters suggested
about assessment is not keeping up with
that the ‘field of educational assessment
advances in the learning sciences? In these
is currently divided and in disarray’ in
cases such a response would be dumb.
Australia (Masters, 2013, p 1). I suspect
The dog in the sub-heading of this section that this is still the case in Australia and
is the school system and the tail that wags still the case for the majority of educational
it is, of course, assessment. Also, mixing jurisdictions across the world today.
my metaphors, the tails which seem to wag
Let us turn now to what is wrong with
so many school systems across the world
educational assessment in more detail.
are the fishing nets with which this section
began.

The problem with educational
assessment today
The measurement of deep learning must be always informed by a wealth of

underlying assessment evidence that captures the complete picture of who students
are, what they know and whether they are prepared to use that knowledge to advance
their lives and others.
(Joanne McEachen, Assessment for Deep Learning, 2017, p 12)
There are many aspects of educational The risk is that schools create students
assessment today which are failing. These dependent on direct instruction,
fall into the four broad areas of cramming, drilling and coaching,
reliant on expert instruction by teachers
what is assessed (focus);
who are expected to guide learners
how it is assessed (methods); through a carefully prescribed body of
the impact of the assessment process knowledge, assessed in predictable ways.
(consequences); and (p 14)
the uses made of the assessment

(validity).
An assessment focus that is
Of course there is also a fifth
too shallow and too narrow
challenge: the degree to which
Complex, higher order whatever we might want Currently, the knowledge that is typically
skills are rarely assessed to measure can be reliably assessed is from a narrow range of subjects,
assessed. rarely explored in depth and almost never
in ways that recognise interdisciplinary. Practical knowledge
the subtleties involved. In a recent review (2020a)
and skill is not much assessed in general
Many dispositions or Sandra Milligan and colleagues
education, and individuals rather than
cut across all of these
capabilities known to teams remain the focus. Complex, higher
categories elegantly when they
be important in life are suggested that
order skills are rarely assessed in ways that
not assessed at all. recognise the subtleties involved (Darling-
Without a focus on mastery of Hammond, 2017). Many dispositions or
generic capabilities, assessment capabilities known to be important in life
and teaching practices tend to privilege are not assessed at all.
memorisation, essay writing, individual Assessments frequently require recall
mastery of set content and solving of of content but rarely demand the kind
problems with formulaic solutions. of deep thinking, problem solving or
application needed in the real world.

Traditional areas, literacy, maths and Assessment methods that
science continue to require considerable
content to be tested, while newer are too blunt
areas such as citizenship, engineering, Most tests used in schools still rely on
sustainable development and ethical paper and pencil. They examine aspects
understanding are only briefly explored. of knowledge and routine skills. They
Except in a very few countries (Finland test students’ ability to remember and
and Singapore are examples) there is little write about something, rather than apply
or no interdisciplinary assessment.4 or do the thing they have been learning.
Concepts and skills are tested in individual
Practical knowledge and skill is rarely subjects and only very rarely across
assessed even in those subjects where disciplines.
it once used to be a central component,
such as science. While tests often purport to be criterion-
based, many countries effectively revert
Students’ capabilities in planning and to norm-referencing either because of the
undertaking extended investigations are scale used (the ATAR in Australia, for
rarely assessed. example), or the external moderation by
Although the ability to collaborate an accountability body that keeps levels
with others is widely valued in the of achievement very similar year on year
workplace it is only acknowledged at (as with GCSE in England). Even where
school on the sports field or in music tests are explicitly criterion-based, grades
and drama performances. often relate to syllabus content, rather
than to more carefully sequenced learning
While dispositions or capabilities are progressions.
becoming more visible in curricula
they are rarely assessed; at a global Traditional assessment methods
level PISA’s innovative domain tests typically fail to measure the high-
of collaborative problem-solving and level skills, knowledge, attributes
creative thinking are exceptions, as is and characteristics of self-directed
the State of Victoria’s testing of critical and collaborative learning that are
and creative thinking. increasingly important for our global
economy and fast-changing world.
(Griffin, McGaw and Care, 2012, p v–vi).
Figure 3. A continuum of assessment methods, adapted from Darling-Hammond

(2017), p 6
Shallow/Narrow Deeper/Wider
Traditional tests Tests with Performance tasks Extended tasks Longer, deeper
pencil and paper open-ended (1 day–1 week) (1–4 weeks) investigations
often multiple- items and short requiring extended involving exhibitions
choice for routine performance problem solving formulation/ (2–3 months)
skills tasks carrying out of in multiple
inquiries and modalities
presentation of
findings

A recent High Resolves report (2020) Assessments need not be done in this
proposes the concept of ‘strings-based way, as ‘Measuring progress provides a
assessment’ (High Resolves, 2020, p 16) deliberate counterpoint to the traditional
to exemplify the kind of blend or ‘strings’ practice of measuring achievement
of immersive, repeated practices and at specific time points’ (Hipkins and
real-world applications that may be Cameron, 2018, p 22).
useful in evidencing high-order skills
in citizenship education. The range of
possible assessment methods educational Consequences that
jurisdictions might choose from is actually are unhelpful
wide (see Figure 3).
In any assessment system there are
Students are tested at set intended and unintended consequences,
times rather than when they but it would seem fundamental to assume
fundamentally, most are ready, often to meet the that an essential principle should be, as the
assessments fail to needs of the next educational USA’s Gordon Commission on assessment
capture the degree to provider or, frequently in 2013 noted, that assessment systems
which students have ineffectively, of employers. should ‘do no harm’.
These inflexible encounters
progressed over time Sadly, the consequences of the focus and
with assessment ignore
the huge variety of student methods of many, especially high-stakes
achievement levels, where ‘in any given assessments, are well-documented and
year of school, the most advanced learners harmful in a number of ways, including
in areas such as Reading and Mathematics leading students to conclude that they
can be as much as five or six years ahead are failures (Education Policy Institute,
of the least advanced learners’ (Masters, 2019);
2013, p 3), the fact that ‘attainment is only
demotivating students to the extent that
loosely related to age’ (Wiliam, 2007, p 248)
they may not stay on at school or find
and the differing levels of maturity found
employment (Milligan et al, 2020a);
in any cohort on account of birth dates.
making it less likely that students will
More fundamentally, most assessments fail
see themselves as learners and want to
to capture the degree to which students
continue learning throughout their lives
have progressed over time. Instead they
(Tuckett and Field, 2016)
… provide snapshots of achievement at
causing negative impact on young
particular points in time, but they do
people’s wellbeing (Howard, 2020);
not capture the progression of students’
conceptual understanding over time, exacerbating inequity (Au, 2016);
which is at the heart of learning. This reducing performance through anxiety,
limitation exists largely because most especially for students of lower ability
current modes of assessment lack an (von der Embse et al, 2018);
underlying theoretical framework of
how student understanding in a content increasing irrelevance to employers
domain develops. (Harvard Business Review, 2015);5
(Pellegrino, Chudowsky and Glaser, 2001, distracting from the huge importance of
p 27–28). assessment for learning and assessment
as learning (Birenbaum et al, 2015);

misunderstanding and undervaluing Dubious validity for many users
wider skills and dispositions by not
measuring them (Heckman and Kautz, Assessment serves many purposes,
2013), and perpetuating the myth that including the following.
soft skills are easy to acquire and of less It certifies, selects and credentials
value than so-called hard skills such as students for universities and colleges.
core literacies;
It is a sifting mechanism for employers.
inviting a lack of trust in teacher
It gives teachers information on the
judgement in some jurisdictions
progress of their students.
(Harlen, 2005; Coe et al, 2020) which,
in an unhelpfully reinforcing loop, It gives students actionable feedback
can lead to lower levels of teacher on their progress and suggests potential
assessment ‘literacy’. next steps.
In The Testing Charade (2015), Koretz Across the world, however, there is a crisis
reminds us of the danger of Campbell’s of validity, with growing dissatisfaction
law, that from each of the main users.
the more any quantitative social Universities and colleges

indicator is used for social decision-
Universities and colleges find the grades
making, the more subject it will be to
or scores they are provided with too crude
corruption pressures and the more
to be helpful, so that many are creating
apt it will be to distort and corrupt
consortia to work with schools to provide
the social processes it is intended to
more rounded information. The Mastery
monitor (p 38) … When test scores
Transcript Consortium,6 the New York
become the goal of the teaching process,
Performance Standards Consortium7 and
they both lose their value as indicators
the Comprehensive Learner Record,8 in
of educational status and distort the
the USA, and New Metrics for Success,
educational process in undesirable
in Australia,9 are indicators of a growing
ways.
unease with the status quo.
(p 39)
The National Academy of Education (2021) Employers

points out that, to avoid unintended and Employers are frustrated that the
sometimes unfair consequences, we need to current crop of academic and vocational
qualifications leave them under-informed
Communicate clearly (and often) about potential employees (Education
the intended purposes and uses of Council, 2020; Confederation of Business
particular assessments as well as any Industry (CBI), 2019). Many employers
relevant context. are now qualification-blind in their
(p 11) recruitment. In England, Rethinking
Assessment has identified many examples
of, predominantly, larger organisations
that operate in this way, including Apple,
Bank of America, BBC, the Civil Service,
Clifford Chance, Google, The Guardian,
Hilton, Microsoft, Penguin Random House,
PricewaterhouseCoopers (PWC) and
Starbucks.

Many employers now develop their When passing tests is high stakes,
own approaches to assessing potential teachers adopt a teaching style which
employees. Often these are ‘strength- emphasises transmission teaching of
based’ aptitude tests, looking to see what knowledge, thereby favouring those
capabilities and values candidates have to students who prefer to learn in this
better enable them to work productively way and disadvantaging and lowering
with others, seeking to establish a more the self-esteem of those who prefer
balanced scorecard than mere exam grades. more active and creative learning
experiences.
As Professor Tristram Hooley,
Chief Research Officer of the (Harlen and Deakin Crick, 2002, p 4)
Teachers ... are
Institute of Student Employers
concerned variously Wherever you are in the world, the
in England, puts it, COVID-19 pandemic has provided
about the way that
a dramatic interruption of normal
tests privilege certain Most employers don’t worry if a
candidate knows a little bit less assessment activity. PISA’s 2021 tests are
subjects over others, currently rescheduled until 2022. Across
about theories of population
especially ‘academic’ the world, school examinations for 18-year-
migration or the nineteenth
over practical, and century novel. But they will olds or 19-year-olds have been cancelled,
how an emphasis on care a lot about candidates’ postponed or simplified.12 In many cases
memorisation can lead ability to learn, to think on these changes have required students to
rely on teacher-assessed grades. While
to shallower and less their feet, to be resilient in the
this can be seen as a positive development
enjoyable learning, face of knock backs, and so
(inviting innovation in methods), in
on.10
especially at upper practice it has caused additional stress
secondary level The old narrative of working
among teachers who may not yet be
hard, getting good grades
assessment literate enough to undertake
at school, going to a good
such testing without an appropriate
university and securing a well-paid job
infrastructure of moderation and training,
is increasingly fractured. Employers are
along with equitable appeals processes.
becoming aware that, ‘when it comes to
predicting job performance, aptitude tests
Students
are twice as predictive as job interviews,
Students are increasingly unsettled. In
three times as predictive as job experience,
one part of their world they have moved
and four times as predictive as education
from an era of television programs to be
level’.11
watched at set times, to unlimited on-
Teachers demand consumption of You-Tube, TikTok
and streaming services; from books which
Teachers have had rising degrees of
needed to be learned, to an Internet which
dissatisfaction with the status quo since
can be searched. Not so their examinations,
the millennium. They are concerned
which mostly require pencil and paper
variously about the way that tests privilege
completion on a set date and considerable
certain subjects over others, especially
feats of memory.
‘academic’ over practical, and how an
emphasis on memorisation can lead to When it comes to high-stakes assessment,
shallower and less enjoyable learning, there is widespread and ongoing stress
especially at upper secondary level. This among students, as this blog13 on the website
was evident in England two decades ago. of Ofqual (The Office of Qualifications

and Examinations Regulation) in England Can dispositions be measured?
highlights. In The Testing Charade, Daniel
Koretz quotes an alarming letter from New In the last few decades we have made
York principals to parents. real progress in understanding how best
to evidence dispositions more generally
We know that many children cried (Soland et al, 2013; Darling-Hammond,
during or after testing, and others 2017; Siarova et al, 2017; Care et al, 2018).
vomited or lost control of their bowels In some cases real progress is being made
or bladders. Others simply gave up. One in developing useful standard measures of
teacher reported that a student kept specific aspects of some key dispositions,
banging his head on the desk … for example of ‘grit’ (Duckworth and
(Koretz, 2017, p 2) Quinn, 2009).
Interestingly, it is through tests like PISA
Educational jurisdictions that we have been able to make significant
An educational jurisdiction’s performance breakthroughs in our understanding of
is also judged through international how two key dispositions/competencies,
assessments. Assessments are used as collaborative problem solving14 and
a means by which society rates, often creative thinking15, can be assessed in an
in very limited ways, the performance online test. (I have been involved in helping
of its schools. Using tests such as the to shape the second of these two tests.)
Programme for International Student
We have been assisted in this process by
Assessment (PISA), Trends in International
advances in assessment technology. For
Mathematics and Science Study (TIMSS)
example, evidence-centred design,
and Progress in International Reading
a way of creating assessments that better
Literacy Study (PIRLS), the success of
demonstrate how test-takers’ inferences
individual jurisdictions can be compared
are made and their reasoning is developed
internationally. These have a powerful
as they approach assessment tasks, is a
impact on both what is tested and how it is
promising approach.
evidenced, but that is beyond the scope of
this discussion.

Table 4. Methods of evidencing progress in creativity
(Lucas and Spencer, 2017, p 160)
Student Teacher Real-world Online
Real-time feedback Criterion-referenced Expert reviews Reliable, validated

Photos grading Gallery critique online tests
Self-report Rating of products Authentic tests Digital badges

questionnaires and processes eg displays, E-portfolios
Logs/diaries/journals Structured interviews presentations,
Performance tasks interviews,
Peer review podcasts,
Group critique Capstone projects films
Badges Exhibitions
Portfolios
In my own work in the UK and in In 2016 the journal Applied Measurement

Australia, working with schools and in Education compiled a special issue
school systems and drawing on a wider focusing on the assessment of so-called
OECD study (Vincent-Lancrin, et al, 2019) 21st century skills.17 It focused on four
with which I was involved exploring the types of dispositions: collaborative problem
assessment of creativity, I have found that solving; complex problem solving; digital
a clear understanding of what creativity is, and information literacy; and creativity,
along with an understanding of learning to which I contributed our research at the
progression, is a necessary starting point. University of Winchester, (Lucas, 2016).
Then, provided a range of different In the spirit of scientific enquiry, the issue
perspectives are acknowledged, it is focused on both what we do know and
possible to provide students and teachers what we do not yet fully understand. It
with robust evidence of progress over time offered some promising approaches, some
(see Table 4). of which are already being used by PISA.
Importantly, we need multimodal Just as these days few contest the notion
assessment to gain an accurate picture, of the learning sciences as a valid lens to
using perspectives from at least three explore teaching, so we need a similar shift
columns in Table 4. in building the science of assessment.
I’ll say more about this in the final section.
However, we have a way to go yet. As
Daniel Willingham reminded us in 2013,
in his blog,
we’re far from agreed-upon measures.

Just how big a problem is that? It
depends on what you want to do. If you
want to do science, it’s not a problem at
all. It’s the normal situation.16

Revisiting the purposes
of assessment
Assessments must fully represent the competencies that the increasingly complex
and changing world demands. The best assessments can accelerate the acquisition of
these competencies if they guide the actions of teachers and enable students to gauge
their progress.
(Gordon Commission, 2013, p 7)
Over the last few decades we have Twenty years ago the Committee on
progressively lost our way with the Foundations of Assessment in the
educational assessment. What we assess USA (National Research Council, 2001)
grows ever further away from what we considered the degree to which advances
want young people to be able to know, do, in the cognitive sciences were impacting
be and become in the complex world in on educational assessment. The central
which they live today. The focus of most problem it identified is that ‘most widely
systems is on summarising used assessments of academic achievement
rather than understanding, are based on highly restrictive beliefs
We seem happier using recalling rather than applying, about learning and competence not fully
noticing deficiencies rather in keeping with current knowledge about
numbers rather than
than celebrating strengths. human cognition and learning’ (p 1). The
narratives, keener on We seem happier using report explores many of the then-current
judging rather than numbers rather than narratives, kinds of assessment and exposes these
prompting improvement. keener on judging rather than to rigorous scrutiny. It concludes with a
prompting improvement. vision of assessment that has still not been
realised in education:
Previous attempts to In the future envisioned by the

committee, educational assessments
rethink assessment will be viewed as a facilitator of high
The Assessment Reform Group in the levels of student achievement. They
UK made ground-breaking progress in its will help students learn and succeed in
exploration of the value of assessment school by making as clear as possible
for learning between 1996 and 2010, to them, their teachers, and other
something which has subsequently spread education stakeholders the nature of
across the world, albeit always in tension their accomplishments and the progress
with the tendency of systems to prefer of their learning.
summative data, (Birenbaum et al, 2015). (p 292)
For a long while (Black and Wiliam, 1998)
we have known that formative assessment In 2006 a group of states in Canada worked
is effective in promoting improvements in with Lorna Earl and Steven Katz to, as they
student learning. described it, ‘rethink classroom assessment

with purpose in mind’. The document In Australia, at the same time as the
provides a framework for thinking about Gordon Commission, the Australian
the purposes of assessment. Its reminders Council for Educational Research (ACER)
about the distinctions between assessment undertook a review of educational
for, as and of learning are clear and succinct: assessment, (Masters, 2013). Geoff Masters
reminds us that ‘the fundamental purpose
Assessment for learning is designed of assessment is to establish where
to give teachers information to learners are in their learning at the time
modify and differentiate teaching and of assessment’ (p 5–6). In a thoughtful
learning activities. It acknowledges and prescient overview Masters points
that individual students learn to the folly of age-related testing, the
in idiosyncratic ways, but it also failure of assessment to enable effective
recognizes that there are predictable differentiation in teaching, the way in
patterns and pathways that many which it supports ‘traditional approaches
students follow. to schooling, including the assembly-
Assessment as learning is a process line model, whole-class teaching, age-
of developing and supporting based curricula’ (Masters, 2013, p 4), its
metacognition for students. Assessment ignoring of important life skills such as
as learning focusses on the role of collaborative working, and its pedestrian
the student as the critical connector use of technology. Not much has changed
between assessment and learning. in the eight years following the ACER review.
Assessment of learning is summative The COVID-19 pandemic, as well as

in nature and is used to confirm requiring educational jurisdictions to
what students know and can do, rethink their approach to assessments, at
to demonstrate whether they have least temporarily, has forced educators to
achieved the curriculum outcomes, think more carefully about issues of equity
and, occasionally, to show how they are as we recover from the pandemic.
placed in relation to others.
Assessments, if used properly, can
(Earl and Katz, 2006, p 13–14) help us to mitigate the impacts of the
The Gordon Commission in the COVID-19 pandemic for years to come.
USA in 2013 made a number of key If used improperly, assessments may
recommendations, about designing and waste precious instructional time and
implementing assessment that supports resources, worsen inequities, reinforce
a more ambitious and expansive vision misperceptions as to sources of inequity,
of education. It is vitally important, the and impede sound education policy.
Commission argued, that assessments (National Academy of Education, 2021, p 13).
best represent the kind of learning New Metrics for Success18 at Melbourne
students will need to thrive in the world University in Australia, the Brookings
that awaits them beyond graduation Institution in the USA19 and Rethinking
(p 8) … Assessments must advance Assessment20 in England are three
competencies that are matched to the examples of organisations trying to find
era in which we live. Contemporary saner solutions to many of the issues noted
students must be able to evaluate the in these earlier attempts to reimagine
validity and relevance of disparate assessment. Also we are starting to move
pieces of information and draw with an urgency and a focus on practical
conclusions from them. solutions.
(p 9)

Many schools that have been
particularly successful in reducing
opportunity and achievement gaps for
traditionally marginalized students
– producing high graduation and
college success rates – have adopted
mastery-oriented performance-based
assessments that build higher order
thinking and performance skills,
collaboration and communication
skills, motivation and engagement,
and a host of co-cognitive skills such
as self-regulation, executive function,
resilience, perseverance and growth
mindset (p 121) … Performance
assessments that encourage higher
order thinking, evaluation, reasoning,
and deep understanding are themselves
tools for learning.
(p 122)
Along with an understanding of the

The science of assessment purpose of any activity to evidence
Over the last two decades there has been a performance and progress in education,
step change in the way in which evidence these kinds of observations form the
about learning is used by schools and bedrock of the assessment sciences that
within school systems; the science of will help us take better decisions. To return
learning is a widely accepted concept in to the metaphor of fishing with which I
education. began this paper, the science of assessment
will help us to cast the right kinds of nets.
However, despite some occasional
thoughtful publications (National Research
Council, 2001), there has not been the
same culture shift among teachers and
Characteristics of high-quality
policy-makers with regard to the science assessment systems
of assessment. A Google search on ‘science In the last decade a significant number of
of learning’ produces 1,840,000,000 results, reviews (Lai and Viering, 2012; Conley
while one on ‘science of educational and Darling-Hammond, 2013; Bennett,
assessment’ yields 655,000,000, the 2013; OECD, 2013; Masters, 2013; Soland,
majority of which on closer scrutiny turn Hamilton and Stecher, 2013; Hill and
out to be about the assessment of science Barber, 2014; Siarova, Sternadel and
education. Mašidlauskaitė, 2017; Care et al, 2018;
In a recent paper exploring implications O’Connell, Milligan and Bentley, 2019;
for educational practice of the science Care, Anderson and Kim, 2019; Milligan et
of learning (Darling-Hammond et al, al, 2020b) have looked at the implications
2020) it is noteworthy that assessment for systems wanting to move towards the
does not merit a discrete section (despite assessment of deeper learning, what Elmore
the fact that the paper is written by a calls ‘Learning 2’ (See page 10).
team including an assessment expert). While analysing implications at a system
Notwithstanding this, there are clear level is complex, and needs to take
messages to be heeded. into account the differences between

the political intentions of educational Authenticity
jurisdictions, the convergence of thinking Increasing interest in strengths-based
across these reviews – combined with the approaches, especially from employers.
slowness with which their suggestions
have been taken up – reminds us of how The need to design better performance-
difficult it is to change assessment systems. based assessments.
Common themes from evidence on high- A move towards scenario-based,

quality assessment systems include the authentic assessment.
following. A move towards assessments of
investigations over longer time periods.
Purpose and consequence
The importance of understanding the Some interest in assessment on demand.
purpose any assessment is intended to Increased opportunities for student
serve. involvement and agency in the process.
A growing recognition of assessment as Progression and improvement
a tool for improvement at individual,
The benefits of assessment for and as
school and system level.
learning.
The tensions that exist between
The need for multimodal approaches
summative and formative approaches.
to assessment, incorporating data from
The many unhelpful consequences of a number of sources.
high-stakes assessment.
Quality infrastructure
Depth and breadth A better understanding of when to use
A need to evidence high-order thinking assessment of, for and as learning.
skills reliably.
The need for new assessment
A requirement for better definitions of partnerships.
dispositions and associated learning
Enhanced teacher capacity in
progressions.
assessment literacy and moderation.
The growing visibility of dispositions
The desirability of international
in the curricula of educational
benchmarking.
jurisdictions.
Three things emerged at the metalevel.
The desirability of assessments
Systems need to decide what they value
being pedagogically sensitive and
before they determine what they seek
educationally valuable.
to evidence. An increasingly scientific
The complexity of designing ways approach to the field of educational
of fairly evidencing student progress assessment is required. While the direction
within dispositions. of travel is increasingly clear, progress
A growing interest in the concept of towards it is glacially slow.
mastery.
The need for flexibility to ensure that
the full range of abilities can be fairly
assessed.
A focus on collaborative rather than just
individual performance.

Promising practices from
across the world
Mā te kimi ka kite, Mā te kite ka mōhio, Mā te mōhio ka mārama.

Seek and discover. Discover and know. Know and become enlightened.
(Maori saying)
There are many examples from which An overarching question here concerns just
we can learn as we rethink the purposes how much knowledge we think students
and practices of assessment. As the need to acquire at school and what kind
Education Council (2020) reminded us in of knowledge that is. Across the world,
its review of senior secondary pathways and reinforced by PISA’s focus, literacy,
into work in Australia, we need to learn numeracy and science are generally
from those who are actually trying things considered to be foundational.
out, ‘Demonstration projects need to have If the answer to this question is ‘less than
greater influence on the traditional core of we do now’, as many of us believe to be the
how we measure educational success’. case, then we will need to understand any
possible consequences for the curriculum
in schools of reducing the core focus of
Interrogating practices assessment.
At Rethinking Assessment (RA) in At a more nuanced level we might want
England, we have been exploring a number to look at the science curriculum to see
of questions to better understand the which concepts are more relevant than
nature of the problems with others given the size of the field; with
which we are grappling, as we maths we might wish to re-emphasise
An overarching question explore promising international content, prioritising, for example, statistics
here concerns just how demonstrations of what might over some aspects of trigonometry.
much knowledge we be adopted in England. We We might want to weigh the benefits
think students need to have developed two sets of of interdisciplinary knowledge, either
questions, one to do with expressed as, for example, STEM (Science,
acquire at school and
knowledge and skills (see Technology, Engineering and Maths) or
what kind of knowledge Table 5), and another relating STEAM (Science, Technology, Engineering,
that is. to dispositions and skills (see Arts and Maths), or through an
Table 6). organisation of the curriculum into projects
Skills, as I have argued earlier, are the requiring more than one discipline. Both
connective tissue between knowledge and of these approaches are increasingly part
dispositions. of university life21 but surprisingly absent
from schools.

Table 5. Some guiding questions for RA when evidencing knowledge and skills
Knowledge 1. What is the core knowledge and skills that students should learn?
and skills 2. How much of a student’s knowledge and skills needs to be evidenced?
3. What consequences will continue to follow from not assessing some aspects of
knowledge and skills?
4. How best can we evidence understanding?
5. How best can we evidence the application of knowledge and skills in familiar situations?
6. How best can we evidence the application of knowledge and skills to new situations?
7. How best can we evidence interdisciplinary knowledge?
8. How best can we evidence practical knowledge and skills?
9. How best can we capture progress in the acquisition and application of knowledge?
10. Who can we learn from to help better evidence the development and application
of the knowledge students need today?
Table 6. Some guiding questions for RA when evidencing dispositions and skills
Dispositions 1. In addition to knowledge, which dispositions should be cultivated in students?

and skills 2. Which dispositions are both learnable and useful to students?
3. What are the benefits of assessing dispositions?
4. How technically reliably can different dispositions be evidenced?
5. How practical are assessments of dispositions to deliver?
6. What will the unintended consequences be of assessing certain dispositions?
7. What consequences will continue to follow from not assessing dispositions?
8. Who can we learn from to help better evidence the cultivation and demonstration
of the dispositions students need today?
In terms of potential assessment methods, With dispositions and skills the questions
Howard Gardner’s words are still powerful. necessarily have a different focus (see
Table 6).
Why talk about performances of
understanding? So long as we examine In terms of potential candidates for
individuals only on problems to which assessment, PISA offers us collaborative
they have already been exposed, we problem solving and creative thinking
simply cannot ascertain whether they as two concepts that it has determined
have truly understood. They might to be sufficiently robust and capable
have understood, but it is just as of being assessed reliably and validly,
likely that they are simply relying on albeit on a relatively short computer test
a good memory. The only reliable way complemented by self-reported data.
to determine whether understanding Collaborative problem solving is only
has truly been achieved is to pose one aspect of collaboration. We do not
a new question or puzzle – one on just learn together to solve problems; we
which individuals could not have been collaborate to generate new ideas, to make
coached – and to see how they fare. art, or to improve the contributions of
(Gardner, 2006, p 34) others through well-chosen feedback and
so forth.

The disposition of creative thinking as In terms of evidencing dispositions,
defined by PISA (OECD, 2019) essentially Rosemary Hipkins reminds us that,
includes two clusters of skills, which ‘Only when students are offered rich
typically are referred to as ‘creativity’ opportunities to demonstrate their
and ‘critical thinking’, both divergent and capabilities will we know what they are
convergent thinking. We might wish to see actually capable of’ (Hipkins, 2018, p 22).
both of these better valued and evidenced.
Sandra Milligan and colleagues (2020a, p 18)
The Australian Council for Educational have elegantly captured the complexity of
Research (Scoular et al, 2020) has made these kinds of assessment processes as a
encouraging progress, in defining and poem (see Box 1).
then developing methods for evidencing
creativity (creative thinking and critical
thinking) and collaboration, when New kinds of nets
developing capabilities within the To return to the fishing metaphor with
Australian Curriculum. which I began this paper, we need better
Many curriculum frameworks also nets to identify the full range of young
increasingly stress Communication as a people’s talents.
candidate for assessment. If this is seen The following is a selection of promising
as interdisciplinary, ubiquitous almost, examples, loosely organised into categories.
then there is an argument for including it
under the broad heading of dispositions Psychometric tests
(although it could equally be seen as a
In the main these are self-reported online
foundational literacy). Educators in the UK
tests or apps often used to evidence
have been focusing on the development
an aspect of character, wellbeing or
and assessment of oracy in its widest sense
metacognition.
and have developed a set of benchmarks
to facilitate more precise understanding of Angela Duckworth’s grit scale22
student progression, (Voice 21, 2019).
The Values in Action (VIA) Survey of
We might also want to look at grit, given Character Strength23
the work that has gone into developing
The Harvard Human Flourishing app24
ways of assessing it.
The DESSA social and emotional
learning assessment25
Box 1. Carol Dweck’s Growth Mindset

assessment26
Poem: Assessment of complex capabilities
The Metacognition Awareness
Assessment is a process
Inventory27
of systematically observing what people say, do, make or write
during a relevant performance These tests as yet have varying degrees
which requires proficiency in the competence of interest of reliability and validity.
and using these observations as evidence
to support an overall judgment
about the position of the person
on a standard scale of expertise from less expert to more
indicating what they know and can do
and what they need to learn next
with a sufficient degree of precision
to allow recognition and reporting of the level of attainment.

Performance-based assessment
Performance-based assessment is a broad
field encompassing traditional approaches
from the Viva to AI simulations. Other
examples include tests of proficiency in
the arts and sports. Increasingly it is being
used to enable students to perform tasks or
activities that are meaningful and engaging.
Performance also includes exhibitions,
presentations and debates, sometimes
associated with pedagogies such as project-
based and problem-based learning.
Alelo’s Oral Language Simulation33 is
a recent example of a computer-based
simulation designed to measure not only
proficiency in a foreign language. The
program allows a student to interact
directly with an avatar in a variety of
languages.
KIPP schools28 (Knowledge is Power Both the PISA test of collaborative problem
Program) in the USA, for example, have solving and of creative thinking are
adopted approaches to developing and performance-based assessments.
assessing character, using resources such as
The Victorian Curriculum and Assessment
the playbooks provided by CharacterLab.29
Authority’s online tests of Critical and
Creative Thinking are examples of
Smart multiple choice
scenario-based tests of performance.34
While some multiple choice tests can be
reductionist and focus on recall or simple A recent European Union overview of
computations, others can be a way of assessment practices (Siarova, Sternadel
evidencing aspects of dispositions such as and Mašidlauskaitė, 2017) suggests that
critical thinking: Performance-based assessment has the
Raven’s Progressive Matrices Test30 potential to measure and foster wide-
ranging competences and higher-order
California Critical Thinking Skills Test31
skills, since it encompasses different
The Partnership for Assessment of assessment techniques and integrates a
Readiness for College and Careers (PARCC) feedback mechanism. The key strengths
in the USA has been moving towards of performance-based assessment
greater freedom for students to construct include its focus on the learners’
their own answers rather than rely on personalised needs, clear definitions of
predetermined responses. the learning goals, and timely feedback.
The Mission Skills Assessment32 is (p 8)
an interesting example of a multiple
choice test looking to evidence a wider
set of constructs – creativity, curiosity,
ethics, resilience, teamwork and time
management.

Extended investigations but, like many such investigations,
Extended investigations are increasingly are assessable only in a written essay
seen by researchers (Soland et al, 2013; format.
Conley and Darling-Hammond, 2013; Alongside the four methods illustrated
Hipkins and Cameron, 2018) as the kinds above, there are two other dimensions
of assessments needed to best evidence worth noting, to do with the availability
higher-order thinking skills present in and timeliness of assessments, micro-
many dispositions. credentialling and on-demand tests.
Many extended investigations are or
contain elements of performance-based Micro-credentialling
assessment. Micro-credentials, sometimes referred to
as digital badges, take an idea long used
In addition to a growing number of schools
by scouts, guides and other informal youth
using such approaches across the world,
organisations and bring it into the digital
there are externally validated examples to
age. Aspects of a larger concept, such as a
consider.
disposition, are reduced to a small number
The Extended Project Qualification35 of skills and ‘badged’ up to enable students
(EPQ), in England and Wales, enables to acquire credential in bite sizes.
students to undertake an investigation
Badges have the advantage, too, of
in the context of a project topic
providing a visual image of a student’s
they have selected. Outcomes can
progress at a glance. The process of using
be a design, performance, report,
digital badges in Franklin School in the
dissertation or artefact. An EPQ ‘counts’
USA is well described by David Niguidula
for half the points value of an A Level
(2020).39 Badges are, in the jargon of
for university entrance.
assessment, ‘stackable’. That is to say they
The Extended Essay in the International can be combined together to evidence
Baccalaureate (IB) is an independent, many different assessment outcomes.
self-directed piece of research,
Digital Promise,40 in the USA, has been
finishing with a 4,000-word paper.
focusing on developing a system of micro-
The Extended Essay is presented as
credentials for use in the professional
practical preparation for undergraduate
learning of teachers. Digital Promise
research – an opportunity for
succinctly articulates the benefits of micro-
students to investigate a topic of
credentialling, including being
personal interest to them taking an
interdisciplinary approach. Although bite-sized, focused on a specific,
encouraging extended investigation, observable competency;
these essays are writing about things subject-adaptable, adaptable to multiple
rather than demonstrating capabilities subject areas;
or dispositions.
research-based, grounded in
The South Australian Certificate of educational research;
Education extended research projects36
can be used to explore aspects of the personal and timely – supporting
Australian general capabilities. This professional growth;
assessment is currently in an interesting portable, can be shared as digital badges
phase of further development.37 in online platforms;
The New South Wales Personal Interest transparent, supported by publicly
Projects,38 as the name suggests, offer available, accessible content, including
the potential for extended investigation criteria for assessment;

performance-based, demonstrated Sugata Mitra suggests that, in future,
through plans, work samples, assessment should precede teaching and
reflections, observations, videos and learning should actively encourage critical
peer and self evaluations. thinking and consensus-building, using
the Internet. He imagines a post-Pandemic
In Europe there is an attempt to build
classroom as follows.
micro-credentials into a reliable system
of credit transfer between schools and Sessions usually start with a set of
universities (Futures, Andersen and questions. In the pre-pandemic times,
Larsen, 2020). this would have been called a test.
In England there are already mechanisms Tests were usually given after the
for awarding credit for discrete ‘teaching and learning’ were over. Not
achievements,41 often in alternative so anymore. Sessions can start with
settings, allowing learners the opportunity tests. The children have no idea what
to have their achievements formally the answers might be, they haven’t been
recognised with a certificate each time ‘taught’. But they can look up things
a short unit of learning is successfully on the Internet and talk to each other.
completed. When the answers come in, the teacher
begins a discussion. She encourages the
On-demand and online children to talk about their answers,
sometimes, very occasionally, she adds
In our learning lives outside school the
a bit. They arrive at a consensus by
idea of only being able to take a driving
the end of the session about what the
test, for example, on a set date would be
answers are and why.
laughed at. In most countries such tests
require some theory (typically an online (Mitra, 2020, p 287)
multiple choice test) and a practical
Mitra’s open style of teaching has an
demonstration on the road. The on-the-
equivalent in assessment, open-book tests.
road test is, unsurprisingly, attempted
Here students are able to have key texts
when we and our driving instructor think
with them as they answer a question.
that we are capable. In similar vein, only
Open-book tests do not just test a student’s
being able to use our memory rather than
ability to recall information but require a
our ability to search and apply knowledge
more critical and analytical and applied
from the Internet or from notes or
approach to answering questions.
materials we have made while studying is
increasingly perverse. Partly as the result of Covid-19, schools
and universities have gone one stage
On-demand testing is already widely used
further and moved to set online open-book
in music, for example. While still at school
tests. Typically, students are required to
many students take grade exams to assess
complete an assessment within a defined
the quality of their playing classical, jazz
time period, for example 24 hours, with
or rock music when they have reached a
questions released to students online at a
standard that they and their teachers deem
specified time and answers required online
them to be ready for.
by the end of the timed period. While there
On-demand testing more generally in are some important equity issues that need
education makes practical sense and a to be considered here (broadband strength,
number of educational jurisdictions are computer availability and the availability
experimenting with it, for example the of quiet spaces in students’ homes), the
Victorian Curriculum and Assessment essence of such approaches could be
Authority (VCAA) in Australia.42 applied and used within the school day.

Far from dumbing down the quality of Comparative judgement
answers, such examinations potentially Comparative judgement uses the
offer opportunities for deeper learning. principle that people are better at making
require different kinds of questions to comparisons between pieces of work than
assess interpretation, evaluation and at making absolute judgements about
critical thinking rather than knowledge their quality. First identified as a useful
recall. Frequently they start from scenario- principle of assessment nearly a century
based or problem-based questions that ago (Thurstone, 1927), the addition of
require students to apply knowledge digital technology enables a kind of
rather than summarise information. As crowd sourcing of teachers’ judgements
a consequence they cannot be gamed by of particular usefulness when reviewing
cutting and pasting from the Internet. written work. Several approaches are being
In a thoughtful recent blog, The searching developed by No More Marking in the UK.45
questions that will allow us to rethink
assessment,43 Guy Claxton suggests that we Profiling
need not spend time worrying about end- Profiling students’ wider achievements
of-school exams but instead could leave at school, using some kind of record
assessment – he calls this process MOECs of achievements, is not new.46 In the
(Methods of Evidencing Capability) – to 1990s, for example, the UK government
a college or university or to an employer encouraged all secondary students to
rather than undertaking it at the exit point develop a National Record of Achievement,
from school. a portfolio of documents showcasing a
student's academic and non-academic
Games-based assessment achievements, including GCSE certificates,
Games-based assessment is in its infancy in certificates from extracurricular activities,
schools. Nevertheless there are examples school reports and anything else
from which we can learn. considered relevant.
Keenville44 is a formative game-based These Records of Achievement did not

assessment initiative for 1st and work because the non-standardised
2nd graders in the state of Georgia, elements were often too long-winded to
a collaboration between the Georgia be assimilated by others; it was not clear
education department, the Georgia Center who was warranting the data; and there
for Assessment at the University of Georgia was not buy-in from employers. They were
and FableVision Studios. ideas before their time and before digital
technology was freely available. Fast forward
Posterlet is a game created by Stanford’s
to today and a number of individual schools,
Graduate School of Education to measure
groups of schools and external providers are
students’ choices in seeking feedback
experimenting with profiles.
and revising work while at the same
time learning graphic design principles In the USA, Panorama Education has
(Cutumisu et al, 2015). developed online dashboards or profiles
to illustrate the range of a young person’s
Critical and Creative Tests developed by
social and emotional and learning skills.47
VCAA in partnership with (initially) ACER
and now NFER in England, use a range of For many years students taking the
engaging scenarios to test students’ critical International Baccalaureate (IB) have
and creative thinking skills. graduated with an IB Profile as well as
their diploma. The IB Learner Profile
contains ten attributes (see Figure 4).

Figure 4. The ten attributes of the IB Learner Profile
Source: The Tacit Dimension: Understanding International Mindedness in

Hong Kong International Baccalaureate Diploma Programme Schools
Figure 5. The Xperiential Learner Profile

In fact the IB Learner Profile is not a The dashboard combines a portfolio –
personalised description of the degree to a space for students to upload evidence
which a learner possesses these attributes of their progress using the language of
or dispositions. It is rather a statement of Australia’s general capabilities – with
intent, of the kind of all-round individuals a goal-setting function, as well as an
the IB values and which, it hopes, opportunity for individuals to compare
universities across the world will also like. their progress to others’ in the school.
Numbers of schools are trying to develop Upskill Me49 is an example of a growing
profiles of student achievement that number of platforms offering school
are closer to the spirit of the Records of students a way of tracking and curating
Achievement three decades ago. Heritage their activities, receiving digital badges
Xperiential Learning School in Doncaster, along the way by use of an app. With
England is an example of this. Modelled Upskill Me, young people can also connect
on High Tech High School in California, with employers and go to events to find job
The Xperiential Learner Profile48 mixes and work experience opportunities. Upskill
dispositions with values and health, (see Me serves as a record of achievement.
Figure 5). Like the IB Profile, Xperiential’s
is a statement of educational intent. It is Standing back from the different nets
currently exploring ways of providing data These examples are illustrative only of
to evidence each of its components. an exciting, emerging field. Back to the
For a number of years Rooty Hill High fishing metaphor; it is tempting to get too
School in Sydney, a school with which I close to the nets and be dazzled by all the
have been privileged to collaborate, has activity. It would be easy to be caught up
been developing a dashboard, of which this in the novelty of some of the methods and
is an early version, for each of its students forget to be clear about the purpose of any
(see Figure 6). assessment.
Figure 6. Rooty Hill High School’s Learner Hub

Table 7. Mapping possible assessment methods to a possible disposition, Creativity
Focus Method Reliability/Validity Pros Cons

Creativity 1. Victorian High, independently Engaging for Expensive to
Curriculum and validated students. develop and
Assessment Authority Useful feedback to validate.
(VCAA) online teachers. Need to quarantine
game-based Assessment as some so that they
learning. can be used for
Can be used benchmarking.
summatively to show
performance data
over time for State.
2. Teacher Medium, depending More precise
observation of on accuracy of scope understanding by
products and and sequence and students of concepts.
processes learning progression Enhanced teacher
documents, moderation capability
processes,
professional support
How best might we curate this array of and cons. Table 7 takes Creativity as an
approaches? Thus far I have loosely clustered example of this process.
them according to the type of test.
Table 7 includes just two examples, as
In Figure 3 we used the continuum of being illustrative of the approach; there are
shallow to deep learning to map how more many others.
complex tests are needed to test deeper
In its recent project exploring the
learning (Darling-Hammond, 2017).
assessment of Australia’s general
Lorna Earl and Steven Katz (2006, p 17) capabilities, the Australian Council
suggest that we can view assessment for Educational Research (ACER) has
processes according to their purposes, of developed a well-evidenced approach to
their assessment using authentic, problem-
1. gathering information, eg, observations, based tasks. ACER’s assessment template
tests, computer-based assessments,
measures multiple skills;
projects, rich tasks;
is problem-based and authentic;
2. interpreting information, eg, is domain-orientated;
developmental continua, rubrics, maps to skill development levels.
learning progressions, self-assessment,
peer-assessment; (Scoular et al, 2020).
3. record-keeping, eg, observations over Whatever the specific ways of evidencing

time, photographs or work, portfolios, dispositions or capabilities, it is important
digital badges; and to remember that research demonstrates
4. communicating, eg, demonstrations, ‘that there is no single method that
presentations, exhibitions, records of would fully measure key competences
achievement, profiles. and transversal skills, nor serve as a best
practice for student assessment’ (Siarova,
Alternatively, it might be easier to start Sternadel and Mašidlauskaitė, 2017, p 8).
from a list of potential dispositions (or As High Resolves (2020) has argued, we
aspects of knowledge, but these have been need to focus on better understanding
better analysed by others already) and the best combinations of multimodal
cross map possible ways of evidencing assessments to select depending on context
these, along with their respective pros and desired outcomes.

Visible progress
The most important assessments that take place in a school building are seen by no-one.
They take place inside the heads of students, all day long. Students assess what they do,
say and produce, and decide what is good enough. These internal assessments govern how
much they care, how hard they work, and how much they learn.
(Berger, Rugen and Woodfin, 2014, p 6)
A little over a decade ago, John Hattie Building on earlier evidence

published a ground-breaking book, Visible We are not starting from scratch. In
Learning: A Synthesis of Over 800 Meta- Australia a Review undertaken by Geoff
Analyses Relating to Achievement, (2008). Masters in 2013 argued for a fundamental
It was remarkable in two ways. It dared to rethink about the purpose of assessments,
use the word ‘meta-analyses’, in a popular that they should be seen as
book for educators. More importantly, it
lifted the lid on educational research for having a single general purpose: to
teachers across the world. In painstaking establish where learners are in their
detail Hattie made the processes of long-term progress, within a domain of
learning visible, clear and actionable. For learning at the time of assessment.
most people in education it was a game- The purpose is not so much to judge as
changer. A similar paradigm shift is now to understand. This unifying principle,
needed in assessment. which has potential benefits for learners,
teachers and other educational decision
We need to explore, in similarly nuanced
makers, can be applied to assessments
and evidenced ways, the different
at all levels of decision making, from
ways in which we can use assessment
classrooms to cabinet rooms.
to improve learning, and make these
discussions visible to all, depending on (Masters, 2013, p 58)
the purpose we have in mind. The Gordon Commission, on assessment
We need to make clearer the kinds of in the USA, spoke similarly.
dispositions and capabilities that we
In our vision of the future of
want all young people to acquire, and
assessment, the improvement of
make visible the processes by which we
learning is its central purpose. It
evidence student progress in acquiring
functions in dynamic interaction with
them.
curriculum and instruction, which
There are already educators beginning themselves have the improvement
to do just this at a grass roots level, of learning as its central purpose.
such as Ron Berger and colleagues, in Decisions about the form and content
Expeditionary Learning schools in the of assessment are informed by a
USA, whose words begin this section. socio-cultural perspective of learning,

curriculum and instruction and its The ‘triad of processes’ that underpin
results are used by both the teacher and effective learning referred to are curriculum,
the learner to guide future teaching and pedagogy and assessment. In too many
learning. systems they have become disconnected.
(Armour-Thomas and Gordon, 2013, p 19)
A paradigm shift
From an English perspective, Peter Hill
The views cited above are just three of
and Michael Barber put it as follows.
many but, as we saw in earlier sections,
Perhaps the most urgent need right now they represent a growing consensus about
in the field of assessment is an overall the direction in which we should be going.
conceptual framework and longer-term In this paper we have also both looked at
vision for its place and purpose within what is wrong with educational assessment
the triad of processes that lie at the today and explored what could offer hope
heart of schooling. for developing better fishing nets in the sea
of education.
(Hill and Barber, 2014, p 9)
Figure 7. Prototype ARC Chrysanthemum Learner Profile

The future is visual and digital
In my horizon-scanning of promising
practices, it is increasingly clear that when
it comes to evidencing the knowledge,
skills and dispositions young people have
acquired in their time at school, rather than
a set of scores or grades, there is a move
towards using Learner Profiles, Transcripts,
Records of Achievement and Portfolios,
alongside whatever standardised data is
felt to be important. Figure 7 (Milligan et al,
2020a, p 24) is illustrative.
The Mastery Transcript Consortium (MTC)50
in the USA, Figure 8, is another example of
this trend.
The Figure 8 example is an early version
and, in discussions with colleagues at
MTC, I was interested to hear that, now
that the transcript is gaining credibility
with university admission staff, the
number at the centre will be removed from
future transcripts in favour of the more
balanced transcript.
The field of profiling is evolving rapidly
and there is as yet no single approach.
The degree to which formal qualification
scores or grades are prominent varies.
Sometimes courses completed are included
as an indication of a student’s dispositions
or interests; in other examples there is an
attempt to represent capabilities in some
kind of spider or radar chart.

Figure 8. Mastery Transcript Consortium
Figure 9. Big Picture Education Australia Profile

Figure 10. New directions in assessment
1. Shallow, narrow, solo 1. Nature of learning Deep, wide, collaborative
2. Dominated by head-work 2. Range of strengths Head, heart and hand
3. Number or grade 3. Style of credential Evidenced narrative
4. Single body 4. Source of credential Broad consortium
5. Predominantly summative 5. Focus of assessment Predominantly formative
6. High-stakes, standardised 6. Integration Ongoing, authentic
7. National/State norms 7. Personalisation Individual progression
8. Employers/HE/FE 8. Ownership For learners and for others
9. Largely for accountability 9. Strategic intent Mainly for improvement
10. Formulaic, mechanistic 10. Approach Carefully evidencing capability
How student agency is exercised is also up the dispositions or capabilities of

variable. Minimally there is a process creativity, critical thinking collaboration
of deciding what evidence should be and aspects of communication. At the same
foregrounded for a particular audience; time we may wish to evidence less of what
in other cases a portfolio of best and most is currently assessed.
beautiful work is being curated over time
We need to have much more nuanced,
by the learner.
strengths-based, multimodal descriptions
A particular challenge for developers and of young people.
consortia is ensuring that the approach is
We need to use some of the many new
seen to be reliable enough. So, for example,
methods being pioneered across the world,
the Big Picture example shown in Figure 9
always seeking to make the processes
is warranted by the University of Melbourne.
of evidencing progress, in all aspects of
learning, visible and evidence-based.
Time for a paradigm shift in
educational assessment As we consider the inclusion of any new
area, we will need to use evidence from the
We have a unique opportunity. The stake-
learning sciences to consider
holders are in agreement. The evidence
is increasingly clear. Figure 10 brings its learnability;
together the direction of shift that the
its usefulness in life;
research has identified, and which
innovative practitioners are prototyping. the validity, reliability and practicality
with which it can be assessed; and
New nets for a sea of opportunity
its likely positive impact on the develop-
To return to the metaphor of fishing with ment of more expansive curricula.
which I began, it is time to value more kinds
of fish, to know more about the fish we catch, In addition, we need to get on with it now
and to use some new nets as we do so. to reunite assessment with curriculum
and pedagogy, from which it has become
The evidence points to finding appropriate harmfully separated.
ways of valuing the skills that go to make

Endnotes
1. pz.harvard.edu/sites/default/files/AssessmentReimagined_Booklet_0.pdf
2. curriculumredesign.org/framework/
3. npdl.global/
4. issues.org/the-world-needs-students-with-interdisciplinary-education/
5. hbr.org/2015/07/ace-the-assessment
6. mastery.org/
7. performanceassessment.org/
8. imsglobal.org/activity/comprehensive-learner-record
9. education.unimelb.edu.au/new-metrics-for-success
10. hepi.ac.uk/2019/08/14/why-employers-dont-care-about-qualifications/
11. criteriacorp.com/resources/definitive-guide-validity-of-preemployment-tests/what-are-pre-employment-tests
12. universityworldnews.com/post.php?story=2021011511142211
13. ofqual.blog.gov.uk/2019/03/08/what-is-it-like-to-experience-exam-stress-a-student-perspective/
14. oecd-ilibrary.org/docserver/9789264281820-8-en.pdf?expires=1615190774&id=id&accname=guest&checksum=
022E0FA0656AA5AC791C134D42F5FB1B
15. oecd.org/pisa/publications/PISA-2021-creative-thinking-framework.pdf
16. danielwillingham.com/daniel-willingham-science-and-education-blog/measurement-of-non-cognitive-factors
17. Applied Measurement in Education, Volume 29, Issue 4 (2016) 21st Century Skill Assessment
18. education.unimelb.edu.au/new-metrics-for-success
19. brookings.edu/blog/education-plus-development/2017/08/31/new-data-on-the-breadth-of-skills-movement-
over-150-countries-included/
20. rethinkingassessment.com/
21. For example nmite.ac.uk/ and https://www.londoninterdisciplinaryschool.org/
22. angeladuckworth.com/grit-scale/
23. viacharacter.org/
24. hfh.fas.harvard.edu/flourishing.app
25. apertureed.com/products-solutions/dessa-system-2/dessa-overview/
26. blog.mindsetworks.com/what-s-my-mindset
27. services.viu.ca/sites/default/files/metacognitive-awareness-inventory.pdf
28. kipp.org/approach/character/
29. characterlab.org/playbooks/
30. pearsonclinical.co.uk/Psychology/AdultCognitionNeuropsychologyandLanguage/AdultGeneralAbilities/
Ravens-Progressive-Matrices/Ravens-Progressive-Matrices.aspx
31. insightassessment.com/article/california-critical-thinking-skills-test-family
32. bbk12e1-cdn.myschoolcdn.com/ftpimages/721/misc/misc_178206.pdf
33. alelo.com/
34. vcaa.vic.edu.au/assessment/f-10assessment/edstateap/Pages/cct-assessments.aspx
35. qips.ucas.com/qip/extended-project-qualification-epq#:~:text=The per cent20EPQ per cent20is per cent20a per
cent20single, per cent2C per cent20report per cent2C per cent20dissertation per cent20or per cent20artefact.
36. sace.sa.edu.au/web/research-project
37. education.sa.gov.au/sites/default/files/sace-stage-2-review.pdf
38. educationstandards.nsw.edu.au/wps/portal/nesa/11-12/stage-6-learning-areas/hsie/society-culture/personal-
interest-project
39. ascd.org/publications/educational-leadership/sept20/vol78/num01/The-Power-of-Digital-Badges.aspx
40. digitalpromise.org/initiative/educator-micro-credentials/
41. aqa.org.uk/programmes/unit-award-scheme
42. vcaa.vic.edu.au/assessment/f-10assessment/ondemandtesting/Pages/benefits.aspx
43. rethinkingassessment.com/rethinking-blogs/the-searching-questions-that-will-allow-us-to-rethink-assessment/
44. fablevisionstudios.com/keenville
45. nomoremarking.com/
46. As an early career teacher in England in the 1970s, I was seconded by the Oxford Certificate of Educational
Achievement (OCEA), a partnership between the University of Oxford Examination Board and Oxfordshire
Education Authority, to help develop a Record of Achievement that combined academic tests with evidence
of extra-curricular and community learning and also included a large element of student self-assessment.
47. panoramaed.com/
48. heritagexperiential.org/about-us/learner-profile/
49. upskillme.io/
50. pilot-transcript.mastery.org/

References
Armour-Thomas, E and Gordon, E (2013) Claxton, G and Lucas, B (2015) Educating

The Gordon Commission on the Future Ruby: What Our Children Really Need
of Assessment in Education. Toward to Learn, Crown House Publishing,
an Understanding of Assessment as Carmarthen.
a Dynamic Component of Pedagogy, Coe, R, Rauch, C, Kime, S and Singleton,
Educational Testing Service, Princeton, D (2020) Great Teaching Toolkit:
NJ. Evidence Review, Evidence-based
ASCL (2019) The Forgotten Third: Final Education, Sunderland.
Report of the Commission of Inquiry, Committee on Developments in the
Association of School and College Science of Learning (2000) How People
Leaders, Leicester. Learn: Brain, Mind, Experience and
Au, W (2016) ‘Meritocracy 2.0: High-stakes, School, The National Academies,
standardized testing as a racial project Washington, DC.
of neoliberal multiculturalism’, Conley, D and Darling-Hammond, L
Educational Policy, 30, 1, p 39–62. (2013) Creating Systems of Assessment
Bennett, E (2013) Preparing for the Future: for Deeper Learning, Stanford Center
What Educational Assessment Must for Opportunity Policy in Education,
Do, Gordon Commission/Educational Stanford, CA.
Testing Service, Princeton, NJ. Cutumisu, M, Blair, K, Chin, D and
Berger, R, Rugen, L and Woodfin, L Schwartz, D (2015) ‘Posterlet: A game-
(2014) Leaders of Their Own Learning: based assessment of children’s choices
Transforming Schools Through to seek feedback and to revise, Journal
Student-Engaged Assessment, Jossey- of Learning Analytics, 2, 1, p 49–71.
Bass, San Francisco, CA. Darling-Hammond, L (2017) Developing
Birenbaum, M, DeLuca, C, Earl, L, and Measuring Higher-Order Skills:
Heritage, M, Klenowski, V, Looney, A, Models for State Performance
Smith, K, Timperley, H, Volante, L and Assessment Systems, Council of Chief
Wyatt-Smith, C (2015) ‘International State School Officers, Washington, DC.
trends in the implementation of Darling-Hammond, L, Flook, L, Cook-
assessment for learning: Implications Harvey, C, Barron, B and Osher, D
for policy and practice’, Policy Futures (2020) ‘Implications for educational
in Education, 13, p 117–140. practice of the science of learning and
Black, P and Wiliam, D (1998) development’, Applied Developmental
‘Assessment and classroom learning’, Science, 24, 2, p 97–140.
Assessment in Education: Principles, Duckworth, A and Quinn, P (2009)
Policy and Practice, 5, 1, p 7–73. ‘Development and validation of the
Care, E, Anderson, K and Kim, H (2016) Short Grit Scale (GRIT–S)’, Journal of
Visualizing the Breadth of Skills Personality Assessment, 91, p 166–174.
Movement Across Education Systems, Earl, L and Katz, S (2006) Rethinking
Brookings Institution, Washington, DC. Classroom Assessment with Purpose
Care, E, Kim, H, Vista, A and Anderson, in Mind: Assessment for Learning,
K (2018) Education System Alignment Assessment as Learning, Assessment
for 21st Century Skills: Focus on of Learning, Manitoba Education,
Assessment, Brookings Institution, Citizenship and Youth, Manitoba.
Washington, DC. Education Council (2020) Looking to
Care, E, Anderson, K and Kim, H the Future: The Report of the Review
(2019) Assessment of Transversal of Senior Secondary Pathways into
Competencies: Current Tools in the Work, Further Education and Training,
Asian Region, UNESCO, Paris. Department of Education, Skills and
CBI (2019) Getting Young People ‘Work Employment, Canberra.
Ready’: Our Vision of How Education Education Policy Institute (2019) ‘The
Should Prepare Young People for Forgotten Third’: A Rapid Review of
the Modern World, Confederation of the Evidence, Association of School
Business Industry, London. and College Leaders, Leicester.

References
Elmore, R (2019) ‘The future of learning High Resolves (2020) The Future of
and the future of assessment’, ECNU Assessment: White Paper 2, High
Review of Education, 2, 3, p 328–341. Resolves, Sydney.
European Parliament (2007) Key Hill, P and Barber, M (2014) Preparing for
Competences for Lifelong Learning: a Renaissance in Assessment, Pearson,
European Reference Framework, London.
Office for Official Publications of the Hipkins, R and Cameron, M (2018) Trends
European Communities, Luxembourg. in Assessment: An Overview of Themes
Fullan, M (2021) The Right Drivers for in the Literature, New Zealand Council
Whole System Success, CSE Leading for Educational Research, Wellington.
Education Series #1, February, Centre Howard, E (2020) A Review of the Literature
for Strategic Education, Melbourne. Concerning Anxiety for Educational
Futures, H, Andersen, T and Larsen, Assessments, Ofqual, Coventry.
K (2020) A European Approach to Koretz, D (2017) The Testing Charade:
Micro-Credentials: Output of the Pretending to Make Schools Better,
Micro-Credentials Higher Education University of Chicago Press, Chicago.
Consultation Group, Publications Office
of the European Union, Luxembourg. Lai, R and Viering, M (2012) Assessing
21st Century Skills: Integrating Research
Gardner, H (2006) Five Minds for the Future, Findings, Council on Measurement in
Harvard Business School Press, Boston, Education, Vancouver, BC.
MA.
Lamb, S, Maire, Q and Doecke, E (2017)
Gordon Commission on the Future of Key Skills for the 21st Century: An
Assessment in Education (2013) To Evidence-Based Review, Centre for
Assess, to Teach, to Learn. A Vision for International Research on Education
the Future of Assessment, Educational Systems, Sydney.
Testing Service, Princeton, NJ.
Lucas, B (2016) ‘A five-dimensional
Griffin, P, McGaw, B and Care, E (Eds) model of creativity and its assessment
(2012) Assessment and Teaching of in schools’, Applied Measurement in
21st Century Skills, Springer, New York. Education, 29, 4, p 278–290.
Gutman, L and Schoon, I (2013) The Lucas, B (2019) Why We Need to Stop
Impact of Non-Cognitive Skills on Talking About Twenty-first Century
Outcomes for Young People: Literature Skills, CSE Seminar Series Paper 283,
Review, Institute of Education, April, Centre for Strategic Education,
University of London, London. Melbourne.
Harlen, W (2005) ‘Trusting teachers’ Lucas, B and Spencer, E (2017) Teaching
judgement: Research evidence of the Creative Thinking: Developing Learners
reliability and validity of teachers’ who can Generate Ideas and Think
assessment used for summative Critically, Crown House Publishing
purposes’, Research Papers in Ltd, Carmarthen.
Education, 20, 3, p 245–270.
Lucas, B, Claxton, G and Spencer, E
Harlen, W and Deakin Crick, R (2002) A (2013) Expansive Education: Teaching
Systematic Review of the Impact of Learners for the Real World, Australian
Summative Assessment and Tests on Council for Educational Research,
Students’ Motivation, Evidence for Melbourne.
Policy and Practice Information and
Co-ordinating Centre, London. Masters, G (2013). Reforming Educational
Assessment: Imperatives, Principles
Hattie, J (2008) Visible Learning: A and Challenges. Australian Council
Synthesis of Over 800 Meta-Analyses for Educational Research, Melbourne.
Relating to Achievement, Routledge, Also presentation for Open University,
Abingdon, Oxon. Maidenhead, UK.
Heckman, J and Kautz, T (2013) Fostering McEachen, J (2017) Assessment for
and Measuring Skills: Interventions Deep Learning: A White Paper, New
that Improve Character and Cognition, Pedagogies for Deeper Learning, Ontario.
NBER Working Paper No 19656,
National Bureau of Economic Research.

References
Milligan, S, Luo, R, Hassion, E and Siarova, H, Sternadel, D and

Johnson, J (2020a) Future-Proofing Mašidlauskaitė, R (2017) Assessment
Students: What They Need to Know and Practices for 21st Century Learning:
How to Assess and Credential Them, Review of Evidence, NESET II Report,
University of Melbourne, Melbourne. Publications Office of the European
Milligan, S, Luo, R, Kamai, T, Rice, S Union, Luxembourg.
and Keang, T (2020b) Recognition of Soland, J, Hamilton, L and Stecher, B (2013)
Learning Success for All: Ensuring Measuring 21st Century Competencies:
Trust and Utility in a New Approach Guidance for Educators, Asia Society,
to Recognition of Learning in Senior New York.
Secondary Education in Australia, Taylor, R, Fadel, C, Kim, H and Care,
Learning Creates Australia, Melbourne. E (2020) Competencies for the 21st
Mitra, S (2020) ‘Children and the internet: Century: Jurisdictional Progress,
Learning in the times to come’, Journal Brookings Institution, Washington, DC.
of Learning for Development, 73, 3, Thurstone, L (1927) ‘A law of comparative
p 286–305. judgment’, Psychology Review, 34,
National Academy of Education (2021) p 273–286.
Educational Assessments in the Tuckett, A and Field, J (2016) Factors
Covid-19 Era and Beyond, National and Motivations Affecting Attitudes
Academy of Education, Washington, DC. Towards and Propensity to Learn
National Research Council (2001) Through the Life Course, Government
Knowing What Students Know: The Office for Science, UK.
Science and Design of Educational Vincent-Lancrin, S, González-Sancho, C,
Assessment, The National Academies Bouckaert, M, de Luca, F, Fernández-
Press, Washington, DC. Barrerra, M, Jacotin, G, Urgel, J and
Niguidula, D (2020) ‘The power of digital Vidal, Q (2019) Fostering Students’
badges’, Educational Leadership, 78,1. Creativity and Critical Thinking: What
O’Connell, M, Milligan, S and Bentley, T it Means in School, OECD Publishing,
(2019) Beyond ATAR: A Proposal for Paris.
Change, Koshland Innovation Fund. Voice 21 (2019) The Oracy Benchmarks,
Accessed at apo.org.au/node/261456. Voice 21, London.
OECD (2019) PISA Creative Thinking von der Embse, N, Jester, D, Roy, D and
Framework, Third Draft, OECD Post, J (2018) ‘Test anxiety effects,
Publishing, Paris. predictors and correlates: A 30-year
OECD (2013) ‘Trends in evaluation and meta-analytic review’, Journal of
assessment’, in Synergies for Better Affective Disorders, 227, p 483–493.
Learning: An International Perspective Wiliam, D (2007) ‘Once you know what
on Evaluation and Assessment, OECD they’ve learned, what do you do next?
Publishing, Paris. Designing curriculum and assessment
Pellegrino, J, Chudowsky, N and Glaser, R for growth’, in R Lissitz (Ed), Assessing
(2001) Knowing What Students Know: and Modeling Cognitive Development
The Science and Design of Educational in Schools, JAM Press, Maple Grove,
Assessment, National Academy Press, MN.
Washington, DC. World Economic Forum (2015) New
Pellegrino, J and Hilton, M (Eds) Vision for Education: Unlocking
(2012) Education for Life and Work: the Potential of Technology, World
Developing Transferable Knowledge and Economic Forum, Geneva.
Skills in the Twenty-First Century, National
Research Council,Washington, DC.
Scoular, C, Ramalingam, D, Duckworth,
D and Heard, J (2020) Assessment of
General Capabilities: Skills for the
21st-Century Learner. Final Report,
Australian Council for Educational
Research, Melbourne.

References
Additional reading
Although not cited explicitly in the text,
the following were used in preparing this
paper and may be of interest to the reader.
Lucas, B and Smith, C (2018) The Capable
Country: Cultivating Capabilities
in Australian Education, Mitchell
Institute policy report No 03/2018,
Mitchell Institute, Melbourne.
OECD (2017) ‘PISA 2015 collaborative
problem-solving framework’, in PISA
2015 Assessment and Analytical
Framework: Science, Reading,
Mathematic, Financial Literacy and
Collaborative Problem Solving, OECD
Publishing, Paris.

B I L L L U CA S
About the Author

Bill Lucas is Professor of Learning and Director of the Centre for Real-World
Learning at the University of Winchester in the UK. Bill is a co-founder
of Rethinking Assessment, a coalition of education leaders, employers,
researchers and policy makers looking to reform assessment in England.
In 2017 Bill was appointed to chair the strategic advisory group for the PISA
2021 test of Creative Thinking, which draws on the five-dimensional model for
creativity that he developed with colleagues in Winchester. In 2019 the OECD
published the results of a four-year, eleven-country study into the teaching and
assessing of creativity, also stimulated by Bill’s research. In the same year Bill
co-authored the Durham Commission on Creativity and Education’s first report.
Bill advises the Victorian Curriculum and Assessment Authority in Australia on
the assessment of critical and creative thinking, and has worked extensively
across Australia over the past eight years with a focus on critical and creative
thinking. In 2018 his report for the Mitchell Institute, Capable Country:
Cultivating Capabilities in Australian Education, suggested a route map for
Australia to make the general capabilities a reality in all states.
Bill is a prolific writer and has authored more than eighty books and many
research reports. His latest books, written with Ellen Spencer, are Zest for
Learning: Developing Curious Learners who Relish Real-World Challenges and
Teaching Creative Thinking: Developing Learners who Can Generate Ideas
and Think Critically. His acclaimed critique in 2015 of the education system
in England, Educating Ruby: What Our Children Really Need to Learn, written
with Guy Claxton, asks challenging questions about the future direction of
schools.
About the Paper

The author argues that in our current approaches to assessment we are
‘using the wrong kinds of nets’, especially if we are wanting to ‘catch’ young
people’s strengths. He discusses issues with the content of school curricula;
models of a more global curriculum and lifelong learning; the roles of skills and
competencies in learning; and related problems with educational assessment.
He revisits the purposes of assessment, explores promising practices from
around the world and provides examples of both visible progress and
emerging new directions in assessment. He concludes that we need urgently
to develop and implement more nuanced, strengths-based, multimodal
descriptions of what young people achieve.
Mercer House 82 Jolimont Street ISSN 1838-8566 ISBN 978-1-925654-57-8

East Melbourne Victoria 3002
Phone +61 3 9654 1200
Fax +61 3 9650 5396
Email [email protected] www.cse.edu.au

Rethinking Assessment 2021

Uploaded by

Copyright:

Available Formats

Rethinking Assessment 2021

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Rethinking Assessment 2021

Uploaded by

Copyright:

Available Formats

02 CSE

ISSN 1838-8566 ISBN 978-1-925654-57-8

Rethinking assessment in education: The case for change / 2

3 / CSE Leading Education Series #02 April 2021

Rethinking assessment in education: The case for change / 4

21st century skills

Foundation literacies Competencies Character qualities

1. Literacy 7. Critical thinking/ 11. Curiosity

3. Scientific 9. Communication 13. Persistence/grit

5 / CSE Leading Education Series #02 April 2021

Table 1. Dispositions for a lifetime of learning (Lucas, 2019)

Rethinking assessment in education: The case for change / 6

Table 2. The prevalence of Center for Curriculum Redesign competencies,

Competency Inclusion Identification Progression Pedagogy Assessment

7 / CSE Leading Education Series #02 April 2021

Differing approaches to that there is a middle way; that these are

Rethinking assessment in education: The case for change / 8

9 / CSE Leading Education Series #02 April 2021

Rethinking assessment in education: The case for change / 10

11 / CSE Leading Education Series #02 April 2021

The measurement of deep learning must be always informed by a wealth of

 the uses made of the assessment

Rethinking assessment in education: The case for change / 12

Figure 3. A continuum of assessment methods, adapted from Darling-Hammond

13 / CSE Leading Education Series #02 April 2021

Rethinking assessment in education: The case for change / 14

the more any quantitative social Universities and colleges

The National Academy of Education (2021) Employers

15 / CSE Leading Education Series #02 April 2021

Rethinking assessment in education: The case for change / 16

17 / CSE Leading Education Series #02 April 2021

Student Teacher Real-world Online

Real-time feedback Criterion-referenced Expert reviews Reliable, validated

Self-report Rating of products Authentic tests Digital badges

In my own work in the UK and in In 2016 the journal Applied Measurement

we’re far from agreed-upon measures.

Rethinking assessment in education: The case for change / 18

Previous attempts to In the future envisioned by the

19 / CSE Leading Education Series #02 April 2021

Assessment of learning is summative The COVID-19 pandemic, as well as

Rethinking assessment in education: The case for change / 20

Along with an understanding of the

21 / CSE Leading Education Series #02 April 2021

Common themes from evidence on high-  A move towards scenario-based,

Rethinking assessment in education: The case for change / 22

Mā te kimi ka kite, Mā te kite ka mōhio, Mā te mōhio ka mārama.

23 / CSE Leading Education Series #02 April 2021

Dispositions 1. In addition to knowledge, which dispositions should be cultivated in students?

Rethinking assessment in education: The case for change / 24

Box 1.  Carol Dweck’s Growth Mindset

25 / CSE Leading Education Series #02 April 2021

Rethinking assessment in education: The case for change / 26

27 / CSE Leading Education Series #02 April 2021

Rethinking assessment in education: The case for change / 28

Keenville44 is a formative game-based These Records of Achievement did not

29 / CSE Leading Education Series #02 April 2021

Source: The Tacit Dimension: Understanding International Mindedness in

Figure 5. The Xperiential Learner Profile

Rethinking assessment in education: The case for change / 30

the uses made of the assessment

Common themes from evidence on high- A move towards scenario-based,

Box 1. Carol Dweck’s Growth Mindset