AQA Science Making Questions Clear

Download as pdf or txt
Download as pdf or txt
You are on page 1of 24

Making

questions clear
GCSE science exams from
summer 2018
Version 2.0 September 2016

How we write accessible questions that give


students of all abilities the best opportunity
to get the results they deserve.

• GCSE Combined Science: Trilogy


(double award)
• GCSE Combined Science: Synergy
(double award)
• GCSE Biology
• GCSE Chemistry
• GCSE Physics

See all our specimen question papers and mark


schemes at aqa.org.uk/gcse-science
Making questions clear – GCSE science exams from summer 2018

Contents
Journey of a question 4

Our accessibility principles 6


Question type 6
Reading age 6
Command words 7
Standard wording/instructions 7
Direct wording 8
Repeated information 9
Layout/spacing 9
Answer line prompts 10
Sentence length 10
Bullet lists 10
Using bold 11
Sub-clauses 11
Contractions 11
Idiom, context and jargon 12
Ellipsis 12
Diagrams 13
Graphs 13
Tables 13
Ramping 14

Some examples 15
Use of command words 15
White space 16
Simplifying graphs 18
Clarity of information 20
Relevant information only 22

3
Journey of a question
Creating a question paper takes a lot of time!

A question paper is put together by an exam team. The members of the team are often teachers, who
have developed assessment expertise as a result of working as examiners. The team is made up of
senior examiners: it includes the Chair of Examiners for the subject; a Chief Examiner for the
specification; and writers for each question paper.

We also have assessment design and product teams who are responsible for managing the process and
ensuring that the question paper complies with rules and regulations. The team draws on the research
and assessment expertise of our Centre for Education Research and Practice: cerp.aqa.org.uk

We analyse the statistics of question papers sat by students. We are continuously improving: by
understanding how individual questions have performed we can learn lessons which help inform any
future decisions on question papers.

We develop a ‘blueprint’ for each question paper. This is essentially the design of the question paper,
for example what areas of the specification and balance of skills are to be assessed. This is one of the
most important parts of the whole process.

The exam team collaborates closely to create and then review the question paper. They scrutinise all
aspects of every question.

This includes:
• relevance to specification
• breadth of specification covered
• depth of knowledge assessed
• scientific accuracy
• answers in the mark scheme accurately reflecting the question asked
• number of marks being appropriate
• level of demand
• allocation of assessment objectives
• mathematics requirements.

One area of reviewing each question encompasses ‘accessibility’. We look at many elements that feed
into the overall accessibility of a question. Individually these elements may sometimes not seem
important but, together, the impact on the question is greater than the sum of its parts.

4
Making questions clear – GCSE science exams from summer 2018

Accessibility is not about:


• ‘dumbing down’ our language
• tailoring our exams to less able students.

Accessibility is about getting the message across clearly without affecting the demand of the question –
the content familiarity requirement is still there. Clear language and layout of question papers is
important for all students.

To ensure that our question papers are clear to all students:


• a specialist team creates modified question papers for visually impaired students
• our question papers are also reviewed by representatives of The British Association of Teachers of
the Deaf. They tell us if certain words or images could make the question more difficult for some
students to access.

Our aim is to assess the science,


rather than making it a test of comprehension.

Our accessibility ethos gives all students the best opportunity to get the results they deserve.

This booklet gives ‘before’ and ‘after’ examples of questions to show how we refine accessibility.

5
Our accessibility principles
To ensure we make our questions as clear as possible we review across a number of different
accessibility elements.

Question type
We use a consistent range of question types:
• closed: multiple-choice, link boxes, sentence completion, labelling diagrams
• open: labelling/drawing diagrams, short answer, calculations, extended response.

Reading age
The different methods used to measure reading age are based on the number of words and the number
of syllables per word.

Generally speaking, a question with a reading age of 16 means that an average 16-year old would score
only 50% on a test of comprehension of what that question is asking. To make sure that most GCSE
students will have a chance of understanding what we are asking them to do, we should be setting the
reading age at about 13.

However, this is only part of the story...

One of the main issues is that the necessary scientific words of three or more syllables (such as
photosynthesis, chromatography, momentum) increase the reading age.

Therefore, we take a balanced approach when ensuring the reading age is appropriate. We will reduce
the reading age by carefully using ‘carrier language’ that supports the scientific words, and by the
sentence structure. This is where accessibility fits in.

6
Making questions clear – GCSE science exams from summer 2018

Command words
Command words are words such as ‘describe’ and ‘explain’ that tell students how to answer a question.
Each command word is part of a command sentence, such as: ‘Explain how passing an alternating
current through the coil creates a sound wave’.

We use a set of command words common to all our GCSE and A-level science question papers. These
can be found on our website on each specification page under ‘teaching resources’:
aqa.org.uk/gcse-science

Command sentences are given on a separate line to the rest of the question text. Most questions use
only one command word. Where two command words are needed we separate them into different
sentences. Having two commands in one sentence can make it difficult for students to decipher what
they must do.

Standard wording/instructions
We will use standard wording for certain question types. This helps students gain familiarity with these
questions. As a result, students will be more confident in what they need to do.

Examples:
• In multiple-choice questions we use: ‘Tick one box’ after the question.
• In link box questions we use: ‘Draw x line(s) from each y to z’.

7
Direct wording
Questions are most effective when they are short and punchy.

Command words are useful as they indicate the response required. However, in some instances it’s
clearer to start questions with:
• what
• why
• when
• how
• where
• who.

For all our multiple-choice questions we use these direct question styles.

This avoids multiple-choice questions becoming unwieldy for students to grasp.

Sometimes we may just refer to picking a correct statement.

Example:

‘Which statement (about...) is true?’

The questioning words can also be combined with our command words, such as: ‘Describe how’ or
‘Explain why’.

We don’t use one-word commands such as ‘Why?’ or ‘How?’ This avoids testing comprehension by
unnecessarily asking students to link back to previous information.

8
Making questions clear – GCSE science exams from summer 2018

Repeated information
We try not to add unnecessary words to a question. One way of doing this is by avoiding repetition.

A common way information can be repeated is by describing a graph in the stem of the question where it
is just as clear to interpret the same information from the graph itself.

Example:

‘Figure 1 shows the population of the red kite in the UK from 1950 to 2010.’

It’s likely the figure for this example will clearly show population, red kite, UK, and the dates.
Therefore there’s no need to introduce them in a statement before the figure.

Sometimes we’ll just say: ‘Look at Table 1/Figure 1.’, where there is no need to increase the number of
words on the page with more detail.

This doesn’t mean we completely abolish where information is in two places! Sometimes it’s useful to
repeat key bits of information to help focus a student’s thinking.

Layout/spacing
We allow for sufficient ‘white space’ on the page. This ‘cleaner’ look is a visual aid to help
with readability.

Within a block of information there are often different groups of information. Where this happens we
space the text so the different groups of information are clear.

9
Answer line prompts
We often use answer line prompts. These make the question more accessible by taking information
already given in the question to help students structure their answer.

These prompts don’t give students any extra clues. They emphasise information given in the question.

Also, using prompts doesn’t disadvantage students if they put all of their answer in one line. Examiners
will mark all work they see (unless it’s been crossed out).

The most common answer line prompt we use is numbering the answer lines. This is to reaffirm the
number of responses needed.

Example:

Coal is a non-renewable energy resource.

Name two other non-renewable energy resources.


[2 marks]

Sentence length
It is preferable to aim for fewer than 20 words per sentence. Sentences longer than this will often
have various ‘strands’ of information and will lose their meaning.

Sticking to fewer than 20 words keeps the information concise, direct and easy to understand.

Bullet lists
Bullets are excellent to use when displaying several strands of information that link to a single idea.
They help to make the uptake of information easier by logically ‘breaking up’ information.

Putting two or more different strands of information in one sentence can be confusing. Often this is
linked to sentences being too long.

Numbered lists also help in the same way. We generally use them to show steps in a process.

10
Making questions clear – GCSE science exams from summer 2018

Using bold
We use bold to emphasise key bits of text.

Some words are always in bold, such as when we ask for a number of responses: ‘Give two reasons...’;
or when referencing figures and tables.

Other times it could be that a certain word has significant impact on the information the question gives.

Example:

‘What is the reason that the student used a water bath for this experiment?’

Bold can be very useful, although we use it with caution – too much bold loses its impact!

Sub-clauses
Sub-clauses are groups of words within a sentence that cannot stand alone as an independent
sentence. Although they’re useful at adding in information related to the sentence, students find them
very difficult to understand. We avoid sub-clauses where possible.

Example:

‘Two children, A and B, are sitting on a see-saw, as shown in Figure 1.’

This sentence contains two sub-clauses. It can be made simpler by shortening the
sentence:

‘Figure 1 shows two children sitting on a see-saw.’

And then label ‘Child A’ and ‘Child B’ on the diagram.

As well as simplifying the language, this solution also shows a neat way of reducing information that has
been repeated.

Interpreting punctuation can be difficult. Therefore, where sensible, we design our wording so that we
have the bare minimum punctuation, such as removing commas.

Contractions
Students can find the use of: don’t’; won’t; didn’t etc difficult to understand. Therefore we don’t use
contractions in our questions.

11
Idiom, contexts and jargon
An idiom is a group of words where the meaning cannot be inferred. Understanding of idiom depends on
experience: one person’s idiom may be another’s ‘I don’t know what you’re talking about’!

Not all students will have the same experience. A student who has led their whole life in an inner city
may never have come across something that a student from a rural area will find commonplace. So, we
try to keep contexts simple and within the experience of most students.

There is a similar problem in the understanding of jargon:


• We use terms from the specification that are relevant to the topic being assessed.
• We explain any unfamiliar terms used in a question.

Ellipsis
Understanding sentences where words are missing or implied is a high-level skill.

Example:

‘There are three main carbohydrate groups. Sugar is one of these.


Name the two other groups.

Many students will not understand what they have to do – they are not able to fill in the missing words
(‘these’: what are ‘these’? ‘other two groups of what’?). You need to fill in the gaps for them so they
demonstrate their knowledge of carbohydrate groups.

Example:

‘There are three main carbohydrate groups. Sugar is one of these groups.

Name the two other carbohydrate groups.’

This question is now much improved. Inevitably the number of words to read is increased, but the
repetition helps the student understand what is required.

Also see how:


• bold is used to emphasise what the question is asking for
• spacing is used to separate the command.

12
Making questions clear – GCSE science exams from summer 2018

Diagrams
Diagrams must be simple, clear and have purpose.

They are great at helping to communicate ideas to students. We ensure that they provide only the
information required to answer a question. Diagrams that are not relevant to the question can cloud a
student’s judgement.

Also, we won’t have photos in simply because they look nice or break up some text! We only use
photographs that have high print quality.

Graphs
We use a variety of graphs to test a range of skills covering all levels of demand.

To ensure our graphs are clear to students we follow these principles:


1. The amount of information displayed should be concise and relevant to the level being tested.
For example, graphs with two y-axes or negative indices are limited to the higher grades.
2. Where suitable, we use simpler gridlines or no gridlines instead of standard 2 mm2 grids. For
example, if a question does not rely on precise graph readings there is no need to have a complex
grid. This is done in GCSE Maths and improves clarity of displayed information.
3. The gridlines and axes will be extended beyond where the data is displayed. This is done in
GCSE Maths and improves clarity of displayed information.

Tables
We test a student’s ability to interpret data from a table.

What we don’t do is expect students to unnecessarily wade through a vast amount of data. This is
counter-productive as it wastes a student’s time during an exam and doesn’t test skills appropriately.

For example, we keep the number of columns and rows to a minimum. Yet, at a sufficient amount to
base questions upon.

We also ensure that the content in a table matches that given elsewhere in the question. So, we’ll keep
to a consistent use of text given in the headings. This is also true of diagrams and graphs.

13
Ramping
Ramping means that a question gets progressively more difficult as you work through it.

Questions for any topic area will be ramped in terms of demand within the question, as well as within the
paper. This allows all students a fair chance of gaining some marks on each topic area throughout
the paper.

Some questions will step up in demand gradually, others quite sharply. In addition the demand also
increases steadily throughout the paper.

We use the model above to structure the ramping of Foundation Tier and Higher Tier question papers.
• Both tiers start with confidence-building questions set at the lowest demand for the paper: ‘Low’ for
Foundation Tier; and ‘Standard’ for Higher Tier.
• The middle of each paper introduces ramping of each question up to the next level of demand. Within
each question the demand increases, then the following question starts again at a lower demand.
• The end of the paper is where the students’ ability is stretched the most. In the Foundation Tier this
means questions are set at standard demand (common with the Higher Tier). In the Higher Tier the
latter questions continue to ramp, but at a much higher level.

14
Making questions clear – GCSE science exams from summer 2018

Some examples
Use of command words
This example shows two command words in one sentence. The second command ‘explain’ is hidden
and could easily be missed by students.

As well as two command words there is a lot going on in the sentence that students have to break down
to understand what’s going on.

Splitting the sentence not only makes the commands and other information clearer but also puts the
sentence length into an acceptable range.

In a scrapyard, an electromagnet is used to lift and release cars so they can be


moved around.

Suggest two ways a solenoid could be made to lift and release cars in a scrapyard
and explain why each suggestion would be useful.
Two [4 marks]
command
words. Total
sentence
length = 24.

In a scrapyard, an electromagnet is used to lift and release cars so they can be


moved around.

Suggest two ways a solenoid could be made to lift and release cars in a scrapyard.

Explain why each suggestion would be useful in the scrapyard.


Command [4 marks]
words
separated Both sentences now
and much short and direct.
clearer to
students.

Both are
at the
beginning
of their
sentence.

15
White space
This example shows how we structure questions and make use of white space.
Steps in the
process not
immediately
clear.
Two students investigated the effect of caffeine on reaction time.

The first student drank a cup of coffee. Another student held a ruler above the
student’s hand then drops the ruler. The first student must catch the ruler as quick
as she can. The distance the ruler falls is recorded.

Suggest how this method could be improved to produce valid results.


[6 marks]
Dense text will
put some
students off.

We use Arial as a typeface.

It’s a well-recognised, plain font that students find easy to read.

We use it at size 11 pt, an acceptable size for reading.

Larger font sizes are available for students who are visually impaired.

16
Making questions clear – GCSE science exams from summer 2018

Two students investigated the effect of caffeine on reaction time.

The referencing of each


This is the method used. student is made clear by
naming them ‘Student A’
1. Student A drinks a cup of coffee. and Student B’.
2. Student B holds a ruler above Student A’s hand. This immediately makes the
process clearer and takes
3. Student B drops the ruler. away the ellipsis issue.

4. Student A must catch the ruler as quickly as she can. The use of bold helps
differentiate clearly between
5. The distance the ruler falls is recorded. the two students.

Suggest how this method could be improved to produce valid results.


[6 marks]

The use of bold and square brackets here


makes it clear to a student the depth of
response required.

We have standardised the way we write methods so that they’re


easier to follow.

We use:
• standard introductory wording: ‘This is the method used.’
• numbered steps to show there’s a process going on
• plenty of white space.

These changes make the information more direct and make it clear
when referring to particular steps of the method in a question.

17
Simplifying graphs
We use a default 2 mm2 grid for our graphs. However, this example shows how there’s no need to stick
to the 2 mm2 grid if unnecessary.

Figure 1 and Figure 2 show how the moose population and the wolf population
changed in one area.

Figure 1

The grids given are far too


detailed for the questions
that follow.
Figure 2
It could be misleading.

18
Making questions clear – GCSE science exams from summer 2018

Figure 1 and Figure 2 show how the moose population and the wolf population
changed in one area.

The fewer gridlines can benefit students Figure 1


who are visually impaired.

The questions that follow


require students to recognise
the peaks and read data from
the graphs.

These grids are now much


more suitable for these
questions.

The years on the x-axes of the graphs


need to align with each other.
Figure 2
By reducing the number of gridlines, this
alignment and trends of both graphs
alongside each other, are clearer.

Also see how extra space has been given around the graph data.

This improves the clarity of displayed information, especially in


graphs where a 2 mm2 grid is used.

19
Clarity of information
This question exemplifies how important the structure of a question is.

There’s a lot of information here that students need to extract and use in their answer. As it stands, it
could be very difficult for students to identify the different strands of information to answer the question.

The percentage atom economy for a reaction is calculated using:


relative formula mass of desired product from equation
× 100
sum of relative formula masses of all reactants from equation

The equation for the reaction of copper carbonate and sulfuric acid is: CuCO3 +
H2SO4 → CuSO4 + H2O + CO2

Relative formula masses: CuCO3 = 123.5; H2SO4 = 98; CuSO4 = 159.5

Calculate the percentage atom economy for making copper sulfate from copper
carbonate.
[3 marks]
This dense mix of text, which
includes equations and
This sole word that chemical symbols, is
has carried over from immediately off-putting to read.
the previous line could
be missed.

20
Making questions clear – GCSE science exams from summer 2018

The percentage atom economy for a reaction is calculated using:

relative formula mass of desired product from equation


× 100
sum of relative formula masses of all reactants from equation
The equations
The equation for the reaction of copper carbonate and sulfuric acid is: are aligned
‘copper’ has centrally to
been moved make them
CuCO3 + H2SO4 → CuSO4 + H2O + CO2
from previous more distinctive.
line to reduce
likelihood of
Relative formula masses: CuCO3 = 123.5; H2SO4 = 98; CuSO4 = 159.5
‘carbonate’
being

Calculate the percentage atom economy for making copper sulfate from
copper carbonate.
[3 marks]

Ample white
Different strands of information space given
are separated to make it to help
clearer to students: readability.
1. calculation
2. symbol equation
3. relative formula masses
4. command.
Atom economy = %

Answer line prompt given.

This takes information already given in the command:


‘atom economy’ and ‘percentage’. It guides students
further.

Also, it helps make the answer clearer to examiners.


Although, a correct answer given elsewhere if not given on
the final answer line will be given full credit.

21
Relevant information only
This example shows where information relating to a diagram has been stripped back.

The heart is supplied with blood by the coronary arteries.

Figure 3 shows a healthy coronary artery and a coronary artery from a person with
coronary heart disease.
The
introductory
Figure 3
information to
the diagram is
also clearly
given in the
diagram.

Consequently,
this increases
the reading
time for a
student.

The coronary arteries supply blood to the heart.


Sometimes we simply use:
Figure 3 shows two coronary arteries. ‘Look at Figure X.’

Surplus
information Figure 3
on the
arteries has
been
removed.

This
immediately
gives a
cleaner
look which
helps
students.

The images used are


simple with clear labelling.

22
Find out more
Visit our website to see our specifications, question
papers, mark schemes and resources:
Want to keep up-to-date with the
aqa.org.uk/gcse-science latest news about our specifications,
You can talk directly to our science team
exams, resources and training?
Telephone: 01483 477 756 Register at aqa.org.uk/ks4-updates
Email: [email protected]

Science for all


Our suite of seven KS4 science qualifications gives you
a real choice for all abilities. Find out more at
aqa.org.uk/ks4-changes

The information in this document is based on the specimen question papers and mark schemes accredited by Ofqual in April 2016.

Copyright © 2016 AQA and its licensors. All rights reserved.


AQA Education (AQA) is a registered charity (registered charity number 1073334) and a company limited by guarantee registered in
England and Wales (company number 3644723). Registered address: AQA, Devas Street, Manchester M15 6EX.

September 2016

You might also like