AQA Science Making Questions Clear
AQA Science Making Questions Clear
AQA Science Making Questions Clear
questions clear
GCSE science exams from
summer 2018
Version 2.0 September 2016
Contents
Journey of a question 4
Some examples 15
Use of command words 15
White space 16
Simplifying graphs 18
Clarity of information 20
Relevant information only 22
3
Journey of a question
Creating a question paper takes a lot of time!
A question paper is put together by an exam team. The members of the team are often teachers, who
have developed assessment expertise as a result of working as examiners. The team is made up of
senior examiners: it includes the Chair of Examiners for the subject; a Chief Examiner for the
specification; and writers for each question paper.
We also have assessment design and product teams who are responsible for managing the process and
ensuring that the question paper complies with rules and regulations. The team draws on the research
and assessment expertise of our Centre for Education Research and Practice: cerp.aqa.org.uk
We analyse the statistics of question papers sat by students. We are continuously improving: by
understanding how individual questions have performed we can learn lessons which help inform any
future decisions on question papers.
We develop a ‘blueprint’ for each question paper. This is essentially the design of the question paper,
for example what areas of the specification and balance of skills are to be assessed. This is one of the
most important parts of the whole process.
The exam team collaborates closely to create and then review the question paper. They scrutinise all
aspects of every question.
This includes:
• relevance to specification
• breadth of specification covered
• depth of knowledge assessed
• scientific accuracy
• answers in the mark scheme accurately reflecting the question asked
• number of marks being appropriate
• level of demand
• allocation of assessment objectives
• mathematics requirements.
One area of reviewing each question encompasses ‘accessibility’. We look at many elements that feed
into the overall accessibility of a question. Individually these elements may sometimes not seem
important but, together, the impact on the question is greater than the sum of its parts.
4
Making questions clear – GCSE science exams from summer 2018
Accessibility is about getting the message across clearly without affecting the demand of the question –
the content familiarity requirement is still there. Clear language and layout of question papers is
important for all students.
Our accessibility ethos gives all students the best opportunity to get the results they deserve.
This booklet gives ‘before’ and ‘after’ examples of questions to show how we refine accessibility.
5
Our accessibility principles
To ensure we make our questions as clear as possible we review across a number of different
accessibility elements.
Question type
We use a consistent range of question types:
• closed: multiple-choice, link boxes, sentence completion, labelling diagrams
• open: labelling/drawing diagrams, short answer, calculations, extended response.
Reading age
The different methods used to measure reading age are based on the number of words and the number
of syllables per word.
Generally speaking, a question with a reading age of 16 means that an average 16-year old would score
only 50% on a test of comprehension of what that question is asking. To make sure that most GCSE
students will have a chance of understanding what we are asking them to do, we should be setting the
reading age at about 13.
One of the main issues is that the necessary scientific words of three or more syllables (such as
photosynthesis, chromatography, momentum) increase the reading age.
Therefore, we take a balanced approach when ensuring the reading age is appropriate. We will reduce
the reading age by carefully using ‘carrier language’ that supports the scientific words, and by the
sentence structure. This is where accessibility fits in.
6
Making questions clear – GCSE science exams from summer 2018
Command words
Command words are words such as ‘describe’ and ‘explain’ that tell students how to answer a question.
Each command word is part of a command sentence, such as: ‘Explain how passing an alternating
current through the coil creates a sound wave’.
We use a set of command words common to all our GCSE and A-level science question papers. These
can be found on our website on each specification page under ‘teaching resources’:
aqa.org.uk/gcse-science
Command sentences are given on a separate line to the rest of the question text. Most questions use
only one command word. Where two command words are needed we separate them into different
sentences. Having two commands in one sentence can make it difficult for students to decipher what
they must do.
Standard wording/instructions
We will use standard wording for certain question types. This helps students gain familiarity with these
questions. As a result, students will be more confident in what they need to do.
Examples:
• In multiple-choice questions we use: ‘Tick one box’ after the question.
• In link box questions we use: ‘Draw x line(s) from each y to z’.
7
Direct wording
Questions are most effective when they are short and punchy.
Command words are useful as they indicate the response required. However, in some instances it’s
clearer to start questions with:
• what
• why
• when
• how
• where
• who.
For all our multiple-choice questions we use these direct question styles.
Example:
The questioning words can also be combined with our command words, such as: ‘Describe how’ or
‘Explain why’.
We don’t use one-word commands such as ‘Why?’ or ‘How?’ This avoids testing comprehension by
unnecessarily asking students to link back to previous information.
8
Making questions clear – GCSE science exams from summer 2018
Repeated information
We try not to add unnecessary words to a question. One way of doing this is by avoiding repetition.
A common way information can be repeated is by describing a graph in the stem of the question where it
is just as clear to interpret the same information from the graph itself.
Example:
‘Figure 1 shows the population of the red kite in the UK from 1950 to 2010.’
It’s likely the figure for this example will clearly show population, red kite, UK, and the dates.
Therefore there’s no need to introduce them in a statement before the figure.
Sometimes we’ll just say: ‘Look at Table 1/Figure 1.’, where there is no need to increase the number of
words on the page with more detail.
This doesn’t mean we completely abolish where information is in two places! Sometimes it’s useful to
repeat key bits of information to help focus a student’s thinking.
Layout/spacing
We allow for sufficient ‘white space’ on the page. This ‘cleaner’ look is a visual aid to help
with readability.
Within a block of information there are often different groups of information. Where this happens we
space the text so the different groups of information are clear.
9
Answer line prompts
We often use answer line prompts. These make the question more accessible by taking information
already given in the question to help students structure their answer.
These prompts don’t give students any extra clues. They emphasise information given in the question.
Also, using prompts doesn’t disadvantage students if they put all of their answer in one line. Examiners
will mark all work they see (unless it’s been crossed out).
The most common answer line prompt we use is numbering the answer lines. This is to reaffirm the
number of responses needed.
Example:
Sentence length
It is preferable to aim for fewer than 20 words per sentence. Sentences longer than this will often
have various ‘strands’ of information and will lose their meaning.
Sticking to fewer than 20 words keeps the information concise, direct and easy to understand.
Bullet lists
Bullets are excellent to use when displaying several strands of information that link to a single idea.
They help to make the uptake of information easier by logically ‘breaking up’ information.
Putting two or more different strands of information in one sentence can be confusing. Often this is
linked to sentences being too long.
Numbered lists also help in the same way. We generally use them to show steps in a process.
10
Making questions clear – GCSE science exams from summer 2018
Using bold
We use bold to emphasise key bits of text.
Some words are always in bold, such as when we ask for a number of responses: ‘Give two reasons...’;
or when referencing figures and tables.
Other times it could be that a certain word has significant impact on the information the question gives.
Example:
‘What is the reason that the student used a water bath for this experiment?’
Bold can be very useful, although we use it with caution – too much bold loses its impact!
Sub-clauses
Sub-clauses are groups of words within a sentence that cannot stand alone as an independent
sentence. Although they’re useful at adding in information related to the sentence, students find them
very difficult to understand. We avoid sub-clauses where possible.
Example:
This sentence contains two sub-clauses. It can be made simpler by shortening the
sentence:
As well as simplifying the language, this solution also shows a neat way of reducing information that has
been repeated.
Interpreting punctuation can be difficult. Therefore, where sensible, we design our wording so that we
have the bare minimum punctuation, such as removing commas.
Contractions
Students can find the use of: don’t’; won’t; didn’t etc difficult to understand. Therefore we don’t use
contractions in our questions.
11
Idiom, contexts and jargon
An idiom is a group of words where the meaning cannot be inferred. Understanding of idiom depends on
experience: one person’s idiom may be another’s ‘I don’t know what you’re talking about’!
Not all students will have the same experience. A student who has led their whole life in an inner city
may never have come across something that a student from a rural area will find commonplace. So, we
try to keep contexts simple and within the experience of most students.
Ellipsis
Understanding sentences where words are missing or implied is a high-level skill.
Example:
Many students will not understand what they have to do – they are not able to fill in the missing words
(‘these’: what are ‘these’? ‘other two groups of what’?). You need to fill in the gaps for them so they
demonstrate their knowledge of carbohydrate groups.
Example:
‘There are three main carbohydrate groups. Sugar is one of these groups.
This question is now much improved. Inevitably the number of words to read is increased, but the
repetition helps the student understand what is required.
12
Making questions clear – GCSE science exams from summer 2018
Diagrams
Diagrams must be simple, clear and have purpose.
They are great at helping to communicate ideas to students. We ensure that they provide only the
information required to answer a question. Diagrams that are not relevant to the question can cloud a
student’s judgement.
Also, we won’t have photos in simply because they look nice or break up some text! We only use
photographs that have high print quality.
Graphs
We use a variety of graphs to test a range of skills covering all levels of demand.
Tables
We test a student’s ability to interpret data from a table.
What we don’t do is expect students to unnecessarily wade through a vast amount of data. This is
counter-productive as it wastes a student’s time during an exam and doesn’t test skills appropriately.
For example, we keep the number of columns and rows to a minimum. Yet, at a sufficient amount to
base questions upon.
We also ensure that the content in a table matches that given elsewhere in the question. So, we’ll keep
to a consistent use of text given in the headings. This is also true of diagrams and graphs.
13
Ramping
Ramping means that a question gets progressively more difficult as you work through it.
Questions for any topic area will be ramped in terms of demand within the question, as well as within the
paper. This allows all students a fair chance of gaining some marks on each topic area throughout
the paper.
Some questions will step up in demand gradually, others quite sharply. In addition the demand also
increases steadily throughout the paper.
We use the model above to structure the ramping of Foundation Tier and Higher Tier question papers.
• Both tiers start with confidence-building questions set at the lowest demand for the paper: ‘Low’ for
Foundation Tier; and ‘Standard’ for Higher Tier.
• The middle of each paper introduces ramping of each question up to the next level of demand. Within
each question the demand increases, then the following question starts again at a lower demand.
• The end of the paper is where the students’ ability is stretched the most. In the Foundation Tier this
means questions are set at standard demand (common with the Higher Tier). In the Higher Tier the
latter questions continue to ramp, but at a much higher level.
14
Making questions clear – GCSE science exams from summer 2018
Some examples
Use of command words
This example shows two command words in one sentence. The second command ‘explain’ is hidden
and could easily be missed by students.
As well as two command words there is a lot going on in the sentence that students have to break down
to understand what’s going on.
Splitting the sentence not only makes the commands and other information clearer but also puts the
sentence length into an acceptable range.
Suggest two ways a solenoid could be made to lift and release cars in a scrapyard
and explain why each suggestion would be useful.
Two [4 marks]
command
words. Total
sentence
length = 24.
Suggest two ways a solenoid could be made to lift and release cars in a scrapyard.
Both are
at the
beginning
of their
sentence.
15
White space
This example shows how we structure questions and make use of white space.
Steps in the
process not
immediately
clear.
Two students investigated the effect of caffeine on reaction time.
The first student drank a cup of coffee. Another student held a ruler above the
student’s hand then drops the ruler. The first student must catch the ruler as quick
as she can. The distance the ruler falls is recorded.
Larger font sizes are available for students who are visually impaired.
16
Making questions clear – GCSE science exams from summer 2018
4. Student A must catch the ruler as quickly as she can. The use of bold helps
differentiate clearly between
5. The distance the ruler falls is recorded. the two students.
We use:
• standard introductory wording: ‘This is the method used.’
• numbered steps to show there’s a process going on
• plenty of white space.
These changes make the information more direct and make it clear
when referring to particular steps of the method in a question.
17
Simplifying graphs
We use a default 2 mm2 grid for our graphs. However, this example shows how there’s no need to stick
to the 2 mm2 grid if unnecessary.
Figure 1 and Figure 2 show how the moose population and the wolf population
changed in one area.
Figure 1
18
Making questions clear – GCSE science exams from summer 2018
Figure 1 and Figure 2 show how the moose population and the wolf population
changed in one area.
Also see how extra space has been given around the graph data.
19
Clarity of information
This question exemplifies how important the structure of a question is.
There’s a lot of information here that students need to extract and use in their answer. As it stands, it
could be very difficult for students to identify the different strands of information to answer the question.
The equation for the reaction of copper carbonate and sulfuric acid is: CuCO3 +
H2SO4 → CuSO4 + H2O + CO2
Calculate the percentage atom economy for making copper sulfate from copper
carbonate.
[3 marks]
This dense mix of text, which
includes equations and
This sole word that chemical symbols, is
has carried over from immediately off-putting to read.
the previous line could
be missed.
20
Making questions clear – GCSE science exams from summer 2018
Calculate the percentage atom economy for making copper sulfate from
copper carbonate.
[3 marks]
Ample white
Different strands of information space given
are separated to make it to help
clearer to students: readability.
1. calculation
2. symbol equation
3. relative formula masses
4. command.
Atom economy = %
21
Relevant information only
This example shows where information relating to a diagram has been stripped back.
Figure 3 shows a healthy coronary artery and a coronary artery from a person with
coronary heart disease.
The
introductory
Figure 3
information to
the diagram is
also clearly
given in the
diagram.
Consequently,
this increases
the reading
time for a
student.
Surplus
information Figure 3
on the
arteries has
been
removed.
This
immediately
gives a
cleaner
look which
helps
students.
22
Find out more
Visit our website to see our specifications, question
papers, mark schemes and resources:
Want to keep up-to-date with the
aqa.org.uk/gcse-science latest news about our specifications,
You can talk directly to our science team
exams, resources and training?
Telephone: 01483 477 756 Register at aqa.org.uk/ks4-updates
Email: [email protected]
The information in this document is based on the specimen question papers and mark schemes accredited by Ofqual in April 2016.
September 2016