Released Test Units: Annex C
Released Test Units: Annex C
Released Test Units: Annex C
Five test units were released to illustrate the cognitive assessment. In what follows, the test units are closely examined, with a
focus on response modalities, levels of difficulty and scoring procedures. Screenshots of every test item are provided, along with
a description of that item. The released test units are also provided on line at www.oecd.org/pisa/test/.
This unit features an excerpt from a lecture by the Nigerian writer Chimamanda Ngozi Adichi entitled «The Danger of a Single
Story». In this excerpt, she describes the experience of realising her roommate did not see her as an individual with unique
experiences and equal worth but instead, had formed a «single story» about her based on preconceived assumptions about
Africa and African life. The unit begins with two questions related to this excerpt and goes on to explore how a «single story»
can be created and to challenge a fictional woman’s assumptions about a man in a market. The content domain of this unit was
categorised as «Culture and intercultural relations», with a subdomain of «Perspective taking, stereotypes, discrimination and
intolerance”.
This item requires the student to reflect on the perspective of Adichie’s roommate and identify a possible reason the roommate
may have created a «single story» of Adichie in which she was shocked by her ability to speak English and disappointed when
she learned that Adichie listened to American pop music instead of «tribal music». The correct answer is C because it is the only
option that explains how the roommate might have already developed an idea of who Adichie was. Here, the student must be
able to accurately identify the perspective of the roommate versus Adichie’s perspective and choose the option that best reflects
the context.
394 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
In this item, the student must evaluate each statement in the table and decide whether it describes a way that a stereotype might
be incomplete. Here, the student must think more broadly than the specific stereotypes of Africa described in the scenario and
consider what a stereotype is and how stereotypes lack critical information that allow them to persist. By identifying the correct
answers in this item, the student demonstrates his/her ability to explain how stereotypes are created. The correct answers to
this question are Yes, No, Yes, Yes, No. The statements that require a Yes response all speak to the fact that stereotypes are
broad generalisations that lack any consideration of individual differences or personal experiences. Within the stereotypes that
are perpetuated, there is no room to consider an individual’s identity or experiences, just like the interaction between Adichie
and her roommate. This item had partial-credit and full-credit scoring. To receive partial credit, four out of five statements had
to be correct. To receive full credit, all five statements had to be correct. If three or fewer statements were correct, no credit was
assigned. The level provided for this item is based on full credit.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 395
Annex C Released test units
This item is similar to the previous item in that the student must think more broadly about stereotypes or single stories and
consider how the media may support the creation of this misinformation. Four examples of media forms and content are
described, and the student must evaluate how each one may or may not support the formation of stereotypes. To receive full
credit, the student needed to select both B and D. Partial credit was assigned if only B or only D was selected. If any other options
were selected, no credit was assigned. By selecting the correct answers, the student demonstrates the ability to identify examples
that address the complex issue of stereotype formation. The level provided for this item is based on full credit.
396 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
Here, a short text is presented about a woman in a market, Alice, who observes a young man’s appearance and behaviour. The
text then describes how Alice perceives the young man. Two independently coded, open-ended items follow the text. In the first
item, the student is are asked to read the text and simply describe, in their own words, one of Alice’s assumptions about the
young man. The test developers identified five possible assumptions that could be considered correct based on the information
provided in the brief text. The coding guide for the correct responses is provided below.
Full Credit
Code 1: Provides one of the assumptions about the young man listed below:
1. The young man is a foreigner.
5. The young man has (or foreigners have) no respect for the rules of society.
• She thinks he’s foreign. [1]
• She thinks he’s poor. [2]
• He can’t pay for his food. [2]
• She thinks he doesn’t have a job. [3]
• He has not paid for the fruit. [4]
• She thinks he has no respect for the rules. [5] – This response includes information provided in the stem. However, in
this case, it is accepted as evidence that the student has correctly identified an assumption that Alice made.
• He wasn’t raised well. [5] – This is an acceptable paraphrase for “no respect for the rules of society”.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 397
Annex C Released test units
After identifying an assumption that Alice makes in the brief text, the student is then asked to explain why that assumption might
be incorrect. To get full credit for this item, the student must provide a more narrow response that explains the assumption he/
she provided in the previous item. For example, if “The young man is stealing” is identified as an assumption, the explanation
could be “He might have already paid for the fruit”. Alternatively, the student can get full credit by providing a broader, more
general response that addresses the problem with making assumptions, such as “She is making a judgement without enough
information”. Both types of responses were given full credit, but coders were asked to attempt to assign different codes in case
researchers were interested in exploring differences between students who take a more narrow approach to answering the
question and those who take a broader approach. For the purpose of the main survey analyses, these categories were all treated
as full credit.
Full Credit
Code 11: Provides an explanation that is specific to the assumption provided in CG123Q04 AND describes why that assumption
might be incorrect. The explanation may provide another interpretation for the behaviour Alice observed or refute
Alice’s assumptions.
1. Assumption: The young man is a foreigner. Explanation must focus on the language he was using.
2. Assumption: The young man is poor or cannot pay for his food. Explanation must focus on his torn clothes OR that
he was grabbing the fruit.
3. Assumption: The young man has no job. Explanation must focus on his torn clothes OR that he was grabbing the fruit.
4. Assumption: The young man is stealing. Explanation must focus on the observation that he was grabbing the fruit.
5. Assumption: The young man has (or foreigners have) no respect for the rules of society. Explanation must focus on
the observation that he was grabbing the fruit.
• Just because he is speaking another language does not mean he is a foreigner. [1]
• He might speak more than one language. [1]
• He might have been born in this country but speaks a different language. [1]
• Maybe it’s the style for young people to wear torn clothes. [2]
• He might work at the fruit stand. [2]
• He might have permission to take the fruit from the owner of the fruit stand. [2]
• He might be asking his friend to help him pay for the fruit. [2]
• He could be wearing torn clothes because of the work he does. [3]
• Just because he is grabbing the fruit doesn’t mean he isn’t working. [3]
• He could have a very low-paying job and not be able to afford the food he needs. [4]
• He might know the owner of the fruit stand and is allowed to take fruit. [4 or 5]
• His family might own the fruit stand. [4 or 5]
Code 11: Provides a general explanation that describes a potential problem about making assumptions.
• She does not have enough information about this young man to make this assumption.
• She is overgeneralising.
• She is stereotyping him. [A stereotype is a type of overgeneralisation.]
• She is racist. [Judging people based on perceived race is a specific type of overgeneralisation. Related words like
discriminating, prejudice, etc. are acceptable.]
• She is rushing to judgment (without enough information/without knowing or talking to this young man).
398 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
• She is judging.
• There might be other good reasons for his behaviour.
• She has a single story about him.
OR: Provides a general explanation of why an assumption based on the young man’s actions might be incorrect.
• He might be acting that way because he has a disability.
• He might be showing off in front of his friend.
This unit contained an introduction screen to provide some initial context about the Refugee Olympic Team, which competed in
the Olympic Games for the first time in 2016. The test developers did not want to assume that all students are familiar with this
team, so background knowledge was provided to ensure that all students would have the same information to start. The rest of
the unit focuses on a fictional character’s participation on the Refugee Olympic Team.
The stimulus for this unit (presented on the next page) introduces Felix, an athlete who fled his homeland and has been living
as a refugee in another country. He was an athlete who trained in his home country before fleeing and has been training in his
new country of residence. In the stimulus, the student learns that Felix participated as a member of the Refugee Olympic Team
and won a medal. The stimulus then presents an interview with Felix about his feelings on accepting the medal for the Refugee
Olympic Team rather than his homeland or his current country of residence. Finally, the student learns that a debate took place
on social media about his decision. The content domain of this unit was categorised as «Institutions, conflicts and human rights»
with a subdomain of «Universal human rights and local traditions».
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 399
Annex C Released test units
In this item, the student must consider the goal of a sports reporter who is writing an article about Felix and the debate about his
Olympic medal. The student needs to evaluate whether information provided by three different sources would give the reporter
the relevant information for the article. By correctly identifying which sources are relevant and which are not, the student is
demonstrating the ability to evaluate and select sources. The correct answers for this item are Yes, No, Yes. Credit is only assigned
if the student gets all three correct.
400 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
This item requires the student to consider the perspective of some residents of the country of Latoona, who feel the medal
should have been awarded to their country, where Felix has refugee status. The correct answer is C because this statement
provides the best support for this claim, the commitment Latoona made to supporting his training that should earn the medal
for Latoona. The other responses are either not relevant to the specific scenario described in the stimulus or they fall short of
recognising the perspective of the people described in the text.
This item is similar to the previous item, but now the student must consider the perspective of some residents of Felix’s home
country, Gondaland. The answer that best demonstrates the recognition of their perspective is B.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 401
Annex C Released test units
This item presents a short text meant to represent a post on social media. In this post, the author makes several statements to
support the argument that the medal should have been awarded to Latoona, Felix’s host country. The student is then asked to
consider four statements from the post and identify which ones are opinions. The correct answer is C and D. If both are selected,
full credit is assigned. If only C or only D is selected, partial credit is assigned. If anything else is selected, the student receives no
credit. The student must evaluate the information carefully and then consider whether the statement is truly a fact or if it goes
beyond a fact and reflects the opinion of the author. In this way, the student must consider the reliability of the statements, which
is related to the cognitive subprocess of «Weighing sources».
402 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
In this last item of the unit, the student must consider Felix’s perspective based on what is provided in the stimulus, go beyond
what is explicitly written in the text and provide a reason for why Felix thought it was appropriate to accept the medal for the
Refugee Olympic Team. Felix never directly states why he made the decision or why he thought it was the appropriate decision to
make. The coding guide for this item specified ways to receive both full and partial credit. The partial-credit description represents
a more literal or fact-based way to answer the question, which only refers to the fact that Felix is a refugee. Such responses are
technically correct but, unlike the full-credit responses, they do not fully demonstrate an attempt to take Felix’s perspective into
account and construct an answer that reflects why he may have felt his decision was the most appropriate one.
Full Credit
Code 2: Refers to one of the following reasons why Felix may have wanted to accept the medal for the Refugee Olympic Team.
1. It helped resolve his conflict about which country to represent. (Note that this reason refers to an internal conflict
within Felix, not a conflict between Latoona and Gondaland.)
2. It reflects the financial, emotional and/or training support of the Refugee Olympic Team. (Note that this information
is not provided in the interview. However, it is factually correct that the Refugee Olympic team provides support for
its athletes. Students may have outside knowledge of this fact and it is acceptable for them to apply this knowledge.)
Partial Credit
Code 1: Refers to Felix’s status as a refugee or that he competed as a member of the Refugee Olympian Team.
• Felix is a refugee so the Refugee Olympic Team best represents his situation.
• He was competing for the Refugee Olympic Team.
• He was a refugee
This unit is about a fictional country, Armaz, where the fictional language Ursk is spoken. A group of Ursk-speaking lawmakers
proposed a policy that would require all public schools to teach all classes, except foreign language classes, in Ursk. There are a
number of citizens in Armaz who speak Jutanese, which is a minority language in Armaz, but is spoken widely outside its borders.
They are concerned about the effects of this policy. In this unit, PISA students must consider the impacts of the policy and reason
through its possible consequences. The content domain of this unit was categorised as «Culture and intercultural relations» with
a subdomain of «Perspective taking, stereotypes, discrimination and intolerance”.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 403
Annex C Released test units
This item presents a short text meant to represent a post on social media. In this post, the author makes several statements to
support the argument that the medal should have been awarded to Latoona, Felix’s host country. The student is then asked to
consider four statements from the post and identify which ones are opinions. The correct answer is C and D. If both are selected,
full credit is assigned. If only C or only D is selected, partial credit is assigned. If anything else is selected, the student receives no
credit. The student must evaluate the information carefully and then consider whether the statement is truly a fact or if it goes
beyond a fact and reflects the opinion of the author. In this way, the student must consider the reliability of the statements, which
is related to the cognitive subprocess of «Weighing sources».
404 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
Here, students must consider four possible consequences and determine which one would be the most serious if the Ursk-only
policy is instituted. All consequences are possible, but one summarises a serious potential consequence of the policy. Here,
B is the correct answer. In order to understand why this is the correct answer, students must consider the fact that a special
school would remove Jutanese-speaking students from the general population. By isolating a group of students like this, the
Ursk-speaking students would have fewer personal interactions with the Jutanese-speaking students, which could lead to Ursk-
speaking students relying on generalisations and stereotypes, rather than on interactions with individuals, to get to know their
Jutanese-speaking peers. This could then lead to widening divisions between Ursk and Jutanese speakers.
For this item, students have the opportunity to express their answer in their own words. The previous items focused on the
effects of a one-language policy within one country. This item broadens the picture to consider a more global community. Earlier,
the unit explained that Jutanese was a minority language within Armaz and not spoken by the majority of citizens. However,
here, the student learns that Jutanese is widely used outside of Armaz, in contrast to Ursk, which is not spoken much outside
Armaz and some neighbouring countries. With this information, the student must describe a possible consequence of having
Ursk-only education in public schools. Students could receive credit by providing two types of responses. Responses that
described a consequence that was more globally focused or expressed an effect on relationships between people or cultures
in Armaz and other countries received a code of 11. Responses that described a consequence that was more locally focused
or expressed an effect on life within Armaz received a code of 12. A code of 13 was applied if the response was not completely
clear with respect to its global or local perspective. All three types of responses received full credit. However, these codes were
developed so that distributions of global versus local responses could be examined by researchers. For the main study scaling,
only full credit compared to no credit was considered.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 405
Annex C Released test units
Full Credit
Code 11: Includes one consequence associated with how interactions between people in Armaz and people in other countries
might be affected as a result of making Ursk the only language used in public schools. Response should refer to one of
the following:
1. It could be difficult for people in Armaz to interact with people from other countries.
• People may have a harder time getting jobs in other countries/with international companies. [3]
• It wouldn’t be good for Armaz tourism if people there only spoke Ursk well. [3]
• It would be hard for Armaz to do business with other countries. [3]
• People interested in learning Ursk might visit Armaz. [4]
Code 12: Includes one consequence associated with how life within Armaz might be affected as a result of making Ursk the only
language used in public schools. Response should refer to one of the following:
1. It could be a benefit for the country of Armaz.
• Jutanese speakers might have to leave Armaz because they can’t communicate well. [4]
• People who speak Jutanese in Armaz may face discrimination. [5]
• Protests might happen as people who are not used to speaking Ursk will feel it is unecessary to learn it. [5]
• There is no problem for students who understand Ursk. But some students who do not get used to Ursk could be
bullied because of wrong use of words. [5]
406 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
Code 13: Includes a correct consequence, but it is not clear whether the response is referring to a consequence that has an effect
within Armaz or a consequence that affects interactions between people in Armaz and people in other countries.
• Discrimination [Acceptable consequence, but it’s not clear whether this refers to discrimination among people within
Armaz or between people in other countries and people in Armaz]
• It would be hard for people to communicate. [Acceptable consequence, but it’s not clear whether this refers to a
communication issue within Armaz or between people in Armaz and other countries.]
• People could become more isolated. [Acceptable consequence, but it’s not clear whether this refers to isolation of a
group of people within Armaz or isolation of Armaz from other countries.]
The stimulus describes four countries that have unique profiles of the language or languages spoken within the country. In
this item, the student must consider where a single-language education system would be the most appropriate and where it
would be the least appropriate. Country 2 is the most appropriate location for a single-language education system because
a majority of the people already speak the official language. A minority of people speak a number of different languages, and
these individuals are spread out across the country in different regions. Thus, in this country, it would be difficult to incorporate a
common second language within the education system. Country 3, however, has only a minority of people that speak the official
language. Here, a majority of the people speak a common language that is not the official language. If a one-language education
policy were instituted in the official language, many citizens would face great difficulties in the education system. Therefore this is
the least appropriate location for a single-language system. The correct answer for this item is Country 2 and Country 3.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 407
Annex C Released test units
This unit begins with a brief introduction that describes the effects of rising temperatures on sea levels. The introduction sets
the stage for the items within the unit, which explores the effects of rising sea levels on individuals who live in areas of low
elevations, such as islands and coastal areas. The unit focuses on a fictional place where sea levels have risen and displaced the
inhabitants of the islands, making them climate refugees. The content domain of this unit was categorised as «Socio-economic
development and interdependence” with a subdomain of «Economic interactions and interdependence”. The experts also felt
that this unit included content relevant to the category “Environmental sustainability” with a subdomain of “Natural resources
and environmental risks”.
The first item in the unit presents a brief text about a fictional film, “Travina: A Paradise Lost”. The documentary focuses on a
fictional island nation, Travina, that has been affected by rising sea levels. Hundreds of Travinians have had to move to higher
ground to escape the changes to the low-lying areas of the islands. The text also states that unless environmental conditions
improve, most of Travina will be underwater by the year 2075.
408 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
With this background, the item introduces the filmmaker’s goal in creating the documentary: “to persuade audiences that rising
global temperatures are a threat by presenting the impact on people’s lives”. The item then presents four reasons that might
explain why the filmmaker focused on Travina. To answer each part of the item correctly, the student must consider the filmmaker’s
goal and evaluate whether each statement could be a reason why Travina would present a persuasive case. In the table, the
second and third statements describe reasons that support the filmmaker’s goal. In both cases, the statements describe why
the situation on Travina could have a broader impact on viewers, even those who live far from Travina or who do not live near
the ocean. By contrast, the first and last statements do not describe why the filmmaker would use Travina as an example. These
statements describe a narrow viewership and one that is likely already persuaded about the effects of rising global temperatures.
Thus, to receive credit for this item, students had to respond No, Yes, Yes, No.
Here, students must be able to identify possible negative consequences of the film’s success and the attention on Travina. For
each example in the table, the student must decide whether it describes a possible negative consequence. The correct answer
is Yes, Yes, No, No. The first two examples describe direct possible consequences of the attention on Travina that could have
additional negative effects on the island nation. The third and fourth examples are not truly consequences of the attention the
film is generating for Travina. In the third example, whether the government can disburse donations to those in need has little to
do with the success of the film and more to do with the government’s capacity. The fourth example expresses a consequence that
is related to Travinians having to move to higher ground, but this is not relevant to the success of the documentary.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 409
Annex C Released test units
This item introduces new information about projects that can be completed to help certain islands within Travina. The brief text
states that Travina cannot afford these projects on its own, so some people have proposed creating an international partnership
of countries that would fund these projects in Travina. The student is then asked to read five arguments and identify whether
each statement is for or against the idea of international funding for projects in Travina. To receive credit on this item, students
had to get all parts of the item correct. The correct answers are: For, Against, Against, Against, For.
410 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
This item asks the student to name one challenge that climate refugees would face when moving to a new place. This item was
one of the easiest items in the Global Competence item pool. While the item is focused on a climate refugee, all refugees face a
similar set of challenges when leaving their home and moving somewhere new. While the majority of PISA students were likely not
refugees, the challenges of moving to a new place are those that many students can imagine or have experienced themselves.
Thus, students could apply their prior knowledge to this context in order to recognise the challenges that affect climate refugees.
The test developers came up with four broad categories for the challenges that would be relevant for climate refugees and others
who need to relocate: communication difficulties; financial/economic difficulties; difficulties adjusting to life in a new place; and
difficulties associated with leaving or losing the community or home and/or finding a new place to live. If students provided a
response that fell within one of those categories, they received full credit.
Full Credit
Code 1: Provides a challenge associated with someone leaving their community or country. Responses should refer to one of the
following categories of challenges:
1. Communication
2. Financial/Economic
4. Difficulties associated with leaving or losing the community or home and/or finding a new place to live
• They may not know the language. [1]
• Language [1] – Minimal response: The word “language” provides a strong enough connection to a communication
challenge.
• They may not know the language which could make it hard to get a job. [1 and 2]
• They might have to move to a place that is more expensive and then life would be harder for them. [2]
• They may be unfamiliar with the culture and not fit in. [3]
• They might have trouble making friends because they are different. [3]
• They may not get used to the temperature or humidity in their new home and get sick easily. [3] – Responses that refer
to adjusting to the climate of a new place are acceptable.
• Discrimination [3] – Minimal response: Related words such as racism, prejudice, etc. are acceptable because they provide
a strong connection to a challenge refugees. might experience in adjusting to life in a new place]
• They may not be able to move with all of their family. [4]
• They might miss their native homeland. [4]
• They would be sad to leave the place they called home. [4]
• They might not be allowed into some countries. [4]
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 411
Annex C Released test units
This final item asks the student to consider a set of proposals and identify which represents a short-term response (to a more
immediate need) and which represents a long-term response (to more systemic causes) to rising sea levels. Here, sea defences,
desalination technologies for drinking water and moving villages are all short-term responses. Each individual response might
require a lot of effort and several years to complete, but they all address a more short-term, immediate response to the problems
people on an island face in the midst of rising sea levels. By contrast, reducing greenhouse gases and supporting research for
new protection strategies are responses that must unfold over a longer period. Each of these solutions could take decades for
the results to affect people and could help tackle the systemic causes of sea level rise. This item had partial-credit and full-credit
scoring. The correct responses were Short term, Long term, Short term, Short term, Long term. To receive partial credit, four out
of five statements had to be correct. To receive full credit, all five statements had to be correct. If three or fewer statements were
correct, no credit was assigned. The level provided for this item is based on full credit.
In this unit, students are introduced to the concept of fast fashion, which is a trend whereby clothing is inexpensive, of lesser
quality and produced to meet the frequent changes in fashion trends. This clothing is not intended to be worn by consumers for
several seasons. Instead, it is likely to be discarded or donated once the style has become less popular. Students also learn about
an alternative concept: durable fashion. Durable clothing is more expensive, of better quality and intended to be worn for longer
periods. Students are also told about three principles of ethical clothing production. Throughout the unit, students are asked
to consider the consequences of clothing production and make connections with these principles. The content domain of this
unit was categorised as “Environmental sustainability” with a subdomain of “Policies, practices and behaviours for environmental
sustainability”. The experts also felt that this unit included content relevant to the category «Socio-economic development and
interdependence” with a subdomain of «Economic interactions and interdependence”.
412 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
A list of four possible consequences of the Fast Fashion trend are presented, and students need to decide whether each
consequence violates one or more of the principles of ethical clothing production. The first and third consequences violate the
principles. The first consequence violates the second principle because more clothing in landfills adds to environmental waste
instead of minimising it. The third consequence violates the first principle because keeping pay rates low means the company or
industry is not working to ensure that workers have fair wages. The second and fourth consequences do not violate the principles.
To receive credit on this item, students had to get all parts of the item correct. The correct answers are: Yes, No, Yes, No.
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 413
Annex C Released test units
Here, the student is asked to think about what might happen if there were a ban on Fast Fashion clothes. They are asked to
provide one possible positive consequence of a ban and one negative consequence. In order for students to provide either kind of
consequence, they first need to think about the current effects of Fast Fashion described in the stimulus. Then they must consider
what would happen if a ban went into effect, which requires the student to be able to think beyond what has been described in
the unit thus far. Test developers came up with several classes of responses for both the positive and negative consequences,
which are provided in the coding guide below with sample responses. For this item, full credit was given if the student could
correctly describe both a positive and a negative consequence. Students received partial credit if they could accurately describe
only a positive or only a negative consequence.
Full Credit
Code 2: Includes a correct response for both the positive AND negative consequences. In general, the two consequences
should appear in the correct boxes. If the student puts both responses in a single box, it must be clear that one is a
positive consequence and one is a negative consequence. Correct possible consequences for each are provided below.
Positive consequences – responses should refer to one of the following types of positive consequences:
1. Positive effects on the environment
• Some Fast Fashion factories might close because people don’t buy the clothes. [1 or 3]
• People won’t have to buy as many clothes because durable clothes last longer, so there will be fewer jobs for clothing
workers. [1]
414 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?
Released test units Annex C
• Clothing prices could go up for everyone if there is more durable clothing available than Fast Fashion. [2]
• There will be fewer style options. [2 or 4]
• Fewer clothes will be donated to charity. [2]
• Companies will make less profit with durable clothes. [3]
• If companies switch to durable clothes, they might not be as successful as they were before. [3]
• Clothes will be more boring. [4]
Partial Credit
Code 1: Includes a correct possible positive consequence OR a correct possible negative consequence. The other possible
consequence is missing, incorrect, vague, insufficient or irrelevant. The correct consequence must appear in the correct box.
Note: For this item, each response is evaluated independently. Therefore, this coding guide is an exception to the general
principle that an incorrect portion of a response leads to a Code 0.
In this item, students have to think about how one action might affect another within the framing of the principles of ethical
clothing. Four actions by a factory are described. Students need to read each one and identify which one causes a conflict
between two of the principles. The correct answer is C. Switching to a type of cotton that needs minimal water addresses the
third principle of ethical clothing (minimise water use). However, this type of cotton requires large quantities of pesticides, which
violates the second principle (minimise environmental waste and pollution).
PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? » © OECD 2020 415
Annex C Released test units
The last item in this unit describes an experiment that took place in Germany. A vending machine offered T-shirts for only two
euros. However, before the machine dispensed the T-shirt, it presented images of the working conditions where the T-shirt was
made. Then, customers were asked if they wanted to go forward with the purchase or donate the two euros to make clothing
production more ethical. Students learn that in this experiment, nine out of ten customers made the donation. They are then
asked to write in their own words why they think most people chose to make the donation. The test developers came up with
two primary ways to receive credit for this item, both of which required students to take the perspective of the customer who just
learned how the T-shirt was made.
Full Credit
Code 1: Describes a reason for making a donation that refers to an awareness of working conditions in the clothing industry or
how consumer actions affect others.
• The images made people aware of the real cost of the t-shirt.
• The images encouraged people to think about how their actions affect other people.
• It made people realise the t-shirt was cheap because factories take advantage of their workers.
• Because they saw the images and they became aware. [Minimal response: Addresses the concept of awareness, but it
doesn’t specify what the customers became aware of.]
416 © OECD 2020 » PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World?