Measuring COVID-19 Information

Articles
https://doi.org/10.1038/s41562-021-01056-1
Measuring the impact of COVID-19 vaccine

misinformation on vaccination intent in the UK
and USA
Sahil Loomba 1,5, Alexandre de Figueiredo 2,5 ✉, Simon J. Piatek2, Kristen de Graaf2 and
Heidi J. Larson 2,3,4 ✉
Widespread acceptance of a vaccine for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) will be the next major
step in fighting the coronavirus disease 2019 (COVID-19) pandemic, but achieving high uptake will be a challenge and may be
impeded by online misinformation. To inform successful vaccination campaigns, we conducted a randomized controlled trial in
the UK and the USA to quantify how exposure to online misinformation around COVID-19 vaccines affects intent to vaccinate
to protect oneself or others. Here we show that in both countries—as of September 2020—fewer people would ‘definitely’ take
a vaccine than is likely required for herd immunity, and that, relative to factual information, recent misinformation induced a
decline in intent of 6.2 percentage points (95th percentile interval 3.9 to 8.5) in the UK and 6.4 percentage points (95th percen-
tile interval 4.0 to 8.8) in the USA among those who stated that they would definitely accept a vaccine. We also find that some
sociodemographic groups are differentially impacted by exposure to misinformation. Finally, we show that scientific-sounding
misinformation is more strongly associated with declines in vaccination intent.
T
he spread of SARS-CoV-2, the causative agent of COVID-19, have negative consequences for community (herd) immunity, as
has resulted in an unprecedented global public health and eco- clustering of non-vaccinators can disproportionately increase the
nomic crisis1,2. The outbreak was declared a pandemic by the needed percentage of vaccination coverage to achieve herd immu-
World Health Organization on 11 March 20203, and development of nity in adjacent geographical regions and encourage epidemic
COVID-19 vaccines has been a major undertaking in fighting the spread24. Estimates of acceptance of a COVID-19 vaccine in June
disease. As of December 2020, many candidate vaccines have been 2020 suggest that 38% of the public surveyed in the UK and 34.2%
shown to be safe and effective at generating an immune response4–6, of the public in the USA would accept a COVID-19 vaccine (a fur-
with interim analysis of phase III trials suggesting efficacies as high ther 31% and 25% were, respectively, unsure that they would accept
as 95%7–9. At least two vaccine candidates have been authorized for vaccination against COVID-19)25. Worryingly, more recent polling
emergency use in the USA10,11, the UK12,13, the European Union14 in the USA (September 2020) has shown significant falls in will-
and elsewhere, with more candidates expected to follow soon. For ingness to accept a COVID-19 among both males and females,
these COVID-19 vaccines to be successful, they need to be not only all age groups, all ethnicities and all major political groups26, pos-
be proven safe and efficacious, but also widely accepted. sibly due to the heavy politicization of COVID-19 vaccination in
It is estimated that a novel COVID-19 vaccine will need to be the run up to the 2020 presidential election on both sides of the
accepted by at least 55% of the population to provide herd immu- political debate27,28. The public’s willingness to accept a vaccine is
nity, with estimates reaching as high as 85% depending on country therefore not static; it is highly responsive to current information
and infection rate15,16. Reaching these required vaccination levels and sentiment around a COVID-19 vaccine, as well as the state of
should not be assumed given well-documented evidence of vac- the epidemic and perceived risk of contracting the disease. Under
cine hesitancy across the world17, which is often fuelled by online these current plausible COVID-19 vaccine acceptance rates, pos-
and offline misinformation surrounding the importance, safety sible levels of existing protective immunity—though it is unclear
or effectiveness of vaccines18–20. There has been widely circulating whether post-infection immunity confers long-term immunity29—
false information about the pandemic on social media platforms, and the rapidly evolving nature of misinformation surrounding the
such as that 5G mobile networks are linked with the virus, that vac- pandemic23,30, it is unclear whether vaccination will reach the levels
cine trial participants have died after taking a candidate COVID-19 required for herd immunity.
vaccine, and that the pandemic is a conspiracy or a bioweapon21–23. Recent studies have examined the effect of COVID-19 misinfor-
Such information can build on pre-existing fears, seeding doubt and mation on public perceptions of the pandemic22,31,32, the tendency
cynicism over new vaccines, and threatens to limit public uptake of of certain sociopolitical groups to believe misinformation33,34 and
COVID-19 vaccines. compliance with public health guidance, including willingness
While large-scale vaccine rejection threatens herd immunity to accept a COVID-19 vaccine35,36. However, to our knowledge,
goals, large-scale acceptance with local vaccine rejection can also there is no quantitative causal assessment of how exposure to
Department of Mathematics, Imperial College London, London, UK. 2The Vaccine Confidence Project, Department of Infectious Disease Epidemiology,
1
London School of Hygiene and Tropical Medicine, London, UK. 3Department of Health Metrics and Evaluation, University of Washington, Seattle, WA,
USA. 4Centre for the Evaluation of Vaccination, Vaccine and Infectious Disease Institute, University of Antwerp, Antwerp, Belgium. 5These authors
contributed equally: Sahil Loomba, Alexandre de Figueiredo. ✉e-mail: [email protected]; [email protected]
Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav 337

Articles NATURe HUmAn BehAVIOUR
misinformation affects intent to receive the vaccine and its implica- virus (COVID-19) vaccine became available, would you accept the
tions for obtaining herd or community immunity if countries adopt vaccine for yourself?’ (SELF) and ‘If a new coronavirus (COVID-
this vaccination strategy. Moreover, it is essential to understand how 19) vaccine became available, would you accept the vaccine if it
misinformation differentially impacts sociodemographic groups meant protecting friends, family, or at-risk groups?’ (OTHERS).
and whether groups at high risk of developing severe complications Responses were on a four-point scale: ‘yes, definitely’, ‘unsure, but
from COVID-19 are more vulnerable to misinformation. leaning towards yes’, ‘unsure, but leaning towards no’ and ‘no, defi-
To fill this gap, we developed a pre–post-exposure study design nitely not’. This scale was chosen to remove subjective ambiguity
and questionnaire to measure the causal impact of exposure to involved with Likert scales and to allow respondents to explicitly
online pieces of misinformation relating to COVID-19 and vac- detail their intent, thereby allowing a more meaningful interpreta-
cines on the intent to accept a COVID-19 vaccine, relative to fac- tion of results.
tual information. In addition to assessing how misinformation All information (misinformation and factual) was identified
might induce changes in vaccination intent, a further aim of this using Meltwater via a Boolean search string eliciting informa-
study is to investigate how exposure to misinformation differen- tion and misinformation around a COVID-19 vaccine (Methods,
tially impacts individuals according to their sociodemographic ‘Selection of images’). A systematic selection approach was used to
characteristics (age, gender, highest education level, employment identify the COVID-19 vaccine information on social media with
type, religious affiliation, ethnicity, income level and political affili- high circulation and engagement between 1 June and 30 August
ation), daily time spent on social media platforms37, and sources of 2020. Information was classified as misinformation or factual
trusted information on COVID-19. Understanding how misinfor- after consulting reputable online sources of knowledge, such as
mation differentially impacts sociodemographic groups and indi- peer-reviewed scientific research, webpages of public health orga-
viduals according to their social media use or sources of trusted nizations and fact-checking websites (or media outlets employing
information can motivate the design of group-specific interventions fact checkers) to verify the content and the context in which it was
to reduce the potential impact of online vaccine misinformation. presented (Methods ‘Selection of images’). A final set of five pieces
Finally, we assess what makes certain information content more of misinformation comprising non-overlapping messaging and
or less likely to influence intent to accept COVID-19 vaccination, themes was selected to represent the diverse messaging found in
which can be used to increase effectiveness of public health com- COVID-19 vaccine misinformation (such as information question-
munication strategies. ing the importance or safety of a vaccine; Supplementary Table 1).
For both the UK and the USA, both the treatment and control As misinformation can be highly country- and context-dependent,
groups were nationally representative samples by gender, age and it was decided to expose UK and US respondents to different sets
sub-national region. The causal impact of misinformation on vac- of misinformation to reflect the different audiences targeted by the
cination intent was assessed on two key vaccination motives: (1) to sources of misinformation, while factual information was the same
accept a COVID-19 vaccine to protect oneself and (2) to accept a for both groups. Each piece of (mis)information was shown on a
COVID-19 vaccine to protect family, friends and at-risk groups. By separate page to facilitate image comprehension. For each exposure
exploring vaccination intent to protect others, we are able to quan- image, respondents were asked to rate the extent that: they agreed
tify how misinformation may affect altruistic vaccination behav- with the information displayed; they were inclined to be vaccinated;
iour—this is particularly important in the UK and the USA, where they believed the information to be trustworthy; they would fact
altruistic messaging prompts have been a feature of COVID-19 check the information; and they would share the image. After expo-
public health messaging campaigns38–41. sure, the respondents were also asked if they had seen similar con-
Our findings are interpreted in the light of vaccination levels tent on social media in the past month. The full questionnaire is
required for herd immunity, and we discuss messaging strategies shown in the Supplementary Materials.
that may help mitigate or counter the impact of online vaccine mis-
information. Throughout this study, misinformation refers to ‘false Misinformation lowers intent to accept a COVID-19 vaccine.
or misleading information’42, which is ‘considered incorrect based Before treatment, 54.1% (95% percentile interval (PI) 52.5 to 55.7)
on the best available evidence from relevant experts at the time’43. of respondents in the UK and 42.5% (95% PI 41.0 to 44.1) in the
Conversely, factual information refers to information that is con- US reported that they would ‘definitely’ accept a COVID-19 vac-
sidered correct based on the best available evidence from relevant cine to protect themselves; whereas 6.0% (95% PI 5.3 to 6.8) and
experts at the time. 15.0% (95% PI 14.0 to 16.1) said they would ‘definitely not’ accept
a COVID-19 vaccine (Table 1). The remaining respondents were
Results ‘unsure’ about whether they would accept a COVID-19 vaccine
For this study, a total of 8,001 respondents were recruited via an (Table 1). Higher intent to accept a COVID-19 vaccine in the UK
online panel and surveyed between 7 and 14 September 2020—4,000 than the USA has been reported recently25.
in the UK and 4,001 in the USA. Following randomized treatment The treatment of misinformation exposure induces a decrease
assignment, 3,000 UK (and 3,001 US) respondents were exposed in the number of respondents who would ‘definitely’ take the vac-
to misinformation relating to COVID-19 and vaccines (treatment cine relative to the control group in both countries by 6.2 percent-
group) in the UK (and the USA) and 1,000 in each country were age points (95% PI 3.9 to 8.5) in the UK and 6.4 percentage points
shown factual information about COVID-19 vaccines (control (95% PI 4.0 to 8.8) in the USA (Table 1). There are corresponding
group). Figure 1 presents an overview of the study design. increases in some lower-intent response categories. In the UK, we
All respondents in both groups were asked to provide their intent observe an increase of 2.7 percentage points (95% PI 1.0 to 4.5)
to receive a COVID-19 vaccine before and after being exposed to in those ‘unsure, but leaning towards no’ and of 3.3 percentage
vaccine information (misinformation or factual): ‘If a new corona- points (95% PI 2.0 to 4.6) in those saying they ‘definitely will not’
Fig. 1 | Overview of pre- and post-exposure study design. A total of 8,001 participants across the USA and the UK were divided into treatment and control
groups and had their intent to accept a COVID-19 vaccine measured. Respondents were then exposed to either misinformation or factual information
before their vaccination intent was re-recorded. Additional survey items asked respondents to detail the frequency with which they use social media, their
sources of trust for information around COVID-19 and their sociodemographic characteristics. The full questionnaire is reproduced in the Supplementary
Information.
338 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

NATURe HUmAn BehAVIOUR Articles
accept a vaccine, while in the USA there is a rise of 2.3 percentage While these values give the net effect of exposure to misinfor-
points (95% PI 0.7 to 4.0) in those ‘unsure, but leaning towards mation compared with the control, they conceal the full picture
no’ (Table 1). of the four post-exposure responses (Y) conditional on the four
8,001 survey participants in the UK and USA
Treatment and control groups: each
nationally representative by gender,

treatment and control group
age and region 4,000 (UK) 4,001 (USA)
3,000 1,000 3,001 1,000

(treatment) (control) (treatment) (control)
(SELF) if a new coronavirus (COVID-19) vaccine became available,

would you accept the vaccine for yourself?
measurement, W
(OTHERS) if a new coronavirus (COVID-19) vaccine became available,

Pre-exposure
would you accept the vaccine if it meant protecting friends, family, or at-
risk groups?
(1 and 2) answered on an ordinal scale:

Yes, definitely
Unsure, but leaning towards yes
Unsure, but leaning towards no
No, definitely not
Q. SOCIAL MEDIA USE FREQUENCY

(SOC MEDIA) in the past month how much time per day have you spent
Pre-exposure
covariates
actively using social media?
Q. SOURCES OF TRUST FOR COVID-19

(SOURCE TRUST) which sources of information do you trust regarding
COVID-19?
Exposure
group, G
Factual Factual
Misinformation Misinformation
information information
(SELF) if a new coronavirus (COVID-19) vaccine became available,

would you accept the vaccine for yourself?
measurement, Y
Post-exposure
(OTHERS) if a new coronavirus (COVID-19) vaccine became available,

would you accept the vaccine if it meant protecting friends, family, or at-
risk groups?
(Same scale as pre-exposure)
Q. PERCEPTIONS OF EXPOSURE IMAGES

(VACCINE INTENT) more or less inclined to be vaccinated
Image perception and
(AGREE WITH INFORMATION) agree or disagree with information in image

socio-demographic
(HAVE TRUST IN) agree or disagree information in image is trustworthy

covariates
(WILL FACT-CHECK) likely or unlikely to fact-check

(WILL SHARE) likely or unlikely to share image
Q. SOCIO-DEMOGRAPHIC CHARACTERISTICS
Age, gender, education, employment, religion, political affiliation,
ethnicity and income

Table 1 | Exposure to COVID-19 vaccine misinformation reduces intent to accept a COVID-19 vaccine relative to exposure to factually
correct information
Distribution of vaccination Vaccination intent UK USA
intent
Self Others Self Others
Pre-exposure P(W) Yes, definitely 54.1 (52.5, 55.7) 63.7 (62.2, 65.1) 42.5 (41.0, 44.1) 53.3 (51.8, 54.9)
Unsure, lean yes 31.9 (30.5, 33.3) 24.7 (23.4, 26.1) 30.0 (28.6, 31.4) 24.7 (23.3, 26.0)
Unsure, lean no 8.0 (7.1, 8.9) 7.0 (6.2, 7.8) 12.4 (11.4, 13.5) 10.0 (9.2, 11.0)
No, definitely not 6.0 (5.3, 6.8) 4.5 (3.9, 5.2) 15.0 (14.0, 16.1) 12.0 (11.1, 13.1)
Post-exposure P(Y|T) Yes, definitely 48.6 (46.9, 50.2) 55.7 (54.1, 57.4) 39.8 (38.2, 41.5) 46.4 (44.8, 48.1)
(treatment group) Unsure, lean yes 31.1 (29.5, 32.6) 27.9 (26.4, 29.4) 28.7 (27.1, 30.3) 26.7 (25.1, 28.2)
Unsure, lean no 11.3 (10.2, 12.4) 9.6 (8.6, 10.6) 14.0 (12.9, 15.2) 11.7 (10.7, 12.8)
Post-exposure P(Y|C) Yes, definitely 54.8 (52.6, 57.1) 61.5 (59.3, 63.7) 46.3 (44.0, 48.5) 52.9 (50.6, 55.0)
(control group) Unsure, lean yes 30.9 (28.6, 33.2) 25.6 (23.4, 27.9) 25.4 (23.0, 27.8) 22.5 (20.2, 24.9)
Unsure, lean no 8.5 (7.0, 10.2) 7.2 (5.8, 8.7) 12.5 (10.7, 14.4) 11.3 (9.5, 13.1)
Treatment effect Yes, definitely −6.2 (−8.5, −3.9) −5.7 (−7.9, −3.5) −6.4 (−8.8, −4.0) −6.5 (−8.8, −4.1)
Δ(Y) = P(Y|T) − P(Y|C) Unsure, lean yes 0.2 (−2.5, 2.8) 2.3 (−0.3, 4.8) 3.3 (0.6, 5.9) 4.2 (1.5, 6.9)
Unsure, lean no 2.7 (1.0, 4.5) 2.3 (0.7, 4.0) 1.5 (−0.6, 3.5) 0.4 (−1.7, 2.4)
No, definitely not 3.3 (2.0, 4.6) 1.1 (−0.1, 2.3) 1.6 (−0.1, 3.3) 1.9 (0.3, 3.5)
The pre- and post-exposure intent to accept a COVID-19 vaccine are shown for the UK and the USA in the first three rows. The treatment effects of exposure to misinformation relative to factual
information on the post-exposure responses Y that is Δ(Y) are shown in the final row as percentage point changes in the number of people with intent Y after exposure to misinformation relative to factual
information. Model probabilities have been quoted as percentages to aid interpretation. Values in parentheses indicate 95% percentile intervals (PI) with values in bold indicating PIs that do not include 0.
pre-exposure response (W) for the treatment group compared to definitely no’ by 10.0 percentage points (95% PI 2.1 to 18.7). The
the control group, since exposure to information on COVID-19 same substantive results hold for the UK.
vaccines may affect those with different prior vaccination intents
differently. The changes in respondents’ post-exposure response The impact of misinformation by sociodemographic characteris-
stratified by pre-treatment response are shown in Fig. 2 and tics. A Bayesian ordered logistic regression model is used to estab-
Supplementary Table 2, where values indicate the percentage point lish whether the treatment of exposure to misinformation relative to
change in the number of people with prior intent W who change factual information differentially impacted subjects’ intent to accept
intent to Y after exposure to misinformation, relative to factual a vaccine for themselves according to their sociodemographic back-
information (Methods, ‘Estimating treatment effects’). ground. We computed the heterogeneous treatment effects (HTEs),
For any pre-treatment response, there is a net movement denoted by the statistic Δ (equation (6), Methods), which represent
towards the response category immediately below (except for the the impact of exposure to misinformation relative to factual infor-
pre-treatment ‘no, definitely not’ where there is a net increase in mation, for a group of interest relative to its reference group. If Δ is
this response after exposure for the treatment group compared greater than 0, then the treatment of exposure to misinformation
with the control). For example, in the UK there is a net increase induces a lowering of vaccination intent, relative to the control for
of 8.5 percentage point (95% PI 5.5 to 11.4) in the post-exposure a specific group relative to the reference group (male, 18–24 years
response ‘unsure, but leaning towards yes’ for respondents with of age, highest education, employed, Christian, white, Conservative
pre-treatment response ‘yes, definitely’. Similarly, there is a 10.6 (UK) or Republican (USA) and highest income). In Fig. 3, we show
percentage point (95% PI 7.1 to 14.0) increase in the post-exposure this statistic for impact on vaccination intent to protect oneself—
response ‘unsure, but leaning towards no’ for respondents with denoted by ΔS—and to protect others—denoted by ΔO—for each
pre-treatment response ‘unsure, but leaning towards yes’ (Fig. 2). sociodemographic characteristic. (Raw parameter values can be
The same substantive results hold for the USA (Fig. 2). found in Supplementary Tables 3 and 4). Below, we describe only
Interestingly, more respondents in both countries would accept those effects where the 95% PIs exclude zero, which we deem sta-
a vaccine if it meant protecting family, friends or at-risk groups tistically credible. Since the HTEs are computed as a difference
(than if the vaccine was for themselves): 63.7% (95% PI 62.2 to of log cumulative odds ratios between the treatment and control
65.1) of respondents in the UK and 54.1% (95% PI 52.5 to 55.7) groups, we include these statistics separately for the treatment and
in the USA say that they would ‘definitely’ get vaccinated to pro- control groups in Supplementary Figs. 1 and 2 and Supplementary
tect others (Table 1). The exposure to misinformation again induces Tables 3 and 4. Although they do not measure causal effects, these
a decrease in intent to accept the vaccine to protect others, by 5.7 log cumulative odds ratios show how sociodemographic groups
percentage points (95% PI 3.5 to 7.9) in the UK and 6.5 percentage respond to misinformation or factually correct information rela-
points (95% PI 4.1 to 8.8) in the USA (Table 1) for the treatment tive to the reference group undergoing the same treatment. This
group relative to the control. The treatment effects when condi- reveals additional knowledge about those sociodemographic groups
tioned on pre-treatment vaccination intent show a similar picture. which—while not displaying a HTE—may be more inclined than
For instance, in the USA there is a net decrease in those who previ- the reference group to change their vaccination intent in the same
ously responded ‘definitely’ by 8.7 percentage points (95% PI 5.3 direction upon exposure to either kind of information (full model
to 12.1) and a net increase in those who previously responded ‘no, details in Methods, ‘Estimating treatment effects’).

UK self UK others US self US others
Yes, definitely (Y ) * * * *
Yes, definitely
Unsure, lean yes (Y ) * * * *
(W ) Unsure, lean no (Y ) * * * *
No, definitely not (Y ) * *
–10 0 10 –10 0 10 –10 0 10 –10 0 10
* *
Unsure, lean yes
Yes, definitely (Y )
Unsure, lean yes (Y ) * *
(W )
Unsure, lean no (Y ) * * *
–10 0 10 –10 0 10 –10 0 –10 0
Unsure, lean no
Unsure, lean yes (Y ) *
(W )
Unsure, lean no (Y )
No, definitely not (Y ) *
–20 0 20 –10 0 10 –10 0 10 –10 0 10
*
No, definitely not
Unsure, lean yes (Y ) * *
(W )
Unsure, lean no (Y ) * *
–25 0 25 –20 0 20 –10 0 10 0 20
Fig. 2 | Exposure to COVID-19 vaccine misinformation induces a net decrease in intent to accept a COVID-19 vaccine for all levels of pre-exposure
intent. Points indicate the relative change in probabilities (denoted as percentage point changes to aid interpretation) in the number of people with prior
intent W who change it to Y after exposure to misinformation, relative to factual information (Methods, ‘Estimating treatment effects’). Bars indicate 95%
PI; asterisks indicate PIs that do not include 0. Values are presented in Supplementary Table 2.
In both countries, we find evidence that some sociodemographic to lower their vaccination intent when exposed to misinformation
groups are differentially impacted by exposure to misinformation, compared with those in the control group (Supplementary Fig. 3
relative to factual information. In the USA, females are less robust and Supplementary Table 5.) In the UK, individuals who trust celeb-
to misinformation than males when considering vaccination intent rities for information about COVID-19 are more robust to COVID-
to protect others: ΔO = 0.42 (95% PI 0.02 to 0.81). There is also evi- 19 misinformation than those who do not (ΔS = −1.31 (95% PI−2.59
dence that lower-income groups (levels 0 to 2) are less likely to lower to −0.03)), whereas in the USA, individuals who indicated trust in
their vaccination intent to protect themselves or others upon expo- family or friends for such information are less robust than those
sure to misinformation than the highest income group (level 4): for who did not (ΔS = 0.52 (95% PI 0.03 to 1.01)) (Supplementary Fig. 4
level 0, ΔS = −0.83 (95% PI −1.57 to −0.12); level 1, ΔO = −0.65 (95% and Supplementary Table 6.)
PI −1.33 to −0.02); level 2, ΔS = −0.86 (95% PI −1.53 to −0.20) and
ΔO = −0.80 (95% PI −1.48 to −0.13). Interestingly, some groups Correlational evidence of the appeal of scientific misinformation.
respond similarly to misinformation to the reference group but show After exposure to misinformation or factual information, respon-
comparatively different inclinations to vaccinate upon exposure to dents were asked to report whether, for each image: it raised their
factual information. Consequently, such groups are differentially vaccination intent; they agreed with the information presented; they
more robust than their reference counterparts to exposure to misin- found the information to be trustworthy; they were likely to fact
formation relative to factual information, such as those from ‘other’ check; and they were likely to share the image with friends or fol-
ethnic minorities in the USA when compared to whites: ΔS = −0.99 lowers (the full questionnaire and further details are provided in
(95% PI −1.65 to −0.31). Similar results are found in the UK, where the Supplementary Materials). These post-exposure self-reported
unemployed respondents are more robust to misinformation than perceptions for all pieces of (mis)information are depicted in Fig. 4.
employed respondents, with ΔS = −0.99 (95% PI −1.78 to −0.19); Overall, it is apparent that in both countries, respondents were less
‘other’ religious affiliations are more robust to misinformation than likely to agree with, have trust in, fact check, share, or say that the
Christians, with ΔS = −0.76 (95% PI−1.29 to −0.23); and those who information raised their vaccination intent when shown misinfor-
are Jewish are more robust to misinformation than Christians, with mation, as opposed to when they were shown factual information.
ΔO = −1.58 (95% PI−3.14 to −0.02). Across both countries, around a quarter of respondents agreed with
Finally, we investigated whether social media use and trust in some of the misinformation or found it trustworthy, although the
sources of COVID-19 information differentially impacts vaccina- majority of respondents did not agree and did not find it trustwor-
tion intent. We remark that due to the similarity of HTEs obtained thy (Fig. 4a,b).
above for vaccination intent to protect oneself and to protect others, This study was not designed to investigate the causal impact of
we report only analysis of vaccination intent to protect oneself here. different kinds of vaccine messaging and cannot be used to draw
We find no strong evidence to suggest that individuals in the UK causal inferences, as all participants rated self-perceptions after
or the USA who use social media more frequently are more likely exposure. However, access to self-reported perceptions provides

a UK US
b
ΔS self ΔO others N ΔS self ΔO others N
25–34 802 25–34 729
35–44 776 35–44 695
Age
45–54 690 45–54 667

55–64 564 55–64 554
65+ 664 65+ 869
–2 –1 0 –1 0 1 0 1 –1 0 1
Gender
Female 2,283 Female 2,204

Other 25 Other 39
0 2 0 2 –4 –2 0 –2 0
Level 0 178 Level 0 189
Education
Level 1 1,151 Level 1 1,256

Level 3 1,045 Level 3 898
Other 307 Other 251
0 1 0 1 –1 0 0 1
Employment
Unemployed 221 Unemployed 374

Student 198 Student 176
Retired 647 Retired 836
Other 401 Other 459
–1 0 1 –1 0 1 –0.5 0 0.5 –0.5 0 0.5
Jewish 44 Jewish 165
Religion
Muslim 151 Muslim 159

Atheist 1,343 Atheist 381
Other 734 Other 896
–2 0 –2 0 –1 0 –2 0
Labour 1,410
Democrat 1,389
Political
Liberal democrat 307

SNP 153
Other 1,323
Other 845
–1 0 1 0 1 0 0.5 –0.5 0 0.5
Black 136 Hispanic 312

Ethnicity
Black 510
Asian 298
Asian 187
Other 73 Other 287
–1 0 1 –1 0 1 –1 0 –1 0 1

Income

Other 247 Other 231
0 1 –1 0 –1 0 –1 0
Fig. 3 | Sociodemographic determinants of change in vaccination intent upon exposure to misinformation about COVID-19 vaccines, relative to factual
information. a,b, Contribution of sociodemographic characteristics to changes in intent to accept a vaccine to protect oneself (left column) and to
protect others (right column) for the UK (a) and the USA (b). The reference category is male, 18–24 years of age, highest education, employed, Christian,
Conservative (UK) or Republican (USA), white and highest income. Values indicate log cumulative odds ratios, such that a value above 0 indicates that the
group is more likely to reject a COVID-19 vaccine than the reference group upon exposure to misinformation, relative to factual information, and a value
below 0 indicates that they are less likely to reject the vaccine. Bars indicate 95% percentile intervals; numbers on the right indicate sample sizes of the
corresponding demographic. Values are presented in Supplementary Tables 3 and 4.
correlational evidence of which pieces of information are associated ceived contributed the least to declines in vaccination intent was
with a greater (or lesser) decline in vaccination intent. For instance image 3 (Supplementary Table 1), in which the University of Oxford
in the UK, image 1 (Supplementary Table 1)—which suggests that announced that their vaccine “produces a good immune response”
“scientists have expressed doubts […] over the coronavirus vac- and that the “teams @VaccineTrials and @OxfordVacGroup have
cine […] after all of the monkeys used in initial testing contracted found there were no safety concerns”—with 39% respondents in the
coronavirus”—appears to be the misinformation piece that is asso- UK and 35% in theUSA ‘agreeing’ that the image raised their vac-
ciated with the largest decrease in vaccination intent (with 39% of cination intent; see Fig. 4.
respondents ‘disagreeing’ that the image raised their vaccination However, people’s self-reported changes in attitude—such as a
intent (Fig. 4)), whereas in the USA, it was image 1 (Supplementary ‘raise’ in vaccination intent—may mistakenly reflect their absolute
Table 1)—which claimed “the new COVID-19 vaccine will liter- levels of the attitude instead44—that is, level of vaccination intent.
ally alter your DNA”—that seems to induce the most impact (with Therefore, to investigate the association of each individual image
40% of respondents ‘disagreeing’ that the image raised their vacci- (information) with vaccination intent, weights (in the range of 0 to
nation intent (Fig. 4)). In the control set—which was identical for 1) were inferred for each image while regressing self-reported image
respondents in both countries—the image that participants per- perceptions against post-exposure vaccination intent (to protect

Raises vaccine intent Agree with information Have trust in Will fact check Will share
a
19 18 50 46 16 13 47 13 8 23 19 36 13 6 10 8 25 22 32 48 10 22 10 8
19 11 57 46 30 12 41 10 6 34 16 33 9 5 18 8 25 17 30 49 9 21 10 8
UK treatment
14 9 63 56 35 11 38 8 6 41 14 31 75 28 9 28 12 20 50 9 22 9 7
14 12 59 57 28 13 34 15 9 32 15 34 11 7 21 10 27 16 24 49 9 22 10 8
16 9 62 47 35 12 38 76 40 14 31 76 23 8 26 15 27 52 8 22 8 7
b
27 13 41 8 9 17 9 44 13 14 20 11 39 15 12 8 5 26 19 39 32 9 26 14 16
23 11 49 6 9 22 11 40 12 12 27 11 36 12 11 17 7 30 15 30 37 8 27 12 14
US treatment
20 12 49 8 8 17 9 46 14 13 19 11 42 14 12 13 7 30 16 32 34 8 29 13 14
21 11 49 8 9 21 9 41 14 12 24 11 39 13 11 14 6 29 16 33 35 8 29 12 14
23 12 49 7 8 22 9 42 12 12 25 12 37 12 12 15 7 30 15 31 36 8 28 11 14
c
7 7 55 16 13 13 38 36 20 23 33 41 19 5 11 30 30 21 26 14 31 18 9
7 8 60 13 10 25 48 28 16 27 44 31 13 10 11 34 24 19 32 12 30 15 9
UK control
56 47 21 18 23 35 36 22 13 31 39 24 10 10 30 28 19 24 11 32 19 11
7 8 61 13 10 36 49 26 13 48 45 28 12 12 10 33 21 20 31 13 31 14 9
7 8 61 13 8 26 41 32 17 35 39 35 16 12 9 34 24 18 29 12 31 15 10
c
15 10 39 16 17 36 35 26 28 47 32 29 25 57 26 24 36 18 9 27 19 24
15 11 44 13 14 46 38 23 26 5 8 34 27 24 8 7 26 22 34 22 9 25 17 23
US control
15 11 37 14 21 56 32 25 29 45 31 26 31 8 7 25 22 35 16 8 26 19 28
17 10 41 14 16 5 9 36 22 25 6 9 35 25 23 9 7 27 21 33 21 8 27 18 25
16 13 40 13 16 65 35 24 29 6 7 32 23 29 10 8 26 21 33 19 8 26 19 26
Strongly disagree Somewhat disagree Neither Somewhat agree Strongly agree
Fig. 4 | Perceptive attitudes of respondents towards the information they were exposed to. a–d, Bars indicate the breakdown of percentage of respondents
providing a given response to each follow-up question to explore their perceptions of each image (information) they were exposed to. Respondents were
asked whether each image raises their vaccination intent (column 1); contains information they agree with (column 2); contains information they find
trustworthy (column 3); is likely to be fact-checked by them (column 4); and is something they will probably share with others (column 5). Rows represent
images shown to the UK treatment group (a), US treatment group (b), UK control group (c) and US control group (d). Bars in each graph are ordered top
to bottom from images 1 to 5. Those responding with ‘do not know’ were grouped with those saying ‘neither/nor’. The response scale for column 1 has been
inverted—from ‘makes less inclined to vaccinate’ to ‘raises vaccine intent’—to facilitate direct comparison across all questions. The relevant questionnaire
subsection is shown in the Supplementary Information.
oneself) and controlling for pre-exposure intent. This simultane- While other images arguably used some scientific messaging
ously reveals the predictive power of self-reported perceptions on (such as image 5 in Supplementary Table 1, “Big Pharma whistle-
actual change in vaccination intent and quantifies the association blower: ‘97% of corona vaccine recipients will become infertile’”),
of each piece of (mis)information to the change in intent (Methods, the misinformation images identified as having the strongest asso-
‘Estimating image impact’). ciation with decreased vaccination intent presented a direct link
Since exactly five images were shown to each respondent, a between the COVID-19 vaccine and adverse effects and cited arti-
weight above 0.2 would indicate a higher association with lowering cles and scientific imagery or links to articles purporting to be repu-
vaccination intent than what would be expected at random, and a table to strengthen their claim. In the UK, this contrasted with more
weight below 0.2 would indicate a lower association. This analysis memetic imaging (for example, ‘striking images with text superim-
confirms that the misinformation image with the largest (and statis- posed on top’42) which showed far weaker associations (images 3
tically credible) association with loss in vaccination intent in the UK and 4 in Supplementary Table 1).
was indeed image 1 (Supplementary Table 1), with weight 0.42 (95%
PI 0.28 to 0.56), while in the USA it was image 1 (Supplementary Discussion
Table 1), with weight 0.41 (95% PI 0.25 to 0.58). Supplementary Using individual-level survey data collected from nationally repre-
Table 7 presents a full description of these results. sentative samples of 4,000 and 4,001 respondents in the UK and the

USA, respectively, we reveal a number of key findings of importance to effects of misinformation exposure on COVID-19 vaccination
to policymakers and stakeholders engaged in either public health intent.
communication or the design of vaccine-rollout programmes. We Although our study indicates the possible impact of COVID-
find that, as of September 2020, only 54.1% (95% PI 52.5 to 55.7) 19 misinformation campaigns on vaccination intent, this study
of the public in the UK and 42.5% (95% PI 41.0 to 44.1) in the USA does not replicate a real-world social media platform environment
would ‘definitely’ accept a COVID-19 vaccine to protect themselves. where information exposure is a complex combination of what is
These values are lower than the proportion of vaccinated people shown to a person by the platform’s algorithms and what is shared
required to achieve the anticipated herd immunity levels, suggest- by their friends or followers51. Online social network structures,
ing that policymakers may need to convince those unsure about governed by social homophily, can lead to selective exposure and
vaccinating to achieve these levels. Higher proportions of individu- creation of homogeneous echo chambers52,53 and polarization of
als in both countries would ‘definitely’ vaccinate to protect family, opinions, which may amplify (or dampen) the spread of misinfor-
friends and at-risk groups, suggesting that effective altruistic mes- mation among certain demographics. Previous work has shown that
saging may be required to boost uptake. However, we also show that there is evidence of echo chambers on real social media platforms
exposure to misinformation lowers individuals’ intent to vaccinate around information on vaccines, in general54,55. If such informa-
to protect themselves and lowers their altruistic intent to vaccinate tion silos also exist for COVID-19 vaccines, then they may lead to
to protect others, which could complicate messaging campaigns self-selection of misinformation or factual information, inducing
focusing on altruistic behaviours. Campaigns may also have to individuals to become progressively more or less inclined to vacci-
compete with misinformation purporting to be based in science or nate. While our study does not directly quantify such social network
medicine, which appears to be particularly damaging to vaccination effects, it emphasizes on the need to do so further. Furthermore, we
intentions. find correlational evidence that misinformation identified by our
These findings are, however, unlikely to be representative of the participants after exposure as having the most impact on lowering
effect of misinformation on uptake rates in real-world social media their vaccination intent was made to have a scientific appeal, such
settings. Individuals are unlikely to experience misinformation in as emphasizing on a direct link between a COVID-19 vaccine and
the same manner as implemented in this survey, and there will be adverse effects while using scientific imagery or links to strengthen
differences in the volume and rate of misinformation people will their claims. However, our design does not allow causal inferences
be exposed to, depending on their online social media preferences and we were limited in the type and volume of misinformation pre-
and demographics. A demographic re-weighting would be required sented to respondents. Future research should examine the causal
to obtain more robust estimates of anticipated COVID-19 vaccine impact of different types of misinformation and identify whether
rejection at sub-national or national levels. Misinformation may there are other types of misinformation that may be far more
have also already embedded itself in the public’s consciousness, impactful on vaccination intent. Therefore, our estimates for the
and studies have shown that brief exposure to misinformation can losses in vaccination intent due to misinformation must be placed
embed itself into long-term memory45. Policymakers may therefore in the context of this study and the correlational evidence it pro-
find challenges ahead to ‘undo’ the impact it may have already had vides, and caution must be exercised in generalizing these findings
and to clearly communicate messages surrounding the safety, effec- to a real-world setting, which may see larger or smaller decreases in
tiveness, and importance of the vaccine. vaccination intent depending on the wider context of influencing
Treatment with exposure to misinformation is found to differen- factors. Addressing the spread of misinformation will probably be a
tially impact individuals’ intent to vaccinate to protect themselves major component of a successful COVID-19 vaccination campaign,
according to some sociodemographic factors. In the UK, the unem- particularly given that misinformation on social media has been
ployed were more robust to exposure to misinformation compared shown to spread faster than factually correct information56 and that,
with those who are employed (before March 2020). Unemployed even after a brief exposure, misinformation can result in long-term
individuals in the UK were recently found to be less undecided attitudinal and behavioural shifts45,57 that pro-vaccination mes-
about whether to vaccinate than employed groups46. In the USA, saging may find hard to overcome57. With regards to COVID-19,
‘other’ ethnicities and lower-income groups are more robust to mis- misinformation has even been shown to lead to information avoid-
information than those of white ethnicity. There is also evidence ance and less systematic processing of COVID-19 information32;
that exposure to misinformation makes those identifying as Jewish however, the amplification of questionable sources of COVID-19
less likely to lower their vaccination intent to protect others com- misinformation is highly platform dependent, with some platforms
pared with Christians in the UK. In the USA, females are more amplifying questionable content less than reliable content58.
likely than males to lower their intent to vaccinate to protect others In conclusion, this study reveals that as of September 2020, in
upon exposure to misinformation. Many recent studies in both the both the the UK and the USA, fewer people would ‘definitely’ take
UK and the USA have highlighted females as less likely to vaccinate a vaccine than is required for herd immunity, and that misinforma-
than males46–48. tion could push these levels further away from herd immunity tar-
We find no evidence that individuals who trust health authori- gets. This analysis provides a platform to help us test and understand
ties are any more or less likely to be impacted by misinformation how more effective public health communication strategies could
(after controlling for their sociodemographic characteristics); how- be designed and on whom these strategies would have the most
ever, trust in experts has been recently found to be associated with positive impact in countering COVID-19 vaccine misinformation.
intent to pursue COVID-19 vaccine in the USA49. Interestingly,
trust of celebrities in the UK is associated with more robustness to Methods
misinformation compared to controls, whereas trust in family and Ethical approval for this study was obtained by the London School of Hygiene
and Tropical Medicine ethics committee on 15 June 2020 with reference 22647.
friends in the USA is associated with a susceptibility to misinforma- A total of 8,001 respondents recruited via an online panel were surveyed by ORB
tion compared to the control. This result aligns with a recent study (Gallup) International (www.orb-international.com) between 7 and 14 September
that associates trust in non-expert sources with dismissal of misin- 2020. Respondent quotas for each country and each group (that is both treatment
formation relating to vaccine decision making50. Some recent work and control) were set according to national demographic distributions for gender,
suggests that those who consume legacy media several times a day age, and sub-national region—the four census regions in the USA59 and first level
of nomenclature of territorial units in the UK60. Following randomized treatment
and online media less frequently exhibit lower COVID-19 vaccine assignment, 3,000 UK and 3,001 US respondents were exposed to images of
hesitancy than those who consume less of both25. We also find no recently circulating online misinformation related to COVID-19 and vaccines
evidence that daily social media usage is associated with robustness (treatment group) and 1,000 respondents in each country were shown images of

factual information about a COVID-19 vaccine to serve as a randomized control are ordered variables (‘yes, definitely’ (1); ‘unsure, but leaning towards yes’ (2);
(control group). All respondents exposed to misinformation were debriefed after ‘unsure, but leaning towards no’ (3), and ‘no, definitely not’ (4)); treatment group
the survey; debriefing information can be found in the questionnaire included G 2 fT; Cg where T denotes the treatment of exposure to misinformation and
in Supplementary Information. Some respondent characteristics were recoded CI is the control of exposure to factual information; and covariates (for example
to reduce their number and facilitate comparison across the two countries. The sociodemographics) are given by X.
recoding is provided in Supplementary Table 8 and a breakdown of respondents’ Since vaccination intent is modelled as an ordered variable, one can expect the
characteristics is provided in Supplementary Table 9. treatment to impact vaccination intent monotonically. To this end, W is modelled
as a monotonic ordered predictor63,64. Using W as a predictor for Y has two
Selection of images. To elicit responses that can be most readily interpreted advantages: it (1) controls for sampling discrepancies between the treatment and
in light of the current state of online misinformation in both the UK and USA, control groups, and (2) allows for the treatment to differentially affect those with
the information shown to respondents—in the form of snippets of social media different prior vaccination intents. We use ordered logistic regression65 to model Y
posts—should satisfy a number of criteria. It should: (1) be recent and relevant to conditional on G, W and covariates X. Then for an individual respondent i we can
a COVID-19 vaccine; (2) have a high engagement, either through user reach or write:
other publicity, and thus represent information that respondents are not unlikely to W ðiÞ  OrderedLogisticð0; ðκ 1 ; κ2 ; κ 3 ÞÞ; ð1Þ
be exposed to through social media use; (3) include posts shared by organizations
or people with whom respondents are familiar (so that, for example, US and UK
audiences are not shown information from people with whom they are unfamiliar); Y ðiÞjðGðiÞ ¼ g; W ðiÞ; X ðiÞÞ
(4) form a distinct set, not replicating content or core messaging, enabling us to !
WP
ðiÞ�1 � g � g g ð2Þ
probe the most impactful types of misinformation. To this end, we followed a  OrderedLogistic
g
βW
g
δj
g
þ f X ðiÞ; βX ; α1 ; α2 ; α3 :
principled approach to select two sets of five images for the treatment and control j¼1
groups, respectively, combining both quantitative and qualitative methods. P3
g g g g g g
For the treatment set, we used a COVID-19 vaccine-specific Boolean search Where βW 2 R, δj 2 R ≥ 0, such that j¼1 δj ¼ 1, �1 < α1 < α2 < α3 < 1
query—corona* OR coronavirus OR covid* OR ‘wuhan virus’ OR wuhanvirus OR and �1I< κ1 <κ 2I< κ 3 < 1. We use the I distribution for k
I ordered logistic
‘Chinese virus’ OR ‘china virus’ OR chinavirus OR ‘nCoV*’ OR SARS-CoV*) AND outcomes
I specified by Z  OrderedLogisticðβ; ðα1 ; α2 ;    ; αk�1 ÞÞ), where
vaccin* AND (Gates OR 5 G OR microchip OR ‘New World Order’ OR cabal OR PðZ ≤ jÞ ¼ σðαj � βÞ and I σðxÞ ¼ 1=ð1 þ e�x Þ is the standard logistic sigmoid
globali*)—to extract COVID-19 vaccine-related online information from 1 June, function.
I We remark that this I operationalizes a proportional-odds assumption65,
2020 to 30 August, 2020 using Meltwater (www.meltwater.com), an online social wherein the difference in log of cumulative odds ratios between successive
media listening platform. This Boolean search term was based on previous research categories
is independent
ofthe slope β, that is, 8j 2 f2; 3;    ; k � 1g, we have:
that used similar search terms obtaining the highest levels of user engagement ≤ jÞ
log PPððZZ > PðZ ≤ j�1Þ I
jÞ � log PðZ > j�1Þ ¼ αj � αj�1 .
with COVID-19 media and social media articles containing misinformation.
I This modelling framework allows us to model (1) the effect of treatment on
This search string returned over 700,000 social media posts that were initially
vaccination intent and (2) the HTEs through the function f. In estimating (average)
filtered by user engagement and reach to provide the most widely shared and
treatment effects (1), f = 0 (we still need to control for pre-exposure intent W);
viewed posts. Two independent coders (S.J.P. and K.d.G.) screened top posts and
whereas when estimating HTEs (2), we assume:
excluded posts that failed criteria 1–4 above. Some posts had relatively low levels
X g X g
of engagement, but were included because they repeatedly appeared in different f ðX ðiÞÞ ¼ β d ð iÞ þ βuðiÞ ;
formats across different outlets and were thus deemed to be influential on social d2D u2U
media. Reputable online sources of knowledge were consulted to determine which
content was classified as misinformation—that is, information that is regarded where D ¼ fAGE; GEN; EDU; EMP; REL; POL; ETH; INCg refers to the set of
false or misleading according to current expert knowledge. A set of five final posts sociodemographic
I characteristics—of age (AGE), gender (GEN), highest education
were obtained for the UK and the USA, respectively. For instance, misinformation qualification received (EDU), (pre-pandemic) employment status (EMP), religion
selected to be shown to the US sample included a post falsely claiming that a (REL), political affiliation (POL), ethnicity (ETH) and income (INC)—such that
g
COVID-19 vaccine will alter DNA in humans, while that in the UK included a 8d 2 D, d(i) corresponds to the category to which i belongs and βdðiÞ 2 R refers
post falsely claiming that COVID-19 vaccine will cause 97% of recipients to toI the slope for that category. Specification of the set U allows us to I investigate the
become infertile. HTEs for (1) sociodemographic characteristics (U = {}); (2) social media use (while
In determining the ‘control set’, the aim was to expose people to factual controlling for possible confounding effects of sociodemographics); and (3) sources
COVID-19 vaccine information to serve as a control against the treatment of trust (while controlling for possible confounding effects of sociodemographics).
exposure of misinformation, since exposure to any information can in principle We thus investigate HTEs for social media use and sources of trusted information
g
cause respondents to change their vaccine inclination (that is control information about COVID-19 by specifying (1) U = {SOCIAL} (where βSOCIALðiÞ 2 R refers to
controls for other elements of our survey), and respondents may misreport the slope when SOCIAL(i) indicates the category of amountI of daily social media
post-exposure vaccination intent due to recall bias or other between-conditions usage for i) and (2) U ¼ fTRUST1 ; TRUST2 ;    ; TRUSTk g for k different sources
differences. Factual information was obtained by a coder (S.L.), also via Meltwater, of information (where I βgTRUSTk ðiÞ 2 R refers to the slope when TRUSTkðiÞ is the
using the same Boolean search term as above, but excluding the last clause category indicating whether I i trusts the kth source of information:
I 1 for no and 2
containing misinformation-specific search keys, which returned over a hundred for yes). We remark that the model for Y specified in equation (2) is equivalent to a
posts. Reputable online sources of knowledge were consulted to determine which traditional linear two-way interaction model for causal estimation under a binary
content is classified as factual information—that is, information that is regarded treatment, composed with a logistic sigmoid function to model the cumulative
correct as per current expert knowledge. A set of five final posts was obtained, distribution of the ordinal categorical outcome variable66.
common to both the UK and the USA. Information was often from authoritative Regularizing hierarchical priors are placed on all primary model parameters
sources (or otherwise referenced to authoritative sources) such as vaccine groups to aid model identifiability and prevent detection of spurious treatment effects:
g g g
and scientific organizations. We ensured that these five posts were not overtly αj  Normalðαj ; 1Þ, βZ  NormalðβZ ; 1Þ, 8z 2 D ∪ U , βW  NormalðβW ; 1Þ and
pro-vaccination and did not reference anti-vaccination campaigns or materials. I
κIj ; αj ; βZ ; βW  Normalð0; 1Þ. Non-informative I hierarchical
I priors were placed on
For instance, information presented included an update on the current state of δIj : ðδ1 ; δ2 ; δ3 Þ  Dirichletððδ1 ; δ2 ; δ3 ÞÞ and δj  Exponentialð1Þ.
g g g g
COVID-19 vaccine trials; the importance of a vaccine to get out of the COVID-
I I I
19 pandemic; and how a candidate vaccine generates a good immune response. Statistics for measuring treatment effects. We are interested in measuring the
Supplementary Table 1 presents further details regarding the treatment and causal effect of exposure to misinformation (G = T), relative to the control of
control image sets, including detailed explanations for classification of posts as exposure to factual information (G = C), on (post-exposure) vaccination intent Y.
misinformation or factual information. When computing average treatment effects, it would be conventional to calculate
a difference in conditional expectations, that is E[Y|T] − E[Y|C]. However, as Y is
Estimating treatment effects. In this study, the outcome of interest, vaccination an ordered categorical, a conditional expectation has no meaningful interpretation.
intent, is measured on a four-level ordered scale. Using a classical approach in the Therefore, we can compute a conditional probability mass function, P(Y|G), and
potential outcomes framework61,62 to determine treatment effects would either define a corresponding statistic for treatment effect67 on vaccination intent as:
necessitate binarizing the outcome—which can lead to loss of information about
vaccination intent—or making a strong assumption of linearity of the outcome ΔðyÞ  PðY ¼ yjG ¼ TÞ � PðY ¼ yjG ¼ CÞ: ð3Þ
scale. Therefore, a hierarchical Bayesian ordered logistic regression framework is Since treatment may also depend on individuals’ pre-exposure vaccination
used here to estimate the impact of (1) treatment of misinformation on change in intent W, we also compute the statistic:
vaccination intentv relative to factual information, and (2) how these treatments
differentially impact individuals by their sociodemographic characteristics (that ΔW ðy; wÞ  PðY ¼ yjG ¼ T; W ¼ wÞ � PðY ¼ yjG ¼ C; W ¼ wÞ; ð4Þ
is, HTEs). Full model details, including the statistics used to describe all effects,
are detailed below. Throughout, the following notation is used: pre- (W) and for 8ðy; wÞ 2 f1; 2; 3; 4g ´ f1; 2; 3; 4g. Using the ordered logistic
post-exposure (Y) intents to accept a COVID-19 vaccine W; Y 2 f1; 2; 3; 4g regression
I model specified in equations (1) and (2), these two
I
P
statistics are given by Δð yÞ ¼ 4w¼1 μðwÞðρðT; y; wÞ � ρðC; y; wÞÞ to each of the 5 images presented along 5 different perception metrics as features
and ΔW ðy; wÞ ¼ ρðT; y; I wÞ � ρðC; y; wÞ, where
  to learn how each image metric and each image itself contributes to the measured
I wÞ  PðY ¼ yjW ¼ w; G ¼ gÞ ¼ σ αg � βg Pw�1 δg
ρðg;y; drop in vaccination intent. As before, let W denote pre-exposure intent, Y is the
y W j¼1 j
P  post-exposure intent and G is the treatment group. Furthermore, let X(i) refer to the
I αgy�1 � βgW w�1
�σ
g
j¼1 δj and μðwÞ  PðW ¼ wÞ ¼ σðκ w Þ � σðκ w�1 Þ. 5 × 5 matrix such that Xjk(i) refers to the ith individual’s rating on the jth image metric
I The interpretation of equationsI (3) and (4) is as follows. (3): If Δ(y) > 0 for the kth image. Then, the model definition here is very similar to when pursuing
(Δ(y) < 0), then the treatment induces an average individual with vaccination intent HTEs analysis, except the function of covariates now corresponds to an aggregation of
y to not change their vaccination intent (to change their vaccination intent)— ratings across images and image metrics:
relative to control. Alternately, 100 × Δ(y) indicates the percentage point change in
Y ðiÞjðGðiÞ ¼ g; W ðiÞ; X ðiÞÞ
the number of people with intent y after exposure to misinformation, relative to !
factual information. (4): If ΔW ðy; wÞ > 0 ( ΔW ðy; wÞ < 0), then the treatment induces g
WP
ðiÞ�1
g
5 P
P 5
g g � g g g
I change theirI vaccination intent from w to y—relative  OrderedLogistic βW δj þ βj Xjk ðiÞγ k ; α1 ; α2 ; α3 ;
an average individual to (not) j¼1 j¼1 k¼1
to control. Alternately, 100 ´ ΔW ðy; wÞ indicates the percentage point rise or drop
in the number of peopleIwith prior intent w who change it to y after exposure to ð7Þ
misinformation, relative to factual information. g g g P3 g g
where
P5 β W ; β j 2 R, δj 2 R ≥ 0 such that j¼1 δj ¼ 1, γ k 2 R ≥ 0 such that
Statistics for measuring HTEs treatment effects may depend on g g g g
k¼1 γ kI ¼ 1, and �1 I < α1 < α2 < α3 < 1 I . As noted above, I Xjk ðiÞ indicates the
sociodemographic groups: misinformation or factual information may cause Likert
I response of the I ith individual’s rating on the jth image Imetric for the kth
some sociodemographic groups to be more or less likely to vaccinate than others. image. Here, we assume a signed response Xjk ðiÞ 2 f�2; �1; 0; 1; 2g corresponding
Following the conditional probability mass function framework, these HTEs would to the negative and positive ratings of a five-level I Likert scale—those reporting ‘do
correspond to computing the following conditional statistic: not know’ were included in the response category of 0. This allows us to gauge both
g
(1) which images have the most impact (from γ k ) and (2) which image metrics or
PðY ¼ yjG ¼ T; X ¼ xÞ � PðY ¼ yjG ¼ C; X ¼ xÞ: ð5Þ g
features have the most impact (from βj ). I
The image metrics considered are,I in order, whether (1) the respondents
Because we consider many covariates, in the interest of being concise we
perceived the image to have made them less inclined to vaccinate, (2) they agreed
cannot estimate conditional treatment effects for every multivariate combination
with the image, (3) they found the image trustworthy, (4) they were likely to
of covariates. However, some progress can be made by considering the following
fact check the information shown in the image, and (5) they were likely to share
modifications. Firstly, we can compute a different statistic that still permits
the image. Regularizing priors are placed on all primary model parameters:
a form expressed as the linear difference of a function over treatments and g g g g g
αj ; βj ; βW  Normalð0; 1Þ. Non-informative priors are placed on γ k ; δj :
controls separately. In particular, since vaccination intent is ordered, we can g g g g g g g g
define a statistic conveniently in terms of the conditional cumulative distribution ðγI 1 ; γ 2 ; γ 3 ; γ 4 ; γ 5 Þ  Dirichletðð1; 1; 1; 1; 1ÞÞ and ðδ1 ; δ2 ; δ3 Þ  Dirichletðð1;
I 1; 1ÞÞ.
function. More precisely,
consider the negative logarithm of cumulative odds ratio I statistics reported in Supplementary Table 7 refer
The I g g g
to βj and γ k . If βj > 0
PðY ≤ yjG¼g;X¼xÞ g
θðg; x; yÞ  �log PðY > yjG¼g;X¼xÞ , which indicates how likely an individual x is
( βj < 0), then a higher (lower) rating on the jth metric is more I predictive
I I of a
toI have a vaccination intent up to level y after exposure to misinformation (G = T) drop
I in vaccination intent in treatment group g, after exposure. Since five images
g g
or factual information (G = C). The larger this statistic is for given y, the less likely were shown to each respondent, γ k > 0:2 ( γ k < 0:2) indicates that the kth image
the individual x in group g is to have a high vaccination intent. Given the ordered contributes more (less) to the dropI in vaccination I intent in treatment group g, after
logistic model specification in equations (1) and (2), and by considering this exposure, than what would be expected at random.
estimand on the latent scale of log of cumulative odds ratio, one can use this latent
continuous variable θ as the de-facto outcome instead of Y (ref. 68). Estimands Statistical inference. Model inference was performed by Hamiltonian Monte
on the latent scale are more difficult to interpret due to non-identifiability of the Carlo with the NUTS sampler using PyStan69, the Python implementation of Stan.
function mapping θ to Y, but the use of regularizing priors in the model makes Samples from the posterior distribution of the model parameters were collected
the function identifiable. Secondly, since we are interested in finding whether a from 4 chains and 2,000 iterations (that is, 4,000 samples excluding warm-up) after
group is more or less susceptible to treatment effects, we can do so by picking ensuring model convergence, with the potential scale reduction factor satisfying
a reference group x0, that is, we compute a difference of conditional treatment ^ ≤1:02 for all model parameters, while ensuring that the smallest effective sample
R
effects when X = x relative to when X = x0. Therefore, we are interested in a statistic I for all model parameters is greater than 500 (refs. 70,71) (Supplementary Table
size
which is the difference in log cumulative odds ratios (or simply log odds ratios), 10). The target average proposal acceptance probability for the NUTS sampler was
ηðg; x; x0 Þ  θðg; x; yÞ � θðg; x0 ; yÞ. This leads to the following (relative) measure set to 0.9 and increased to 0.99 to remove any divergent transitions if they were
for
I heterogeneous effects: encountered. The maximum tree depth for the sampler was set to 10 but increased
to 15 if the limit was reached for any model. Relevant statistics for parameters of
ΔX ðx; x0 Þ  ηðT; x; x0 Þ � ηðC; x; x0 Þ: ð6Þ interest (coefficients, contrasts, log odds ratios, percentages and weights) were
extracted from the samples, and all results report the mean estimate—the effect
Given the model definition in equations
(1) and (2), the statistic η is size—alongside 95% PIs (that is, values at 2.5% and 97.5% percentiles) to indicate
g g
given by ηðg; x; x0 Þ ¼ f ðx; βX Þ � f x0 ; βX , which is simply given by a credible values of the statistic.
differenceIin the log cumulative odds ratios. For example, the statistic of
g g
difference in log odds for gender is ηðg; Female; MaleÞ ¼ βFemale � βMale Reporting Summary. Further information on research design is available in the
and the corresponding heterogeneous I effect is given by Nature Research Reporting Summary linked to this article.
ΔX ðFemale; MaleÞ ¼ ðβTFemale � βCFemale Þ � ðβTMale � βCMale Þ. In this model,
parameters
I exist for every sociodemographic group—with regularizing priors
ensuring identifiability—allowing for posterior contrast distributions of η Data availability
and Δ with regards to any reference group x0. In our analysis, for categorical The data used in this study are available at https://github.com/sloomba/
characteristics we pick the most populated— ‘employed’ for employment, covid19-misinfo/. A copy of the materials used in this study, as displayed to
‘Christian’ for religion, ‘white’ for ethnicity—or second-to-most populated respondents, can be obtained from the authors upon request. Source data are
group— ‘male’ for gender, ‘Conservative’ (UK) or ‘Republican’ (USA) for political provided with this paper.
affiliation—as the reference group, which allows for a natural comparison of
minority groups against the majority. For ordinal characteristics, we pick one of Code availability
the end groups as the reference group— ‘18–24’ (lowest) for age, ‘level 4’ (highest) The code developed for this study is available at https://github.com/sloomba/
for income, ‘level 4’ (highest) for education, ‘none’ (lowest) for social media use— covid19-misinfo/.
which allows for a natural comparison to the extrema of the characteristic. For
binary characteristics, we pick the null group as the reference—indicating no trust Received: 19 October 2020; Accepted: 19 January 2021;
in a source of COVID-19 information.
Published online: 5 February 2021
The interpretation of the HTE (equation (6)) is as follows. If ΔX ðx; x0 Þ >0
( ΔX ðx; x0 Þ <0), then the treatment makes an individual of group xI more (less)
likely
I to move from lower vaccine hesitancy to a higher one when compared to an References
individual of group x0—relative to control. The interpretation of equation (5) is as 1. Andersen, K. G., Rambaut, A., Lipkin, W. I., Holmes, E. C. & Garry, R. F. The
follows: if ηðg; x; x0 Þ > 0 ( ηðg; x; x0 Þ < 0), then the exposure within treatment group proximal origin of SARS-CoV-2. Nat. Med. 26, 450–452 (2020).
g makes an Iindividual of group I x more (less) likely to move from lower vaccine 2. Zhou, P. et al. A pneumonia outbreak associated with a new coronavirus of
hesitancy to a higher one when compared to an individual of group x0. probable bat origin. Nature 579, 270–273 (2020).
3. World Health Organization. WHO Director-General’s opening remarks at the
Estimating image impact. To study which images, corresponding to misinformation media briefing on COVID-19–11 March 2020 https://www.who.int/
or factual information, are perceived by participants to induce a larger drop in director-general/speeches/detail/who-director-general-s-opening-remarks-
vaccination intent upon exposure, we make use of ratings given by the respondents at-the-media-briefing-on-covid-19---11-march-2020 (2020).

4. Jackson, L. A. et al. An mRNA vaccine against SARS-CoV-2—preliminary 29. Altmann, D. M., Douek, D. C. & Boyton, R. J. What policy makers need to
report. N. Engl. J. Med. 383, 1920–1931 (2020). know about COVID-19 protective immunity. Lancet 395, 1527–1529 (2020).
5. Mulligan, M. J. et al. Phase I/II study of COVID-19 RNA vaccine BNT162b1 30. Zarocostas, J. How to fight an infodemic. Lancet 395, 676 (2020).
in adults. Nature 586, 589–593 (2020). 31. Islam, M. S. et al. COVID-19–related infodemic and its impact on
6. Folegatti, P. M. et al. Safety and immunogenicity of the ChAdOx1 nCoV-19 public health: a global social media analysis. Am. J. Trop. Med. Hyg. 103,
vaccine against SARS-CoV-2: a preliminary report of a phase 1/2, 1621–1629 (2020).
single-blind, randomised controlled trial. Lancet 396, 467–478 (2020). 32. Kim, H. K., Ahn, J., Atkinson, L. & Kahlor, L. A. Effects of COVID-19
7. Pfizer. Pfizer and BioNTech conclude phase 3 study of COVID-19 vaccine misinformation on information seeking, avoidance, and processing: a
candidate, meeting all primary efficacy endpoints https://www.pfizer.com/ multicountry comparative study. Sci Commun 42, https://doi.
news/press-release/press-release-detail/pfizer-and-biontech-conclude-phase- org/10.1177/1075547020959670 (2020).
3-study-covid-19-vaccine (2020). 33. Kreps, S. E. & Kriner, D. Medical misinformation in the COVID-19
8. US National Institute of Allergy and Infectious Diseases. Promising interim pandemic. SSRN https://doi.org/10.2139/ssrn.3624510 (2020).
results from clinical trial of NIH-Moderna COVID-19 vaccine https://www. 34. Murphy, J., Vallières, F. & Bentall, R.P. et al. Psychological characteristics
nih.gov/news-events/news-releases/ associated with COVID-19 vaccine hesitancy and resistance in Ireland and
promising-interim-results-clinical-trial-nih-moderna-covid-19-vaccine (2020). the United Kingdom. Nat. Commun. 12, 29 (2021).
9. Voysey, M. et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine 35. Roozenbeek, J. et al. Susceptibility to misinformation about COVID-19
(AZD1222) against SARS-CoV-2: an interim analysis of four randomised around the world. R. Soc. Open Sci. 7, 201199 (2020).
controlled trials in Brazil, South Africa, and the UK. Lancet https://doi. 36. Romer, D. & Jamieson, K. H. Conspiracy theories as barriers to controlling
org/10.1016/S0140-6736(20)32661-1 (2020). the spread of COVID-19 in the U.S. Soc. Sci. Med. 263, 113356 (2020).
10. US Food and Drug Administration. FDA takes key action in fight against 37. Ernala, S. K., Burke, M., Leavitt, A. & Ellison, N. B. How well do people
COVID-19 By issuing emergency use authorization for first COVID-19 report time spent on Facebook? In Proc. 2020 CHI Conference on Human
vaccine https://www.fda.gov/news-events/press-announcements/ Factors in Computing Systems (ACM, 2020); https://doi.
fda-takes-key-action-fight-against-covid- org/10.1145/3313831.3376435
19-issuing-emergency-use-authorization-first-covid-19 (2020). 38. National Health Service England. Protect your loved ones https://www.nhs.
11. US Food and Drug Administration. FDA takes additional action in fight uk/apps-library/nhs-covid-19/ (2020).
against COVID-19 by issuing emergency use authorization for second 39. National Health Service England. We must keep on protecting each other
COVID-19 vaccine https://www.fda.gov/news-events/press-announcements/ https://coronavirusresources.phe.gov.uk/Hands-Face-Space-/resources/
fda-takes-additional-action-fight-against-covid- co-branded-posters/ (2020).
19-issuing-emergency-use-authorization-second-covid (2020). 40. US Centers for Disease Control and Prevention. Do it for yourself and your
12. UK Medicines and Healthcare products Regulatory Agency. UK medicines friends. https://www.cdc.gov/coronavirus/2019-ncov/downloads/
regulator gives approval for first UK COVID-19 vaccine https://www.gov.uk/ Young-Mitigation-recommendations-resources-toolkit-COL.pdf (2020).
government/news/uk-medicines-regulator-gives-approval-for-first-uk-covid- 41. US Centers for Disease Control and Prevention. Wear a mask to protect you
19-vaccine (2020). and your friends. https://www.cdc.gov/coronavirus/2019-ncov/downloads/
13. UK Department of Health and Social Care. Oxford University/AstraZeneca Young-Mitigation-recommendations-resources-toolkit.pdf (2020).
vaccine authorised by UK medicines regulator https://www.gov.uk/ 42. Lazer, D. M. J. et al. The science of fake news. Science 359, 1094–1096 (2018).
government/news/oxford-universityastrazeneca-vaccine-authorised-by-uk-medicines- 43. Vraga, E. K. & Bode, L. Defining misinformation and understanding its
regulator (2020). bounded nature: Using expertise and evidence for describing misinformation.
14. European Medicines Agency. EMA recommends first COVID-19 vaccine for Political Commun. 37, 136–144 (2020).
authorisation in the EU https://www.ema.europa.eu/en/news/ 44. Graham, M. & Coppock, A. Asking about attitude change. Public Opinion
ema-recommends-first-covid-19-vaccine-authorisation-eu (2020). Quarterly (in the press).
15. Kwok, K. O., Lai, F., Wei, W. I., Wong, S. Y. S. & Tang, J. W. T. Herd 45. Zhu, B. et al. Brief exposure to misinformation can lead to long-term false
immunity—estimating the level required to halt the COVID-19 epidemics in memories. Appl. Cogn. Psychol. 26, 301–307 (2012).
affected countries. J. Infect. 80, e32–e33 (2020). 46. Paul, E., Steptoe, A. & Fancourt, D. Attitudes towards vaccines and intention
16. Sanche, S. et al. High contagiousness and rapid spread of severe acute to vaccinate against COVID-19: implications for public health
respiratory syndrome coronavirus 2. Emerg. Infect. Dis. 26, 1470–1477 (2020). communications. Lancet Reg. Health Eur. https://doi.org/10.1016/j.
17. de Figueiredo, A., Simas, C., Karafillakis, E., Paterson, P. & Larson, H. J. lanepe.2020.100012 (2020).
Mapping global trends in vaccine confidence and investigating barriers to 47. Malik, A. A., McFadden, S. M., Elharake, J. & Omer, S. B. Determinants of
vaccine uptake: a large-scale retrospective temporal modelling study. Lancet COVID-19 vaccine acceptance in the US. EClinicalMedicine 26, 100495
396, 898–908 (2020). (2020).
18. Bellaby, P. Communication and miscommunication of risk: understanding 48. de Figueiredo, A. Sub-national forecasts of COVID-19 vaccine acceptance
UK parents’ attitudes to combined MMR vaccination. BMJ 327, across the UK: a large-scale cross-sectional spatial modelling study. Preprint
725–728 (2003). at medRxiv https://doi.org/10.1101/2020.12.17.20248382 (2020).
19. Burki, T. Vaccine misinformation and social media. Lancet Digit. Health 1, 49. Callaghan, T. et al. Correlates and disparities of COVID-19 vaccine hesitancy.
E258–E259 (2019). SSRN https://doi.org/10.2139/ssrn.3667971 (2020).
20. Lo, N. C. & Hotez, P. J. Public health and economic consequences of vaccine 50. Motta, M., Callaghan, T. & Sylvester, S. Knowing less but presuming more:
hesitancy for measles in the United States. JAMA Pediatr. 171, 887–892 Dunning–Kruger effects and the endorsement of anti-vaccine policy attitudes.
(2017). Soc. Sci. Med. 211, 274–281 (2018).
21. Megget, K. Even COVID-19 can’t kill the anti-vaccination movement. BMJ 51. Bakshy, E., Messing, S. & Adamic, L. A. Exposure to ideologically diverse
369, m2184 (2020). news and opinion on Facebook. Science 348, 1130–1132 (2015).
22. Geldsetzer, P. Knowledge and perceptions of COVID-19 among the general 52. Del Vicario, M. et al. The spreading of misinformation online. Proc. Natl
public in the United States and the United Kingdom: a cross-sectional online Acad. Sci. USA 113, 554–559 (2016).
survey. Ann. Intern. Med. 173, 157–160 (2020). 53. Del Vicario, M., Quattrociocchi, F., Scala, S. & Zollo, F. Polarization and fake
23. Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G. & Rand, D. G. Fighting news: early warning of potential misinformation targets. ACM Trans. Web 13,
COVID-19 misinformation on social media: experimental evidence for a 10 (2019).
scalable accuracy-nudge intervention. Psychol. Sci. 31, 770–780 (2020). 54. Schmidt, A. L., Zollo, F., Scala, A., Betsch, C. & Quattrociocchi, W.
24. Salathé, M. & Bonhoeffer, S. The effect of opinion clustering on disease Polarization of the vaccination debate on Facebook. Vaccine 36, 3606–3612
outbreaks. J. R. Soc. Interface 5, 1505–1508 (2008). (2018).
25. McAndrew, S. & Allington, D. Mode and frequency of COVID-19 55. Cossard, A. et al. Falling into the echo chamber: the Italian vaccination
information updates, political values, and future covid-19 vaccine attitudes. debate on Twitter. In Proc. 14th International AAAI Conference on Web and
Preprint https://doi.org/10.31234/osf.io/j7srx (2020). Social Media 130–140 (AAAI, 2020).
26. Tyson, A., Johnson, C. & Funk, C. U.S. public now divided over whether to 56. Vosoughi, S., Roy, D. & Aral, S. The spread of true and false news online.
get COVID-19 vaccine. Pew Research Center https://www.pewresearch.org/sci Science 359, 1146–1151 (2018).
ence/2020/09/17/u-s-public-now-divided-over-whether-to-get-covid- 57. Pluviano, S., Watt, C. & della Sala, S. Misinformation lingers in memory:
19-vaccine/ (2020). failure of three pro-vaccination strategies. PLOS ONE 12, e0181640 (2017).
27. Peretti-Watel, P. et al. A future vaccination campaign against COVID-19 58. Cinelli, M. et al. The COVID-19 social media infodemic. Sci. Rep. 10, 16598
at risk of vaccine hesitancy and politicisation. Lancet Infect. Dis. 20, (2020).
769–770 (2020). 59. US Census Bureau. Census regions and divisions of the United States.
28. Galvão, J. COVID-19: the deadly threat of misinformation. Lancet Infect. Dis. https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf
https://doi.org/10.1016/S1473-3099(20)30721-0 (2020). (2010).

60. An Overview of the 3 NUTS and 2 LAU Layers in the UK (UK Office for study and had final responsibility for the decision to submit for publication. The Vaccine
National Statistics, accessed September 2020); https://www.ons.gov.uk/ Confidence Project acknowledges its long partnership and collaboration with ORB
methodology/geography/ukgeographies/eurostat (Gallup) International who collected all data for this study.
61. Rubin, D. B. Estimating causal effects of treatments in randomized and
nonrandomized studies. J. Educ. Psychol. 66, 688–701 (1974). Author contributions
62. Holland, P. W. Statistics and causal inference. J. Am. Stat. Assoc. 81, 945–960 All authors contributed to questionnaire design, with A.d.F. obtaining ethical approval
(1986). for the study via the London School of Hygiene and Tropical Medicine ethics committee.
63. Bürkner, P. & Charpentier, E. Modelling monotonic effects of ordinal S.L. and A.d.F. designed the statistical analyses. S.L. performed all model calculations
predictors in Bayesian regression models. Br. J. Math. Stat. Psychol. 73, and inference and created all figures for publication. S.J.P., K.d.G. and S.L. performed
42–451 (2020). image selection. A.d.F. and S.L. wrote the final manuscript with input from all authors.
64. McElreath, R. Statistical Rethinking: A Bayesian Course with Examples in R All authors contributed to the interpretation of the results. A.d.F. and H.J.L. supervised
and STAN (CRC Press, 2020). the project.
65. McCullagh, P. Regression models for ordinal data. J. R. Stat. Soc. 42, 109–142
(1980).
66. Wang, C., Dominici, F., Parmigiani, G. & Zigler, C. M. Accounting for Competing interests
uncertainty in confounder and effect modifier selection when estimating H.J.L. and A.d.F. are involved in Vaccine Confidence Project collaborative grants
average causal effects in generalized linear models. Biometrics 71, 654–6565 with GlaxoSmithKline, Janssen and Merck. S.J.P. and H.J.L. are involved with Vaccine
(2015). Confidence Project collaborative grants with Janssen. H.J.L. has also received
67. Boes, S. Nonparametric analysis of treatment effects in ordered response honoraria as a member of the Merck Vaccine Confidence Advisory Board and
models. Empir. Econ. 44, 81–109 (2013). GlaxoSmithKline advisory roundtables. H.J.L. is a member of the Merck Vaccine
68. Volfovsky, A., Airoldi, E. M. & Rubin, D. B. Causal inference for ordinal Confidence Advisory Board.
outcomes. Preprint at arXiv https://arxiv.org/abs/1501.01234 (2015).
69. PyStan: the Python Interface to Stan Version 2.17.1.0 (Stan Development
Team, 2019). Additional information
70. Gelman, A. & Rubin, D. B. Inference from iterative simulation using multiple Supplementary information The online version contains supplementary material
sequences. Stat. Sci. 7, 457–472 (1992). available at https://doi.org/10.1038/s41562-021-01056-1.
71. Vehtari, A., Gelman, A., Simpson, D., Carpenter, B. & Bürkner, P.-C. Correspondence and requests for materials should be addressed to A.d.F. or H.J.L.
Rank-normalization, folding, and localization: an improved Ȓ for assessing Peer review information Nature Human Behaviour thanks Paul Bürkner, Matt Motta
convergence of MCMC. Bayesian Anal. https://doi.org/10.1214/20-BA1221 and the other, anonymous, reviewer(s) for their contribution to the peer review of this
(2020). work. Primary Handling Editor: Stavroula Kousta.
Reprints and permissions information is available at www.nature.com/reprints.
Acknowledgements
This project was funded by the United Nations’ Verified Initiative. The funders had Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
no role in data collection, questionnaire design, data analysis, data interpretation or published maps and institutional affiliations.
writing of this report. The corresponding authors had full access to all the data in the © The Author(s), under exclusive licence to Springer Nature Limited 2021

nature research | reporting summary
Corresponding author(s): A. de Figueiredo & H. J. Larson
Last updated by author(s): Jan 4, 2021
Reporting Summary
Nature Research wishes to improve the reproducibility of the work that we publish. This form provides structure for consistency and transparency
in reporting. For further information on Nature Research policies, see our Editorial Policies and the Editorial Policy Checklist.
Statistics
For all statistical analyses, confirm that the following items are present in the figure legend, table legend, main text, or Methods section.
n/a Confirmed
The exact sample size (n) for each experimental group/condition, given as a discrete number and unit of measurement
A statement on whether measurements were taken from distinct samples or whether the same sample was measured repeatedly
The statistical test(s) used AND whether they are one- or two-sided
Only common tests should be described solely by name; describe more complex techniques in the Methods section.
A description of all covariates tested

A description of any assumptions or corrections, such as tests of normality and adjustment for multiple comparisons
A full description of the statistical parameters including central tendency (e.g. means) or other basic estimates (e.g. regression coefficient)
AND variation (e.g. standard deviation) or associated estimates of uncertainty (e.g. confidence intervals)
For null hypothesis testing, the test statistic (e.g. F, t, r) with confidence intervals, effect sizes, degrees of freedom and P value noted
Give P values as exact values whenever suitable.
For Bayesian analysis, information on the choice of priors and Markov chain Monte Carlo settings
For hierarchical and complex designs, identification of the appropriate level for tests and full reporting of outcomes
Estimates of effect sizes (e.g. Cohen's d, Pearson's r), indicating how they were calculated
Our web collection on statistics for biologists contains articles on many of the points above.
Software and code

Policy information about availability of computer code
Data collection n/a
Data analysis
Python version 3.7.3 was used for all analysis with the following libraries:
-- pystan
-- pandas
-- numpy
-- matplotlib
The custom code for this project is available at https://github.com/sloomba/covid19-misinfo/.

For manuscripts utilizing custom algorithms or software that are central to the research but not yet described in published literature, software must be made available to editors and
reviewers. We strongly encourage code deposition in a community repository (e.g. GitHub). See the Nature Research guidelines for submitting code & software for further information.
Data
April 2020
Policy information about availability of data

All manuscripts must include a data availability statement. This statement should provide the following information, where applicable:
- Accession codes, unique identifiers, or web links for publicly available datasets
- A list of figures that have associated raw data
- A description of any restrictions on data availability
The data used in this study are available at https://github.com/sloomba/covid19-misinfo/.
1
nature research | reporting summary
Field-specific reporting
Please select the one below that is the best fit for your research. If you are not sure, read the appropriate sections before making your selection.
Life sciences Behavioural & social sciences Ecological, evolutionary & environmental sciences
For a reference copy of the document with all sections, see nature.com/documents/nr-reporting-summary-flat.pdf
Behavioural & social sciences study design

All studies must disclose on these points even when the disclosure is negative.
Study description Quantitative cross-sectional study
Research sample Nationally representative (by age, sex, subnational region) samples are obtained via online panels of participants in the UK and US.
4,000 respondents in both countries are surveyed to quantify national-level summaries of individuals' vaccine intent and
susceptibility of misinformation. All respondents surveyed were 18 or over and informed consent was obtained by ORB (Gallup)
International.
Sampling strategy Survey participants were recruited via online survey panels by ORB (Gallup) International using fixed quotas at the national level for
sex, age, and sub-national region.
Data collection n/a
Timing The survey was in field between 7 and 14 September, 2020.
Data exclusions No data was excluded from the analyses.
Non-participation Individuals who did not wish to participate simply did not complete the survey. We (the authors) do not have access to this
information as ORB (Gallup) International provided a quota-matched sample of respondents.
Randomization Of the 4,000 respondents in each country, 75% were exposed to misinformation, while 25% were exposed to factual information.
Reporting for specific materials, systems and methods

We require information from authors about some types of materials, experimental systems and methods used in many studies. Here, indicate whether each material,
system or method listed is relevant to your study. If you are not sure if a list item applies to your research, read the appropriate section before selecting a response.
Materials & experimental systems Methods

n/a Involved in the study n/a Involved in the study
Antibodies ChIP-seq
Eukaryotic cell lines Flow cytometry
Palaeontology and archaeology MRI-based neuroimaging
Animals and other organisms
Human research participants
Clinical data
Dual use research of concern
Human research participants

Policy information about studies involving human research participants
Population characteristics See above
Recruitment Participants were recruited via an online panel of respondents (see above). There may be technological literacy biases
April 2020
associated with this sample, however, we do control for highest level of education.
Ethics oversight Approval for this study was obtained by the LSHTM ethics committee on 15 June 2020 with reference 22130.
Note that full information on the approval of the study protocol must also be provided in the manuscript.

Measuring COVID-19 Information

Uploaded by

Copyright:

Available Formats

Measuring COVID-19 Information

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Measuring COVID-19 Information

Uploaded by

Copyright:

Available Formats

Articles

Measuring the impact of COVID-19 vaccine

Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav 337

338 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

8,001 survey participants in the UK and USA

Treatment and control groups: each

nationally representative by gender,

age and region 4,000 (UK) 4,001 (USA)

3,000 1,000 3,001 1,000

(SELF) if a new coronavirus (COVID-19) vaccine became available,

(OTHERS) if a new coronavirus (COVID-19) vaccine became available,

(1 and 2) answered on an ordinal scale:

Q. SOCIAL MEDIA USE FREQUENCY

actively using social media?

Q. SOURCES OF TRUST FOR COVID-19

(SELF) if a new coronavirus (COVID-19) vaccine became available,

(OTHERS) if a new coronavirus (COVID-19) vaccine became available,

(Same scale as pre-exposure)

Q. PERCEPTIONS OF EXPOSURE IMAGES

(AGREE WITH INFORMATION) agree or disagree with information in image

(HAVE TRUST IN) agree or disagree information in image is trustworthy

(WILL FACT-CHECK) likely or unlikely to fact-check

Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav 339

340 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav 341

45–54 690 45–54 667

Female 2,283 Female 2,204

Level 1 1,151 Level 1 1,256

Unemployed 221 Unemployed 374

Muslim 151 Muslim 159

Liberal democrat 307

Black 136 Hispanic 312

Level 0 584 Level 0 561

Level 2 748 Level 2 625

342 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

Strongly disagree Somewhat disagree Neither Somewhat agree Strongly agree

Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav 343

344 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

346 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav 347

348 Nature Human Behaviour | VOL 5 | March 2021 | 337–348 | www.nature.com/nathumbehav

A description of all covariates tested

Software and code

The custom code for this project is available at https://github.com/sloomba/covid19-misinfo/.

Policy information about availability of data

The data used in this study are available at https://github.com/sloomba/covid19-misinfo/.

Behavioural & social sciences study design

Data collection n/a

Timing The survey was in field between 7 and 14 September, 2020.

Data exclusions No data was excluded from the analyses.

Reporting for specific materials, systems and methods

Materials & experimental systems Methods

Human research participants

You might also like