HLS19 International Report

International Report
on the Methodology, Results, and

Recommendations of the European
Health Literacy Population Survey
2019-2021 (HLS19) of M-POHL
Report
The HLS19 Consortium of the WHO Action Network M-POHL

International Report
on the Methodology, Results, and
Recommendations of the
European Health Literacy Population
Survey 2019-2021 (HLS19) of
M-POHL
Report
Authors: The HLS19 Consortium of the WHO Action Network M-POHL
represented by the members of the ICC and the PIs and PRs of the 17 participating countries:
ICC: Jürgen M. Pelikan (International PI), Christa Straßmayr (Coordination), Thomas Link,
Dominika Mikšová, Peter Nowak; Austria: Robert Griebler, Christina Dietscher; Belgium: Stephan
Van den Broucke, Rana Cheraffedine; Bulgaria: Antoniya Yanakieva, Nygyar Dzhafer; Czech
Republic: Zdeněk Kučera, Alena Šteflová; Denmark: Henrik Bøggild, Andreas Jull Sørensen;
France: Julien Mancini, Geneviève Chêne; Germany: Doris Schaeffer, Alexander Schmidt-Gernig;
Hungary: Éva Bíró, Péter Csizmadia; Ireland: Lucy Bruton, Sarah Gibney; Israel: Diane Levin-
Zamir, Orna Baron-Epel; Italy: Luigi Palmieri, Daniela Galeone; Norway: Kjell Sverre Pettersen,
Christopher Le; Portugal: Andreia Silva da Costa, Miguel Telo de Arriaga; Russian Federation:
Maria Lopatina, Oxana Drapkina; Slovakia: Zuzana Klocháňová; Slovenia: Mitja Vrdelja, Tamara
Štemberger Kolnik; Switzerland: Saskia De Gani and Karin Gasser.
Chapters were written by author teams, which are named at the beginning of each individual
chapter.
Vienna, December 2021

M-POHL chairs
Christina Dietscher (Policy) Jürgen M. Pelikan (Research)
ICC team members who contributed to the HLS19 Project

Jürgen M. Pelikan (International PI) Christa Straßmayr (Coordination) Thomas Link
Dominika Mikšová Peter Nowak
Julia Bobek Bettina Engel Lisa Mayer
Tobias Alfers Sandra Peer Karin Waldherr
National HLS19 team members who contributed to the HLS19 Project
Austria
Robert Griebler (PI) Christina Dietscher (PR) Thomas Link
Dominika Mikšová Peter Nowak Christa Straßmayr
Belgium
Stephan Van den Broucke (PI) Rana Cheraffedine (PR)
Bulgaria
Antoniya Yanakieva (PI) Nygyar Dzhafer (PR) Todor Kundurdzhiev
Karolina Liubomirova Elisaveta Petrova-Geretto Alexandrina Vodenicharova
Czech Republic
Zdeněk Kučera (PI) Alena Šteflová (PR)
Denmark
Henrik Bøggild (PI) Andreas Jull Sørensen (PR) Anna Aaby
Rasmus Baagland Carsten Kronborg Bak Christina Ersbøl
Karina Friis Lars Kayser Helle Terkildsen Maindal
Marie Germund Nielsen Niels Sandø Pedersen Kristine Sørensen
Majbritt Tang Svendsen
France
Julien Mancini (PI) Geneviève Chêne (PR) Cécile Allaire
Pierre Arwidson Francis Guillemin Youssoufa Ousseine
Laurent Rigal Alexandra Rouquette Rajae Touzani
Stéphanie Vandentorren
Germany
Doris Schaeffer (PI) Alexander Schmidt-Gernig (PR) Eva-Maria Berens
Lennert Griese
Hungary
Éva Bíró (PI) Péter Csizmadia (PR) Róza Ádány
Gabriella Mátyás Ferenc Vincze
Ireland
Lucy Bruton (PI) Sarah Gibney (PI), (PR) Teresa Maguire (PR)
Greg Straton (PR) Gerardine Doyle
Israel
Diane Levin-Zamir (PI/PR) Orna Baron-Epel Micha Laron
Tamar Medina Artom Shirly Mor-Anavy Shahar Lev-Air
Italy
Luigi Palmieri (PI) Daniela Galeone (PR) Chiara Cadeddu
Roberto D’Elia Paola De Castro Simona Giampaoli
Valeria Mastrilli Aldo Rosano
Norway
Kjell Sverre Pettersen (PI) Christopher Le (PR) Hanne Søberg Finbråten
Pål Joranger Øystein Guttersrud
Portugal
Andreia Silva da Costa (PI) Miguel Telo de Arriaga (PR) Rita Francisco
Jorge Oliveira Paolo Nogueira Carlota Ribeiro da Silva
Russian Federation
Maria Lopatina (PI) Oxana Drapkina (PR)
Slovakia
Zuzana Klocháňová (PI/PR)
Slovenia
Mitja Vrdelja (PI) Tamara Štemberger Kolnik (PR) Nejc Berzelak
Sanja Vrbovšek
Switzerland
Saskia Maria De Gani (PI) Karin Gasser (PR) Rebecca Jaks
Isabelle Villard
Further M-POHL members who contributed to the HLS19 Project

Altyn Aringazina Gill Rowlands Josefin Wångdahl
Working group leads of Optional Packages

Navigational HL: Doris Schaeffer Lennert Griese
Communicative HL: Peter Nowak Hanne Søberg Finbråten
Digital HL: Diane Levin-Zamir Stephan Van den Broucke
Vaccination HL: Robert Griebler Stephan Van den Broucke
Cost and Economics of HL: Gill Rowlands Gerardine Doyle
To be cited as: The HLS19 Consortium of the WHO Action Network M-POHL (2021): International
Report on the Methodology, Results, and Recommendations of the European Health Literacy
Population Survey 2019-2021 (HLS19) of M-POHL. Austrian National Public Health Institute,
Vienna
Online publication: https://m-pohl.net/Results
This report contributes to the implementation of the 2030 Agenda for Sustainable Development,
in particular to the Sustainable Development Goal (SDG) 3 “good health and well-being”, and the
Sustainable Development Goal (SDG) 10 “social inequity”.
Owner, editor and publisher: The HLS19 Consortium of the WHO Action Network M-POHL,
homepage https://m-pohl.net/
Preface
Health Literacy cannot be separated from the development of health

promotion. In the Ottawa Charter for Health Promotion 1986 many of
issues we have later come to explore more deeply through “health
literacy” were referred to under the action point “personal skills”. In
true health promotion thinking these skills were never seen separately
from issues of context, equity, participation, and empowerment.
© provided by Ilona Kickbusch

I was able in the late 1990ties - when I joined Yale University - to
learn from the US colleagues, who had been developing the field of
health literacy, but in contrast to the health promotion debate, initially
very much focused on patients within the health care system, but later
also on measuring health literate health care organisations. Their fo-
cus on tools, data and measurement was very critical.
Returning to Europe I found that in Switzerland where I lived a very first health literacy survey had
been developed and my idea to initiative a European survey quickly began to take shape. With the
support of many different stakeholders – from industry, the European Commission, public health
and academia –this idea was promoted for example at the European Health Forum Gastein and the
IUHPE meetings. Finally, the EU was willing to support HLS-EU and a group of highly motivated
researchers came together to conduct this groundbreaking work. WHO Europe published the Solid
Facts – Health Literacy and this that gave both the concept and results of HLS-EU a high visibility
and underscored its relevance for Health for All.
As a next step the German speaking health ministers and WHO-Europe initiated M-POHL, which
set the framework for measuring HL regularly in Europe; HLS19 could now be initiated and admin-
istered. The results of HLS19 confirm the initial results of HLS-EU for more countries and for new
specific measures. HLS19 also offers recommendations for policy, practice and research based on
empirical data.
I hope that these results and recommendations will initiate actions in Europe for improving HL and
will convince decision makers to plan interventions for strengthening HL on local, national, and
European level. Possibly there is even the potential to take the European experiences to other
regions of WHO.
It will be critical to invest in measuring population HL again in 2024, as well as support projects
for measuring organizational HL. This will require many new dimensions integrating the experi-
ences gained in the COVID19 pandemic, which has driven home the high relevance of health lit-
eracy
Prof. Ilona Kickbusch, Founding Director and Chair of the Global Health Centre, Graduate Institute
for International and Development Studies Geneva
Preface III
Preface
The publication of this Health Literacy Survey (HLS19) report is timely.
Low levels of health literacy among population groups are a chal-

lenge for public health, associated with riskier health behaviours and
poorer health outcomes. Today, we understand this more than ever
before.
Health literacy is an important element of the WHO Regional

Office for Europe’s flagship initiative on incorporating behavioural
© WHO/Europe
and cultural insights, which is central to delivering the European
Programme of Work 2020-2025. Prioritizing health literacy along-
side other behavioural and cultural determinants of health builds on
the commitment made by Member States in 2019, when they adopted a resolution on health lit-
eracy, and as part of this agreed to strengthen its measurement, monitoring and evaluation at
country and regional levels.
I am pleased that the WHO Action Network on Measuring Population and Organizational Health
Literacy (M-POHL) has been leading in this area, supporting a cross-national survey in 17 countries
of the European Region, and acknowledge the leadership of Austria in this area.
The data in this report can help build a strong foundation for future action. I hope that its data-
driven recommendations will support evidence-informed policy and interventions, and activities
we are currently developing in a broader context of behavioural and cultural insights.
This report underlines the value of engaging with and listening to populations to identify and
address the challenges that people face in their daily lives, and in using health services. In so
doing, we can help to make healthy practices possible, acceptable and attractive. Ultimately, this
will contribute to better health and well-being for the people we serve.
Hans Henri P. Kluge, MD, WHO Regional Director for Europe
IV © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Foreword
Health literacy is important for health and health equity. Therefore,

Austria decided to join forces with Germany and Switzerland to initiate
a European network on measuring health literacy. Austria is grateful
and proud that WHO-Europe has adopted this idea by supporting the
foundation of the WHO Action Network on Population and Organiza-
tional Health Literacy (M-POHL) that came to life in 2018.
The recent European health literacy survey HLS19 with 17 participating

countries from the WHO-European region is the first project of M-
POHL. The results demonstrate that low health literacy is still an area
© BKA / Andy Wenzel 1

of concern in all participating countries, and that health literacy is
associated with a social gradient. This has consequences for personal
health-related decisions in the domains of health promotion, disease
prevention and health care.
But in times of a pandemic like COVID-19, health-related decisions do not only concern people’s
individual health; they affect societies at large. Investments in strengthening health literacy are
therefore also investments in the overall well-being and prosperity of societies.
Austria has supported the HLS19 study because data are important to guide evidence-informed
policy and practice in identifying the target groups in most need of support and the areas in most
need of interventions. In order to moderate the existing gaps in health literacy, we must specifi-
cally focus on the at-risk groups for low health literacy – for example, by taking account of the
diverse health information and communication needs of these groups of the population.
Personally, I am convinced that the monitoring of health literacy at regular intervals is an important
prerequisite for effectively investing in strengthening health literacy over time.
Therefore, I would like to thank all those whose engagement made M-POHL and HLS19 a success.
And I do hope that the sustained commitment of the policy and research teams that participated
in HLS19 will enable the M-POHL network to achieve its aim to institutionalize regular health liter-
acy surveys in a growing number of countries. Austria strongly values the continued guidance of
WHO-Europe in further strengthening health literacy in Europe.
Wolfgang Mückstein, MD, Austrian Federal Minister of Social Affairs, Health, Care and Consumer
Protection
Foreword V
Short Summary
Background/Research Topics/Partner Countries
In Europe, interest has grown in measuring the health literacy (HL) of the adult population in re-
lation to public health, disease prevention, and health promotion to inform health policy in the
new millennium, partly building on a longer tradition of measuring HL, but with a focus on pa-
tients, in the US. The European Health Literacy Survey (HLS-EU, 2009-2012) confirmed the rele-
vance of HL for people’s health for eight countries in the European Union, and this was replicated
in follow up-studies for more European and Asian countries. All of these studies demonstrated
that HL is limited in a considerable proportion of the general population, with a social gradient for
HL and problematic consequences of limited HL for healthy lifestyles, self-reported health, and
the utilization of health care services. Following the recommendations of the WHO’s Health Liter-
acy: The solid facts (2013), WHO/Europe initiated the Action Network on Measuring Population
and Organizational Health Literacy (M-POHL), with 28 countries involved, to measure HL regularly,
starting with the Health Literacy Survey 2019 (HLS19). This survey was carried out in 17 countries
in the WHO European Region (Austria, Belgium, Bulgaria, Czech Republic, Denmark, France, Ger-
many, Hungary, Ireland, Israel, Italy, Norway, Portugal, Russian Federation, Slovakia, Slovenia, and
Switzerland). In the HLS19 not only General HL was investigated but also specific HLs, namely Nav-
igational HL, Communicative HL with physicians, Digital HL, and Vaccination HL, using newly de-
veloped and validated instruments. Furthermore, HL and health-related quality of life was ana-
lyzed as a mediator for health costs.
Methods
The HLS19 applied a cross-sectional multi-center survey study design. The study population was
defined as all permanent residents aged 18 and above living in private households in the partici-
pating countries. A total of 42,445 interviews were included in the study. National sample sizes
were expected to be at least 1,000 but varied from 865 to 5,660 respondents. The participating
countries used a multi-stage random sampling procedure or quota sampling, and most countries
stratified samples by gender, age group, population density, and geographical areas/units. Data
were collected in personal, telephone, or web-based interviews, or by using a mix of these. The
timeframe for data collection was from November 2019 to June 2021.
To measure General HL across the 17 countries, a short form of the original HLS-EU-Q47 instru-
ment – the HLS19-Q12 – was validated and used. At the same time, new instruments were devel-
oped and validated to measure Navigational HL, Communicative HL with physicians, Digital HL,
and Vaccination HL which were used by between 7 to 13 volunteering countries. Reflecting the
HLS-EU definition for comprehensive, general HL, explicit definitions were drawn up for these
specific HLs, and items were selected or constructed based on these. Relevant correlates of HL
were also measured in the HLS19. The HLS19 instruments were translated into their national lan-
guage(s) by 16 out of the 17 countries and also into migrant languages by a few countries. For HL
VI © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
measures, a score was calculated by combining the categories “easy” and “very easy”, and stand-
ardizing it from 0 to 100, with higher values indicating a higher level of HL. For General HL, four
categorial levels were also constructed (excellent, sufficient, problematic, inadequate).
Results/Findings
Psychometric properties of the HLS19 instruments: The internal consistency (Cronbach’s alpha) of
all HL scales was good (Cronbach’s alpha). For all measures, and for most countries, the distribu-
tion of scores was negatively skewed, with a ceiling effect. Thus, the instruments are sensitive
especially for respondents with lower HL. Unidimensionality of the scale was confirmed by CFA
and Rasch models, with acceptable results for all scales. Based on moderate correlations with the
General HL measure and with each other, all four specific measures are deemed sufficiently inde-
pendent to measure a specific aspect of HL.
The most difficult tasks identified were as follows: General HL: judging different treatment options,
protecting oneself from illness using information from the mass media, finding information on
how to handle mental problems; Navigational HL: understanding information on health care re-
forms, judging the suitability of health services, finding out about patients’ rights, judging the
extent of health insurance coverage; Communicative HL: getting enough time from physicians,
expressing personal views and preferences; Digital HL: judging the reliability of information, judg-
ing whether information is offered with commercial interests, using information to help solve a
health problem; Vaccination HL: judging which vaccinations one needs, finding information on
recommended vaccinations.
Disadvantaged vulnerable subpopulations at risk of lower General HL and lower specific HLs than
their national averages were identified. Having poor self-perceived health, being financially de-
prived, and having a lower level in society were indicators for lower General and lower specific
HLs. People with a low education also had, on average, lower General, Digital, and Vaccination HL.
To investigate a social gradient, the indicators gender, age, education, perceived social status,
and financial deprivation were used in a linear regression model. A social gradient was demon-
strated for all countries, albeit differing in degree across countries. The strongest predictors in
the model for a social gradient were financial deprivation and self-perceived level in society for all
HLs. Further relevant predictors were education for Navigational, Communicative, and Vaccination
HLs as well as age for Digital HL.
Using multiple linear regression analyses, the potential effects of General HL and specific HLs were
tested. Concerning General HL, out of the five lifestyle indicators, significant positive potential
effects were shown for physical activity (12 countries), fruit and vegetable consumption (8 coun-
tries), alcohol consumption (4 countries), BMI (2 countries), and smoking behavior (2 countries).
For three indicators of health status, significant positive potential effects were demonstrated for
self-perceived health (all 17 countries), limitations in activities due to health problems (13 coun-
tries), and long-term illness/health problems (7 countries). For the utilization of five types of
health services, fewer contacts with higher HL were found for GPs/family doctors (9 countries),
emergency services (8 countries), medical or surgical specialists (4 countries), inpatient hospital
Short Summary VII

services (4 countries), and day patient hospital services (2 countries). Thus, General HL is relevant
for people’s health but to a different degree concerning indicators and for individual countries.
Concerning specific HLs, Navigational HL, measured in eight countries, had a significant potential
positive effect on self-perceived health (7 countries), limitations in activities due to health prob-
lems (5 countries), and long-term illness/health problems (2 countries). Higher Communicative
HL, measured in nine countries, had a significant potential positive effect on self-perceived health
in seven countries. Digital HL, measured in 13 countries, had a significant potential positive effect
on self-perceived health (9 countries) and the utilization of GPs/family doctors (7 countries), when
General HL was also included in the regression models. Vaccination HL had a significant potential
positive effect on self-reported vaccination behavior in five out of the seven countries, a relation-
ship which is at least partly mediated by confidence in vaccinations, risk knowledge, and risk
perception.
To sum up, the results demonstrated the relevance of General and specific HLs for considerable
proportions of adult residents with low HLs, a social gradient for HLs, and significant potential
effects of HLs with health-relevant indicators. For all results, there were considerable variations
across countries, which confirms that HL is a contextual concept and must be measured for each
country. Due to the different methods and times of data collection, however, differences across
individual countries must be interpreted with caution, as must any causal assumptions about po-
tential effects due to the cross-sectional study design.
Recommendations
Based on the HLS19 results, the HLS19 consortium agreed on a set of recommendations, presented
here in a shortened format.
Regarding General HL
» Health policy should include an investment in longitudinal studies, measuring and monitoring
population HL regularly, and should systematically implement interventions to improve HL.
» Interventions should be specifically targeted at at-risk groups for low HL to reduce the health
gap between groups.
» Interventions to improve HL should focus on all four aspects of processing health-related
information (accessing, understanding, appraising, and applying information) within the do-
mains of healthcare, disease prevention, and health promotion.
» For interventions related to specific, concrete HL tasks, the tasks that are experienced as
being more difficult should be prioritized.
» The quality of health information in the mass media should be improved.
» Interventions to improve HL in relation to mental health should be prioritized and supported
by specific research.
VIII © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Regarding specific HLs
» Health policy should develop strategies to improve people’s Navigational HL, specifically in-
terventions on systemic and organizational levels to make health systems more health-lit-
erate, user-friendly, and easier to navigate.
» Interventions to improve the communication of health professionals with patients should
have high priority. Specifically, support for health professionals, especially physicians, in
dedicating more time to person-centered communication is needed.
» Regarding Digital HL, emphasis on providing easily accessible, high quality, trustworthy, un-
derstandable, assessable, and applicable health information, as well as communication via
digital sources should be increased.
» Improving Vaccination HL should have top priority, with a focus on judging vaccination in-
formation by improving the trustworthiness of information and communication on vaccina-
tions.
Regarding research on HL
» The HL of the adult resident population should be measured regularly in as many countries
as possible.
» The next wave of measuring should be planned for data collection in 2024.
» In preparation for this next wave, more specific research should be funded to analyze existing
data in more depth as well as to revise, extend, and apply the tools for measuring HL and
relevant correlates.
» For the four specific HLs, more detailed analyses and publications on the HLS19 data are
needed as well as further research and development on improvements for later applications.
» Additionally, further specific health literacies or relevant topics of General HL should be re-
viewed, selected, and researched to be included in the next wave of measuring HLs.
» More detailed analyses are needed regarding the costs and economics of HL.
» Further dissemination of the results of the HLS19 through peer-reviewed scientific publica-
tions is required.
Keywords
Health literacy survey, HLS19, recommendations, health literacy measurement
Short Summary IX
Executive Summary
1. Background/Introduction (Chapter 1)
Relevance of measuring HL in general adult populations and pre-existing research
The relevance of Health Literacy (HL) was first demonstrated for patients’ utilization and the out-
comes of health care services, primarily by research in the United States of America. The findings
encouraged politicians to develop a national action plan for the improvement of HL and practi-
tioners and researchers to develop the concept of a health-literate healthcare organization to deal
better with patients with low HL.
Later, the importance of HL was also demonstrated for public health, more specifically for disease
prevention and health promotion, and this in relation to the general population and not just for
actual patients. In Europe, adult population HL was first measured in a few countries which par-
ticipated in a US American led study, using the Health Activities Literacy Study (HALS) instrument,
and in a single study in Switzerland, using a newly developed experience-based instrument.
The HLS-EU study (2009-2012) offered an integrated conceptual and generic model and definition
of comprehensive General HL with a theory-based measurement instrument. Data were collected
and analyzed, originally for eight European Union countries, but there were many follow-up stud-
ies in individual European countries and in a group of Asian countries. The results of these studies
demonstrated the relevance of general, comprehensive HL for public health and health policy.
Therefore, the WHO’s report Health Literacy: The solid facts (2013) recommended the regular,
standardized measurement of general population HL, as well as of organizational HL, to investi-
gate how responsive health care and other organizations are to HL. The WHO’s Action Network
Measuring Population and Organizational Health Literacy (M-POHL) since 2018, followed up on
this recommendation and initiated the Health Literacy Survey 2019-2021 (HLS19).
International and national policy documents have highlighted the relevance of HL and recommend
measuring and improving HL in practice, both by investing in research and implementing HL pol-
icy. By that, global leaders in public health are paying increasing attention to the potential of HL.
In 2009, the United Nations Economic and Social Council (ECOSOC) recognized the concept of HL
as an “important factor for ensuring significant health outcomes” and called for action plans to
promote it. Within the European Region, the WHO’s publication Health Literacy: The solid facts
(2013) summarized important evidence relating to the topic and highlights HL as a key dimension
for implementing the WHO’s European strategy Health 2020, not least in relation to its potential
for promoting empowerment and participation in communities and in health care. At the WHO’s
9th Global Health Promotion conference in Shanghai, China (2016), HL was prominently featured,
resulting in the Shanghai Declaration on promoting health in the 2030 Agenda for Sustainable
Development, declaring HL a critical determinant of health. The Declaration established the link
between HL and the United Nation’s Sustainable Development Goals (SDGs), calling for the devel-
opment, implementation, and monitoring of intersectoral strategies at national and local levels for
X © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
strengthening HL in all populations. The Organisation for Economic Co-operation and Develop-
ment (OECD) has also worked on HL, drafting a fast-track paper on how HL is addressed by OECD
Member States, which was published in 2018. Also in 2018, the Executive Board of the Interna-
tional Union of Health Promotion and Education (IUHPE) ratified a position statement, A Practical
Vision for a Health Literate World, supporting HL policy, practice, and research at a global level.
Specific attention has also been paid to the potential of HL to reduce the prevalence and impact
of noncommunicable diseases (NCDs), as reflected in the Montevideo Roadmap 2018–2030 on
NCDs as a Sustainable Development Priority. Within the European Region, former Regional Director
Zsuzanna Jakab defined HL as one of the enablers for implementing the Sustainable Development
Goals during the 67th WHO Regional Committee for Europe meeting in Budapest in September
2017. In 2019 the Region launched the resolution Towards the implementation of health literacy
initiatives through the life course (EUR/RC69/R9) which demands, among others, the promotion
of HL with a focus on reducing health inequities and the strengthening of HL measurements and
action.
2. Methods
2.1 Study Design (Chapter 2)
In the HLS19, a cross-sectional multi-center survey study design was applied in 17 countries in the
WHO European Region: Austria (AT), Belgium (BE), Bulgaria (BG), Czech Republic (CZ), Denmark
(DK), France (FR), Germany (DE), Hungary (HU), Ireland (IE), Israel (IL), Italy (IT), Norway (NO), Por-
tugal (PT), Russian Federation (RU), Slovakia (SK), Slovenia (SI) and Switzerland CH).
The study population was defined as all permanent residents aged 18 and above, living in private
households in the countries participating in the study. In total interviews from 42,445 respondents
were included in the study. National sample sizes varied as follows: AT: 2,967, BE: 1,000, BG: 865,
CH: 2,502, CZ: 1,599, DE: 2,143, DK: 3,602, FR: 2,003, HU: 1,195, IE: 4,487, IL: 1,315, IT: 3,500,
NO: 2,855, PT: 1,247, RU: 5,660, SI: 3,360, and SK: 2,145.
The participating countries used a multi-stage random sampling procedure or quota sampling,
and most countries stratified samples by gender, age group, population density, and geographical
areas/units. As a rule, at least 80% of the HLS19 core items, consisting of the 12 items measuring
General HL and the 31 correlate items, needed to be answered to be accepted as a completed
interview. Data collection was carried out in most of the participating countries by national data
collection agencies and in three cases by the HLS19 National Study Centers. The following methods
of data collection were used: paper-and-pencil personal interviews (PAPI) in DE and RU; computer-
assisted personal interviews (CAPI) in SK; computer-assisted telephone interviews (CATI) in AT,
HU, IE, NO, and PT; and computer-assisted web interviews (CAWI) in BE, CH (with a few CATI), DK,
and FR. There were also mixed types of data collection, namely CAWI + CATI (by BG, CZ, IL, and
IT) and CAPI + CAWI (by SI). Response rates varied considerably across methods of data collection,
from 4% (FR using CAWI) to 94% (RU using PAPI). Due to the Covid-19 pandemic, the original
timeframe for collecting data was extended, and the data collection phase lasted from November
2019 to June 2021. All participating countries ensured compliance with ethical guidelines and data
Executive Summary XI
protection and explicitly obtained informed consent from respondents before carrying out any
interviews.
Due to the differences in methodology, the time of data collection, and the potential effects of
Covid-19, differences in results between countries, and any comparison among them, must be
interpreted with caution.
2.2 The HLS19 Instruments (Chapter 3)
To measure General HL, based on the HLS-EU instruments, an adapted 47-item instrument, the
HLS19-Q47, and two adapted short forms, the HLS19-Q12 and the HLS19-Q16, were developed to
collect data. New instruments were developed to measure Digital HL, Communicative HL with phy-
sicians in healthcare, Navigational HL, Vaccination HL, and the Costs and Economics of HL, namely,
respectively, the HLS19-DIGI, the HLS19-COM-P-Q11 (long form) and HLS19-COM-P-Q6 (short
form), the HLS19-NAV, the HLS19-VAC, and an item set to measure HL and health-related quality
of life as a mediator for health costs. Additionally, 31 core correlates, and 18 optional correlates
were also made available in the HLS19. Participating countries had to implement at least the HLS19-
Q12 and the 31 core correlates; all other parts were optional. National add-ons were possible. The
HLS19 instruments were translated into their national language(s) by 16 out of the 17 countries
(Ireland used the original English version), thereby creating a rich spectrum of languages in which
the instruments are now available: Arabic, Bulgarian, Czech, Danish, Dutch, French, German, He-
brew, Hungarian, Italian, Norwegian, Portuguese, Russian, Slovenian, and Slovak. Additionally,
some countries translated the instruments into migrant languages. In 16 out of the 17 countries
a field test was performed.
2.3 Methods of Data Analyses (Chapter 4)
Scores were calculated for the following HL measures:
HLS19-Q12 General Health Literacy (12 items)

HL-DIGI Digital Health Literacy (8 items)
HL-DIGI-INT Digital Interaction Literacy (2 items)
HL-NAV Navigational Health Literacy (12 items)
HL-COM-Q11 Communicative Health Literacy with physicians in health care services (11 items)
HL-COM-Q6 Communicative Health Literacy with physicians in health care services (6 items
short form)
HL-VAC Vaccination Health Literacy score (4 items, which were all taken from the HLS19-
Q47)
The score value was calculated as the percentage (ranging from 0 to 100) of items with valid
responses that were answered with “very easy” or “easy”, provided that at least 80% of the individ-
ual items contained valid responses. Thus, the scores measure HL as the percentage of health-
related tasks being experienced as “very easy” or “easy” by a respondent, with higher values indi-
cating a higher level of General HL.
XII © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
For General HL (HLS19-Q12), each respondent was assigned to one of four levels of HL: excellent,
sufficient, problematic, or inadequate.
For each measure, the Cronbach alpha coefficient was calculated and a confirmatory factor analysis
with a single latent variable as well as a Rasch analysis were conducted to confirm the internal
consistency and the unidimensionality of the scale. For some measures of specific HL, this was
complemented by models for two latent variables, or rather two dimensions.
Associations (1) between HL and a pre-defined set of potential determinants of HL or (2) between
potential consequences or outcomes of HL and associated determinants were estimated by means
of Spearman correlation coefficients and multivariable linear regression models. The following
variables were analyzed as potential determinants: gender, age, education, self-perceived level in
society, financial deprivation/difficulties, migration background, long-term illnesses/health prob-
lems, and training in a health care profession.
3. Key Results for General Health Literacy

3.1 The Instrument of measuring General HL (Chapter 5)
For the concept and definition of comprehensive, general HL, the HLS19 followed the concept and
definition of the HLS-EU, but instead of the long form Q47, the HLS19 used a specially developed
HLS19-Q12 shorter form to investigate General HL. (For those countries that used the Q47 or the
Q22 data set for also constructing the Q16, data for the Q12 were extracted from this measure or
data set.) The HLS19-Q12 measure represents the HLS-EU matrix by using one indicator for each
cell; the wording of its items was adapted slightly and its answer categories to “very easy”, “easy”,
“difficult”, and “very difficult”.
3.1.1 Difficulty of Individual Items
To rank the individual items by difficulty, the response categories “very difficult” and “difficult”
were combined. There were some common patterns for ranking the difficulties of HL-related tasks
across countries, but there were also considerable differences between countries. The overall per-
centage of respondents ticking “very difficult” or “difficult” varied between 8% and 43% for the
HLS19-Q12 items. On average, the most difficult tasks were item 3 “to judge the advantages and
disadvantages of different treatment options” (42%, varying from 26% to 71%), item 8 “to decide
how you can protect yourself from illness using information from the mass media” (40%, varying
from 26% to 62%), item 5 “to find information on how to handle mental health problems” (36%,
varying from 19% to 50%), and item 12 “to make decisions to improve your health and well-being”
(26%, varying from 12% to 42%).
Executive Summary XIII

3.1.2 Construction of Scores, Validation, and Psychometric Properties
The HLS19-Q12 shows adequate internal consistency, with an average Cronbach alpha coefficient
of 0.78 (varying from 0.67 to 0.87 for individual countries). With the twelve dichotomized HLS19-
Q12 items loading onto a single factor, the confirmatory factor model resulted in fit indices indi-
cating a good model fit for each country. Three rather easy items differed considerably in relation
to the standardized parameter estimates across countries. These were item 4 “to act on advice
from your doctor or pharmacist” (“very difficult” or “difficult”, 8% on average, ranging from 4% to
17%), item 9 “to find information on healthy lifestyles such as physical exercise, healthy food, or
nutrition” (9.9% on average, ranging from 6% to 21%), and item 10 “to understand advice concern-
ing your health from family or friends” (17% on average, ranging from 7% to 27%).
When testing data against the Partial Credit Model (PCM) by country, the HLS19-Q12 displayed
good overall data-model fit in eight participating countries. With a reduced sample size (n=360),
the HLS19-Q12 had acceptable overall data-model fit in an additional four participating countries.
It had an acceptable reliability index in each country. The HLS19-Q12 was somewhat “off target”
as the items referred to tasks which most respondents in the participating survey studies perceived
as manageable.
Using a principal component analysis of Rasch model residuals, two possible subscales or item
subsets of the HLS19-Q12 were identified empirically. However, these two subsets seem to meas-
ure “the same”, and so it may be concluded that the HLS19-Q12 is sufficiently unidimensional and
measures one latent trait.
No evidence of response dependency or “too similar” items was observed. Most HLS19-Q12 items
displayed acceptable data-model fit. Several items displayed differential item functioning (DIF)
even when the sample size was reduced to 1,080. This could affect comparisons of subpopulations
across countries, age groups, or employment status.
In conclusion, the HLS19-Q12 is a psychometrically rather sound instrument for measuring com-
prehensive General HL in adult populations as intended in the HLS19.
The statistical representation of the HLS19-Q47 by the HLS19-Q12 was tested in six countries with
a Pearson correlation of 0.93 (ranging from 0.90 to 0.95 for individual countries). Accordingly, the
HLS19-Q12 represents the total score of the HLS19-Q47 very well from a statistical perspective.
3.1.3 Distributions of Scores and APRPs
The distribution of the scores was negatively (left) skewed for all countries. There was also a con-
siderable ceiling effect, which partly indicates that the scale included tasks that many respondents
found manageable. Thus, the instrument is sensitive especially for respondents with lower HL.
For all countries, the median score of the HLS19-Q12 was 83, varying across countries from 67 to
91; the mean score was 76 and varied across countries from 65 to 86.
XIV © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
The ceiling effect and its variation across countries was also shown by the Average Percentage
Response Patterns (APRPs) for HLS19-Q12, where, on average, 24% answered the items as being
either “very difficult” or “difficult” (varying from 14% to 35% between countries).
Based on defined cutting points, like those in the HLS-EU, categorical levels were constructed for
the HLS19-Q12. Accordingly, across all participating countries, about 40% of respondents had a
“sufficient” level of HL, with about 15% being “excellent”. In contrast, about 33% had a “problem-
atic” level and for 13% it was “inadequate”. There was considerable variation in level values across
countries. Following the example of the HLS-EU study, the HL categorical levels of “inadequate”
and “problematic” were combined and defined as “limited” HL, with a range of 25% to 72%. That
means that between one in four and three out of four residents in countries participating in the
HLS19 have limited General HL. Compared to the HLS-EU, with one out of three up to two out of
three, the variation between countries is even more pronounced, which could be due to the dif-
ferent methodology used but also by different countries being included in the two studies.
3.1.4 Disadvantaged/Vulnerable Subpopulations
The HLS19-Q12 mean score was considerably lower for selected predefined disadvantaged or vul-
nerable subpopulations than for the total population. This was, on average, especially true for
respondents with “poor self-perceived health” (-14%, varying across countries from -5 to -27),
respondents who are “financially deprived” (-8%, varying from +1 to –14), respondents reporting
a “low self-perceived level in society” (-8%, varying from -2 to -18), or respondents with “low
education” (-6% points, varying from +1 to -22).
3.3 Social Gradient and Determinants (Chapter 6)
To investigate the social gradient, indicators were used including gender, age, education, self-
perceived level in society, and financial deprivation. The existence of a social gradient was con-
firmed by multivariable linear regression models explaining on average 7% of the variance of the
HLS19-Q12 score, ranging from 4% to 25% across countries. The strongest predictors of the social
gradient were financial deprivation, with, on average, ß=-0.21 (varying from ß=-.15 to ß=-0.32
and significant (p<0.01 is referred to here and elsewhere in this summary) for all countries with
one exception), followed by the respondents’ self-perceived level in society, with ß=0.10 (signif-
icant for 14 countries and varying for these from ß=0.08 to ß=0.26).
Models including additional predictors (respondents’ migration background, long-term illness,

training in a health care profession) did not, on average, improve the explained variance of the
models. In fact, respondents’ migration background had a significant effect on HL in just two
countries (with migrants having a slightly higher level of General HL in those two countries), long-
term illness in eight countries (respondents with at least one long-term illness having lower Gen-
eral HL), and training in a health profession in eight countries (respondents trained in a health
profession having higher General HL).
Executive Summary XV
Thus, the HLS19 confirms earlier results that there is a social gradient for General HL across coun-
tries which varies to a considerable extent and that both financial deprivation and level in society
are the strongest predictors.
3.3 Consequences for Health-Related Outcome Indicators
3.3.1 Health Behaviors and Lifestyles (Chapter 7)
The potential effects of General HL on five indicators - BMI, smoking behavior, alcohol consump-
tion, physical activity, and fruit and vegetable consumption - were investigated. Multivariable lin-
ear regression models showed significant effects of General HL on physical activity explaining, on
average, 3% of the variance (varying across countries from 1% to 9%) with, on average, ß=0.11
(significant for 12 countries, varying for these from ß=0.08 to ß=0.27). Compared to the five
social indicators, General HL is the strongest predictor of physical activity. For fruit and vegetable
consumption, the same models explained 4% of the variance on average (varying from 4% to 9%
across countries), with General HL being the second strongest predictor at ß=0.09 (significant for
eight countries, varying for these from ß= 0.07 to ß=0.18). However, while similar models ex-
plained 5% of the variance for BMI, (varying from 1% to 14%), General HL was the predictor with
the lowest ß on average, at ß=-0.01 (and was significant for only two countries at ß=-0.06). For
smoking behavior, similar models explained, on average, 4% of the variance (varying from 1% to
14%); the results for General HL were significant, but inconsistently so, for only four countries,
with either ß=-0.04 and ß=-0.06 or ß=+0.08 and ß=+0.09). For alcohol consumption, similar
models explained 8% on average (varying from 5% to 16%), with General HL showing on average a
low ß=-0.05, which was significant for only five countries: ß ranged from -0.05 to -0.13 in four
countries but for the fifth ß=+0.09, which was in an unexpected direction.
Thus, according to the HLS19, General HL was shown to have potentially positive effects on only
two lifestyle indicators, namely physical activity and fruit and vegetable consumption. In contrast
to some earlier research, no relevant and consistent effects on BMI, smoking behavior, and alcohol
consumption could be demonstrated for most countries.
3.3.2 Health Status (Chapter 8)
The three Minimum European Health Module (MEHM) indicators (self-perceived health, long-term
illness/health problems, and limited in activities due to long-term illness/health problems) were
used to investigate the potential effects of General HL on respondents’ health status. In all coun-
tries (equally weighted), a positive linear association was found between General HL and self-
perceived health, while negative linear associations were demonstrated between General HL and
long-term illness/health problems as well as between General HL and limited in activities due to
long-term illness/health problems. These associations varied considerably in extent (and con-
sistency) across participating countries.
In multivariable linear regression models for self-perceived health, including the five core social
indicators and General HL as predictors, on average, 21% of the variance (varying from 11% to
38%), was explained, with General HL being the predictor with the third highest ß=-0.15 (varying
XVI © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
from ß=-0.07 to ß=-0.22; significant for each country). Similar models for long-term ill-
nesses/health problems explained 15% of the variance on average (varying across countries from
8% to 37%), with General HL again being the third highest predictor on average at ß=-0.09 (sig-
nificant for seven countries, varying for these from ß=-0.06 to ß=-0.19 across countries). For
being limited in activities due to health problems, the models explained, on average, 10% (varying
from 3% to 22%), and General HL was the predictor with the second highest ß=0.14 on average,
(significant for 13 countries, varying for these from ß=0.06 to ß=0.21).
Thus, this study confirmed earlier research that there is a potentially direct, relevant effect of
General HL on respondents’ health status. This pattern applied to most of the countries, with some
variation in extent for the different health status indicators.
3.3.3 Extent of Healthcare Services Utilization (Chapter 9)
The potential effects of General HL on the extent of the utilization of healthcare services were
investigated for five types of health services (emergency services, GPs/family doctors, medical or
surgical specialists, inpatient hospital service, and day-patient hospital service).
Multivariable linear regression models, with General HL and five social indicators as predictors,
explained just 2% of the variance on average (varying from 1% to 11%) for the utilization of emer-
gency services, with General HL being the second highest predictor on average, at ß=-0.06 (sig-
nificant for eight countries and varying for these from ß=-0.05 to ß=-0.20). Similar models for
the utilization of GPs/family doctors explained, on average, 6% of the variance (varying from 4%
to 14%), with General HL being the predictor with the second highest ß=-0.09 on average (signif-
icant for nine countries, varying for these from ß=-0.05 to ß=-0.14). For the utilization of medical
and surgical specialists, similar models explained, on average, 3% of the variance (varying from
1% to 12%), with General HL being the predictor with the fourth highest significant ß=-0.05 on
average (but significant for only four countries and varying for these from ß=-0.05 to ß=-0.10).
Similar models for the utilization of inpatient hospital services explained, on average, just 2% of
the variance (varying from 0% to 9% across countries), with General HL, on average, being the third
highest predictor at ß=-0.04 (significant for just four countries and varying for these from ß=-
0.05 to ß=-0.06). For the utilization of day patient hospital services, the models explained, on
average, just 1% (varying from 1% to 4%), with General HL, on average, being the predictor with
the second highest significant ß=-0.04 (significant for only two countries, varying from ß=-0.04
to ß=-0.06).
Thus, as expected from earlier research, a potentially direct, relevant effect of General HL on the
utilization of health care services could be demonstrated just for specific indicators and for a
smaller number of countries. While regression models did not explain much of the variance intro-
duced by the classical social determinants included, in comparison to these, General HL was rel-
atively relevant and is a better modifiable predictor of health care service utilization.
Executive Summary XVII

4. Specific Health Literacy Measures
In contrast to the HLS-EU study, in the HLS19, besides measuring General HL, optional work pack-
ages were included for especially relevant specific aspects of HL, namely Navigational HL, Com-
municative HL with physicians, Digital HL, and Vaccination HL. These were developed and validated
jointly, translated, and administered for the first time, with 12 countries participating in Digital
HL, nine countries in Communicative HL with physicians in health care services, eight countries in
Navigational HL, and 11 countries in Vaccination HL.
4.1 Navigational HL (Chapter 10)
4.1.1 Relevance
In the last few decades, the structures, and regulations of healthcare systems in many countries
have become increasingly complex for patients and users and thus ever more difficult for them to
navigate. Thus, more than ever before, specific Navigational HL is needed by patients and users
alike. In response to a lack of measurement tools and population-based data on Navigational HL,
one aim of the HLS19 was to develop and introduce a theory-based instrument for measuring
Navigational HL and to provide data on the topic by the same instrument in a set of different
countries for the first time.
4.1.2 Definition and Instrument
Using the conceptual framework of the HLS-EU Consortium and the HLS19 study, Navigational HL
was defined as “people’s knowledge, motivation and skills to access, understand, appraise and
apply the information and communication in various forms necessary for navigating healthcare
systems and services adequately to get the most suitable health care for oneself or related per-
sons”. An instrument for measuring Navigational HL was developed based on a scoping review of
the literature, an expert and stakeholder survey, focus group discussions, personal interviews,
and continuous discussions in the HLS19 Consortium. This led to a questionnaire with twelve items
measuring self-perceived difficulties in accessing, understanding, appraising, and applying navi-
gation-related information primarily for selected tasks on the macro (societal) and meso (organi-
zational) levels of navigating health care services.
4.1.3 Data Collection
The Navigational HL was applied in eight countries (AT, BE, CH, CZ, DE, FR, PT, and SI) in seven
languages in samples using CATI, CAWI, or mixed methods for data collection for a total of over
16,000 respondents.
The percentages of the combined “difficult” or “very difficult” answers to the 12 HLS19-NAV items
ranged from 19.5 % to 56.6 %, with considerable variation across countries. The most difficult
XVIII © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
tasks were dealing with information on health care reforms, the suitability of a particular health
service, patients’ rights, and health insurance coverage of specific health services.
A score for Navigational HL was calculated by combining the response categories “very easy” or
“easy”, adding these up over the twelve items, and standardizing the raw score for a range from 0
to 100. The scale proved to be a valid measure with acceptable psychometric properties concern-
ing internal consistency (Cronbach’s alpha between 0.83 and 0.92), and unidimensionality by CFA
and polytomous partial credit Rasch models. Nevertheless, the instrument worked better in some
countries than in others, and limitations exist regarding differential item functioning (DIF). With
correlations on average between 0.40 and 0.56 with the other specific HLs and General HL, the
instrument is related closely enough to be interpreted as being an instrument of HL and inde-
pendent enough to measure a specific aspect of HL.
With, on average, a mean score of 55 (varying from 42 to 67), Navigational HL is low in most
countries, at least compared to the measures of other health literacies. In terms of Average Per-
centage Response Patterns, 45% of the answers (varying from 33% to 59% across countries) were,
on average, either “very difficult” or “difficult”.
In most countries, participants with poorer health (on average -12%), financial deprivation (-10%),
and self-perceived level in society (-9%)) had mean scores for Navigational HL which were consid-
erably below the population’s average.
4.1.8 Social Gradient and Determinants
In multivariable linear regression models with five social determinants, explaining, on average, 6%
of the variance (from 4% to 13% across countries), a social gradient was demonstrated for Naviga-
tional HL, with, on average, financial deprivation (ß=-0.15), self-perceived level in society
(ß=0.14), and education (ß=-0.11) being the predictors with the highest ß values. When General
HL was also introduced into the regression model, General HL was found to be the strongest pre-
dictor with, on average, ß=0.53.
4.1.9 Consequences for Health-Related Outcome Indicators
With similar regression models, higher HL-NAV was a significant predictor of self-perceived health
with, on average, ß=-0.13 (significant in seven out of the eight countries and varying for these
from ß=-0.06 to ß=-0.13). For limited in activities due to health problems this was the case, with,
on average, ß=0.11 (significant for only five countries and varying for these from ß=0.07 to
ß=0.10) and for long-term illnesses or health problems, with, on average, ß=-0.07 (significant
Executive Summary XIX

for only two countries, with ß=-0.06 to ß=-0.10). For indicators of utilization of GPs/family doc-
tors, and utilization of medical and surgical specialists, navigational HL was a significant indicator
for only two or one countries.
4.1.10 Discussion and Conclusions
With the HLS19-NAV, a new and extensively tested instrument with some potential for improvement
is available, the implementation of which has provided important information for the specific field
of managing health information in the context of navigating healthcare systems. The results con-
firm the need to strengthen Navigational HL (and General HL) through target group-specific, tai-
lored strategies but also to reduce the demands placed on individuals by realizing health-literate
healthcare systems and anchoring Navigational HL at all levels of the system.
4.2 Communicative HL with Physicians (Chapter 11)
4.2.1 Relevance
Communicative HL is recognized as being critical for patients to actively participate in health com-
munication with health professionals, to obtain and understand information, to achieve successful
outcomes from health care, and to use the information to manage health. Communication is a
core task for health professionals and patients when making diagnosis, deciding on and imple-
menting treatments, organizing appropriate health care, and maintaining good health. Commu-
nication in health care settings is becoming increasingly important due to changes in the patients’
role, the expectation for more patient participation and for shared decision making. But for a start,
the working group on this optional package developed an instrument just for Communicative HL
with physicians.
A comprehensive definition of Communicative HL is provided: “Communicative HL refers to pa-

tients’ communicative and social skills that enable them to actively engage in face-to-face en-
counters with health care professionals, to give and seek information, derive meaning from it and
apply this information in decision making and in co-producing their health care”. However, the
focus of this instrument is on physician-patient communication within the healthcare system. For
this, a long form (11 items) and a short form (6 items) were constructed based on a comprehensive
theoretical framework that integrates the communicative literacy concept of Nutbeam, the basic
competencies of information processing included in the HL framework of the HLS-EU Consortium),
and the main communicative tasks of the Calgary-Cambridge Guide framework.
The HLS19-COM-P instrument was successfully applied in nine countries: Austria, Belgium, Bul-
garia, Czech Republic, Germany, Denmark, France, Hungary, and Slovenia, in seven languages (in
a total sample of around 20,000 for the HLS19-COM-P-Q6) using different formats of data collec-
tion (PAPI, CATI, CAWI, or mixed methods).
XX © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Perceived difficulties ranged on average from 4% to 25% for the HLS19-COM-P-Q11 items and from
9% to 26% for the HLS19-COM-P-Q6 items. In general, getting enough time in the consultation
with the physician and expressing personal views and preferences were experienced as being the
most difficult tasks, while explaining personal health concerns was the easiest.
Scores for the long and short scales of Communicative HL were calculated by combining the re-
sponse categories “very easy” and “easy”, adding these up over the eleven or six items respectively,
and standardizing the raw scores for a range from 0 to 100. Both instruments displayed accepta-
ble psychometric properties for internal consistency with Cronbach’s alpha (for HLS19-COM-P-
Q11: mean 0.83, from 0.79 to 0.87; for HLS19-COM-P-Q6: mean 0.78, from 0.69 to 0.81), as well
as for unidimensionality by CFA and polytomous partial credit Rasch models. Both instruments
correlated moderately with General HL (mean 0.46 and 0.43 respectively) and with Navigational
HL (mean 0.47 and 0.43 respectively), indicating that Communicative HL and General HL or Navi-
gational HL are related but still distinctive constructs.
For all countries, the distributions of scores were rather left-skewed. Communicative HL with phy-
sicians, in the general population under investigation, was relatively good with a mean score for
Q11 of 85 (ranging from 74 to 92) and a mean score for Q6 of 83 (from 72 to 90), but about 10-
20% of the population have problems communicating with their physician. In terms of APRPs, 15%
(from 8% to 26%) of the answers for the Q11 or 17% (from 9% to 27%) of the Q6 were either “very
difficult” or “difficult”.
In most countries, participants with poorer self-perceived health (on average -11%) or lower socio-
economic status (financial deprivation (-9%) and self-perceived level in society (-8%)) were found
to have lower Communicative HL mean scores than their national population.
In multivariable linear regression models with five social determinants for the short form of Com-
municative HL, explaining on average 5% of the variance (varying across countries from 2% to 18%)
a social gradient was identified for communicative HL, with, on average, level in society (ß=0.14),
financial deprivation (ß=-0.13), and education (ß=-0.07) being the predictors with the highest ß
values. When General HL was also introduced into the regression model, General HL was found to
be the strongest predictor with, on average, ß=0.42.
Executive Summary XXI

In most countries Communicative HL was a significant predictor of self-perceived health, while in

some countries higher Communicative HL was associated with somewhat lower use of medical
care.
The HLS19-COM-P instrument was used successfully in research on different national adult general
populations to map communicative HL. Physician-patient communication is an important aspect
of HL and a relevant issue for health policy and practice. It is of utmost importance for patient
satisfaction and participation but also for health outcomes and health equity. The restriction of
the study to physician-patient interaction indicates that further research is needed for Communi-
cative HL focusing on other health professionals.
4.3 Digital HL (Chapter 12)
4.3.1 Relevance
The increasing availability and use of health-related digital/electronic resources such as electronic
health records, telehealth initiatives, digital health applications, and interactive communication
options with health care providers (e.g., for making appointments or reporting medical results)
places a growing demand on the population's skills in relation to Digital Hl to adequately use these
applications and resources. Researchers, practitioners, and policy makers should therefore realize
the importance of understanding and improving people’s proficiency in using digital resources for
managing disease and/or promoting their health by measuring Digital HL.
The concept and definition of Digital HL in the HLS19 is based on the HLS-EU Consortium’s concept
and definition of General HL but aligned with existing research on the scope and diversity of digital
health resources. Digital HL includes the ability to search for, access, understand, appraise, vali-
date, and apply online health information as well as the ability to formulate and express questions,
opinion, thoughts, or feelings when using digital devices. This concept relates strongly to the
frequency with which people use different health resources from digital sources and resources
such as online video consultations, digital personal health records, social media, and health re-
lated apps, etc. for promoting health. One scale was constructed with eight items measuring the
skills related to dealing with health information digitally and two items for the interactive use of
digital devices.
The following countries included the optional package on Digital HL in their national assessment:
Austria, Belgium, Czech Republic, Denmark, France, Germany, Hungary, Ireland, Israel, Norway,
Portugal, Slovakia, and Switzerland. Analyses were based on 29,060 respondents, with country
XXII © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
specific sample sizes ranging from 1,000 to 3,602. There was variation in the data collection
method administered, by using CAPI, CATI, CAWI, PAPI, and mixed formats.
The ranked difficulty of single tasks across countries is rather similar, with some exceptions. On
average, the difficulty of items varies (for the combined response categories “very difficult or “dif-
ficult”) between 22% and 54%, with considerable variation across countries. The three most difficult
tasks were: “to judge whether the information is reliable”, “to judge whether the information is
offered with commercial interests”, and “to use the information to help solve a health problem”.
A score was calculated for Digital HL by combining the response categories “very easy” and “easy”,
adding them up across the eight items, and standardizing the raw score for a range from 0 to 100.
The internal consistency of the Digital HL scale is acceptable with, on average, Cronbach’s alpha
of 0.83, varying across countries from 0.77 to 0.87. A single-factor confirmatory factor model
with dichotomized items loading onto a single latent variable provided fit indices which indicate
an acceptable fit for all countries. According to a principal component analysis (PCA) of Rasch
model residuals combined with dependent t-tests to identify possible empirical subscales, the
Digital HL scale was sufficiently unidimensional. The thresholds, and thus the response categories,
were ordered and well-functioning. On average, Digital HL correlated with General HL (r=0.53),
Navigational HL (r=0.55), Communicative HL (Q11: r=0.39, Q6: r=0.31), and Vaccination HL
(r=0.38), and was thus related enough to the other HLs to measure HL and independent enough
to measure a specific aspect of HL.
For all but one country, the distributions of Digital HL scores were left-skewed, with a clear ceiling
effect. The mean score was, on average, 62, varying from 42 to 79 across countries. In terms of
the APRPs, on average, 38% of the answers were either “very difficult” or “difficult”, varying between
22% and 58% across countries.
Disadvantaged or vulnerable subpopulations with lower mean scores of Digital HL than the na-
tional population were identified as respondents with bad or very bad self-perceived health (on
average -11%), with considerable or severe financial deprivation (-9%), with low education (-8%),
with six or more contacts to a GP/family doctor (-7%), and with low self-perceived level in society
(-7%).
A social gradient for Digital HL was demonstrated by multivariable linear regression models with
five social predictors; explained variance varied by country (6% on average, varying from 2% to
Executive Summary XXIII

23%). On average, financial deprivation was the predictor with the highest ß=-0.15 (significant
with ß between -0.08 and -0.27 for 10 countries), followed by age with ß=-0.13 (significant for
six countries with ß between -0.15 and -0.26), self-perceived level in society with ß=0.08 (sig-
nificant for 10 countries with ß between 0.05 and 0.13). Including the extent of use of digital
resources in the model did not improve explained variance much, but the use of digital resources
with, on average, ß=0.11 (significant for 10 countries with ß between 0.06 and 0.22) was the
predictor with the third highest value, after financial deprivation and age. When General HL is also
included in the original model, it is by far the strongest predictor of Digital HL with ß=0.51 (sig-
nificant for all countries, varying from ß=0.43 to ß=0.67), followed by age (ß=-0.13), financial
deprivation (ß=-0.06), and education (ß=0.06).
In a multivariable linear regression model for explaining self-perceived health with the five social
determinants and Digital HL as predictors, Digital HL was significant for nine out of 13 countries
(with ß=-0.05 or -0.10). A similar model for the utilization of GPs/family doctors as dependent
variable showed significant effects of Digital HL for seven countries (ß between -0.07 and -0.11).
A new, short eight-item scale for measuring experience-based Digital HL was jointly developed,
validated, and used for investigating Digital HL in thirteen countries. The measure showed ac-
ceptable psychometric properties for all countries, but further development is recommended. The
results demonstrated the relevance of Digital HL by revealing that a considerable proportion of
respondents have lower Digital HL, that there is a social gradient for Digital HL in most countries,
and that Digital HL is associated with the health-relevant indicators self-perceived health also in
most countries and with utilization of GPs/family doctors in some countries. By identifying espe-
cially difficult, concrete tasks relating to Digital HL and subpopulations with lower Digital HL than
the national adult population, the results offer an orientation for tailoring strategies to improve
Digital HL by health policy.
4.4 Vaccination HL (Chapter 13)
4.4.1 Relevance
Vaccine hesitancy is a pressing public health issue, especially in Europe, and it poses an increasing
challenge to health authorities. Credible and tailored information about vaccination could help
regain individuals’ confidence in vaccinations. However, as information on vaccination is often
difficult to access, complex, not always easy to understand, and challenged by biased and one-
sided information, a high level of Vaccination HL is a prerequisite for assessing the trustworthiness
and quality of information and for dealing competently with false and misleading information.
XXIV © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Based on the definition of General HL the working group defined Vaccination HL as referring to
people’s knowledge, motivation, and skills to find, understand, and evaluate immunization-re-
lated information to make informed decisions on immunization. A context-independent measure
of Vaccination HL was developed, measuring the process dimensions of finding, understanding,
judging, and applying vaccination information for better immunization, based on a partial adap-
tation of the four vaccination-related items included in the HLS-EU survey. In addition, the optional
package on HL-VAC also included one item on personal vaccination behavior during the last five-
year period, four items referring to personal confidence in vaccinations (Confidence), three items
on myths about possible risks of getting vaccinated (Calculation/Conspiracy), and one item on the
risk of getting a disease for which a vaccine exists if not vaccinated (Complacency).
Seven countries (AT, BE, CZ, HU, IE, PT, SI) implemented the complete optional package on Vac-
cination HL, and four additional countries (BG, DE, IT, and NO) collected data on at least the four
HL-VAC items and general background variables. Differing by country, data were collected by PAPI,
CAPI, CATI, and CAWI, or combinations of these. In total, data on Vaccination HL are available for
just over 25,000 respondents in Europe.
There was a rather common ranking of difficulty of the four items across countries, with “judging
which vaccinations you or your family needs” as the most difficult item, followed by “finding in-
formation on recommended vaccinations”, “deciding if you should have a flu vaccination”, and
“understanding why you or your family may need vaccinations”.
4.4.5 Construction of scores, Validation, and Psychometric Properties
A score was calculated for Vaccination HL by combining the response categories “very easy” and
“easy”, adding these up over the four items, and standardizing the raw score for a range from 0
to 100. The internal consistency of the Vaccination HL scale with an, on average, Cronbach’s alpha
of 0.72, varied between 0.60 and 0.85, indicating that the reliability of the scale is acceptable for
most countries. Confirmatory factor and discriminant analyses revealed that the HL-VAC measures
a different but related trait or competencies than the overall HL scale (HLS19-Q12). The overall
data-model fit to the Rasch model was sufficient for the Vaccination HL scale for five countries,
acceptable for four countries, but poor for two countries. The scale did not measure invariantly
across countries since the “difficulty order” of the items varied between countries.
The distribution of the Vaccination HL score was negatively skewed across all countries, suggest-
ing a ceiling effect. The mean score for all countries (equally weighted) was 75, varying from 58
Executive Summary XXV

to 87. In terms of APRPs, on average 25% of the responses rated the Vaccination HL items as either
“very difficult” or “difficult”, varying between countries from 13% to 45%.
Respondents with low education (in six countries), low self-perceived level in society in ten coun-
tries), some or severe financial deprivation in all 11 countries), and limited by health problems (in
eight countries) had lower Vaccination HL mean scores compared to the corresponding compari-
son groups.
In multivariable linear regression models with five potential social determinants, there is a weak
social gradient for Vaccination HL, with financial deprivation being the predictor with the highest
ß=-0.17 on average (significant for all but one countries), followed by level in society (ß=0.05,
significant for three countries), and education (ß=-0.04, significant for five countries). In a model
with General HL added, General HL is by far the predictor with the highest ß=0.51 (varying across
countries from ß=0.39 to ß=0.70).
In all but one country, Vaccination HL is positively correlated with confidence in vaccinations,
knowledge about the risks of vaccines, and risk assessment of developing a specific disease if not
vaccinated. Vaccination behavior, defined as the odds of someone in the family being vaccinated
in the last five years, increased as a function of Vaccination HL for five out of seven countries,
when controlling for socio-demographic and socio-economic variables, and being trained in a
health profession. Mediation analysis using the Baron and Kenny approach showed that the rela-
tionship between Vaccination HL and vaccination behavior is at least partly mediated by confidence
in vaccinations, risk knowledge, and risk perception.
As such, the measure is suitable for measuring Vaccination HL in different countries but could be
further developed with a focus on additional, specific HL-VAC tasks. The results demonstrated the
relevance of Vaccination HL by revealing that a considerable proportion of respondents have lower
Vaccination HL, that there is a social gradient for Vaccination HL in most countries, and that Vac-
cination HL is associated with vaccination behavior in most countries.
XXVI © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
4.5 Health literacy and health-related quality of life as a mediator
for health costs (Chapter 14)
4.5.1 Relevance
The objectives/research questions of this chapter were twofold: first to explore whether there is
a relationship between general, comprehensive HL (as measured by HLS19-Q12), and health-re-
lated quality of life (as measured by EQ-5D-5L), and secondly to explore the relationship between
HL and work absenteeism.
Both health-related quality of life and absenteeism have cost and economic implications for health
services, for individuals and for society. Such costs are in the context of rising health care ex-
penditure and limited health budgets globally. The findings provide evidence to inform policy
makers of the importance of interventions to enhance HL as a disease prevention strategy so as
to improve health-related quality of life and reduce the incidence of absenteeism amongst citi-
zens. Such outcomes will assist in more efficient use of scarce resources for health care expendi-
ture, a better quality of life for citizens, with consequent implications for society.
4.5.2 Specific Instruments and Indicators
Health Related Quality of life (HRQoL) denotes the impact of health on a person’s ability to live a
fulfilling life, defined by the World Health Organisation (WHO) as an individual's perception of their
position in life in the context of the culture and value systems in which they live and in relation to
their goals, expectations, standards, and concerns. HRQoL thus represents a broad concept of
physical, psychological, and social functioning and well-being including both positive and nega-
tive aspects.
Absence from work was measured by the number of days of absenteeism per year due to health
problems
Specific data for this chapter was collected for the measure EQ-5D-5L in three countries Denmark,
Ireland, and Norway. Differing by country, data were collected by CATI (Ireland and Norway) and
CAWI (Denmark). In total data on EQ-5D-5L are available for nearly 6,000 respondents in the three
countries.
4.5.4 Method
A freely available English syntax file for transforming the responses to the EQ-5D-5L question-
naires into an EQ-5D-5L estimate for individual respondents was used. For regression analyses
with EQ-5D-5L as the dependent outcome variable, it was relied on Ordinary Least Squares (OLS)
estimation with “robust” estimates of variance to account for violations of homoscedasticity.
Executive Summary XXVII

The analysis of number of days of absenteeism per year due to health problems was based on the
two-step model (“Two-part model”) where the first step is based on “probit” and the second step
on General Linear Modelling (GLM) with gamma “distribution family”, log link function and “robust”
estimation of variance.
4.5.5 Results
Research Question 1
The analysis of Health-Related Quality of Life (HRQoL), as measured by EQ-5D-5L, in the three
countries Denmark, Ireland and Norway, shows a significant association between General HL and
health related quality of life. In general, as HL increases so does HRQoL.
The magnitude of the association between General HL and HRQoL is larger than that for the asso-
ciation between education and HRQoL highlighting the importance of HL interventions to improve
General HL levels for adult populations. The associations observed between HRQoL and General
HL and other social determinants of health appears to be additive for education level, gender and
employment status.
Research Question 2
For both Norway and Denmark, a negative correlation between HL and absenteeism was observed.
As General HL increases there is a decrease in absenteeism from work due to health problems. For
Ireland when General HL score increases from 0 to about 70, absenteeism increases, however
absenteeism decreases thereafter, noting that 73% of the respondents have a General HL score
between 70% - 100%. These findings suggest that further research and analysis of the HLS19 in-
ternational data is required to fully understand the complexities surrounding the correlation be-
tween HL and absenteeism from the workplace.
4.5.6 Discussion/Conclusion/Recommendations
This is the first European study to explore and measure the relationship between General HL (as
measured by HLS19-Q12), and health-related quality of life (as measured by EQ-5D-5L) which
have consequential cost and economic implications for the health services, for individuals and for
society. The sample size within this study was large across the three countries where the health-
related quality of life data were collected for EQ-5D-5L. Unlike prior studies of the health eco-
nomic implications of HL all the data were directly measured, with no data inferred.
The findings of the HLS19 study are sufficiently strong for national and local governments to rec-
ognise the importance of General HL for the health and well-being of their citizens and in the
utilization of health services. Investment in HL interventions as a disease prevention strategy at
local, national and regional levels may lead to significant benefits to citizens for their quality of
life alongside more effective use of expensive health services.
XXVIII © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
5. Recommendations
One of the aims of the HLS19 was to gather evidence to inform policy, practice, and further re-
search. The data allow to distinguish aspects and domains of HL that need more attention than
others. The same holds true for different population groups, identifying those at the lower end of
the social gradient who are in more need of support in relation to their HL. The data, however, do
not provide evidence for specific concrete interventions to address the areas that need to be im-
proved.
On these grounds, the HLS19 consortium agreed on a set of recommendations, presented here in
a shortened format.
Regarding General HL
» Health policy should include an investment in longitudinal studies, measuring and monitoring
population HL regularly, and should systematically implement interventions to improve HL.
» Interventions should be specifically targeted at at-risk groups for low HL to reduce the health
gap between groups.
» Interventions to improve HL should focus on all four aspects of processing health-related
information (accessing, understanding, appraising, and applying information) within the do-
mains of healthcare, disease prevention, and health promotion.
» For interventions related to specific, concrete HL tasks, the tasks that are experienced as
being more difficult should be prioritized.
» The quality of health information in the mass media should be improved.
» Interventions to improve HL in relation to mental health should be prioritized and sup-
ported by specific research.
Regarding specific HLs
» Health policy should develop strategies to improve people’s Navigational HL, specifically in-
terventions on systemic and organizational levels to make health systems more health-lit-
erate, user-friendly, and easier to navigate.
» Interventions to improve the communication of health professionals with patients should
have high priority. Specifically, support for health professionals, especially physicians, in
dedicating more time to person-centered communication is needed.
» Regarding Digital HL, emphasis on providing easily accessible, high quality, trustworthy, un-
derstandable, assessable, and applicable health information, as well as communication via
digital sources should be increased.
» Improving Vaccination HL should have top priority, with a focus on judging vaccination in-
formation by improving the trustworthiness of information and communication on vaccina-
tions.
Executive Summary XXIX

Regarding research on HL
» The HL of the adult resident population should be measured regularly in as many countries
as possible.
» The next wave of measuring should be planned for data collection in 2024.
» In preparation for this next wave, more specific research should be funded to analyze existing
data in more depth as well as to revise, extend, and apply the tools for measuring HL and
relevant correlates.
» For the four specific HLs, more detailed analyses and publications on the HLS19 data are
needed as well as further research and development on improvements for later applications.
» Additionally, further specific health literacies or relevant topics of General HL should be re-
viewed, selected, and researched to be included in the next wave of measuring HLs.
» More detailed analyses are needed regarding the costs and economics of HL.
» Further dissemination of the results of the HLS19 through peer-reviewed scientific publica-
tions is required.
XXX © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Table of Contents
Short Summary ......................................................................................................................... VI
Executive Summary.................................................................................................................... X
Abbreviations ..................................................................................................................... XXXVI
1 Background of the HLS19 ................................................................................................. 1

1.1 The relevance of measuring health literacy in health care & public health
(before the HLS-EU) ............................................................................................. 1
1.2 Policy recommendations concerning HL by the WHO and other international
national stakeholders ........................................................................................... 2
1.3 The original HLS-EU study and its follow-up studies ............................................ 4
Concept and definition of Health Literacy .............................................. 4
Methods................................................................................................ 8
Results .................................................................................................. 9
Short forms of the HL measure ............................................................ 10
Follow-up studies ............................................................................... 12
1.4 The European Health Literacy Survey 2019-2021 (HLS19) – the first project
for M-POHL ....................................................................................................... 14
Establishment of the WHO Action Network Measuring Population
and Organizational Health Literacy (M-POHL)....................................... 14
HLS19: M-POHL’s first project .............................................................. 14
Main aim and objectives of HLS19 ......................................................... 15
Principal research questions of the HLS19 ............................................. 16
1.5 References ......................................................................................................... 17
2 Study design .................................................................................................................. 24

2.1 Development of the HLS19 study design .............................................................. 25
2.2 The HLS19 study design ...................................................................................... 26
2.3 Translation and field testing of instruments ....................................................... 28
Description of the national translation processes ................................. 28
Description of national field-testing procedures .................................. 30
2.4 Data collection and processing ........................................................................... 33
Survey sampling procedures and response rates .................................. 33
Study population and representativeness ............................................. 40
Fieldwork ............................................................................................ 44
Data quality checks and data weighting ............................................... 49
Ethical considerations, data protection, and informed consent
by country .......................................................................................... 51
2.5 Summary of study design and its implementation in the HLS19: Discussion
and conclusions ................................................................................................. 52
2.6 References ......................................................................................................... 55
3 The HLS19 instruments ................................................................................................... 57

3.1 Development of the HLS19 instruments ............................................................... 57
The HLS19 instruments measuring General HL: long and short
versions .............................................................................................. 57
Core and optional correlates of General HL .......................................... 60
Specific HL measures and their correlates: optional packages .............. 63
3.2 Overview of the final HLS19 instruments.............................................................. 64
3.3 Description of the generic HLS19-Q12 instrument ............................................... 65
Table of Contents XXXI

3.4 Overview of the implementation of the HLS19 instruments in the participating
countries ........................................................................................................... 66
3.5 References ......................................................................................................... 68
4 Methods ........................................................................................................................ 70
4.1 Introduction ....................................................................................................... 71
4.2 Construction of the HL scales ............................................................................. 72
4.3 Weighting of data for analyses ........................................................................... 74
4.4 Univariate statistics ............................................................................................ 74
4.5 Bivariate associations ......................................................................................... 75
Identification of vulnerable or disadvantaged groups ........................... 76
4.6 Regression analyses ........................................................................................... 76
4.7 Validity analyses ................................................................................................ 81
Cronbach’s alphas............................................................................... 81
Confirmatory factor analyses ............................................................... 82
Rasch analyses .................................................................................... 82
Predictive validity ................................................................................ 84
Comparisons of the HLS19-Q12 short form with the HLS19-Q16
and HLS19-Q47 long forms................................................................. 84
4.8 Discussion and conclusions................................................................................ 84
4.9 References ......................................................................................................... 86
5 The HLS19-Q12 measure ................................................................................................ 88

5.1 Distributions of the HLS19-Q12 items ................................................................. 89
5.2 Average Percentage Response Patterns (APRP) .................................................... 91
5.3 Internal consistency of the HLS19-Q12 ................................................................ 92
5.4 Results of Confirmative Factor Analyses for the HLS19-Q12 ................................. 94
5.5 Rasch analyses ................................................................................................... 97
5.6 How well the short forms HLS19-Q12 and HLS19-Q16 represent each other
and the long form HLS19-Q47 statistically .......................................................... 99
5.7 Distributions of the HLS19-Q12 scores .............................................................. 102
5.8 Differences in mean HLS19-Q12 scores between the country sample and
selected vulnerable subpopulations .................................................................. 103
5.9 Categorical levels for the HLS19-Q12 ................................................................ 106
5.10 Discussion and conclusions.............................................................................. 109
5.11 References ....................................................................................................... 110
6 Determinants and a social gradient of General Health Literacy measured by the

HLS19-Q12 ................................................................................................................... 111
6.1 Background ..................................................................................................... 112
6.2 Distribution of selected determinants, their associations, and correlations
with HL ............................................................................................................ 114
Socio-demographic determinants: Gender and age ............................ 117
Socio-economic determinants: Level in society, education, and
financial deprivation .......................................................................... 121
Further determinants: Migration background, long-term illness,
training in a healthcare profession, and status of employment ........... 128
6.3 Regression analyses ......................................................................................... 136
Base model 1: Effects on HL of three measures of socio-economic
status, gender, and age ..................................................................... 136
Model 2: Effects on HL of migration background, three measures
of socio-economic status, gender, and age ....................................... 138
Model 3: Effects on HL of long-term illness, three measures of socio-
economic status, gender, and age ..................................................... 140
XXXII © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Model 4: Effects on HL of training in a healthcare profession, three
measures of socio-economic status, gender, and age ........................ 142
6.5 References ....................................................................................................... 145
7 General Health Literacy as a predictor of health behaviors and lifestyles ....................... 148
7.1 Background ..................................................................................................... 149
7.2 Spearman correlations among indicators of health behaviors and lifestyles,
with General HL and selected socio-demographic and socio-economic
determinants ................................................................................................... 151
7.3 General HL and BMI .......................................................................................... 152
7.4 General HL and smoking behavior .................................................................... 155
7.5 General HL and alcohol consumption ............................................................... 157
7.6 General HL and physical activity ....................................................................... 159
7.7 General HL and fruit and vegetable consumption .............................................. 161
7.9 References ....................................................................................................... 164
8 General Health Literacy as a predictor of health status.................................................. 165

8.1 Background ..................................................................................................... 166
8.2 Spearman correlations among indicators of health status with General HL
and selected socio-demographic and socio-economic determinants................. 167
8.3 General HL and self-perceived health ............................................................... 168
8.4 General HL and long-term illnesses/health problems ....................................... 171
8.5 General HL and limitations due to health problems ........................................... 174
8.7 References ....................................................................................................... 177
9 General Health Literacy as a predictor of health care utilization .................................... 179

9.1 Background ..................................................................................................... 180
9.2 Spearman correlations among indicators of health care utilization, with
General HL and selected socio-demographic and socio-economic
determinants ................................................................................................... 182
9.3 General HL and the utilization of emergency services ....................................... 183
9.4 General HL and the utilization of GPs/family doctors ........................................ 186
9.5 General HL and the utilization of medical or surgical specialists ....................... 189
9.6 General HL and the utilization of inpatient hospital services ............................. 192
9.7 General HL and the utilization of day patient hospital services .......................... 195
9.9 References ....................................................................................................... 199
10 Navigational Health Literacy ......................................................................................... 201

10.1 Background and development of instrument .................................................... 202
Overview of the relevance, existing research, and measures of
Navigational Health Literacy .............................................................. 202
Arguments for providing a new measure and the procedure for
developing a measure for Navigational HL ......................................... 203
Objectives and research questions on Navigational HL ....................... 206
Countries participating in Navigational HL ......................................... 207
10.2 Methods of analyses and results....................................................................... 207
Distributions of individual HLS19-NAV items by country ..................... 208
Psychometric validity analyses ........................................................... 209
Table of Contents XXXIII

Distribution of the Navigational HL scores by country ........................ 214
Identification of specific vulnerable/disadvantaged subpopulations ... 215
Determinants of Navigational HL ....................................................... 216
Consequences of Navigational HL ...................................................... 218
10.3 Discussion and Conclusion............................................................................... 225
10.4 References ....................................................................................................... 229
11 Communicative Health Literacy with physicians in health care services ......................... 233
11.1 Background and development of instrument .................................................... 234
Communicative Health Literacy in health care services ....................... 234
Arguments for providing a new measure and the procedure for
developing the measure .................................................................... 236
Objectives and research questions on Communicative HL .................. 240
Countries using the HLS19-COM-P ..................................................... 241
11.2 Methods and results ........................................................................................ 241
Distributions of individual items by country ....................................... 242
Psychometric validity analyses ........................................................... 246
Distributions of scores ...................................................................... 253
General and specific determinants of Communicative HL ................... 257
Communicative HL and self-perceived health .................................... 262
11.3 Discussion and Conclusions ............................................................................. 266
11.4 References ....................................................................................................... 268
12 Digital Health Literacy .................................................................................................. 275

12.1 Background and development of the new instrument........................................ 276
Rationale for providing a new measure and procedure of developing
the measure and specific correlates for the measure in HLS19 ............. 278
Definition and the Concept of Digital HL in HLS19 ............................... 278
Objectives and research questions of the Digital HL optional
package ............................................................................................ 279
Countries participating in the Digital HL Optional Package ................. 280
12.2 Methods of analyses and results....................................................................... 280
Distributions of individual items of measuring Digital HL ................... 281
Validity and psychometric properties of the Digital HL scale ............... 287
Distributions of HL-DIGI score measures ........................................... 291
Use of digital resources ..................................................................... 294
Determinants and social gradient of Digital HL .................................. 295
Health related consequences of Digital HL ......................................... 301
12.3 Summary, Discussion and Conclusions ............................................................. 307
12.4 References ....................................................................................................... 308
13 Vaccination Health Literacy .......................................................................................... 312

13.1 Background and instrument ............................................................................. 313
Vaccination Health Literacy................................................................ 313
Overall objective and research questions ........................................... 317
Measuring vaccination-related HL and vaccination-specific
background variables in the HLS19 ..................................................... 317
Countries implementing the optional package on vaccination-related
HL in their HLS19 national assessment ................................................ 319
13.2 Methods and results ........................................................................................ 320
Responses at item level by country .................................................... 322
Psychometric properties of the Vaccination HL scale .......................... 324
Distributions of the Vaccination HL score by country ......................... 328
Vulnerable/disadvantaged subpopulations ........................................ 328
XXXIV © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Determinants of Vaccination HL......................................................... 331
Consequences of Vaccination HL ....................................................... 333
13.4 References ....................................................................................................... 339
14 Health literacy and health-related quality of life as a mediator for health costs ............. 344
14.1 Introduction ..................................................................................................... 345
14.1.1 HL and health-related quality of life .................................................. 348
14.2 Overall objective and research questions .......................................................... 349
14.3 Methods .......................................................................................................... 349
14.3.1 The statistical analyses ...................................................................... 350
14.3.2 The variables included....................................................................... 351
14.1 Results ............................................................................................................ 352
14.1.1 HL and Health-related quality of life .................................................. 352
14.1.2 HL and absenteeism .......................................................................... 357
14.2 Discussion and Conclusions ............................................................................. 359
14.2.1 Main Findings ................................................................................... 359
Strengths of the investigation of HL and health-related quality of
life as a mediator for health costs ...................................................... 360
Limitations of the investigation of HL and health-related quality of
life as a mediator for health costs ...................................................... 361
14.2.2 Implications for future research ......................................................... 361
14.2.3 Implications for policy ....................................................................... 362
14.2.4 Conclusion ........................................................................................ 362
14.3 References ....................................................................................................... 363
15 Recommendations for policy, practice, and research .................................................... 366

15.1 Recommendations and proposed actions relating to General HL ....................... 368
Recommendations and proposed actions concerning specific
population groups most affected by limited General HL ..................... 369
Recommendations and proposed actions addressing limited HL in
specific aspects and domains of General HL....................................... 371
Recommendations and proposed actions addressing concrete
tasks of General HL ........................................................................... 372
15.2 Recommendations and proposed actions for specific health literacies ............... 376
Navigational HL ................................................................................. 377
Communicative HL: Communicating with physicians .......................... 378
Digital HL .......................................................................................... 380
Vaccination HL .................................................................................. 382
15.3 Recommendations and actions proposed for research ...................................... 384
15.4 References ....................................................................................................... 386
Figures and Tables ................................................................................................................ 387
Table of Contents XXXV

Abbreviations
General abbreviations
APRP Average percentage response patterns

BMI Body mass index
DIF Differential item functioning
EHIS European Health lnterview Survey
ESS European Social Survey
EU-SILC The European Union Statistics on Income and Living Conditions
HL Health literacy
HLS19 The International Health Literacy Population Survey 2019-2021 of M-POHL
ICC International Coordination Center
GÖG Austrian National Public Health Institute
NSC National Study Center
M-POHL Action Network on Measuring Population and Organizational Health Literacy
n.a. Not applicable
OP Optional package
PI Principal investigator
PR Policy representative
Country codes
AT Austria
BE Belgium
BG Bulgaria
CH Switzerland
CZ Czech Republic
DE Germany
DK Denmark
EL Greece
ES Spain
FR France
HU Hungary
IE Ireland
IL Israel
IT Italy
NL The Netherlands
NO Norway
PL Poland
PT Portugal
RU Russia
SK Slovakia
SI Slovenia
XXXVI © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
Methods of data collection
CAPI Computer-assisted personal interviews

CATI Computer-assisted telephone interviews
CAWI Computer-assisted web interviews
PAPI Paper-and-pencil interviews
Health literacy measures mainly used in tables and figures
GEN-HL General Health Literacy score

HL-DIGI Digital Health Literacy score
HL-DIGI-INT Digital Interaction Literacy score
HL-NAV Navigational Health Literacy score
HL-COM-Q6 Health literacy relating to communication with physicians in health care services
score (6 items)
HL-COM-Q11 Health literacy relating to communication with physicians in health care services
score (11 items)
HL-VAC Vaccination Health Literacy score
HL tools
HLS19-Q47 The generic English instrument for measuring health literacy in the general pop-
ulation, 47 items
ulation, 16 items
ulation, 12 items
HLS19-COM-P-Q6 The English HLS19 instrument for measuring health literacy relating to com-
munication with physicians in health care services in the general population, six
items
HLS19-COM-P-Q11 The English HLS19 instrument for measuring health literacy relating to com-
munication with physicians in health care services in the general population, 11
items
HLS19-DIGI The English HLS19 instrument for measuring Digital Health Literacy in the general
population
HLS19-ECON The English HLS19 instrument for measuring the cost and health economics of
health literacy in the general population
HLS19-NAV The English HLS19 instrument for measuring Navigational Health Literacy in the
general population
HLS19-VAC The English HLS19 instrument for measuring Vaccination Health Literacy in the
general population
NVS Newest Vital Sign Test
Abbreviations XXXVII
1 Background of the HLS19
Authors:
Jürgen M. Pelikan (ICC)
Christa Straßmayr (ICC)
for the HLS19 Consortium of the WHO Action Network M-POHL
1.1 The relevance of measuring health literacy in health care

& public health (before the HLS-EU)
The measurement of health literacy (HL) started in the US and mostly involved assessing patients’
HL using different (short) tests of functional HL (e.g., TOFHLA, REALM, NVS) (Rudd 2017). It was
soon observed that low HL had detrimental health outcomes (DeWalt et al. 2004; Nielsen-Bohlman
et al. 2004; Parker 2000). Later, a few studies used the functional HL tests on general populations.
In 2003, the National Assessment of Adult Literacy (NAAL) (Kutner et al. 2006), included a specific
section on measuring HL for the first time. The HL section focused on the ability to read, under-
stand, and apply health-related information in English and focused on health tasks that were
grouped into categories relating to clinical and preventive contexts as well as navigation of the
health system. Concerning general populations and for matters of public health, a broader under-
standing of HL is preferable, as are more comprehensive instruments for measurement. This was
pursued by researchers, who aggregated items relating to health issues from the available pre-
2003 general large-scale adult literacy tests and constructed the Health Activities Literacy Study
(HALS), which contained 191 health-related items and represented a comprehensive measurement
instrument (Rudd et al. 2007). HALS included health activities in five dimensions: health promo-
tion, health protection, disease prevention, health care and maintenance, and health care systems
navigation. Although HALS was utilized in the first decade of this century on samples in the US
(data collection 2003, (Rudd 2007; Rudd et al. 2004)), Canada (data collection 2003, (Murray et
al. 2008), and in a few European countries (data collection 2003 in Italy, Norway, and Switzerland
(Notter et al. 2006)), it was not administered again.
In Europe, the measurement of HL in general populations began when Ilona Kickbusch and Don
Nutbeam recognized the potential of HL for health promotion and public health, in addition to its
importance for clinical healthcare (Kickbusch 2002; Kickbusch/Maag 2008; Kickbusch 2001;
Kickbusch et al. 2006; Nutbeam 2000; Nutbeam 2008; Nutbeam/Kickbusch 2000). After 2004,
Kickbusch advocated for HL at the European Health Forum Gastein (Kickbusch 2004) and was
successful in initiating a HL population survey (the HLS-CH) in Switzerland in 2006 (Wang/Schmid
2007; Wang et al. 2014). The HLS-CH did not use HALS since participant interviews took a signif-
icant amount of time and were not suitable for telephone interviews (CATI). Instead, a new, mainly
self-reporting instrument was developed which was rather comprehensive and yet still took less
time (about 30 minutes in a telephone interview). The HLS-CH was a multidimensional instrument
that contained 127 questions on 30 competences with various response formats that measured
Chapter 1 / General background 1

participant knowledge, behaviors/skills, motivation, and attitudes. As a result, the HLS-CH pro-
vided a profile for different dimensions of HL rather than one general measure for HL (as provided
by HALS). The approach underlying the HLS-CH for measuring HL was “closer to the public health
asset model than the clinical risk factor one” (Wang et al. 2014). As mentioned above, this per-
ception-based instrument was first applied using CATI in a randomly selected sample of n=1,250
respondents from the resident population aged 15 and above in Switzerland in 2006.
Between the proponents of ‘objective’ performance-based tests and those of ‘subjective’ percep-
tion-based instruments, a debate persists as to how best measure HL. Most of the existing tests,
except for HALS, measured – and still measure – a narrow understanding of functional health
literary and not HL’s interactive and critical aspects (Nutbeam 2000). Of course, it can be argued
that a test instrument is preferable if decisions about the individual participants are based on the
results of their assessments. To measure population HL for public health, however, it is more
important to assess a comprehensive concept of HL, which is accomplished more efficiently by
using a self-reporting perception-based instrument.
The results of the HLS-CH also stimulated a public and political debate in Switzerland regarding
health policy. This suggested that HL was more acceptable as a bona fide public health issue once
evidence-based data on distributions of and associations with HL became available within the
general population, as was the case earlier in the US, as well as in Canada, and Australia. The
experiences of measuring population HL in Switzerland (and the health policy debate that the HLS-
CH study triggered off) stimulated interest in measuring population HL in some of the European
Union’s member states. While representatives of the European Commission became convinced of
the relevance of investing in an assessment of population HL in 2006, it took three years until a
research consortium was established and a research proposal was developed, accepted, and sup-
ported by the Executive Agency for Health and Consumers (EAHC) of the European Union (EU). So,
work on a new, international study, called the HLS-EU, began in 2009.
1.2 Policy recommendations concerning HL by the WHO and

other international and national stakeholders
In the WHO’s Ottawa Charter (1986), HL was not mentioned but, as one of its Health Promotion
Action strategies, “Develop personal skills” was introduced, with the following explanation: “Health
promotion supports personal and social development through providing information, education
for health and enhancing life skills. By so doing, it increases the options available to people to
exercise more control over their own health and over their environments, and to make choices
conducive to health. Enabling people to learn throughout life, to prepare themselves for all of its
stages and to cope with chronic illness and injuries is essential. This has to be facilitated in school,
home, work and community settings. Action is required through educational, professional, com-
mercial and voluntary bodies, and within the institutions themselves.” In the WHO’s Health Pro-
motion Glossary (WHO 1998) a definition of HL was included as: “The cognitive and social skills
which determine the motivation and ability of individuals to gain access to, understand and use
information in ways which promote and maintain good health” (Nutbeam 1998). This was followed
2 © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19)
by the WHO’s publication Health Literacy: The solid facts (Kickbusch et al. 2013; Kickbusch et al.
2016), which partly built on the concepts and results of the HLS-EU study (2012) and collected
evidence on the measurement of and interventions in HL in different settings. It also recommended
measuring population and organizational HL in Europe on a regular basis.
HL was also a topic at several of the WHO’s Global Conferences on Health Promotion and will be
included in the program of the WHO’s 10th Global Health Promotion Conference and was high-
lighted in documents related to these conferences, e.g., The Nairobi Call to Action for Closing the
Implementation Gap in Health Promotion (2009), or the Shanghai Declaration on promoting health
in the 2030 Agenda for Sustainable Development (2016). It was also mentioned in documents like
the Bangkok Charter for Health Promotion in a Globalized World (2005), the Mexico City Political
Declaration on Universal Health Coverage (2012), the Helsinki Statement on Health in All Policies
(2013), the Montevideo Roadmap 2018-2030 on NCDs as Sustainable Development Priority
(2017), the Declaration of Astana (2018), and the resolution “Towards the implementation of HL
initiatives through the life course” (2019) (EUR/RC69/R9) which demands, among others, the pro-
motion of HL with a focus on reducing health inequities and the strengthening of HL measure-
ments and action. Also, two of the WHO’s HEN reports were dedicated to HL (Rowlands et al. 2018;
WHO Regional Office for Europe 2019).
While the concept of ‘health literacy’ was originally used in the United States and Canada; it was
later used internationally not only in connection with health care but also in the context of public
health (Pleasant/Kuruvilla 2008), also in Europe. HL was explicitly mentioned as an area of priority
action in the European Commission’s Health Strategy 2008-2013 (European Commission 2007).
It was linked to the core value of citizen empowerment, and the priority actions proposed by the
European Commission included the promotion of HL programs for different age groups. HL has
since gained momentum on the European health agenda. Closely linked to empowerment, it can
be defined as ‘the ability of citizens to make sound decisions concerning health in daily life - at
home, at work, in health care, at the marketplace, and in the political arena’ (Kickbusch/Maag
2008). This is exemplified by the inclusion of HL in European policy documents such as in the
European Commission’s White Paper entitled ‘Together for Health’ (European Commission 2007),
the Health 2020 strategy of the (WHO 2013) WHO European Region (2012). the Vilnius Declaration
on Sustainable Health Systems for Inclusive Growth in Europe, agreed to by health ministers during
the Lithuanian Presidency of the European Union (The Lithuanian Presidency of the Council of the
European Union 2013).
Besides these international initiatives, from early on, national programs and action plans, etc. were
initiated to take better account of HL in health care and public health and to strengthen it with the
help of specific measures. Stimulated by measuring HL, action plans were published for Canada
(Public Health Association of British Columbia 2012; Rootman/Gordon-El-Bihbety 2008), the US
(U.S. Department of Health and Human Services 2010), Australia (Australian Bureau of Statistics
2008); (Australian Commission on Safety and Quality in Health Care 2014), and New Zealand
(Ministry of Health 2010; Ministry of Health 2015).
Some countries in Europe also issued policy documents for improving HL, e.g., Wales (Puntoni
2010), Ireland (National Adult Literacy Agency/O'Connor 2012), Scotland (Scottish Government

2014). England (Public Health England 2015), and Portugal (Grupo de Trabalho do Plano Nacional
de Saúde 2013). Thanks to the HLS-EU, at least two countries introduced sustainable policy initi-
atives, Austria (Federal Ministry of Health and Women's Affairs 2012), and Germany (Schaeffer et
al. 2020; Schaeffer et al. 2018). Several countries also initiated alliances or coordination platforms
(e.g., Austria, Germany, Switzerland) or prizes for excellent HL practice and research (e.g., Ireland,
Netherlands, Switzerland), and in many countries official websites related to HL were established.
Later, already within the context of M-POHL, a few countries already initiated policies and strate-
gies to improve HL, e.g., the Czech Republic (integration of HL into the Health 2020 Roadmap,
founding a National HL Institute), and Norway (Ministry of Health and Care Services 2019). Con-
cerning national policy initiatives for promoting HL, Adriaenssens et al. (2021) provide an analysis
on lessons to be learned based on six countries (Australia, Austria, Ireland, the Netherlands, Por-
tugal, and Scotland) for developments of HL policy plans, implementation of these, their content,
opportunities and threats, and evaluation of the HL policies and plans.
1.3 The original HLS-EU study and its follow-up studies
This section is mainly based on the following publications: Pelikan/Ganahl (2017); Pelikan et al.
(2019); Pelikan et al. (2020a); Pelikan (2020b).
Concept and definition of Health Literacy
While studies had demonstrated the prevalence of limited HL in the US and across the world (for
an overview see Pleasant (2013)), population data on HL levels in the European Union (EU) were
unavailable. Therefore, starting in 2004, some researchers spent several years trying to initiate an
international comparative survey (Pelikan et al. 2019), which finally resulted in a grant agreement
with the European Agency for Health and Consumers in 2008 (HLS-EU Consortium 2008). The
HLS-EU Project was then co-financed by the European Commission’s Health Programme (Grant
no. 2007-113) and the national organizations taking part in the project. Due to limited founding,
a consortium of nine organizations from only eight EU member states (Austria, Bulgaria, Germany,
Greece, Ireland, the Netherlands, Poland, and Spain) launched the European Health Literacy Project
(HLS-EU) to conduct the first comparative European HL survey.
Notable aims of the project included developing a model instrument for measuring HL and gen-
erating first-time data on HL across diverse populations in the EU. This would then facilitate a
comparative assessment and provide an empirical basis for European, national, and regional health
policies.
A systematic literature review of 17 existing explicit HL definitions and 12 conceptual models

resulted in an integrated definition of the comprehensive concept of HL: “Health literacy is linked
to literacy and encompasses people’s knowledge, motivation and competencies to access, under-
stand, appraise and apply information to form judgments and take decisions in terms of
healthcare, disease prevention and health promotion to improve quality of life during the life
course” (Sørensen et al. 2012).
Figure 1.1: The HLS-EU Conceptual Model of Health Literacy
Source: (Sørensen et al. 2012)
In addition, a conceptual and generic model (Figure 1.1) was developed that captures the most
comprehensive evidence-based dimensions of HL with its main antecedents and consequences. In
the definition and the model, health promotion is understood in the broad sense defined by the
World Health Organization in the Ottawa Charter ((WHO Regional Office for Europe 1986). This
conceptual model and the related definition are more comprehensive than most of their forerun-
ners since they follow a broader understanding of “literacy” as well as of “health”.
This model and definition of HL served as a basis for developing a multidimensional conceptual
matrix to operationalize a questionnaire aimed at measuring comprehensive HL in general popu-
lations. It was named the HLS-EU-Q47 and was developed in eight steps (item generation, focus
groups, pre-testing, expert consultation, finalization of the questionnaire, plain language check,
translation, field test) (Sørensen et al. 2013).

Table 1.1:
Matrix of subdimensions of HL based on the HLS-EU Conceptual Model (Sørensen et al. 2012)
used for developing the HLS19 instruments
Access/obtain in- Understand infor- Appraise/judge/eval- Apply/use infor-

Health Literacy formation rele- mation relevant uate information rel- mation relevant
vant for health for health evant for health for health
1) Ability to ac- 2) Ability to un- 4) Ability to make

3) Ability to interpret
cess information derstand medical informed deci-
Health Care and evaluate medical
on medical or information and sions on medical
information
clinical issues derive meaning issues
6) Ability to un-
8) Ability to judge
5) Ability to ac- derstand infor- 7) Ability to interpret
Disease Preven- the relevance of
cess information mation on risk and evaluate infor-
tion information on
on risk factors factors and derive mation on risk factors
risk factors
meaning
10) Ability to un-

11) Ability to inter- 12) Ability to form
9) Ability to up- derstand health-
pret and evaluate in- a reflected opin-
Health Promotion date oneself on related infor-
formation on health- ion on health is-
health issues mation and derive
related issues sues
meaning
Source: Sørensen et al. (2012)
This model distinguishes between four aspects of health-related information management or four
cognitive dimensions: to access/obtain, understand, appraise/judge/evaluate, and apply/use in-
formation relevant for health. For each of the four aspects, three domains are considered in which
dealing with health-relevant information is necessary: healthcare (or managing illness), disease
prevention, and health promotion (Table 1.1).
» The domain of healthcare is where HL refers to the ability to access information on medical
or clinical issues, to understand medical information and derive meaning, to interpret and
evaluate medical information, and to make informed decisions on medical issues as well as
comply with medical advice.
» The domain of disease prevention is where HL involves the ability to access information on
risk factors for health, to understand information on risk factors and derive meaning, to in-
terpret and evaluate information on risk factors, and to make informed decisions to protect-
ing against risk factors for health.
» The domain of health promotion is where HL refers to the ability to regularly update oneself
on determinants of health in the social and physical environment, to understand health-re-
lated information and derive meaning, to interpret and evaluate information on determinants
of health in the social and physical environment, and to make informed decisions on health
determinants in the social and physical environment, and also engage in joint action.
Figure 1.2:
The Vienna Model of Health Literacy defining the principal determinants and consequences of HL
Source: HLS-Consortium
To analyze associations of HL with relevant correlates, determinants, or consequences of HL, the

Vienna HLS-EU team specified the original generic/logical model in more detail (Figure 1.2). The
Vienna Model distinguishes between personal and situational factors influencing personal HL as
well as between direct and indirect effects of HL on an individual’s health-relevant behaviors,
health status, and illness behaviors. A main direction for causality is proposed as well as the pos-
sibility of reciprocal or cyclical causality (Pelikan/Ganahl 2017a).
According to the Vienna Model (see Figure 1.2), individual or personal HL (2.) is influenced by
personal determinants (1.) like socio-demographic and socio-economic factors (such as gender,
age, educational level, migration status) on the one hand and situational determinants on the
other hand. Personal HL, in turn, can directly influence lifestyle-related health behaviors (3.) like
tobacco and alcohol consumption, physical activity, and nutrition. Personal HL can have a direct
and indirect influence on indicators of health status (4.) like self-perceived health and long-term
illnesses/health problems. Long-term illnesses/health problems could also be considered a per-
sonal determinant of HL. Next, illness behaviors (5.) like the extent of utilization of health care
services can be directly influenced by HL, and indirectly by its effects on health behaviors and
health status. Furthermore, health behaviors, health status, and illness behaviors can also be in-
fluenced by personal and situational determinants. The model also includes (albeit smaller) causal
or cyclical effects in the opposite direction.

Methods
The European Health Literacy Survey (HLS-EU, 2009-2012) was conducted during the summer of
2011 across eight European countries (Austria, Bulgaria, Germany (North Rhine-Westphalia),
Greece, Ireland, Netherlands, Poland, and Spain). In each country, a random sample of approxi-
mately 1,000 EU citizens, 15 years and older, was interviewed, yielding a total sample of approx-
imately 8,000 respondents. On behalf of the HLS-EU consortium, TNS Opinion collected the data,
applying Eurobarometer standards in methodology and sampling procedures, i.e., for EU citizens
only, not for residents with other nationalities living in these countries. Data were collected face
to face via a standardized questionnaire using a Computer Assisted Personal Interviewing (CAPI)
mode in all countries except for Bulgaria and Ireland, where a Paper Assisted Personal Interview
(PAPI) was used. To measure HL, the 47 items of the instrument labelled HLS-EU-Q47 were as-
sessed using a 4-point self-report scale (very easy, easy, difficult, very difficult) to measure the
perceived difficulty of selected health-relevant tasks. Therefore, the HLS-EU-Q47 measures self-
perceived HL and reflects the fit between individual competences and situational complexities or
demands. This must be borne in mind when interpreting the survey results, and especially when
comparing these results across the participating countries.
Based on the 47 items, a general index and seven sub-indices were constructed (healthcare, dis-
ease prevention, and health promotion as well as finding, understanding, appraising, and using
health-related information) as well as sub-sub-indices for the 12 cells of the conceptual matrix.
Indices represent the sum of the values for the answer categories of single items (very easy = 4,
rather easy = 3, rather difficult = 2, very difficult = 1) for individuals answering at least 80% of
the items included in the specific index. Following this procedure, elevated index values suggest
higher HL. To facilitate comparability between indices, the general index and the sub-indices were
standardized into a scale from 0 to 50 and the sub-sub-indices into a scale from 0 to 5. These
indices were used for data analysis. Reliability using Cronbach’s alpha was well above 0.7 for the
general index, sub-indices, and most of the sub-sub-indices as well, with only a few at least close
to 0.7. The index values for the general index and the sub-indices had normal distribution, with
some ceiling effects for higher HL; this was more pronounced for the sub-sub-indices. The some-
what skewed normal distributions indicate that the HLS-EU-Q47 indices are more sensitive and
provide more information for lower rather than higher HL scores, which makes sense for most of
the research questions. Pearson correlations among and between indices were rather high, namely
from r=.89 to r=.93 for the 7 sub-indices with the general index, from r=.70 to r=.81for the sub-
indices among each other, from r=.72 to r=.82, r=.54 to r=.84, and r=.42 to r=.69 for the sub-
sub-indices with the general index, with the sub-indices, and among each other respectively.
These correlations suggest that all indices measure HL, or at least have something in common,
but that there is also variation in the perceived difficulty of tasks by specific sub-dimensions of
HL.
Concurrent discriminant validity for the general index was investigated for the NVS test. Pearson
correlations with the NVS were r=.27 for the general index and r=.24 respectively r=.25 for the
domain-specific sub-indices. For the stages of information management, the correlation with the
NVS was somewhat higher for “accessing” and “understanding” (r=.29) than for “appraising”
(r=.18) or “applying” (r=.22). A similar pattern was found for the sub-indices. Hence, functional
HL measured by a performance-based test explained a certain amount of variation among the
comprehensive, self-assessed HLS-EU HL measures but not significantly more than the level of
education of the respondents.
Furthermore, categorial levels for the HLS-EU HL measures were defined, as for other HL measures
(e.g., the HALS or the NVS), for comprehensive general HL, domain-specific sub-indices, and
stage-specific sub-indices; for the sub-sub-indices (due to the small number of items and skewed
distributions) it was not advisable to construct these levels. The likelihood of experiencing tasks
as being difficult guided the criterion for fixing thresholds. In addition, when constructing the
general comprehensive HL index, thresholds were set to minimize ‘external’ information loss to
guarantee that categorized and metric HL indices produced similar correlation strengths and pat-
terns (with the NVS, age, financial deprivation, social status, self-perceived health, and frequency
of utilization of doctors). Thresholds were also set to minimize ‘internal’ information loss by max-
imizing their correlations with their corresponding metric indices. Like for the four categories of
the original items, four levels of HL were defined: inadequate (0-25 pts. or up to 50%), problematic
(>25-33 pts. or 51%-66%); sufficient (>33-42 pts. or 67%-80%), and excellent (>42-50 pts. or
81%-100%). For some analyses, the levels of “inadequate” and “problematic” HL were combined
and defined as “limited” HL.
Results
This first HL survey in Europe (HLS-EU) indicated that limited HL concerns large groups of the
general population in different countries but to varying degrees, namely from 1 out of 3 to 2 out
of 3 citizens depending on the country (HLS-EU Consortium 2012; Sørensen et al. 2015). Thus,
limited HL is not just a problem for a small minority of citizens. Furthermore, in all countries, there
were disadvantaged or vulnerable sub-populations including senior citizens, those with low edu-
cation, in financial difficulties, or with low self-assessed social status as well as those with low
self-assessed health status or frequent use of health care services. Accordingly, a social gradient
was found for the HLS-EU HL measures, like for most other HL measures, again varying consider-
ably across countries, with the strongest effects for financial deprivation, self-assessed social
status, level of education, and age. The survey also revealed somewhat better HL for women than
men.
Concerning the potential consequences of HL, correlation and regression models were investi-
gated for the HLS-EU-Q47 for health behaviors or health risks relating to four variables or indi-
cators: frequency of physical activity, body mass index (BMI), alcohol consumption, and smoking
behavior. The strongest significant associations with HL were found for frequency of physical ac-
tivity, which increased continuously with grouped categories of the index of General HL (for more
detailed results, see Pelikan/Ganahl (2017a)).

To investigate health status, the three indicators of the Minimum European Health Module (MEHM)
were taken: self-assessed health with an item of SF36, the number of chronic diseases, and the
existence of disease-related restrictions. For all three indicators, significant associations were
demonstrated with the HLS-EU-Q47, as well as potential direct effects in linear multiple regression
models (for more detailed results, see Pelikan/Ganahl (2017a)).
To investigate disease-related behavior, the frequency of using four different kinds of health care
services were chosen, and emergency units, hospitals, doctors, and other health care professionals
(such as dentists, physiotherapists, psychologists, dieticians, or opticians) were selected as indi-
cators. The associations of comprehensive HL with all four indicators were significant in the HLS-
EU data but only to a moderate degree (for more detailed results, see Pelikan/Ganahl (2017a)).
These results demonstrate that HL is relevant for health policy, practice, and research in all par-
ticipating European countries but should also be measured for each individual country due to the
considerable differences in results across countries. The HLS-EU received the European Health
Award in 2012 for its societal impact on health policy. In their Health Literacy: The solid facts, the
WHO already took up the conceptualization and definition of HL and the main results of the HLS-
EU survey in 2013.
Short forms of the HL measure
Compared to a performance-based comprehensive test of HL like the HALS, the HLS-EU-Q47 is

an efficient instrument. However, even a quick battery of 47 Likert scale items (taking about 10
minutes interview time), which can also be self-administered (Duong et al. 2017) or done online
(Nakayama et al. 2015), is regarded as too time-consuming for some HL research intended for
general or specific populations. As a result, two short forms, the short-scale HLS-EU-Q16 (3
minutes) and the short-short scale HLS-EU-Q6 (1 minute), were developed using the HLS-EU
measure. For the short instruments, the psychometric properties of a unidimensional scale were
required to measure comprehensive HL which should represent the scope and underlying theo-
retical construct of the long form as far as possible. To guarantee unidimensionality, a one-par-
ametric dichotomous Rasch model was used to select items. To do so, the items had to be dichot-
omized into two categories, “easy” (“fairly” or “very” easy = 1) and “difficult” (“fairly” or “very”
difficult = 0). Rasch analyses were then conducted for every country and for the total sample.
Three split criteria were used: the median of the distribution, gender, and dichotomized level of
education within each country. This resulted in a sub-set of the same 16 items for all countries
which satisfied Rasch characteristics for each country although the order of the items occasionally
varied. Altogether, these 16 items represented the HLS-EU matrix well, with one exception as no
item in the “applying information” cell for “health promotion” in the matrix fulfilled the Rasch
criteria. Hence, this cell was not represented in the short form HLS-EU-Q16. Scale values were
calculated as simple sum scores and varied between 0 and 16. It was recommended to only cal-
culate scale values for respondents who answered at least 14 items. Correlations with the index
of the long form were very high, namely r=.82 for the total sample and varying by country between
r=.73 and r=.88. Correlations with functional HL (NVS test) were similar to these of the index of
the long form (r=.25 overall and varying between r=.14 and r=.38 for individual countries). The
distribution of the HLS-EU Q-16 was J-shaped, with a clear ceiling effect for better HL. Therefore,
only three levels were defined: inadequate HL (scale values 0-8), problematic HL (9-12), and ad-
equate HL (13-16).
As expected, the short form was a less robust measure than the long form, having fewer items as
well as dichotomized answer categories. Overall, the levels of the long and the short form corre-
sponded in 76% of all cases (varying for countries between 68% and 79%). It was possible to cal-
culate score values for sub-scales of the short form, but levels could not be defined. Correlation
patterns with important determinants and potential consequences of HL were very similar for the
long and short form of the instrument. In further studies, the Rasch homogeneity of the 16 items
was confirmed, e.g., for Austrian adolescents (Röthlin et al. 2013) and migrant populations in
Austria (Ganahl et al. 2016), as well as by data from studies of general populations, e.g., for the
Czech Republic and for Hungary.
The HLS-EU-Q16 short form contained about a third of the original items and consequently took
only a third of the interviewing time, about 3 minutes on average. Nevertheless, even this was
regarded as too long for some types of studies. Therefore, another ‘short-short-form’ containing
6 of the 16 items, was constructed and validated, which took about one minute of interviewing
time (for details see Pelikan et al. (2014)). Thanks to these two short forms, the HLS-EU instrument
to measure comprehensive HL became available for efficient studies with the option to benchmark
them with studies that use the long form of the instrument.
In follow-up studies to the original HLS-EU, the HLS-EU-Q16 and HLS-EU-Q6 measures were in-
vestigated further (see Subsection 1.3.5), and additional short forms were developed. First, an
Asian short form Q12, based on factor analyses and including one item for each of the 12 cells in
the conceptual matrix of HL, was developed using data from Taiwan (Duong et al. 2017) and was
later applied to the data of the six countries (Indonesia, Kazakhstan, Malaysia, Myanmar, Taiwan,
Vietnam) in the HLS-Asia study (Duong et al. 2019a) as well as to Vietnamese data (Duong et al.
2019b). Later, a Norwegian Q12 short form, also representing the conceptual matrix and based
on Norwegian data as well as factor and Rasch analyses, was developed (Finbraten et al. 2018).
Inspired by the Asian and Norwegian short forms and to overcome the shortcomings of the HLS-
EU-Q16 and HLS-EU-Q6, a new short form, the HLS-EU-Q12 with 12 items, one for each of the
cells in the conceptual HLS-EU matrix, was constructed using the original HLS-EU data for eight
countries and two additional countries based on Rasch analyses (Waldherr et al. 2021), https://m-
pohl.net/Results). The three short forms with 12 items are similar in as far as in all three the cells
in the conceptual matrix are represented by one item, but the individual items selected only partly
overlap in the three versions. For the HLS19 some of the original wording of the HLS-EU-Q12 was
changed as well as the wording of the categories and this HLS19-Q12 was then the short form that
was primarily used and validated in the HLS19.

Follow-up studies
The languages used in the original HLS-EU study for the complete HLS-EU-Q86 questionnaire
including the 39 correlates, or rather the HLS-EU-Q47 instrument for measuring HL, were Bulgar-
ian, Dutch, English, German, Greek, Polish, and Spanish. Later the complete HLS-EU survey ques-
tionnaire or just the HL measure itself were translated into and validated for additional languages
(e.g., Albanian, Czech, Danish, French, Hebrew, Hungarian, Italian, Maltese, Portuguese, Russian,
Serbo-Croatian, and Turkish). The translated, and partly also extended, questionnaires have been
used in population surveys and smaller research projects in many different countries, and versions
have also been developed for languages outside Europe (including Indonesian, Japanese, Kazakh,
Malay, Burmese (Myanmar), Russian, Traditional Mandarin, and Vietnamese.
Many follow-up studies using the HLS-EU instruments for general populations have been admin-
istered and published (based on: Pelikan/Ganahl (2017); Pelikan et al. (2019); Pelikan et al.
(2020a); Pelikan et al. (2020b).) The references of this section (1.3.5) can be found in Annex 1,
Table 1.1 (this table in the Annex is based on Table 1 in Pelikan et al. (2020) but has been ex-
panded; in Table 8.2 in Pelikan et al. (2019), the studies on general populations are described by
the year of the survey, commissioning institution(s), executing institution(s), design and sampling
method, regional definition of population, age definition of population, sample size, instrument
used for measuring HL, and publications).
For (WHO) Europe:
With the HLS-EU-Q47: Austria, extended sample (Pelikan et al. 2013); Germany HLS-GER (Berens
et al. 2016; Schaeffer et al. 2016; Schaeffer et al. 2017a; Schaeffer et al. 2017b; Berens et al.
2018); Greece (Michou et al. 2019a), Ireland (Doyle et al. 2012), Netherlands (van der Heide et al.
2013), Spain (Catalonia (Contel et al., 2015, Garcia-Codina et al., 2019)), Czech Republic (Kučera
et al. 2016); Hungary (Koltai/Kun 2016); Italy (Palumbo et al. 2016); Portugal (Espanha/Ávila 2016;
Pedro et al. 2016); just one municipality (Azevedo Alves et al. 2018); Kazakhstan (see also Asia)
(Baisunova et al. 2016; Duong et al. 2017b); Switzerland (Bieri et al. 2016).
With the HLS-EU-Q47 and a focus on methodological development or validation: HLS-EU total
(Pelikan et al. 2018; Lorini et al. 2018); Austria (Gerich/Moosbrugger 2018); Germany, adaption
for children (26 items) (Bollweg et al. 2020); Netherlands (van der Heide et al. 2013; van der Heide
et al. 2016); Albania (Toçi et al. 2015); Norway (Finbraten et al. 2017; Finbraten et al. 2018);
Switzerland, food literacy (Grea Krause et al. 2018); Turkey (Abacigil et al. 2019).
With the HLS-EU-Q16: Belgium, health insured people (Van den Broucke/Renwart 2014; Vanden-
bosch et al. 2016; Avalosse et al. 2017); Denmark (Svendsen et al. 2020); Israel (Levin-Zamir et
al. 2016; Levin-Zamir/Bertschi 2018; Baron-Epel et al. 2019); Malta (Office of the Commissioner
for Mental Health 2014).
With the HLS-EU-Q16 (or HLS-EU-Q6) and a focus on methodological development or validation:
Netherlands (Pander Maat et al. 2014; Storms et al. 2017); Spain (Nolasco et al. 2018); Belgium,
people with low literacy (Storms et al. 2017); Denmark (Fransen et al. 2014); France (Rouquette et
al. 2018); Iceland (Gustafsdottir et al. 2020); Italy (Lorini et al. 2017; Lorini et al. 2019); Sweden
(Wångdahl et al. 2019; Wångdahl et al. 2020); Turkey (Emiral et al. 2018).
For Africa:
HLS-EU-Q16: Ghana, Ashanti region (Amoah/Phillips 2019).
For Asia:
With the HLS-EU-Q47: six Asian countries (Indonesia, Kazakhstan, Malaysia, Myanmar, Taiwan,
Vietnam) (Duong et al. 2017b); Japan Q47 (Nakayama et al. 2015); Kazakhstan (Baisunova et al.
2016); Taiwan (Duong et al. 2015)
With the HLS-EU-Q47 in Asia and a focus on methodological development or validation: six Asian
countries, Q12 (Duong et al. 2019a); Japan, validating the HL-SDHQ (Matsumoto/Nakayama 2017);
Malaysia, Q18 (Mohamad et al. 2020); Samoa (Bollars et al. 2019); Taiwan (Huang et al. 2018a),
Q47>Q12 (Duong et al. 2017c), Rasch model (Huang et al. 2018b), Q12>e-healthy diet literacy
(Duong et al. 2020), Rasch model (Huang et al. 2018b); Vietnam Q47>Q12 (Duong et al. 2019b).
With the HLS-EU-Q16 (or HLS-EU-Q6) in Asia and a focus on methodological development or
validation: Indonesia, Q16 & HLS-EU-SQ10-IDN (Rachmani et al. 2019).
HLS-EU instruments have also been applied to investigate the HL of specific populations (for de-
tails, see Annex 1, Table A 1.1):
» young people/students,
» old people/seniors,
» migrants/refugees/asylum seekers,
» other sub-populations,
» Patients.
Due to the wide-ranging acceptance and up-take of the HLS-EU study design in further HL re-
search projects, it was decided to take this study design as the foundation for the HLS19 study and
to develop it further, to the extent that this was necessary.

1.4 The European Health Literacy Survey 2019-2021 (HLS19) –
the first project for M-POHL
Establishment of the WHO Action Network Measuring

Population and Organizational Health Literacy (M-POHL)
Building on the European HL survey (HLS-EU), the WHO’s publication Health Literacy: The solid
facts recommended measuring population and organizational HL regularly in Europe. This was
strongly promoted by the health ministers of the German-speaking “quintet countries” (Austria,
Germany, Liechtenstein, Luxemburg, and Switzerland), who convinced WHO/Europe to establish
an Action Network on Measuring Population and Organizational HL (M-POHL) under the umbrella
of its European Health Information Initiative (EHII). M-POHL was launched in February 2018, with
currently 28 participating member countries from the WHO European Region, as a network of
researchers and policy representatives.
M-POHL’s overall aim, as defined in its Concept Note (M-POHL 2018), is to “support the availability
of high-quality internationally comparative data on HL as a comprehensive and relational concept”.
This entails both the institutionalization of periodical comparative surveys on population HL and
of organizational health literate or HL-sensitive healthcare organizations, settings, and systems,
to facilitate the identification and selection of specific aspects of HL that can be best improved
with health policies and strategies.
HLS19: M-POHL’s first project
After the publication of the HLS-EU study, many countries who could not participate in the original
study did individual follow-up surveys, using the study design of the HLS-EU, and partly extended
its methodology by including additional instruments and variables as well as more complex anal-
yses. Their results were published in individual reports or journals meaning that it was difficult to
use the results for benchmarking across countries and for consented further development of the
methodology. Therefore, it seemed important to again offer the opportunity to participate in a
multinational standardized study and to establish a framework for regular follow-up surveys. In
the years following the HLS-EU, work on the internal differentiation of the concept of HL was
continuing. It was considered important to follow this trend and to use more specific concepts and
instruments to measure relevant selected aspects of the comprehensive concept of HL in general
populations. Thus, in the HLS19, the intention was once again to measure general, comprehensive
population HL in all participating countries but also four specific HLs as optional packages, by
volunteering countries. To allow for the inclusion of further specific HL instruments and correlates,
it was decided to use the short form HLS19-Q12 of the HLS19-Q47 long form as a common measure
for general comprehensive HL across participating countries.
The Health Literacy Population Survey Project 2019-2021 (HLS19) is M-POHL’s first project. Its aim
was to measure population HL in as many member states of the WHO European Region as possible.
To plan and carry out this project, a consortium was established within M-POHL comprising coun-
tries planning to participate in the survey, and working groups were formed to develop individual
modules in the study design, including optional packages for Navigational HL, Communicative HL
with physicians, Digital HL, Vaccination HL, and the costs and economics of HL.
In the end, 17 countries in the WHO European Region - Austria (AT), Belgium (BE), Bulgaria (BG),
Czech Republic (CZ), Denmark (DK), France (FR), Germany (DE), Hungary (HU), Ireland (IE), Israel
(IL), Italy (IT), Norway (NO), Portugal (PT), Russian Federation (RU), Slovakia (SK), Slovenia (SI), and
Switzerland (CH) - participated in the HLS19 project in the period 2019-2021.
Main aim and objectives of HLS19
The main aim of the HLS19 was to prepare, and – as far as possible – standardize, a European
survey of population HL that can also be taken up at regular intervals to report comparative trends
over time.
The specific objectives of the HLS19 were to:
» provide empirical data for evidence-based HL policies in the HLS19 countries, other countries
in M-POHL (with observer status), and the larger WHO European Region,
» use the findings to inform policymakers from different sectors and levels about the rele-
vance of HL in their field of decision making,
» provide evidence for knowledge-based recommendations for HL interventions in the fields
of health promotion, disease prevention, and healthcare,
» contribute to theoretical, conceptual, and methodological developments in research on pop-
ulation HL in Europe,
» contribute to the knowledge base on which factors and covariates explain variation in HL
within and between countries,
» disseminate findings to relevant stakeholders, e.g., policymakers, researchers, healthcare
staff, health promotion practitioners, non-governmental organizations, and others, as well
as the general public,
» stimulate policies that build further capacities and provide resources for implementing
measures to improve HL as well as for researching HL and how it can be implemented effec-
tively,
» support M-POHL in becoming an international resource and expert group on population HL,
» support M-POHL in developing future projects that improve the quality of population HL
measurements in the WHO European Region.

Research and development-related activities for the HLS19
To meet the main aim and objectives set in the HLS19, the following main research and develop-
ment-related activities for the HLS19 were initiated, accomplished, and are still ongoing at the
time this report was being written:
» development of an HLS19 study protocol and survey instruments,

» implementation of a cross-national comparative survey among the HLS19 project partner
countries,
» establishment of an international HLS19 database,
» distribution of results across the HLS19 project partner countries for benchmarking,
» publication and dissemination of the findings in reports, scientific peer-reviewed journals,
books, factsheets, policy briefing papers, and via the M-POHL public website.
While building conceptually on the HLS-EU study (see Chapters 2 to 4 for how the HLS-EU study
design was followed and expanded), the HLS19 project had the ambition to consider major trends
in HL in response to public health challenges.
1. Navigational HL: Health care systems are becoming more and more complex and difficult to
navigate for patients and consumers. Thus, Navigational HL is needed to do this success-
fully. (Chapter 10)
2. Communicative HL with physicians: Patient/physician communication has a long tradition in
health and social sciences but is still seen as problematic. (Chapter 11)
3. Digital HL: Due to digital transformation, Digital HL is already important and will become
even more so in the future. (Chapter 12)
4. Vaccination HL: At the time of writing, the Covid-19 pandemic is still ongoing, so we do not
have to explain why Vaccination HL is important. But it was included in HLS19 long before we
knew about Covid-19. (Chapter 13)
5. HL and health-related quality of life as a mediator for health costs: The relation of HL to the
costs and economics of health care and beyond is of specific interest for health policy.
(Chapter 14)
For more detailed reasons for selecting these topics and argumentation relating to the need for
new measures in these fields, see Chapters 10 to 14.
Principal research questions of the HLS19
The principal research questions in the HLS19 cover the core project in which General HL is meas-
ured and the optional packages which relate to specific HL domains. The overriding research ques-
tion was:
How are the different HL measures distributed and how are they associated with relevant deter-
minants and health consequences in the participating countries?
The specific research questions for the HLS19 were:
» Validation of the new HL measures: How well do they function in different countries? What
further developments are necessary?
» Distributions and aggregate measures of the new HL measures: How do these differ across
countries?
» Distributions of individual items: Which tasks for dealing with health-related information are
most problematic?
» Which specific disadvantaged vulnerable sub-populations can be identified?
» Is there a social gradient for these HL measures?
» How strongly is HL associated with socio-demographic, socio-economic, and additional se-
lected determinants?
» What are the potential health-related consequences of the HL measures? How is HL associ-
ated with selected indicators of health behaviour and lifestyle, health status, and health care
utilization?
» To what extent does General HL have implications for health-related quality of life and for
absenteeism from work due to health problems?
» Which recommendations for policy interventions to improve HL can be given based on the
research findings?
1.5 References
Adriaenssens, Jef; Rondia, Karin; Van den Broucke, Stephan; Kohn, Laurence (2021): Health liter-
acy: What lessons can be learned from the experiences and policies of different countries? In:
The International Journal of Health Planning and Management
Australian Bureau of Statistics (2008): Adult literacy and life skills survey, summary results. Aus-
tralian Government Publishing Service, Canberra
Australian Commission on Safety and Quality in Health Care (2014): Health literacy: Taking action
to improve safety and quality. ACSQHC, Sydney
DeWalt, Darren A; Berkman, Nancy D.; Sheridan, Stacey; Lohr, Kathleen N.; Pignone, Michael P
(2004): Literacy and Health Outcomes. A Systematic Review of the Literature. In: JGIM
2004/19:1228-1239
Duong, T. V.; Aringazina, Altyn; Baisunova, Gaukhar; Nurjanah; Pham, Thuc V.; Pham, Khue M.;
Truong, Tien Q.; Nguyen, Kien T.; Myint Oo, Win; Mohamad, Emma; Tin Su, Tin; Huang, Hsiao-
Ling; Sørensen, Kristine; Pelikan, Jürgen M.; Van den Broucke, Stephan; Wushou Chang, Peter
(2017): Measuring health literacy in Asia: Validation of the HLS-EU-Q47 survey tool in six
Asian countries. In: Journal of Epidemiology 27/2:80-86
Duong, T. V.; Aringazina, Altyn; Kayupova, Gaukhar; Nurjanah; Pham, Thuc V.; Pham, Khue M.;
Truong, Tien Q.; Nguyen, Kien T.; Oo, Win Myint; Su, Tin Tin; Majid, Hazreen Abdul; Sørensen,

Kristine; Lin, I. Feng; Chang, Yuwen; Yang, Shwu-Huey; Chang, Peter W. S. (2019a): Develop-
ment and Validation of a New Short-Form Health Literacy Instrument (HLS-SF12) for the Gen-
eral Public in Six Asian Countries. In: HLRP: Health Literacy Research and Practice 3/2:e91-
e102
Duong, T. V.; Nguyen, T. T. P.; Pham, K. M.; Nguyen, K. T.; Giap, M. H.; Tran, T. D. X.; Nguyen, C.
X.; Yang, S. H.; Su, C. T. (2019b): Validation of the Short-Form Health Literacy Questionnaire
(HLS-SF12) and Its Determinants among People Living in Rural Areas in Vietnam. In: Int J En-
viron Res Public Health 16/18:3346-doi:10.3390/ijerph16183346
European Commission (2007): Together for Health: A Strategic Approach for the EU 2008-2013:
White Paper. OOPEC
Federal Ministry of Health and Women's Affairs (2012): Health Targets Austria – Relevance – Op-
tions – Contexts: Abbreviated Version. Federal Ministry of Health and Women's Affairs
Finbraten, H. S.; Wilde-Larsson, B.; Nordstrom, G.; Pettersen, K. S.; Trollvik, A.; Guttersrud, O.
(2018): Establishing the HLS-Q12 short version of the European Health Literacy Survey Ques-
tionnaire: latent trait analyses applying Rasch modelling and confirmatory factor analysis. In:
BMC Health Serv Res 18/506
Ganahl, Kristin; Dahlvik, Julia; Röthlin, Florian; Alpagu, Faime; Sikic-Fleischhacker, Ankica; Peer,
Sandra; Pelikan, Jürgen M. (2016): Gesundheitskompetenz bei Personen mit Migrationshinter-
grund aus der Türkei und Ex-Jugoslawien in Österreich. Ergebnisse einer quantitativen und
qualitativen Studie. Ludwig Boltzmann Institut Forschungsbericht, Wien
Grupo de Trabalho do Plano Nacional de Saúde (2013): Plano Nacional de Saúde 2012-2016-
Versão Resumo. Direçao-Geral da Saùde
HLS-EU Consortium (2012): Comparative Report on Health Literacy in Eight EU Member States
(Second Extended and Revised Version, Date July 22th, 2014). The European Health Literacy
Survey HLS-EU, Vienna
Kickbusch, Ilona (2002): Health literacy: a search for new categories. In: Health Promotion Inter-
national 17/1:1-2
Kickbusch, Ilona (2004): Improving health literacy-A key priority for enabling good health in Eu-
rope. European Health Forum Gastein, Presentation. Special Interest Session
Kickbusch, Ilona; Maag, Daniel (2008): Health Literacy. In: International Encyclopedia of Public
Health. Hg. v. Kris Heggenhougen, Stella Quah, San Diego: 204-211
Kickbusch, Ilona; Pelikan, Jürgen M.; Apfel, Franklin; Tsouros, Agis D. (2013): Health literacy: The
solid facts. World Health Organization (WHO) Regional Office for Europe, Copenhagen
Kickbusch, Ilona; Pelikan, Jürgen M.; Haslbeck, Jörg; Apfel, Franklin; Tsouros, Agis D. (2016): Ge-
sundheitskompetenz. Die Fakten. Deutsche Fassung, veröffentlicht von der Careum Stiftung,
Schweiz
Kickbusch, Ilona S. (2001): Health Literacy: addressing the health and education divide. In: Health
Promotion International
Kickbusch, Ilona; Wait, Suzanne; Maag, Daniel (2006): Navigating health: the role of health literacy.
Alliance for health and the future, International Longevity Centre-UK, London
Kutner, Mark; Greenberg, Elizabeth; Jin, Ying; Paulsen, Christine; White, Sheida (2006): The Health
Literacy of America’s Adults: Results From the 2003 National Assessment of Adult Literacy
(NCES 2006–483). U.S. Department of Education.Washington, DC:National Center for Educa-
tion Statistics
M-POHL (2018): The Vienna Statement on the measurement of population and organizational
health literacy in Europe. [Online]. M-POHL. Vienna [Zugriff am 20.09.2018]
Ministry of Health (2010): Kōrero Mārama: Health Literacy and Māori: Results from the 2006 Adult
Literacy and Life Skills Survey. Ministry of Health, Wellington
Ministry of Health (2015): Health Literacy Review. A guide. Ministry of Health, Wellington
Ministry of Health and Care Services (2019): National Strategy on Health Literacy (2019-2023).
Murray, TS; Hagey, Jane; Willms, Douglas; Shillington, Richard; Desjardins, Richard (2008): Health
literacy in Canada: a healthy understanding 2008. Ottawa (ON): Canadian Council on Learning.
Ottawa
Nakayama, Kazuhiro; Osaka, Wakako; Togari, Taisuke; Ishikawa, Hirono; Yonekura, Yuki; Sekido,
Ai; Matsumoto, Masayoshi (2015): Comprehensive health literacy in Japan is lower than in
Europe: a validated Japanese-language assessment of health literacy. In: BMC Public Health
1/505
National Adult Literacy Agency; O'Connor, Tom (2012): Health Literacy in Ireland: Benchmarking
the Present State of the Art and Examining Future Challenges and Opportunities. National
Adult Literacy Agency (NALA)
Nielsen-Bohlman, Lynn; Panzer, Allison M; Kindig, David A (Hg.) (2004): Health Literacy. A
Prescription to End Confusion. Washington
Notter, Philipp; Arnold, Claudia; von Erlach, Emanuel; Hertig, Philippe (2006): Lesen und Rechnen
im Alltag Grundkompetenzen von Erwachsenen in der Schweiz. Neuchatel

Nutbeam, Don (1998): Health promotion glossary. In: Health Promotion International 13/4:349-
364
Nutbeam, Don (2000): Health literacy as a public health goal: a challenge for contemporary health
education and communication strategies into the 21st century. In: Health Promotion Interna-
tional 15/3:259-267
Nutbeam, Don (2008): The evolving concept of health literacy. In: Social Science & Medicine
67/12:2072-2078
Nutbeam, Don; Kickbusch, Ilona (2000): Advancing health literacy: a global challenge for the 21st
century. In: Health Promotion International 15/3:183-184
Parker, Ruth (2000): Health literacy: a challenge for American patients and their health care pro-
viders. In: Health Promotion International 15/4:277-283
Pelikan, Jürgen M.; Röthlin, Florian; Ganahl, Kristin (2013): Die Gesundheitskompetenz der öster-
reichischen Bevölkerung - nach Bundesländern und im internationalen Vergleich. Abschluss-
bericht der Österreichischen Gesundheitskompetenz (Health Literacy) Bundesländer-Studie.
Abschlussbericht der Österreichischen Gesundheitskompetenz (Health Literacy) Bundeslän-
der-Studie. Ludwig Boltzmann Institut. Forschungsbericht, Wien
Pelikan, Jürgen M.; Ganahl, Kristin (2017a): Die europäische Gesundheitskompetenz-Studie: Kon-
zept, Instrument und ausgewählte Ergebnisse. In: Health Literacy Forschungsstand und Per-
spektiven. Hg. v. Doris Schaeffer, Jürgen M. Pelikan, hogrefe, Bern: 125-93
Pelikan, Jürgen M.; Ganahl, Kristin (2017b): Measuring Health Literacy in General Populations: Pri-
mary Findings from the HLS-EU Consortium's Health Literacy Assessment Effort. In: Stud
Health Technol Inform 240/34-59
Pelikan, Jürgen M.; Ganahl, Kristin; van den Broucke, S; Sørensen, Karin (2019): Measuring health
literacy in Europe: Introducing the European Health Literacy Survey Questionnaire (HLS-EU-Q)
In: International handbook of health literacy Research, practice and policy across the life-span.
Hg. v. O. Okan et al., Policy Press: 138-115
Pelikan, Jürgen M.; Röthlin, Florian; Ganahl, Kristin; Peer, Sandra (2014): Measuring comprehensive
health literacy in general populations – the HLS-EU instruments. International Conference of
Health Literacy and Health Promotion. Taipei/Taiwan, 6.–8. Oktober 2014
Pelikan, Jürgen M.; Straßmayr, Christa; Ganahl, Kristin (2020a): Health Literacy Measurement in
General and Other Populations: Further Initiatives and Lessons Learned in Europe (and Be-
yond). In: Health Literacy in Clinical Practice and Public Health. Hg. v. R. A. Logan, Siegel E. R.,
IOS Press: 191-170
Pelikan, Jürgen M.; Straßmayr, Christa; Sorensen, Kristine; Nowak, Peter (2020b): Technical Report
on Health Literacy Measurement. Mandated by WHO-HQ. Gesundheit Österreich GmbH. Vi-
enna. Unpublished
Pleasant, Andrew (2013): Health literacy around the world: Part 1. Health literacy efforts outside
of the United States. In: Health Literacy
Pleasant, Andrew; Kuruvilla, Shyama (2008): A tale of two health literacies: public health and clin-
ical approaches to health literacy. In: Health Promotion International 23/2:152-159
Public Health Association of British Columbia (2012): An inter-sectoral approach for improving
health literacy for Canadians. A discussion paper. Victoria
Public Health England (2015): Local action on health inequalities: improving health literacy to re-
duce health inequalities practice resource summary. Public Health England, London
Puntoni, Sarah (2010): Health literacy in Wales. A scoping document for Wales. Welsh Assembly
Government Wales
Rootman, Irving; Gordon-El-Bihbety, Deborah (2008): A vision for a health literate Canada. Report
of the Expert Panel on Health Literacy. Ottawa, ON: Canadian Public Health Association. Ca-
nadian Public Health Association, Ottawa
Röthlin, Florian; Pelikan, Jürgen M.; Ganahl, Kristin (2013): Die Gesundheitskompetenz der 15-
jährigen Jugendlichen in Österreich. Abschlussbericht der österreichischen Gesundheitskom-
petenz Jugendstudie im Auftrag des Hauptverbands der österreichischen Sozialversicherungs-
träger. Ludwig Boltzmann Institut für Health Promotion Research, Wien
Rowlands, GP; Russell, S; O'Donnell, A; Kaner, E; Trezona, A; Rademakers, J; Nutbeam, D (2018):

What is the evidence on existing policies and linked activities and their effectiveness for im-
proving health literacy at national, regional and organizational levels in the WHO European
Region? Health Evidence Network (HEN) synthesis report 57, WHO Regional Office for Europe
Copenhagen,
Rudd, RE; Anderson, JE; Oppenheimer, Sarah; Nath, Charlotte (2007): Health literacy: an update of
public health and medical literature. In: Review of adult learning and literacy 7/175-204
Rudd, Rima E (2007): Health literacy skills of US adults. In: American journal of health behavior
31/1:S8-S18
Rudd, Rima E. (2017): Health Literacy Developments, Corrections, and Emerging Themes. In:
Health Literacy Forschungsstand und Perspektiven. Hg. v. Doris Schaeffer, Jürgen M. Pelikan,
hogrefe, Bern: 19-32

Rudd, Rima; Kirsch, Irwin; Yamamoto, Kentaro (2004): Literacy and Health in America. Policy In-
formation Report. Princeton
Schaeffer, Doris; Gille, Svea; Hurrelmann, Klaus (2020): Implementation of the national action plan
health literacy in Germany—lessons learned. In: International Journal of Environmental Re-
search and Public Health 17/12:4403
Schaeffer, Doris; Hurrelmann, Klaus; Bauer, Ullrich; Kolpatzik, Kai; Altiner, Attila; Dierks, Marie-
Luise; Ewers, Michael; Horn, Annett; Jordan, Susanne; Kickbusch, Ilona; Klapper, Bernadette;
Pelikan, Jürgen M.; Rosenbrock, Rolf; Schmidt-Gernig, Alexander; Schmidt-Kaehler, Sebastian;
Weishaar, Heide; Woopen, Christiane (2018): National Action Plan Health Literacy. Promoting
health literacy in Germany. KomPart Verlagsgesellschaft, Berlin
Scottish Government, NHS Scotland (2014): Making it easy: a health literacy action plan for Scot-
land. The Scottish Government, Scotland
Sørensen, K.; Van den Broucke, S.; Fullam, J.; Doyle, G.; Pelikan, J.; Slonska, Z.; Brand, H.; Consor-
tium Health Literacy Project, European (2012): Health literacy and public health: a systematic
review and integration of definitions and models. In: BMC Public Health 12/80:http://www.bi-
omedcentral.com/1471-2458/1412/1480
Sørensen, K.; Pelikan, J. M.; Rothlin, F.; Ganahl, K.; Slonska, Z.; Doyle, G.; Fullam, J.; Kondilis, B.;
Agrafiotis, D.; Uiters, E.; Falcon, M.; Mensing, M.; Tchamov, K.; Broucke, S. V.; Brand, H.; Con-
sortium, HLS-EU (2015): Health literacy in Europe: comparative results of the European health
literacy survey (HLS-EU). In: European journal of public health1-6
Sørensen, K.; Van den Broucke, S.; Pelikan, J. M.; Fullam, J.; Doyle, G.; Slonska, Z.; Kondilis, B.;
Stoffels, V.; Osborne, R. H.; Brand, H.; Consortium, HLS-EU (2013): Measuring health literacy
in populations: illuminating the design and development process of the European Health Lit-
eracy Survey Questionnaire (HLS-EU-Q). In: BMC Public Health Oct/13:948
The Lithuanian Presidency of the Council of the European Union (2013): The Vilnius Declaration.:
Call for Action. Sustainable Health Systems for Inclusive Growth in Europe
U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion
(2010): National Action Plan to Improve Health Literacy, Washington, DC
Waldherr, Karin; Alfers, Tobias; Peer, Sandra (2021): Development and validation of the HLS-EU-
Q12. https://m-pohl.net/Results
Wang, Jen; Schmid, Margareta (2007): Regional differences in health literacy in Switzerland. Insti-
tute of Social and Preventive Medicine. University of Zurich, Zürich
Wang, Jen; Thombs, Brett D; Schmid, Margareta R (2014): The Swiss Health Literacy Survey: devel-
opment and psychometric properties of a multidimensional instrument to assess competen-
cies for health. In: Health Expectations 17/3:396-417
WHO (1998): Health Promotion Glossary. Bd. WHO/HPR/HEP/98.1. World Health Organization, Ge-
neva
WHO (2013): Health 2020, A European Policy Framework Supporting Action Across Government
and Society for Health and Well-Being. World Health Organisation Regional Office for Europe,
Copenhagen
WHO Regional Office for Europe (1986): Ottawa Charter for Health Promotion, WHO Regional Office
for Europe, Copenhagen.
WHO Regional Office for Europe (2019): What is the evidence on the methods frameworks and
indicators used to evaluate health literacy policies programmes and interventions at the re-
gional national and organizational levels?

2 Study design
Authors:
Thomas Link (ICC)
Dominika Mikšová (ICC)
Eva-Maria Berens (DE)
Stephan Van den Broucke (BE)
Oxana Drapkina (RU) (Section 2.3)
Hanne Søberg Finbråten (NO) (Section 2.3)
Rita Francisco (PT) (Section 2.4)
Øystein Guttersrud (NO) (Section 2.3)
Zuzana Klochanova (SK) (Section 2.3 and 2.4)
Christopher Le (NO) (Section 2.3)
Diane Levin-Zamir (IL) (Section 2.3)
Maria Lopatina (RU) (Section 2.3)
Paulo Nogueira (PT) (Section 2.4)
Jorge Oliveira (PT) (Section 2.4)
Kjell Sverre Pettersen (NO) (Section 2.3)
Carlota Ribeiro da Silva (PT) (Section 2.4)
Doris Schaeffer (DE)
Kristine Sørensen (DK) (Section 2.3)
Miguel Telo de Arriaga (PT) (Section 2.4)
2.1 Development of the HLS19 study design
The HLS19 was planned as a cross-sectional multi-center survey study that was meant to be as
standardized as possible to allow comparisons across the national surveys that were part of the
international study. To ensure this, adequate structures, and procedures for cooperation in the
HLS19 were developed by the M-POHL consortium. M-POHL relied on the experience and example
of the European Commission’s HLS-EU study (HLS-EU-Consortium 2012; Sørensen et al. 2012;
Sørensen et al. 2013; Sørensen et al. 2015) and was guided by the WHO’s Health Behavior in
School-Aged Children study (HBSC) (http://www.hbsc.org/) to establish the necessary standards
for working together and to create a framework for repeating the study over time. This involved
defining the rights and duties of the project partners and implementing certain governance and
work structures.
For the HLS19, an International Coordination Center (ICC) was established to enable international
coordination and support for the national HLS19 Partners. The ICC was situated within the WHO CC
for Health Promotion in Hospitals and Health Care in the Department of Health Literacy and Health
Promotion at the Austrian National Public Health Institute (Gesundheit Österreich GmbH, GÖG).
The HLS19 Member Countries (the national partners in the project) had to ensure that national
participation took place in accordance with the internationally agreed set of rules, especially those
laid down in the HLS19 Study Protocol.
The decisions on developments for the HLS19 Project were taken jointly by two assemblies of rep-
resentatives of the HLS19 Partners (i.e., the HLS19 Policy Assembly and the HLS19 Research Assem-
bly). M-POHL members who did not participate in the HLS19 Project had observer status for the
HLS19 Project (i.e., the HLS19 Observer Group).
To fulfill the research part, a National Study Center (NSC) was contracted to conduct the HLS19
Project in each participating country, providing national data, compiling a country report, and
representing their HLS19 Member Country at the HLS19 Research Assembly. To fulfill the policy role
in the HLS19 Project, Ministries of Health sent a member of staff or nominated a representative
from another suitable institution for the HLS19 Policy Assembly.
Following the examples of the HLS-EU and HBSC, specific tasks for preparing the survey were
conducted by temporary HLS19 Working Groups. These groups were established to deal with un-
derlying research concepts and theories, methodological issues, and thematic areas as well as to
translate research findings into policy recommendations.
An HLS19 Study Protocol was developed step by step by the HLS19 Consortium as a guiding docu-
ment for any decisions relevant for the research.
A study design for data collection and sampling was developed to keep a balance between stand-
ardizing the data collection and sampling within an international multi-center study to enable
benchmarking between the data collection in the participating countries. At the same time, it
should leave enough room and flexibility for country-specific needs and accommodate countries’
Chapter 2 / Study Design 25

practical and financial conditions and necessities. As a result, the study design was sufficiently
flexible to facilitate the participation of all interested countries. Regarding the mode of data col-
lection, this flexibility became an asset in the light of the Covid-19 pandemic, which forced some
countries to change their previously chosen type of data collection (i.e., face to face) to telephone
or online data collection. On the other hand, the different types of data collection limit the com-
parability of results across participating countries to a considerable degree.
In this chapter, the study design is described in terms of defining the population and sample size,
methods of sampling, and the procedures for data collection as well as for translating and field
testing the instruments. The development of the instrument itself is then dealt with in Chapter 3.
2.2 The HLS19 study design
Definition of the population
The HLS19 study population was defined as all permanent residents aged 18 and above living in
private households in the countries participating in the study. (This is different from the HLS-EU,
which included European citizens aged 15+ years.)
Sample size
A sample size was determined of at least 1,000 persons for each country, which is the same as
the standard sample size per country in the HLS-EU. However, most participating countries in the
HLS19 study chose a larger sample.
As a rule, at least 80% of the HLS19 core items, consisting of the 12 items measuring General HL
and the 31 correlate items, needed to be answered to be accepted as a complete interview.
Methods of sampling
A multi-stage random sampling procedure was recommended as the main method of sampling,
using the Eurobarometer standard as a reference, like in the HLS-EU study. For PAPI/CAPI, it was
recommended that the sampling be based on a random selection of sampling points after strati-
fication for gender, age groups, population density, and geographical areas/units. Quota sampling
was considered acceptable if the quota were representative of the population regarding the above
criteria. For data collection using CATI or CAWI, the methods of sampling were adapted to the
characteristics of these methods of data collection (see Subsection 2.4.1 Survey sampling proce-
dure and response rates).
Time frame of data collection
Originally the intention was to collect data between November 2019 and April 2020, but due to
the Covid-19 pandemic, the time frame had to be extended, and data were collected between
November 2019 and June 2021.
Procedure for data collection
For the HLS19 survey, data collection was performed separately in each participating country. In
most countries, large data collection agencies were contracted by the National Study Centers to
collect the data, but in some countries data collection was done by the National Study Center it-
self. In contrast, in the original HLS-EU study, data collection was performed by an international
agency utilizing Eurobarometer standards for all participating countries.
The recommendation to use personal face-to-face interviews (PAPI or CAPI) as the preferred
method for collecting data was given in the HLS19 Study Protocol. The reasons for this preference
were that personal face-to-face interviews would allow for better comparability with the original
HLS-EU study, which had also used face-to-face data collection via either CAPI or PAPI. In addition,
face-to-face interviewing makes it possible to use visual cue cards and to include the NVS test, a
test of functional HL, for validation purposes. Other advantages of personal face-to-face inter-
views are that individuals with poor literacy skills are easier to reach and that unfamiliar words
and questions may be clarified by the interviewers. Face-to-face data collection also has its dis-
advantages: data collection is more expensive and time consuming than CATI or CAWI, and re-
spondents might feel more reluctant to share truthful answers on sensitive topics (e.g., financial
deprivation). In addition, people may not open the door to interviewers anymore.
For these reasons, CATI and CAWI were also included in the HLS19 Study Protocol as acceptable
data collection methods. Due to the Covid-19 pandemic, many countries changed their initial
preference for CAPI to CATI or CAWI.
Section 2.4.1 gives a detailed description of the guiding standards for data collection and sampling
procedures as well as the actual methods of data collection and sampling procedures used by each
country.

2.3 Translation and field testing of instruments
Description of the national translation processes
The HLS19 instruments were developed in English by the HLS19 Consortium (see Chapter 3) and
translated into the national languages by the participating countries.
Recommended procedure for the translation process
The HLS19 Study Protocol suggested the following procedure for translating the HLS19 instruments:
The translation process should have two stages: first, two forward translations should be per-
formed, one by the National Study Center (NSC) and one by the data collection agency. In a second
stage, the NSC should compare the two translations and decide, in a consensus process with the
agency, on the most appropriate translation in case of differences.
The questions on the correlates of HL in the HLS19 instrument are partly identical to items in large
international surveys (EHIS, ESS). Notes referring to these initial surveys are provided in the HLS19
instruments. A supporting document on the correlate items has also been produced by the ICC to
provide guidance and to support the implementation of the correlate items. It is recommended
that NSCs should use already existing translations of these items by consulting documents relating
to the above-mentioned international surveys.
Many translations of the HLS19 instruments were created by the HLS19 project partners (see Table
2.1 and Table 2.2 for details of the translated instruments by country). The HLS19 instruments
were translated into their national language(s) by 16 out of the 17 countries (Ireland used the
original English version). Each NSC organized the translation process. This was mostly done by the
data collection agencies and/or other professional translation services.
The translation processes are listed in Table 2.1 and Table 2.2. Different national versions of the
instruments were created for the German, French, Israeli and Italian translations, namely three
German versions (AT, CH, DE), three French versions (BE, CH, FR), three versions for the Israeli
population (Hebrew, Arabic and Russian) and two Italian versions (CH, IT), as each participating
country adapted the instruments to its cultural, linguistic, and healthcare context. Countries with
common languages cooperated in translating the instruments.
The suggested procedure of performing two forward translations was implemented by ten coun-
tries (AT, BE (Dutch translation), CH (German translation), DE, DK, HU, IT, NO, SI, and SK). One
forward translation was chosen by countries which cooperated with other countries using the
same language (BE for the French translation, CH for the French and Italian translations). Alt-
hough back translation was not required according to the study protocol, it was performed by
four countries (IL, NO, RU, and SI).
Moreover, 14 countries chose additional ways of ensuring the quality of the translations, such as
» cognitive interviewing (NO, SI),

» focus groups (RU),
» expert discussion/review (CH, CZ, RU, PT, and SK),
» three forward translations (DK, SI),
» comparison of the translation with the (national) HLS-EU translation (BG, CH, CZ, HU, DK,
and NO), and
» cooperation with other countries (AT, CH, DE, BE, and FR).
The final versions of the translated instruments were based on consensus processes in all coun-
tries. The different translations and, if applicable, the additional quality assurance methods for
the translated instruments were agreed on by the different actors in the translation process.
Although the countries participating in the study did not face any major problems with the trans-
lations, some challenges were reported, such as adapting the wording of items to the national
context without changing their meaning. A way of dealing with this was to add some examples to
support the respondents’ understanding. Adapting health service terminology to the national
healthcare system was another challenge that was sometimes reported, but solutions were found
each time.
Table 2.1:
Overview of the different processes used by each country to translate the instruments
AT BE BG CH CZ DE DK FR HU
Translated languages French,
Ger- Dutch, Bul- Ger- Dan- Hun-
German, Czech French
man French garian man ish garian
Italian
One forward translation x x (French)
x x
(French) x (Italian)
Two forward translations x x (Dutch) x (German) x x x
Back translation
Based on consensus pro-
x x x x x x x x x
cesses
Other quality assurance
methods for the transla- x x x x x x x x x
tion
The translation procedure applied is marked with an “x”
Source: HLS19 Consortium

Table 2.2:
Overview of the different processes used by each country to translate the instruments (continued
from Table 2.1)
IE IL IT NO PT RU SI SK
Translated lan- Hebrew,
Norwe- Portu-
guages Arabic, Italian Russian Slovenian Slovak
gian guese
Russian
One forward
x x x
translation
Two forward
x x x x
translations
Back transla- n.a.
x x x x
tion
Based on con-
sensus pro- x x x x x x
cesses
Other quality
assurance
x x x x x
methods for
the translation
The translation procedure applied is marked with an “x”
Description of national field-testing procedures
Recommended procedure for field testing
The HLS19 Study Protocol suggested the following procedure for field testing the HLS19 instru-
ments:
In each partner country a field test is to be performed with the HLS19 instruments. Countries with
non-English national languages have to test the translated version of their national HLS19. The
field test must be performed by the data collection agency.
Sample size of the field test: 30 interviews. Purposeful sampling is suggested to ensure an equal
distribution of participants in terms of age, gender, and education.
The data collection agency should compile a field-test report and send it to the NSC. The field-
test report should include information on the participants, the comprehensibility of each item, and
the length in minutes of each part of the survey and the full interview, as well as any problems
experienced and suggestions for changes, e.g., of the wording of an item.
In the case of an online survey, the aim of the field (or pilot) test is specifically to: (1) check that
the questions asked and the proposed response modalities were fully understood by the respond-
ents, (2) validate the average time needed to complete the survey, and (3) consider the need to
include short instructions that can be accessed with a simple click, thus allowing respondents to
obtain additional information in case the terminology used is not clear.
Field testing was performed to check whether the items in the survey were fully understood by
respondents and to assess the feasibility of its implementation using the data collection method
chosen for the country. Table 2.3 to Table 2.6 provide an overview of the procedures implemented
for field testing in the participating countries, grouped by the main type of data collection. All the
countries, except Bulgaria, performed a field test (16 out of 17). The sample size for these field
tests varied by country, from six interviews in Belgium to 161 interviews in Italy, but for most
countries, the field tests involved at least 30 completed interviews (8 out of 16 countries). Germany
was the first country to conduct the national field test (in November 2019). The German field-test
results were taken into consideration when developing the final English version of the HLS19 in-
struments. None of the countries experienced major difficulties during the field test, and only
slight changes were made due to the results of the field testing and interviewers’ feedback in 12
countries (AT, BE, CH, CZ, DE, FR, HU, IT, NO, PT, SK, and SI). Such changes concerned language
adaptions, removing brackets around examples, adding examples, and clarifying instructions for
the interviewers.
Countries that selected interviewees by purposeful sampling used gender, age groups, educational
levels, degrees of urbanity, and administrative regional units (whereby different mixes of these
criteria were used). Switzerland made use of intentional oversampling for younger and older age
groups. Portugal decided to select Portuguese-speaking immigrants as interviewees for field test-
ing to ensure that the language was accessible to all population groups. In eight countries (BE, CZ,
DK, FR, IL, NO, RU, and SK), respondents were selected by convenience sampling. Additional test
methods relating to all or part of the HLS19 instruments were applied in six countries; these in-
cluded in-depth cognitive analyses (CH), setting up focus groups (PT, RU), additional testing of
the instruments by staff (DE) or a team of researchers (FR, IE), and expert interviews (DE, RU).
Countries that used CATI reported that they needed to adapt the wording of the instrument slightly
because the visual cue cards for response categories could not be used (such visual cue cards
were used with CAPI/PAPI). An example of an adaption is changing the interviewer’s instruction
for items measuring HL from “On a scale from very easy to very difficult, how easy would you say
it is?” to “On a scale from very easy, easy, difficult, and very difficult, how easy would you say it
is?”.
Countries that used CAWI had to find a solution to deal with the possible response “don’t
know/does not apply”. When using CAPI/PAPI and CATI, the “don’t know/does not apply” category
was not a direct option for the respondents but had to be coded by the interviewer as “no answer”.
Some countries using CAWI did not provide the option for “don’t know/does not apply”. In France,
the poll company suggested keeping all items mandatory as the participants were members of a
panel and keen to respond to all items. In Switzerland, the “don’t know/does not apply” option
was not displayed on the screen to prevent participants from choosing it without trying to under-
stand the question. The option only appeared if participants did not tick any of the four answer
categories and tried to continue, with the questions not answered indicated in red. The message
text was: “please answer all questions; if this is not possible, click continue”. In case a participant

clicked “continue” without filling in an answer, the software automatically coded the answer as
“don’t know/does not apply”.
In Italy both CATI and CAWI were used. The response category “don’t know/does not apply” was
explicitly offered for both data collection formats. The Italian field test revealed that CATI re-
spondents showed a lower propensity to indicate the “very difficult” and “difficult” response op-
tions, despite being older and having a lower level of education than the interviewees targeted for
CAWI. To reduce this effect, interviewers were instructed to specify at the beginning of the inter-
view that there were no right or wrong answers and that it was particularly important that re-
spondents answered each item truthfully.
Table 2.3:
Overview of the different field-testing procedures used in countries where CAPI/PAPI was the
main method of data collection
Countries using CAPI/PAPI

DE RU SK
No. of interviews >=30 x x x
Purposeful sampling x
Convenience sampling x x
Additional test method(s) applied x x
The procedure applied is marked with an “x”
Table 2.4:
Overview of the different field-testing procedures used in countries where CATI was the main
method of data collection
Countries using CATI

AT HU IE NO PT
No. of interviews >=30 x x x x
Purposeful sampling x x x x
Convenience sampling x
Table 2.5:
Overview of the different field-testing procedures used in countries where CAWI was the main
Countries using CAWI

BE CH* DK FR
No. of interviews >=30 x
Purposeful sampling x
Convenience sampling x x x
* CH: CAWI was the main type of data collection and a small number of CATI interviews were carried out. In field testing
both CAWI and CATI were used.
Table 2.6:
Overview of the different field-testing procedures used in countries where mixed methods were
the main method of data collection
Countries using mixed methods

BG CZ IL IT SI
Type of method(s) used for field test No field test CAPI CAWI CAWI, CATI CAPI
performed
Type of method(s) used in main data CAPI, CAWI CAWI, CATI CAWI, CATI CAWI, CATI CAPI, CAWI,
collection paper and
pencil
No. of interviews >=30 x x x
Purposeful sampling x x
Convenience sampling x x
Additional test method(s) applied
2.4 Data collection and processing
Survey sampling procedures and response rates
Preliminary remark: The planning phase for the HLS19 started in 2018 with the aim of collecting
data in 2019 and 2020. As the Covid-19 pandemic progressed, some partner countries which had
initially planned face-to-face data collection decided to change to methods without physical con-
tact.
In Table 2.7 and Table 2.8Table 2.7, details of survey sampling and response rates are presented
by country and grouped by the main type of data collection respectively, providing an overview of

the standard procedures used as well as on differences and similarities in procedures and perfor-
mances. For each procedure, the approach recommended in the HLS19 Study Protocol is shown in
an information box.
Detailed recommendations when using PAPI/CAPI
For countries using CAPI or PAPI for data collection, it is suggested to draw samples from the
general population using a multi-stage random sampling procedure. Quota samples are also ac-
ceptable. In each country a number of sampling points is to be drawn with probability proportional
to population size and density. The drawing of sampling points has to be done in a systematic
way in each stratum so that the number of interviews is equal to the proportion of the strata in
the population aged 18 years and over. As a next step, it is suggested to randomly draw a starting
address in each of the selected sampling points. Then, further addresses are to be selected as
every Nth address using the standard random route procedure from the initial address. A respond-
ent in each household is to be drawn using “the nearest birthday” method or the Kish method. Up
to four attempts were allowed to obtain an interview with the selected respondent. No more than
one interview should be conducted in each household.
Details of survey sampling and response rates in the countries using CAPI/PAPI as the main
method of data collection can be found in Table 2.7.
In Germany, multi-stage random and quota sampling were combined to ensure the representa-
tiveness of the sample. First, sampling points in administrative regional units (NUTS2 level) were
selected randomly bearing population density in mind. In those areas, respondents were sam-
pled/recruited using a community-based quota sampling approach, with quota for gender in com-
bination with age, size of household, and education.
In the Russian Federation, a multi-stage random sampling procedure was applied in three selected
regions. Out of a list of all medical institutions in each of the three regions, 13 medical institutions
(for Novosibirsk and Tatarstan) and ten medical institutions (for Karelia) were randomly selected,
taking the proportion of people living in urban and rural areas into account. In each medical in-
stitution selected, four therapeutic areas were randomly selected for each region and then in each
therapeutic area, the sampling of addresses (households) was also selected randomly. In Slovakia
the sample was calculated by NUTS2. However, the data collection was organized via regional
public health offices (36 across Slovakia) and the samples were then calculated by catchment areas
to fulfill the national sample.
The number of net interviews in which 80% of the core items were answered are the basis for the
data analyses in this international report. These varied from 2,143 in Germany to 5,660 in the
Russian Federation. The response rates (calculated as the number of net interviews/number of
individuals contacted) varied between 94% in the Russian Federation and 64% in Germany, while
Slovakia had a response rate of 67%. The especially high response rate in the Russian Federation
might be a result of the interviewers, i.e., doctors (therapists and paramedics) working in the
Centers for Medical Prevention, who are well known and trusted by the population.
Table 2.7:
Details of survey sampling and response rates in the countries using CAPI/PAPI as the main

DE (PAPI) RU (PAPI) SK (CAPI)
Sampling procedure Multi-stage Multi-stage Multi-stage
random and quota random sampling random sampling
sampling
Source for the selection of households Randomly selected List of household ad- 36 regional public
sampling points com- dresses in four thera- health offices
bined with a commu- peutic areas of se-
nity-based quota sam- lected medical institu-
ple with 558 sampling tions in the three re-
points, which equaled gions: Novosibirsk,
the number of inter- Karelia, and Tatarstan
viewers
Number of sampling points (=interview- 558 48 42
ers)
Number of individuals contacted 3,394 6,926 3,220
Number of net interviews 2,162 6,493 2,171
Number of net interviews 2,143 5,660 2,145
with 80% of the core items answered
Response rate % 64 94 67
Detailed recommendations when using CATI
For the telephone surveys, households should be selected using a random selection procedure in
stages as described in PAPI/CAPI (see above). A CATI system should randomly generate landline
numbers using the Random Digit Dialing (RDD) procedure. A dual-frame approach should be ap-
plied to ensure the inclusion of both landline phone and mobile phone users.
Details of survey sampling and response rates in the countries that used CATI as the main method
of data collection can be found in Table 2.8. Of these countries, Austria and Hungary used multi-
stage random sampling procedures. While in Austria the Randomized Last Digit Method was used,
in Hungary a hybrid form of sampling was used in which stratification was based on regions/type
of settlements (as in the face-to-face mode) to define the case numbers to achieve while ensuring
random selection within the sample frame by using a pool of phone numbers drawn from public
registries. Complemented by a quota controlling for gender/age/region/type of settlement during
fieldwork, this ensured a representative sample with the same level of random selection as in the
case of face-to-face surveys. In Ireland, random selection of mobile phone numbers using a Ran-
dom Digit Dialing approach was used as the sampling procedure. In Norway, sample management
was significantly improved by stratification of the random sample. This methodology splits the
population into smaller groups from which a random sample is drawn from each stratum. This
method ensures the correct proportion of the different population groups. In Portugal, the sam-
pling procedure was also based on stratified random sampling, with placement according to the
Portuguese population on the following variables: number of residents by NUTS III, gender, and

large age groups. A dual-frame approach including landline phone and mobile phone was imple-
mented in four countries (AT, HU, NO, and PT), while in Ireland only mobile phone numbers were
included.
The number of net interviews with 80% of the core items answered, serving as the basis for the
analyses presented in this international report, varied from 1,195 in Hungary to 4,487 in Ireland.
The response rate (calculated as the number of net interviews/number of individuals contacted)
was highest in Portugal at 69% and lowest in Hungary at 14%.
Table 2.8:
Details of survey sampling and response rates in the countries using CATI as the main method of
data collection

AT* HU IE NO PT
Sampling procedure Multi- Multi- Random Random Random
stage ran- stage ran- digit dial- sampling stratified
dom sam- dom sam- ing ap- procedure sampling
pling pro- pling pro- proach within each procedure
cedure cedure stratum
Dual-frame approach applied Yes Yes No Yes Yes

Source for the selection ofof tele- Registry of Online Registry of Registry of Registry of
phone numbers landline public mobile landline landline
and mobile phonebook phone and mobile and mobile
telephone numbers telephone telephone
numbers numbers numbers
Number of telephone contacts 28,558 13,381 22,866 39,371 6,749
Number of individuals contacted 22,704 8,654 16,730 15,008 2,257
Number of net interviews 3,535 1,221 4,530 3,000 1,555
Number of net interviews
with 80% of the core items answered
2,967 1,195 4,487 2,855 1,247
and included in the HLS19 international
analyses
Response rate % 16 14 27 20 69
* In AT, data collection included additional interviews on a regional level (n=509) which were not used in the
international analyses but which could not be extracted when calculating the response rate.
Detailed recommendations when using CAWI
For CAWI, a representative sample of the national population is usually constituted from a panel
of internet users. To that effect, the following stepwise sampling procedure is to be followed: (1)
A first group of potential respondents is selected from the panel based on established inclusion
criteria (residents of the country aged 18-75 years); (2) Preselected potential respondents are
invited by e-mail to participate in the survey; (3) For those who agree to participate, compliance
with the inclusion criteria is verified; those who comply with the criteria can access the instrument
via a personalized login; (4) After a first wave of responses, a second wave of potential respondents
is selected from the panel according to the pre-established quotas (defined by gender, age group,
population density, and geographical areas/units) that are underrepresented; (6) This procedure
is repeated until the sample is representative of the population for the pre-established quotas.
Details of survey sampling and response rates for the countries that used CAWI as the main method
of data collection can be found in Table 2.9. Four countries used CAWI as the main type (BE, CH,
DK, and FR). Quota sampling was used in Belgium and in France, where respondents were sampled
from a pre-existing online access panel. Switzerland and Denmark used multi-stage random
sampling to select respondents. Respondents in Switzerland were selected from the SRPH registry
and an invitation letter was sent to all sampled individuals including a CAWI link; in case they
preferred a telephone interview, a toll-free phone number was provided. In Denmark the sample
was based on the Register of Civil Registration Numbers.
The number of net interviews with 80% of the core items answered, which are the basis of the
analyses presented in this international report, varied from 1,000 in Belgium to 3,602 in Denmark.
The response rate (calculated as the number of net interviews/number of invited participants) was
highest in Switzerland, where 54% of the invited participants completed the survey. For Belgium,
where data collection was carried out by an external organization using automated multi-source
sampling, the number of invited participants (and response rate) is not available.
Table 2.9:
Details of survey sampling and response rates in the countries using CAWI as the main method
of data collection

BE CH* DK FR
Sampling procedure Quota sampling Multi-stage ran- Multi-stage ran- Quota sampling
dom sampling dom sampling
Source for the selection of re- Online panel SRPH registry Register of Civil Online panel
spondents Registration
Numbers
Number of participants invited - 4,764 27,727 47,102
(e-mail, letter)
Number of invited participants - 4,560 4,605 15,262
who agreed to participate
Number of net interviews 1,000 2,348 (+ CATI 3,679 2,003
210)
Total: 2,558
Number of net interviews 1,000 2,312 (+ CATI 3,602 2,003
with 80% of the core items an- 190)
swered and included in the Total: 2,502
HLS19 international analyses
Response rate % - 54** 13 4
* CH: CAWI was the main type of data collection; additionally, a small number of CATI interviews were conducted.
** Response rate for both types of data collection.

Detailed recommendations when using multiple or mixed methods
When mixed methods are used for data collection, the recommendations for the specific data
collections described above apply.
Details of survey sampling and response rates for the countries using mixed or multiple methods
can be found in Table 2.10.
Bulgaria used a random quota sample for CAWI and a proportional stratified sampling procedure
for CATI. The Czech Republic used CAWI and CATI for data collection, with random quota sampling
for CAWI and a random digital procedure for CATI. Israel, where a combination of CAWI and CATI
was used, and Slovenia, where CAPI, CAWI, and paper and pencil were applied, both chose a multi-
stage random sampling strategy. In Slovenia, respondents were sampled from the Central Popu-
lation Registry. In Italy, a proportional stratified sampling procedure selecting respondents ran-
domly was applied for CATI and CAWI data collection.
The number of net interviews with 80% of the core items answered, which are the basis for this
international report, was 865 for Bulgaria, 1,315 for Israel, 1,599 for the Czech Republic, 3,360
for Slovenia, and 3,500 for Italy.
In Bulgaria, the CAWI sample of net interviews with 80% of the core items answered (463) was
slightly higher than the CATI sample (402). In the Czech Republic, the CAWI sample of net inter-
views with 80% of the core items answered (1,067) was twice as high as the CATI sample (532),
CATI being selected to contact the older population, which was less represented in the e-mail
registry. In Israel, CATI sampling focused on the Arab population, which was expected to be more
difficult to reach via the internet panel; 311 net interviews with 80% of the core items answered
were collected by CATI and 1,004 by CAWI. In Italy, the CATI survey was targeted primarily at over
65 year olds and CAWI at younger people. However, to allow a comparative analysis of the possible
effects derived from using two data collection tools, both methodologies were applied to all age
groups. Out of the 3,500 Italian net interviews with 80% of the core items answered, 2,949 were
collected by CAWI and 551 by CATI. In Slovenia, 1,860 net interviews with 80% of the core items
answered were collected by CAPI, 1,488 by CAWI, and 12 by self-administered paper-and-pencil
questionnaire, with the mixed-method mode used to increase the response rates to a maximum
level despite the Covid-19 pandemic. The procedure in Slovenia was as follows: Invitation letters
were sent to all 6,000 people in the sample, inviting them to participate online and informing them
that an interviewer would come and visit them for personal interviews if they did not complete the
online questionnaire within the next seven days. Interviewers were instructed to make up to four
contact attempts in person, at different times on different days, and then visited the selected
respondents at their home addresses. If the latter did not want to complete the questionnaire with
an interviewer, they were again reminded that they could do it online or were even offered a self-
administered paper-and-pencil questionnaire with a return prepaid envelope to send it back.
Table 2.10:
Details of survey sampling and response rates in the countries using mixed methods for data collection

BG CZ IL IT SI
Type of method(s) used in the Paper and
main data collection CAWI CATI CAWI CATI CAWI CATI CAWI CATI CAPI CAWI pencil, self-
administered
Sampling procedure Random Propor-
Random Random dig-
quota tional Multi-stage random sam- Proportional stratified sam-
quota ital proce- Multi-stage random sampling
sam- stratified pling pling
sampling dure
pling sampling
Dual-frame approach applied
- - x - x x - - -
(CATI only)
Source for the selection of re- Central
Main public Main public Central
spondents Internet Telephone Online Online Telephone SWG Panel Popula- Central Popu-
telephone di- telephone Population
panel register panel panel register Community tion lation Registry
rectory directory Registry
Registry
Number of initial contacts (ad-
dresses/telephone numbers) or 1,654 1,238 3,455 9,952 9,400 1755 13,441 11,075 6,000
invited participants (e-mail, letter)
Number of individuals contacted in
CAPI/CATI/ paper and pencil or
948 912 1,797 4,635 1,861 300 7,627 4,973 5,585
number of invited participants who
agreed to participate in CAWI
Number of net interviews 555 402 1,070 580 1,004 300 3,357 582 3,412
Number of net interviews with 80%
463 402 1,067 532 1,004 311 2,949 551 1,860 1,488 12
of the core items answered and
included in the HLS19 international
analyses Total: 865 Total: 1,599 Total: 1,315 Total: 3,500 Total: 3,360
Response rate % 59 44 31 13 54 18 25 12 61*

* Response rate for all types of data collection calculated as the number of net interviews/number of individuals contacted in CAPI/CATI/paper and pencil or the number of invited participants who
agreed to participate in CAWI.
“x”=yes, “-”=no

Study population and representativeness
Definition of study population
The HLS19 study target population is permanent residents aged 18 years and older living in private
households.
Recommended procedure for stratification
To increase representativeness, stratification is suggested for gender, age groups, population

density, and geographical areas/units. For stratification by age group, seven age groups are sug-
gested: 25 or younger, 26-35, 36-45, 46-55, 56-65, 66-75, 76 or older. As for stratification for
population density, three categories are suggested: predominately urban, intermediate, and pre-
dominately rural. According to this definition, predominantly urban regions have a rural popula-
tion of less than 20%, intermediate regions between 20% and 50%, and predominantly rural regions
more than 50%. For non-EU countries a similar classification should be used. NUTS2 1 areas/units
are suggested for stratification by geographical area/unit. Non-EU countries should use equivalent
standards.
Table 2.11 to Table 2.14 provide details regarding the representativeness of the samples, includ-
ing the key demographics used for stratification when selecting the sample and the data on which
stratification was based. All countries targeted the permanent resident population aged 18 years
and older living in private households. None of the countries carried out purposeful oversampling
or undersampling of subpopulations.
Details regarding the study population and representativeness of the sample for countries using
CAPI/PAPI as the main method of data collection are shown in Table 2.11. Limitations regarding
representativeness must be considered for the sample of the Russian Federation, for which re-
spondents were selected from only three regions, Novosibirsk, Karelia, and Tatarstan. The Russian
Federation has highly heterogeneous areas and populations, and therefore it is not possible to
draw conclusions regarding the whole country based on data from just three regions. The Russian
Federation also reported that the age group of 75+, especially men in rural areas, was the least
covered, since men in rural areas usually die at a younger age.
Stratification for key demographics was applied to different degrees. Further variables for strati-
fication were used in Germany and Slovakia. Target distributions were based on the 2018 micro-
census in Germany, on national census data from 2019 in the Russian Federation, and on national
census data from 2018 in Slovakia.
1 NUTS: nomenclature of territorial units for statistics as used by the statistical office of the European Union (EUROSTAT).
Table 2.11:
Details of study population and representativeness in countries using CAPI/PAPI as the main

DE RU SK
Limitations on representativeness (1) Representative for
the three regions,
not the whole coun-
try
(2) Age group 75+
less covered
Variables considered for stratification/quota sampling
» Gender x x x
» Age groups (no. of groups) x (5) x (7) x (7)
» Population density x - x
» Administrative regional units (NUTS x (NUTS2) - x (NUTS2)
level or other)
» Other variables, described 3 levels of education - 36 Regional Public
and Health Office areas
2 groups by size of
household
Stratification/quota based on (which data) Microcensus 2018 National census 2019 National census 2018
Details regarding the study population and representativeness of the sample for countries that
used CATI as the main method of data collection are shown in Table 2.12. One major limitation in
representativeness in CATI was that people with secret or non-listed phone numbers (in Norway)
or those who did not make their phone numbers and names available for the purpose of phone
surveys (in Hungary) could not be contacted. In Ireland, only mobile phone numbers were used,
but as there is near-universal ownership of mobile phones in Ireland, this is unlikely to have im-
pacted representativeness. Portugal included the mainland population and not that of its islands.
Stratification for key demographics is shown in Table 2.12. In Ireland no stratification was carried
out on the target population with target distributions based on census data.

Table 2.12:
Details of study population and representativeness in countries using CATI as the main method
of data collection
AT HU IE NO PT
Limitations on representativeness People who Only mobile People with Mainland
did not allow phone num- secret or population
the authori- bers were non-listed included/no
ties to make used (but phone num- population
their phone near-univer- bers were from the is-
numbers sal owner- excluded lands in-
and names ship of mo- cluded
available for bile phones
the purpose in Ireland)
of phone
surveys were
excluded
» Gender X x - x x
» Age groups (no. of groups) x (7) x (3) - x (7) x (3)
» Population density X - - - -
» Administrative regional units (NUTS x (11 re- x (NUTS3)
level or other) x (NUTS2) x (NUTS2) - gional
units)
» Other variables, described 3 levels of 3 levels of -
- -
education education
Stratification/quota based on (which National National National
Microcen-
data) census n.a. census census
sus 2017
2011 2020 2011
Details regarding the study population and representativeness of the sample for countries that
used CAWI as the main method of data collection are shown in Table 2.13. In France and Belgium,
one limitation was that only participants belonging to a pre-existing online panel were sampled.
In France people aged 76 or older were not covered. Denmark reported that individuals without
an official e-mail (“e-box”) were excluded, but since this concerns only a small proportion of
Danish residents, this is considered a minor limitation. Stratification for key demographics was
applied to different degrees in the countries. Target distributions were based on census data (see
Table 2.13).
Table 2.13:
Details of study population and representativeness in countries using CAWI as the main method
of data collection

BE CH* DK FR
Limitations on representativeness Only participants Individuals with- Only participants
of an online out official e- of an online
panel mail “e-box” panel (IPSOS i-
were excluded say),
-
(=small propor- people aged 76
tion of Danish or older were
residents) not covered

» Gender x x x x
» Age groups (no. of groups) x (3) x (7) x (7) x (6)
» Population density - x x x
» Administrative regional units (NUTS x (Region)
x (NUTS3) - x (NUTS2)
level or other)
» Other variables, described - - - -
Stratification/quota based on (which National cen- National cen- National cen- National cen-
data) sus 2011 sus, 2020 sus Q4, 2020 sus 2015
Details about the study population and representativeness of the sample for countries that used
mixed or multiple methods for data collection are shown in Table 2.14. A possible limitation on
representativeness is that the internet panel might oversample people with high Digital Health
Literacy. Another factor affecting representativeness might be the timing of the data collection.
Variables considered for stratification/quota sampling are detailed in Table 2.14.

Table 2.14:
Details of study population and representativeness in countries using mixed methods for data
collection

BG CZ IL IT SI
Limitations on representa- The internet panel probably oversamples people with high Digital Health Literacy.
tiveness

» Gender x x x x x
» Age groups (no. of X (12) x (4) x (7) x (7) x (13)
groups)
» Population density - x - x -
» Administrative regional X (NUTS2) x (NUTS3) - X (NUTS2) x (NUTS3)
units (NUTS level or
other)
» Other variables, de- Location 3 levels of Geographical - 4 levels of
scribed (city/ village) education periphery education
measure
Stratification/quota based National National cen- Israel Central National cen- National cen-
on (which data) Census 2011 sus 2011 and Bureau of sus 2020 sus 2020
2019 Statistics
Fieldwork
Recommended procedure for fieldwork
For performing data collection, it is suggested that the HLS19 National Study Centers sign a con-
tract with data collection agencies. The data collection agency should ensure high quality data
collection and provide detailed documentation of the fieldwork.
Table 2.15 to Table 2.18 provide information on who performed the data collection, the time taken
for data collection, and the average length of interviews in minutes as well as the shortest and
longest interview times. Data collection was carried out in most of the participating countries by
national data collection agencies and in three cases by the HLS19 National Study Centers (BG, DK,
and SK).
The planning phase for the HLS19 started in 2018 with the aim of collecting data in 2019 and 2020.
Due to the Covid-19 pandemic, the timeframe for collecting data was extended and the data col-
lection phase lasted from November 2019 until June 2021.
The timeframe for data collection in the Russian Federation was from November to December
2019 and in Germany 2 from December 2019 to January 2020, so in both countries, data were
collected before the Covid-19 pandemic. All other participating countries started data collection
during the pandemic, which reached different phases of intensity in the different countries at
different points in time. Slovenia started data collection in March 2020 and had to stop after six
days due to the pandemic, starting a second wave of data collection from June 2020 to August
2020. For Belgium, the first part of data collection took place just before the pandemic, with a
second part during the pandemic.
In the Czech Republic there was a spike in the second Covid-19 pandemic wave in November
2020, with many restrictions and measures. At that time the public debate on vaccination also
started. Many health- and vaccination-related items might have unusual results due to this dis-
tortion. Some segments of the population (e.g., elderly, people with chronic diseases) might have
reacted more sensitively than others.
The average length of the interviews varied between 13 min in Israel and Italy for CAWI respond-
ents and 65 min in Germany (PAPI). Differences in the average length of the interviews depend on
the number of items included in the questionnaire and the different types of data collection. In
Denmark, where CAWI was used for data collection, the length was measured from opening the
survey to finishing the last page (the median length was 22 minutes).
DE conducted an additional round of data collection in August and September 2020 in order to assess the effects of Covid-
19 on HL. These data were not included in this report but were analyzed and reported on a national level.

Table 2.15:
Details of fieldwork in countries using CAPI/PAPI as the main method of data collection
DE (PAPI) RU (PAPI) SK (CAPI)
Data collection performed by Institut fuer Demosko- Center of Medical Pre- Regional Offices of
pie Allensbach GmbH vention of the Republic Public Health managed
of Karelia, by the Public Health
Regional Center of Authority of the Slovak
Medical Prevention of Republic (HLS19 Na-
the Novosibirsk re- tional Study Center)
gion,
Regional public organ-
ization “Golden Heart”,
Republic of Tatarstan
Time of data collection 13.12.2019- 01.11.2019- 22.06.2020-
27.01.2020 20.12.2019 14.09.2020
Average interview length (in minutes) 65 28 28
Shortest interview (in minutes) 15 8 10
Longest interview (in minutes) 180 120 95
Table 2.16:
Details of fieldwork in countries using CATI as the main method of data collection
AT HU IE NO PT
Data collection performed by Das Öster- TÁRKI Social Ipsos MRBI Norstat Universidade
reichische Research In- Norge AS de Aveiro -
Gallup Insti- stitute CIMAD -
tut Centro de
Investigação
em Market-
ing e Análise
de Dados
Time of data collection 16.03.2020- 02.12.2020- 24.07.2020- 04.04.2020- 10.12.2020–
26.05.2020 20.12.2020 07.12.2020 13.05.2020 13.01.2021
Average interview length (in minutes) 26 23 29 21 15
Shortest interview (in minutes) 15 10 20 8 9
Longest interview (in minutes) 60 84 60 59 30
Table 2.17:
Details of fieldwork in countries using CAWI as the main method of data collection
BE CH* DK FR
Data collection performed by Medistrat M.I.S. Trend Aalborg Univer- IPSOS
sity and Aalborg
University Hos-
pital (HLS19 Na-
tional Study
Center)
Time of data collection 30.01.2020- 05.03.2020- 11.12.2020- 27.05.2020-
28.02.2020 and 29.04.2020 05.02.2021 05.06.2020 and
01.10.2020- 08.01.2012-
26.10.2020 18.01.2021
Average interview length (in minutes) - 16 (CATI: 21) 39 22
Shortest interview (in minutes) - 5 (CATI: 8) 2 10
Longest interview (in minutes) - 110 (CATI: 104) 1,435 329

Table 2.18:
Details of fieldwork in countries using mixed methods for data collection
BG CZ IL IT SI
Type of method(s) used CATI CAWI CAWI CATI CAWI CATI CAWI CATI CAPI CAWI Paper and pencil
in the main data collec-
tion
Data collection per- Faculty of Public Health, STEM - Ústav empirických Sekernet Statnet SWG spa IPSOS Slovenija
formed by Medical University, Sofia výzkumů, z.ú
Time of data collection 15.08.202 1.04.2021 10.11.2020-24.11.2020 15.12.2020-10.1.2021 08.04.2021-08.05.2021 9.3.2020-15.3.2020 (stopped due to Covid-19) and
0- - 9.6.2020-10.8.2020
30.11.202 1.06.2021
0
Average interview 30 15 19 34 13 19 13 14 28 31 -
length (in minutes)
Shortest interview (in 15 10 7 12 - 14 7 7 7 6 -
minutes)
Longest interview (in 45 30 111 102 - 24 27 33 124 - -
minutes)
“-”=no information available
Data quality checks and data weighting
Recommendation to perform quality checks on data and to weight the sample
The HLS19 Study Protocol suggests that national samples should be weighted by gender, age
group, population density, and geographical areas/units, based on national census data to in-
crease representativeness. Quality checks for data entry and outcomes are recommended.
Table 2.19 to Table 2.22 provide an overview of the data quality checks on data entry and out-
comes, information on weighting by demographics, and the source of data used for applying
weights. Quality checks on data entry and outcomes were performed by all participating countries.
Weight factors used to weight respondents by gender were included in all country data sets. Four-
teen countries weighted their data for different socio-demographic factors; two countries (PT and
SK) did not weight their data. For weighting by age groups, the number of age groups varied
between three and thirteen, with most countries using seven groups as suggested in the HLS19
Study Protocol. Population density was either weighted using three groups, predominately urban,
intermediate, and predominately rural (AT, CH, DE, DK, and IT) or by two groups differentiating
between urban and rural areas (FR, RU), or by four quota categories (CZ). For weighting by admin-
istrative regional units, NUTS2 was used in Austria, France, and Italy. Switzerland and Slovenia
used NUTS3. Other administrative geographical units were used in Germany (17 federal states,
whereby West Berlin and East Berlin were included separately), Norway (11 administrative geo-
graphic units), and Belgium (three regions). Levels of education were used for weighting the data
by six countries (AT, CZ, DE, HU, IE, and SI).
Table 2.19:
Details of data quality checks and data weighting in countries using CAPI or PAPI as the main
DE RU SK
Quality checks on data entry performed x x x
Quality checks on data outcomes performed x x x
Weighted by gender x x -
Weighted by age group (no. of groups) x (7) x (7) -
Weighted by population density (no. of groups) x (3) x (2) -
Weighted by administrative regional units x (17 federal states, - -
(NUTS level or other) whereby West Berlin
and East Berlin were
included separately)
Other weighting 8 levels of education - -
Weights are based on (which data) Microcensus 2018 National census 2019 -

Table 2.20:
Details of data quality checks and data weighting in countries using CATI as the main method of
data collection
AT HU IE NO PT
Quality checks on data entry performed x x x x x
Quality checks on data outcomes per- x x x x x
formed
Weighted by gender x x x x -
Weighted by age group (no. of groups) x (7) x (3) x (7) x (7) -
Weighted by population density (no. of
x (3) - - - -
groups)
Weighted by administrative regional x (NUTS2) - x (11 units) -
-
units (NUTS level or other)
Other weighting 3 levels of 3 levels of 9 levels of - -
education education, 3 education
settlement
types
Weights are based on (which data) Microcensus National Labour Force National -
2017 census 2011 Survey 2020 census 2020
Table 2.21:
Details of data quality checks and data weighting in countries using CAWI as the main method of
data collection

BE CH* DK FR
Quality checks on data entry performed x x x x
Quality checks on data outcomes per-
x x x x
formed
Weighted by gender x x x x
Weighted by age group (no. of groups) x (3) x (7) x (7) x (7)
Weighted by population density (no. of
- x (3) x (3) x (2)
groups)
Weighted by administrative regional
x (regions) x (NUTS3) - x (NUTS2)
units (NUTS level or other)
Other weighting - - - -
Weights are based on (which data) National census National census National census National census
2011 2020 Q4, 2020 2015
* CH: CAWI was main type of data collection; additionally, a small number of CATI interviews were conducted.
Table 2.22:
Details of data quality checks and data weighting in countries using mixed methods for data
collection

BG CZ IL IT SI
Quality checks on data entry x x x x x
performed
Quality checks on data out- x x x x x
comes performed
Weighted by gender x x x x x
Weighted by age group (no. x (12) x (4) x (10) x (7) x (13)
of groups)
Weighted by population - x (4) - x (3) -
density (no. of groups)
Weighted by administrative x (NUTS2) - - x (NUTS2) x (NUTS3)
regional units (NUTS level or
other)
Other weighting Location (city/ 3 levels of ed- Religion - 4 levels of ed-
village) ucation ucation
Weights are based on (which National Cen- Data from de- Israel Central National cen- National cen-
data) sus 2011 mographic Bureau of Sta- sus 2020 sus 2020
yearbook tistics
Ethical considerations, data protection, and informed

consent by country
Recommendations for ethical approval, data protection, and informed consent
In countries where national regulations foresee a review of ethical conduct requirements (e.g.,
through ethics committees at universities), it has to be ensured that these countries meet these
requirements. Where such requirements and ethics committees are not in place, countries are
required to adhere to national ethical guidelines concerning population surveys and submit their
protocol to any relevant board at national level.
Furthermore, each country participating in the HLS19 was required to:
» comply with applicable national data protection legislation (EU countries additionally needed
to comply with the EU General Data Protection Regulation (GDPR) (Regulation (EU) 2016/679
2016),
» guarantee that study participants were fully informed about the research and procedures in
place to enable them to withdraw from the study easily,
» employ written and/or oral procedures for informed consent, and
» fully document their national procedures.

In Table 2.23 and Table 2.24, details of ethical approval, compliance with data protection rules,
and informed consent are shown by country. National regulations on the need to perform a sci-
entific ethical review vary and ten countries had to get approval from an ethics committee. One
country (DE) was not obliged to obtain such approval but applied for it nevertheless and was
granted it. All participating countries ensured compliance with data protection and explicitly ob-
tained informed consent from respondents before carrying out any interviews.
Table 2.23:
Details of ethical approval, compliance with data protection rules, and informed consent
AT BE BG CH CZ DE DK FR HU
Ethical approval needed? - - - - - - - x x
Ethical approval obtained? - - - - - x - x x
Compliance with data pro- x x x x x x x x x
tection ensured?
Informed consent obtained? x x x x x x x x x
Table 2.24:
Details of ethical approval, compliance with data protection rules, and informed consent
(continued from Table 2.23)
IE IL IT NO PT RU SI SK
Ethical approval needed? x x x x x x x x
Ethical approval obtained? x x x x x x x x
Compliance with data pro- x x x x x x x x
tection ensured?
Informed consent obtained? x x x x x x x x
2.5 Summary of study design and its implementation in the

HLS19: Discussion and conclusions
This chapter describes the HLS19 study design, especially the translation and field testing of the
instruments as well as methods of data collection and processing.
The main features of the cross-sectional study design developed and decided on by the HLS19
Consortium and its working groups were to do a national survey for adults aged 18 and above
living in private households in the 17 participating countries. A minimum sample size of 1,000
was defined, but most countries used larger samples. A multi-stage random sampling procedure
was recommended as a main standard for the methods of sampling, using stratification for gender,
age group, population density, and geographical areas/units. Quota sampling was considered ac-
ceptable if the quota were representative of the population regarding the above criteria. For data
collection using CATI or CAWI, the methods of sampling were adapted to the characteristics of
these methods of data collection. Using personal face-to-face interviews (PAPI or CAPI) was rec-
ommended, but CATI and CAWI were also included in the HLS19 Study Protocol as being acceptable
and were used more often than originally planned due to Covid-19. The time frame for data col-
lection was originally planned for November 2019 to April 2020 but had to be extended to June
2021 due to the Covid-19 pandemic. Countries had to find and commission a suitable agency to
carry out data collection.
Although in principle based on the HLS-EU, the HLS19 deviates from the HLS-EU study protocol in
more than one aspect, which makes a comparison of results with the HLS-EU rather difficult. Of
the 17 participating countries, four had already participated in the original HLS-EU study (AT, BG,
DE, and IE), another nine had administered their own national survey at a later date using the HLS-
EU study design and instrument (BE, CH, CZ, DK, HU, IL, IT, NO, and PT), while four countries
carried out their first national population survey on HL in the HLS19 study (FR, RU, SI, and SK).
For the translation and field testing of instruments, recommendations were provided, which were
adhered to by countries to varying extents.
The same holds true for the methods of data collection administered by the individual countries.
Four “pure” types of data collection were used: PAPI (by DE and RU), CAPI (by SK), CATI (by AT, HU,
IE, NO, and PT), and CAWI (by BE, CH (with a few CATI), DK, and FR). There were also mixed types,
namely CAWI + CATI (by BG, CZ, IL, and IT) and CAPI + CAWI (by SI). In addition to these different
types of data collection, individual types were administered quite differently in detail by the coun-
tries employing them (e.g., selecting and approaching respondents, kind of interviewers, present-
ing questions in interviews, realized response rates).
Therefore, the comparability of results between countries participating in the HLS19 is severely
limited, more so for univariate measures than for measures of association. Due to further differ-
ences, a comparison of results from earlier HLS-EU studies with results from the HLS19 is even
more restricted.
The different kinds of data collection used in the countries participating in the HLS19 and in the
HLS-EU studies can affect the results in different ways:
First, different kinds of data collection use different procedures and sources of data to construct
the samples, which can therefore represent different populations (Robling et al. 2010). Hoebel et
al. (2014) found that socio-demographic characteristics differed significantly between study par-
ticipants in paper, telephone, and online data collection procedures. Cornesse/Bosnjak (2018)
found that web surveys have a significantly lower representativeness compared to other survey
methods. It can be hypothesized that for personal interviews, better representation can be ob-
tained of vulnerable groups in the population than for CATI or CAWI. This can partly be demon-
strated by looking at the raw distributions of representativeness indicators in samples. Possibly,
these effects can be dealt with partly by using weighting procedures.

Second, while most countries used probability samples, quota sampling was also used by some
countries, partly due to the kind of data collection chosen, e.g., CATI, which can have effects on
the representativeness of the raw sample.
Third, the response rates can differ for the three modes of data collection, which is clearly demon-
strated by the considerable differences in response rates achieved for national samples in the
HLS19 study.
Fourth, the different communication situations when the interviewer is personally present, on the
telephone, or when a computer is used can have different effects. According to the literature, one
consistent result is that people interviewed by telephone tend to give more positive answers to
scale questions than people completing online questionnaires (Christian et al. 2008; Lugtig et al.
2011; Ye et al. 2011) and that online surveys yield more accurate and complete responses than
telephone surveys (Chang/Krosnick 2009; Kreuter et al. 2009). Fessler et al. (2018) looked at
variation in interview modes in the Austrian EU SILC panel and found that a switch from CAPI to
CATI led to major changes in response behavior (unit and item non-response) and answering
behavior (potential misreporting), leading to large differences in estimated inequality measures.
In addition, differences in sample size may affect the precision of the results.
Collecting data in different seasons of the year, with the respondents’ experiences of illness at the
time of the survey differing, may also influence responses on health-related issues. Moreover, the
national surveys took place in different stages of the Covid-19 crisis, and the communication
strategies used by the countries also differed.
Conclusions
The differences in the way the study design was fulfilled across countries severely restrict direct
and precise comparisons between the HLS19 countries, and even more so with the results of earlier
HLS-EU studies. Nevertheless, this international study of 17 countries trying to follow a common
study design as far as possible offers a rare opportunity to study general trends and ranges in
variation for HL across countries in the WHO European Region.
2.6 References
Chang, Linchiat; Krosnick, Jon A. (2009): National Surveys Via Rdd Telephone Interviewing Versus
the Internet. In: Public Opinion Quarterly 73/4:641-678
Christian, Leah Melani; Dillman, Don A; Smyth, Jolene D (2008): The effects of mode and format
on answers to scalar questions in telephone and web surveys. In: Advances in telephone survey
methodology 12/:250-275
Cornesse, Carina; Bosnjak, Michael (2018): Is there an association between survey characteristics
and representativeness? A meta-analysis. In: European Survey Research Association 12/1:1-
13
Fessler, Pirmin; Kasy, Maximilian; Lindner, Peter (2018): Survey mode effects on measured income
inequality. In: The Journal of Economic Inequality 16/4:487-505
HLS-EU-Consortium (2012): Comparative Report on Health Literacy in Eight EU Member States.

The European Health Literacy Survey. The international Consortium of the HLS-EU Project,
Online
Hoebel, Jens; von der Lippe, Elena; Lange, Cornelia; Ziese, Thomas (2014): Mode differences in a
mixed-mode health interview survey among adults. In: Archives of Public Health 72/1:1-12
Kreuter, F.; Presser, S.; Tourangeau, R. (2009): Social Desirability Bias in CATI, IVR, and Web Sur-
veys: The Effects of Mode and Question Sensitivity. In: Public Opinion Quarterly 72/5:847-865
Lugtig, Peter; Lensvelt-Mulders, Gerty JLM; Frerichs, Remco; Greven, Assyn (2011): Estimating
nonresponse bias and mode effects in a mixed-mode survey. In: International Journal of Mar-
ket Research 53/5:669-686
Regulation (EU) 2016/679 (2016): Regulation (EU) 2016/679 of 27 April 2016 on the European
Parliament and the Council on the protection of natural persons with regard to the processing
of personal data and on the free movement of such data, and repealing Directive 95/46/EC
(General Data Protection Regulation)
Robling, Michael R; Ingledew, David K; Greene, Giles; Sayers, Adrian; Shaw, Chris; Sander, Lesley;
Russell, Ian T; Williams, John G; Hood, Kerenza (2010): Applying an extended theoretical
framework for data collection mode to health services research. In: BMC Health Services Re-
search 10/1:1-12

sortium, Hls-Eu (2015): Health literacy in Europe: comparative results of the European health
literacy survey (HLS-EU). In: European journal of public health:1-6
Ye, Cong; Fulton, Jenna; Tourangeau, Roger (2011): More positive or more extreme? A meta-anal-
ysis of mode differences in response choice. In: Public Opinion Quarterly 75/2:349-365
3 The HLS19 instruments
Authors:
Thomas Link (ICC)
3.1 Development of the HLS19 instruments
Working groups were established by the HLS19 Consortium to develop the HLS19 instruments for
measuring General HL as well as selected specific forms and aspects of HL (Navigational HL, Com-
municative HL, Digital HL, Vaccination HL, the costs and economics of HL), and to identify corre-
lates relevant for potential determinants and consequences of HL. The proposals of the working
groups were discussed and approved by the HLS19 consortium.
The HLS19 instruments on General HL were based on the HLS-EU instrument and its conceptual
framework and definition of HL so the HLS19 instruments on General HL cover three domains in
which dealing with health-relevant information is needed: healthcare (or managing illness), dis-
ease prevention and health promotion, and the four key competencies of health-related infor-
mation management: to access/obtain, understand, appraise/judge/evaluate, and apply/use in-
formation relevant for health (see Chapter 1, Figure 1.1 for the matrix of subdimensions of HL
based on the HLS-EU conceptual model). The HLS19 instruments for measuring specific forms and
aspects of HL used the format of the instrument for measuring General HL as much as possible.
The HLS19 instruments measuring General HL: long

and short versions
Building on the HLS-EU-Q47
The generic HLS19 instruments, which measure General HL and consist of 12, 16, or 47 items (and
are termed HLS19-Q12, HLS19-Q16, and HLS19-Q47 respectively), were based on the HLS-EU in-
struments.
The HLS-EU instrument consists of 47 items for measuring comprehensive General HL (referred
to as the HLS-EU-Q47) and 39 correlates of HL (HLS-EU-Consortium 2012; Sørensen et al. 2012;
Sørensen et al. 2015; Sørensen et al. 2013b). The HLS-EU-Q47 - the long form - was first devel-
Chapter 3 / The HLS19-Instruments 57

oped, validated, and applied in eight countries (Austria, Bulgaria, Germany (North Rhine-West-
phalia), Greece, Ireland, Netherlands, Poland, and Spain) in seven languages (Bulgarian, Dutch,
English, German, Greek, Polish, and Spanish) by the initial HLS-EU study and was subsequently
validated and used for general population samples in follow-up studies of the survey in the fol-
lowing countries in the WHO European Region: Albania, Czech Republic, Hungary, Israel (Hebrew,
Arab, Russian), Italy, Lithuania, Norway, Portugal, Switzerland (German, French, Italian, Portu-
guese, Turkish), and Turkey. It was since been used in Asia in Indonesia, Japan, Kazakhstan, Ma-
laysia, Myanmar, Taiwan (Mandarin), and Vietnam, as well as for research in other European coun-
tries such as in Serbia, in the Americas such as in Brazil (Brazilian Portuguese) and Mexico, and in
Asia, such as in Hong Kong and Sri Lanka (Sinhalese) (for details, see Pelikan/Ganahl (2017a);
Pelikan/Ganahl (2017b); Pelikan et al. (2019); Pelikan et al. (2020)).
Experiences with the HLS-EU-Q47 showed that it was perceived as taking a long time to complete
(10 minutes). As a result, two short forms of the instrument were constructed to measure HL: the
HLS-EU-Q16 (about three minutes) and the HLS-EU-Q6 (about one to two minutes).
Besides the original eight countries with seven languages, the HLS-EU-Q16 short version was also
validated and used in population studies in Belgium (Dutch, French), Denmark, France, Iceland,
Israel (Hebrew, Arabian, Russian), Italy, Kazakhstan, Malta (Maltese, English), Sweden (Swedish,
Arabic), and Turkey. Translated versions have also been used for research in Finland, Egypt (Ara-
bic), Ethiopia (Amharic), Ghana (Twi), and China (Mainland Mandarin).
The 6-item version of the HLS (HLS-EU-Q6) has been used in the original eight countries (Austria,
Bulgaria, Germany, Greece, Ireland, Netherlands, Poland, and Spain) representing seven languages
(Bulgarian, Dutch, English, German, Greek, Polish, and Spanish) but also in France and Italy (for
details, see Pelikan et al. (2020)).
Independently, another two short forms based on the HLS-EU-Q47 containing twelve items each
were developed: an Asian version (Duong et al. 2019a; Duong et al. 2017b) and a Norwegian
version (Finbraten et al. 2017; Finbraten et al. 2018). The Asian version was developed by explor-
ative and confirmatory factor analyses (EFA, CFA) while the Norwegian one was validated by testing
the data against the unidimensional Rasch model for polytomous data, also referred to as the
Partial Credit Model (Masters 1982)
The HLS-EU-Q12 was developed by taking the disadvantages of the HLS-EU-Q16 into account and
building on the two available 12-item short forms (the Norwegian HLS-Q12 and the Taiwanese
HL-SF12). On the basis of data from the eight HLS-EU countries, Item Response Theory analyses
were conducted to achieve maximum overlap with the HLS-EU-Q16 and to identify a 12-item set
with the lowest deviance from the assumptions of the Partial Credit Model (PCM; (Masters 1982))
when analyzed separately for each HLS-EU-8 country (Waldherr et al. 2021). Data from the fol-
lowing countries went into the construction of the HLS-EU-Q12: Austria, Bulgaria, Germany (North
Rhine-Westphalia), Greece, Ireland, Netherlands, Poland, and Spain (from the HLS-EU study) as
well as survey data from Hungary and the Czech Republic.
Criteria in the development of the HLS-EU-Q12 were that the 12 items should reflect the 12 cells
of the HLS-EU matrix and form a locally independent scale (unidimensional and no response de-
pendency) with acceptable data-model fit when using the Partial Credit Model (Waldherr et al.
2021).
As such, the HLS-EU-Q12 is not only slightly shorter than the HLS-EU-Q16 but also better repre-
sents the underlying model and definition of the HLS-EU instruments.
The HLS19-Q47 and the short forms HLS19-Q12 and HLS19-Q16
Based on the different versions of the HLS-EU, in the HLS19 different forms for measuring HL were
offered: the HLS19-Q47, the HLS19-Q12, and a set of 22 items which, besides being used for the
HLS19-Q12, were used to construct the HLS19-Q16. For countries measuring HL for the first time,
it was recommended to use the HLS19-Q47, but a minimum for each country was to measure the
HLS19-Q12.
Changes in the wording of response categories and selected items
The decision was made to remove the original qualifier “fairly” from the 4-point rating scale as
it is prone to be translated and interpreted differently in different countries. This ensures that a
more uniform understanding is likely after translation into different languages.
Therefore, in the HLS19 instruments, the phrases anchored in the 4-point rating scale response
categories were changed from “very difficult” - “fairly difficult” - “fairly easy” - “very easy” (as used
in the HLS-EU) to “very difficult” - “difficult” - “easy” - “very easy”. Removing the word “fairly” from
the two response categories in the middle of the 4-point rating scale was informed by empirical
evidence from field testing in Norway, which showed fewer problems with unordered response
categories after removing the phrases anchored with the middle/central categories).
A “don’t know” category was not included, as in the HLS-EU but was coded by the interviewer as
“no answer” (for CAPI/PAPI and CATI).
In addition, some of the HLS-EU items were revised for the HLS19 instruments measuring General
HL. Revisions involved rewording items as well as adding or removing examples within items. For
changes in the HLS19, the following criteria were applied:
Criteria for revising the wording of items included:
» wording which was too complex or words which are difficult to understand (based on expert
views and qualitative studies) (Domanska et al. 2018; Finbraten et al. 2018; Storms et al.
2017),
» the harmonization of similar terms (e.g., health and well-being, examples of types of me-
dia),

» indications based on quantitative aspects such as high non-response rates in the HLS-EU,
response dependency, item fit (under-discriminating items, differential item functioning
(Finbraten et al. 2017; Finbraten et al. 2018; Huang et al. 2018) and additional analyses of
HLS-EU (unpublished).
» items should clearly relate to dealing with health information.
Criteria for removing or adding examples:
» use of gender-neutral examples (e.g., removing breast exam),

» adaptation to societal changes in lifestyle and health-related or medical practices.
Table A 3.1 in Annex 3 highlights the differences between the HLS-EU-Q47 and the HLS19-Q47
instruments.
The HLS19 instruments were first field tested by Germany in November 2019. The German field-
test results were considered for the development of the final English version of the HLS19 instru-
ments (see Subsection 2.3.2).
Core and optional correlates of General HL
Theoretical framework for selecting correlates
The theoretical framework for selecting correlates (determinants and consequences) for the HLS19
was the HLS-EU conceptual model of HL (see Figure 1.1, Chapter 1) (Sørensen et al. 2012), which
was refined to create the Vienna Model of Health Literacy (Pelikan/Ganahl 2017a). According to
the Vienna Model (see Figure 1.2, Chapter 1), individual or personal HL is influenced by personal
determinants like socio-demographic and socio-economic factors (such as gender, age, educa-
tional level, migration background) on the one hand and situational determinants on the other
hand. Personal HL, in turn, can directly influence lifestyle-related health behaviors, like tobacco
and alcohol consumption, physical activity, and nutrition. Personal HL can have a direct and indi-
rect influence on indicators of health status, like self-perceived health and long-term ill-
nesses/health problems. Long-term illnesses/health problems could also be considered a per-
sonal determinant of HL. Illness-related behaviors (e.g., the extent of utilization of health care
services) can be directly influenced by HL, and indirectly by its effects on health behaviors and
health status. Furthermore, health behaviors, health status, and illness behaviors can also be in-
fluenced by personal and situational determinants. The model also allows (albeit smaller) causal
or cyclical effects in the opposite direction.
Further criteria for selecting correlates in the HLS19
Indicators for correlates were selected based on the conceptual model, on specified research ques-
tions, and also by following analyses done in previously published studies using the HLS-EU data
as well as other analyses in international publications. The following research questions were
agreed upon by the Consortium:
» How valid are the HL measures and what are their psychometric characteristics?
» How are the HL measures and their individual items distributed in general and in relevant
subpopulations in the participating countries?
» How strongly are the HL measures associated with socio-demographic, socio-economic, and
additional, pre-selected potential determinants?
» Is there a social gradient for the HL measures?
» How are the HL measures associated with potential consequences of HL in connection with
the
» indicators of health behaviors and lifestyles,
» health status indicators, and
» indicators of health care utilization?
The selection criteria required that the correlates to be included either had to be potential deter-
minants or consequences of HL, moderating or mediating determinants, or consequences of HL
(see Vienna Model of Health Literacy (Pelikan/Ganahl 2017a)). If possible, they should have already
been used in the HLS-EU or in other HL or health-related studies (e.g., EHIS). In addition, a theory-
based hypothesis regarding an association with HL (not just with health) had to be ensured. For
correlates already used in the HLS-EU, indicators were omitted if they did not work well, were
rephrased, or if there were similar, better options in other studies.
Criteria for the wording of items:
» inclusion in international/national standard instruments, like the EHIS, ESS, and EU-SILC,
» easy to interpret by respondents and researchers,
» easy to answer for respondents (questions and categories not too wordy),
» simple and efficient; not taking too much time in interviews,
» categories adequate for statistical data analysis,
» possibility of building indices for a set of indicators by standardizing the format of item
wording and categories (e.g., for health care utilization, healthy lifestyle).
Core and optional correlates
The HLS19 differentiated between core correlates, which all countries had to use, and optional
correlates, which could be chosen purposely by countries to deal with problems of instrument
length, interview time, and costs as well as to allow for specific national adaptations and the ad-
dition of items. The NSCs of participating countries needed to make sure that, if possible, they
used already existing translations of included items by consulting documents relating to relevant
international surveys such as the EHIS. The final HLS19 instruments for General HL include 31 core
correlates and 18 optional correlates.

Determinants of HL with selected indicators investigated in this International Report:
The following core determinants were used in the HLS19 analyses in this International Report (for
General HL in Chapter 6):
Socio-demographic determinants:
» gender
» age.
Socio-economic determinants:
» education
» level in society
» financial deprivation.
Additional determinants:
» migration
» training in a healthcare profession
» status of employment
» long-term illnesses/health problems (whereby long-term illnesses/health problems was an-
alyzed in the HLS19 in two ways, both as a determinant and as a possible consequence of HL
(see below)).
Indicators of potential consequences investigated in this International Report:
Associations of HL with the following indicators of potential consequences were investigated in

this International Report (for introductions on detailed measurements of the indicators and results,
see the relevant chapters):
Health behavior and lifestyles (for General HL in Chapter 7):
» BMI
» physical activity
» smoking behavior
» alcohol consumption
» fruit and vegetable consumption.
Health status (for General HL in Chapter 8):
» self-reported health
» long-term illnesses/health problems (whereby long-term illnesses/health problems was an-
alyzed in the HLS19 in two ways, both as a determinant (see above) and as a possible conse-
quence of HL)
» limitations due to health problems.
Health care utilization (for General HL in Chapter 9):
» utilization of emergency services

» utilization of GPs/family doctors
» utilization of medical or surgical specialists
» utilization of inpatient hospital services
» utilization of day patient hospital services.
Specific HL measures and their correlates: optional

packages
In the HLS19 additional, optional packages were developed to measure specific aspects of HL which
participating countries could select. Working groups were established for each of the six optional
packages (the instrument identifiers are in brackets):
» Digital HL (HLS19-DIGI),
» Communicative HL with physicians in healthcare services (HLS19-COM-P-Q11 (long
form)/HLS19-COM-P-Q6 (short form)),
» Navigational HL (HLS19-NAV),
» Vaccination HL (HLS19-VAC),
» Cost and health economics of HL (HLS19-ECON), and
» Functional HL measured by the Newest Vital Sign Test (NVS).
Each working group provided a rationale, including background information and an overview of
selected literature, research questions, and suggestions for analyses. An instrument (partly based
on existing tools) was developed for each optional package.
The instruments, analyses, and results of the optional packages are described in detail in Chapters
10 to 14.
National add-ons
Participating countries added items to their survey which were important for health policy issues
in their country.
Items on concern about the Covid-19 pandemic and the Corona HL Questionnaire were such na-
tional add-ons: During the data collection phase for the HLS19 Project, the Covid-19 pandemic
started, and in most countries data collection was performed during the pandemic. Therefore, an
additional question “How much are you personally concerned about the situation caused by Co-
rona?”, with the response categories “strongly” – “fairly” – “somewhat” – “not at all”, was recom-
mended for inclusion in the questionnaire and was at least used by Austria and Israel. Furthermore,
an additional questionnaire on Covid-19, the Corona Health Literacy Questionnaire (Griebler
2020), including 16 items on obtaining, understanding, judging, and basing one’s own decisions
on information about being infected with the coronavirus, was developed and proposed to the

participating countries, with the response categories “very easy” – “easy” – “difficult” – “very diffi-
cult”. The Corona Health Literacy Questionnaire was used in Italy.
3.2 Overview of the final HLS19 instruments
The final HLS19 national instruments comprised different components (see Table 3.1), including
core (mandatory) and optional packages or optional items:
1. A core measurement of comprehensive General HL:
» the HLS19-Q12 (consisting of 12 items) or
» a set of 22 items (consisting of the HLS19-Q12 plus 10 further items from the HLS-
EU-Q47 to allow the HLS19-Q16 to be analyzed as well), or
» the full version, the HLS19-Q47, consisting of 47 items.
2. 31 core correlates on determinants and potential consequences of HL.
3. Optional packages and optional items to be selected:
» 18 optional correlates on determinants and consequences of HL,
» optional packages on measures of specific aspects of HL including correlates of
these specific aspects of HL,
» national add-ons, referring to country-specific questions related to topics of im-
portance for national policy.
Table 3.1:
Overview of the HLS19 instruments including core and optional parts/items
Instrument Number of Number of Total num-

items correlates ber of items
measuring
HL
General HL measures
HLS19-Q12 12 12
Set of 22 items (Q12 & Q16) 22 22

Core
HLS19-Q47 47 47
Correlates
Core correlates 31 31
Optional correlates 18 18
Measures of specific aspects of HL/Optional packages
HLS19-NAV 12 12
HLS19-COM-P-Q11 (long form)/ 11 11
HLS19-COM-P-Q6 (short form) 6 6
Optional
HLS19-DIGI 10 6 16
HLS19-VAC 4 10 14
HLS19-ECON 18 18
NVS 7 7
Additional national items
To be defined by the participating countries
In Section 3.4 an overview of the implementation of the HLS19 instruments, correlates, and addi-
tional optional items by country is shown.
3.3 Description of the generic HLS19-Q12 instrument
Since countries had to include at least the 12 items from the HLS19-Q12 in their surveys, the
analyses in this International Report are mainly based on the HLS19-Q12 scores. Therefore, the
HLS19-Q12 instrument is presented here in more detail. For an overview of the items of the HLS19-
Q47 as well as the set of 22 items which permit calculation of the HLS19-Q16 and the HLS19-Q12,
see Table A 3.2 in Annex 3, (for a comparison of the short forms HLS19-Q12 and HLS19-Q16 with
the long form HLS19-Q47, see Section 5.6.).
Table 3.2 introduces the HLS19-Q12 instrument and its item numbers as well as the related item
numbers in Q22 and Q47. The wording of the items is also included.
Table 3.2:
The HLS19-Q12 instrument, item numbers in the Q12 and Q47, and item wording
Item
Item wording
no. in Item no. in Q47
On a scale from very easy to very difficult, how easy would you say it is:
Q12
1 CORE-HL4 …to find out where to get professional help when you are ill?
[instructions: such as doctor, nurse, pharmacist, psychologist]
2 CORE-HL7 …to understand information about what to do in a medical emergency?
3 CORE-HL10 …to judge the advantages and disadvantages of different treatment options?
4 CORE-HL16 ...to act on advice from your doctor or pharmacist?
5 CORE-HL18 …to find information on how to handle mental health problems?
[Instruction: stress, depression or anxiety]
6 CORE-HL23 …to understand information about recommended health screenings or examina-
tions?
[Instructions: e.g., colorectal cancer screening, blood sugar test]
7 CORE-HL24 ...to judge if information on unhealthy habits, such as smoking, low physical activity
or drinking too much alcohol, are reliable?
8 CORE-HL31 …to decide how you can protect yourself from illness using information from the
mass media?
[Instructions: e.g., Newspapers, TV or Internet]
9 CORE-HL32 …to find information on healthy lifestyles such as physical exercise, healthy food or
nutrition?
10 CORE-HL37 …to understand advice concerning your health from family or friends?
11 CORE-HL42 ...to judge how your housing conditions may affect your health and well-being?
12 CORE-HL44 …to make decisions to improve your health and well-being?
In Table 3.3, the items in the HLS19-Q12 are positioned according to the conceptual matrix model
(Sørensen et al. 2012). One item is allocated to every cell of the matrix. For the positioning of the
items of the HLS19-Q47 in relation to the definitions and conceptual matrix model of the HLS-EU
consortium (Sørensen et al. 2012), see Table A 3.3, Annex 3.

Table 3.3:
Categorization of the items in the HLS19-Q12 (using the item numbers for the Q12 and Q47) in
relation to the conceptual model of HL according to SØrensen et al. (2012)
Access/obtain in- Understand infor- Appraise/judge/evalu- Apply/use infor-

Health literacy formation relevant mation relevant to ate information rele- mation relevant to
to health health vant to health health
Health care 1/CORE-HL4 2/CORE-HL7 3/CORE-HL10 4/CORE-HL16
Disease prevention 5/CORE-HL18 6/CORE-HL23 7/CORE-HL24 8/CORE-HL31
Health promotion 9/CORE-HL32 10/CORE-HL37 11/CORE-HL42 12/CORE-HL44
3.4 Overview of the implementation of the HLS19 instruments

in the participating countries
Participating countries could choose between the HLS19-Q12, HLS19-Q16, and HLS19-Q47 as well
as select different optional items according to national interests (such as optional correlates, the
“concern about Corona” item, and other national add-ons). The also could choose to include op-
tional packages to measure specific aspects of HL.
Table 3.4 provides an overview of the implementation of all core and optional parts of the HLS19
instruments in the participating countries. The core instruments and selected items included in
the main analyses of the International Report are marked in bold. The instrument on measuring
Navigational HL (HLS19-NAV) was included in the national data collection by eight countries. The
optional package on Communicative HL with physicians in health care services was implemented
by nine countries; three countries included the 11-item long version HLS19-COM-P-Q11, and six
countries included the short version HLS19-COM-P-Q6, which is a subset of the long form. Twelve
countries chose all of the items in the optional package on Digital HL (HLS19-DIGI), and one country
(NO) used only the items on the HL measures (but not the correlates). Therefore, it was possible
to compute a comparable score for 13 countries. The full optional package on Vaccination HL
(HLS19-VAC), which includes a 4-item VAC-HL measure and 10 correlates, was used by seven
countries; as four more countries used the HLS19-Q47, information at least on VAC-HL could be
gained. Items on Cost and health economics of HL (HLS19-ECON) were used by three countries,
whereby none of them used the full instrument. The instrument on functional HL (NVS) was im-
plemented by two countries (Table 3.4).
Table 3.4:
Overview of the implementation of the HLS19 instruments, correlates, and additional optional items by country
No. of
AT BE BG CH CZ DE DK FR HU IE IL IT NO PT RU SI SK
countries
HLS19-Q47 x x x x x x 6
Set of 22 items (Q12 & Q16) x x x x x x x x x x x x x x 14
HLS19-Q12 x x x x x x x x x x x x x x x x x 17
Additional HL items when using 22 items or Q12 x x x x 4
31 core correlates x x x x x x x x x x x x x x x x x 17
Selected optional correlates x x x x x x x x x x 11
HLS19-NAV x x (x) x x x x (x) x x 8
HLS19-COM-P-Q11 x x (x) x 3 + (1)
HLS19-COM-P-Q6 x x x x x x x x (x) x 9 + (1)
HLS19-DIGI x x (x) x x x x x x x x (x) x x 12 + (1) +(1)
HLS19-VAC x x (x)* x (x)* x x (x)* (x)* x x 7+ (4)
HLS19-ECON (x) (x) (x) (3)
NVS x x 2
National items x x x x x x x x x x x** x x x 13
Total number of items used 119 132 94 90 108 176 134 148 97 159 75 94 149 94 71 156 82
* In 4 counties 4 items on VAC-HL were measured by using HLS19-Q47
** IT used the full ‘Corona Health Literacy Questionnaire’ (Griebler/Nitsche 2020) after translating it in Italian language
x=all items of an instrument used, (x)=only selected items of an instrument used
items included in the analyses of the main part of the International Report are in bold

3.5 References
Domanska, O. M.; Firnges, C.; Bollweg, T. M.; Sørensen, K.; Holmberg, C.; Jordan, S. (2018): Do
adolescents understand the items of the European Health Literacy Survey Questionnaire (HLS-
EU-Q47) - German version? Findings from cognitive interviews of the project "Measurement
of Health Literacy Among Adolescents" (MOHLAA) in Germany. In: Arch Public Health 76/:46
Duong, T. V.; Aringazina, Altyn; Kayupova, Gaukhar; Nurjanah; Pham, Thuc V.; Pham, Khue M.;
Truong, Tien Q.; Nguyen, Kien T.; Oo, Win Myint; Su, Tin Tin; Majid, Hazreen Abdul; Sørensen,
Kristine; Lin, I. Feng; Chang, Yuwen; Yang, Shwu-Huey; Chang, Peter W. S. (2019):
Development and Validation of a New Short-Form Health Literacy Instrument (HLS-SF12) for
the General Public in Six Asian Countries. In: HLRP: Health Literacy Research and Practice
3/2:e91-e102
Duong, T. V.; Chang, P. W.; Yang, S. H.; Chen, M. C.; Chao, W. T.; Chen, T.; Chiao, P.; Huang, H. L.
(2017): A New Comprehensive Short-form Health Literacy Survey Tool for Patients in General.
In: Asian Nurs Res (Korean Soc Nurs Sci) 11/:30-35
Finbraten, H. S.; Pettersen, K. S.; Wilde-Larsson, B.; Nordstrom, G.; Trollvik, A.; Guttersrud, O.
(2017): Validating the European Health Literacy Survey Questionnaire in people with type 2
diabetes: Latent trait analyses applying multidimensional Rasch modelling and confirmatory
factor analysis. In: J Adv Nurs 73/11:2730-2744
Finbraten, H. S.; Wilde-Larsson, B.; Nordstrom, G.; Pettersen, K. S.; Trollvik, A.; Guttersrud, O.
(2018): Establishing the HLS-Q12 short version of the European Health Literacy Survey
Questionnaire: latent trait analyses applying Rasch modelling and confirmatory factor analysis.
In: BMC Health Serv Res 18/:506
Griebler, Robert; Nitsche, Michael (2020): The Austrian Corona Health Literacy Questionnaire.
Gesundheit Österreich GmbH & Das Österreichische Gallup Institut, Vienna
HLS-EU-Consortium (2012): Comparative Report on Health Literacy in Eight EU Member States.

The European Health Literacy Survey. The international Consortium of the HLS-EU Project,
Online
Huang, Yi-Jing; Lin, Gong-Hong; Lu, Wen-Shian; Tam, Ka-Wai; Chen, Chiehfeng; Hou, Wen-Hsuan;
Hsieh, Ching-Lin (2018): Validation of the European Health Literacy Survey Questionnaire in
women with breast cancer. In: Cancer nursing 41/2:E40-E48
Masters, Geoff N (1982): A Rasch model for partial credit scoring. In: Psychometrika 47/2:149-
174
Pelikan, Jürgen M.; Ganahl, Kristin (2017a): Die europäische Gesundheitskompetenz-Studie:
Konzept, Instrument und ausgewählte Ergebnisse. In: Health Literacy Forschungsstand und
Perspektiven. Hg. v. Schaeffer, Doris; Pelikan, Jürgen M. hogrefe, Bern. S. 93-S. 125
Pelikan, Jürgen M.; Ganahl, Kristin (2017b): Measuring Health Literacy in General Populations:
Primary Findings from the HLS-EU Consortium´s Health Literacy Assessment Effort. In: IOS
Press. Hg. v. Logan, G. D.; Siegel, Elliot R. S. 34-S. 59
Pelikan, Jürgen M.; Ganahl, Kristin; van den Broucke, S; Sørensen, Karin (2019): Measuring health
literacy in Europe: Introducing the European Health Literacy Survey Questionnaire (HLS-EU-Q)
In: International handbook of health literacy Research, practice and policy across the life-span.
Hg. v. Okan, O. et al. Policy Press. S. 115-S. 138
Pelikan, Jürgen M.; Straßmayr, Christa; Ganahl, Kristin (2020): Health Literacy Measurement in
General and Other Populations: Further Initiatives and Lessons Learned in Europe (and
Beyond). In: Health Literacy in Clinical Practice and Public Health New Initiatives and Lessons
Learned at the Intersection with other Disciplines. Hg. v. Logan, G. D.; Siegel, E. R. . IOS PressS.
170-191
Sørensen, K.; Van den Broucke, S.; Fullam, J.; Doyle, G.; Pelikan, J.; Slonska, Z.; Brand, H.;
Consortium Health Literacy Project, European (2012): Health literacy and public health: a
systematic review and integration of definitions and models. In: BMC Public Health
12/80:http://www.biomedcentral.com/1471-2458/1412/1480
Agrafiotis, D.; Uiters, E.; Falcon, M.; Mensing, M.; Tchamov, K.; Broucke, S. V.; Brand, H.;
Consortium, Hls-Eu (2015): Health literacy in Europe: comparative results of the European
health literacy survey (HLS-EU). In: European journal of public health:1-6
in populations: illuminating the design and development process of the European Health
Literacy Survey Questionnaire (HLS-EU-Q). In: BMC Public Health Oct/13:948
Storms, H.; Claes, N.; Aertgeerts, B.; Van den Broucke, S. (2017): Measuring health literacy among
low literate people: an exploratory feasibility study with the HLS-EU questionnaire. In: BMC
Public Health 17/:475
Waldherr, Karin; Alfers, Tobias; Peer, Sandra (2021): Development and validation of the HLS-EU-
Q12. https://m-pohl.net/Results

4 Methods
Authors:
Thomas Link (ICC)
Nejc Berzelak (SI)
Henrik Bøggild (DK)
Hanne Søberg Finbråten (NO)
Øystein Guttersrud (NO)
Christopher Le (NO)
Marie Germund Nielsen (DK)
Paulo Jorge Nogueira (PT)
Kjell Sverre Pettersen (NO)
4.1 Introduction
The results of the HLS19 study will be made available through three different types of media:
(1) through this International Report, aimed primarily at health policy makers in the WHO European
Region; (2) through peer-reviewed publications, aimed primarily at the international research
community; and (3) through fact sheets on HL measurement tools, aimed primarily at practitioners
and researchers.
The methods of data analysis and formats of the presentation of results used in this International
Report aim to be as simple as possible but as complex as necessary to answer the research ques-
tions concerned. As with the entire HLS19 study, the methods and presentations of data analyses
are based partly on the procedures used in the HLS-EU study. The International Report aims to
demonstrate general trends in the results across all participating countries and the extent of var-
iation in these trends across countries. To do that, methods were used that worked well enough
across all countries.
At the same time, the authors of the International Report abstained from optimizing methods for
individual countries, which must be left to the National Reports of those countries. Furthermore,
the International Report does not aim to explain differences in the results among countries, which
may be due in part to different methods of data collection. The International Report should, nev-
ertheless, present the results for individual countries in such a way that allows countries to com-
pare their results with general trends and with the results of countries with which they want to
compare themselves. More sophisticated methods and detailed analyses are planned for peer-
reviewed publications on selected topics.
It is intended that each chapter in the International Report can be understood independently, while
cross-references to other chapters with more detailed information are indicated where necessary.
With the support of appropriate captions, labels, and footnotes, both tables and figures should be
self-explanatory.
After showing the distributions of HL items (and their correlations in the annex to each chapter)
and constructing scales, the statistical analysis was first based on univariate statistics to represent
the distributions of HL measures (Chapter 5 for General HL). Second, bivariate associations be-
tween the scale scores and socio-demographic and socio-economic variables were conducted to
help identify the main health literacy determinants, which were investigated further using multi-
variable linear regression analyses (Chapter 6 for General HL). Multivariable linear regression mod-
els were also used to examine the predictive value of General HL concerning potential health-
related consequences of HL (Chapters 7 to 9 for General HL). For specific measures of HL, all the
steps of the analyses are included in the relevant chapters (Chapters 11 to 15).
The construct validity of the general and specific HL measures was established using methods
from Classical Test Theory (e.g., Cronbach’s alpha for internal consistency and Confirmatory Fac-
tor Analysis for the factor structure) and Item Response Theory (e.g., the Rasch model).
Chapter 4 / Methods 71
4.2 Construction of the HL scales
General Health Literacy (GEN-HL) scores
For the International Report, General Health Literacy (GEN-HL) scores were calculated primarily for
the HLS19-Q12 and, in specific cases, also for the long form HLS19-Q47 and the short form HLS19-
Q16 (cf. Chapters 5 to 7).
The construction of the score values deviates from the HLS-EU study because using the mean of
items with an ordinal, bipolar 4-point Likert scale was criticized when the equidistance of the four
response categories is not guaranteed. In the HLS19 study, the scores are calculated as the per-
centage (ranging from 0 to 100) of items with valid responses that were answered with “very easy”
or “easy” provided that at least 80% of the items contained valid responses; in other words, if less
than 80% of the items contained valid responses, the score is set to “missing”. A higher score value
signifies a higher level of health literacy.
Specific HL measures
In addition to the GEN-HL score for the HLS19-Q12 that was used throughout the International
Report, Chapters 11 to 15 on optional packages introduce measures for specific health literacies.
These measures and scores are:
HL-NAV Navigational Health Literacy score (12 items)

HL-COM-Q11 Communicative Health Literacy with physicians in health care services score
(11 items)
HL-COM-Q6 Communicative Health Literacy with physicians in health care services score
(6 items, i.e., a short form of HL-COM-11 just including six of the 11 items)
HL-DIGI Digital Health Literacy score (8 items)
HL-DIGI-INT Digital Interaction Literacy score (2 items)
HL-VAC Vaccination Health Literacy score (4 items, which all are included in the HLS19-
Q47 as well)
Following the method for calculating the GEN-HL score described above, these scores are also
calculated as the percentage (ranging from 0 to 100) of items with valid responses that were
answered with “very easy” or “easy” given that at least 80% of the items contained valid responses.
The scores for the specific health literacy measures are explained in detail in the relevant chapters.
Categorical levels for the HLS19-Q12
To visualize the distribution of health literacy within populations by percentages, it is common

practice for many HL measures to also define categorical levels of health literacy. This was the
case for the HLS-EU measures as well, and countries that had already used these were interested
in having such a measure for the HLS19 too. Therefore, categorical levels for General HL, as meas-
ured by the HLS19-Q12, are provided but not for the new specific HL measures. It was decided to
use four categorical levels, as in the HLS-EU, and in accordance with the four response categories
offered. Since the calculation of health literacy scores in the HLS19 (which is based on dichotomized
items) deviates from the HLS-EU methodology, where a mean score of the polytomous items was
used, the calculation of the levels had to be adapted. For the levels “excellent” and “inadequate”
the percentage of items answered with “very easy” was used in addition to the score (i.e., percent-
age of items that were answered with “easy” or “very easy”, or rather 100 minus the percentage of
“difficult” or “very difficult”). Consequently, in some cases, respondents with equal scores are as-
signed to different levels. Another challenge with the definition of the four levels of health literacy
described above is that in some cases, respondents who reported a higher health literacy level
when using the original 4-point rating scale for all the HLS19-Q12 items may be assigned to a
lower categorical health literacy level than respondents who reported a lower level of health liter-
acy based on the same rating scale.
For reasons of comparability and easier comprehensibility, the category labels used in the HLS-EU
were retained. These normative labels are defined in a transparent way following a simple ruleset,
namely that these labels should be easy to understand and suggest an intuitive ranking of lower
or higher levels of health literacy. The level of “inadequate”, for example, should be used to de-
scribe people for whom most of tasks included in the HLS19-Q12 were “difficult” or “very difficult”,
with one task at the most being “very easy”.
The following definitions of cut-off points 3 for the categorial levels of the HLS19-Q12 were used
(as far as possible based on the HLS-EU study):
» Excellent: “very easy” ≥ 50 AND “very difficult” + “difficult” < 8.334

For “excellent”, the number of answers with “very easy” should be above 1/2 and the answers
for “very difficult” + “difficult” should be no more than 1/12.
» Sufficient: “very easy” + “easy” > 83.33
For a level of “sufficient” health literacy, at least 10 out of the 12 items should be answered
with “very easy” or “easy” and not more than 2 out of 12 with “very difficult” or “difficult”.
» Problematic: all respondents who are not in the groups “excellent”, “sufficient”, or “inade-
quate” (i.e., once the three other categories have been calculated)
The level of “problematic” is the intersecting set of not “excellent”, not “sufficient” and not
“inadequate”.
» Inadequate: “very easy” < 8.334, “very difficult” AND “difficult” ≥ 50
For “inadequate”, the number of answers with “very difficult” + “difficult” should be above 1/2
and for “very easy” should be no more than 1/12.
Of course, categories for the HLS19-47 or HLS19-Q16 can be calculated with similar threshold val-
ues.
3 Identical cut-offs are used for all data collection methods.
4.3 Weighting of data for analyses
Unless otherwise stated, post-stratification weights were applied to the univariate, bivariate, and
regression analyses described below. The base weights were calculated by the national teams and
differ depending on the survey procedure (cf. Table 2.18 to Table 2.21, Subsection 2.4.4).
For country-by-country analyses, the effective weights are scaled so that their sum equals the
number of valid cases in a country dataset. For analyses across all participating countries, the
weights are rescaled so that the sum of weights by country equals 1,000, i.e., the countries have
equal weights.
4.4 Univariate statistics
To present the results for individual items on the HL scales, two procedures are used. The per-
centages for the four response categories as well as for missing and total n for each item per
country are shown in the annexes to the chapters concerned. In the chapters themselves, for easier
comparisons across countries, the percentages for the dichotomized items are visualized as line
charts to display variations in the ranking of task difficulty between countries and are presented
in tables (e.g., Table 5.1).
Categorical data is described by the relative distribution of its levels (as percentages). Unless oth-
erwise stated, the percentages are calculated based on the number of valid values. The percentage
of missing values and the number of total respondents are reported separately.
The scores of HL measures were calculated based on dichotomized answer categories. To com-
pensate for the resulting loss of information, item sets of categorical variables are also described
by the average percentage response patterns (APRP), or the average percentage of how often a
response category was selected within an item set. Given a data matrix consisting of m categorical
items with k identical response categories for n respondents, an APRP is a compositional measure
consisting of k percentages that describe for each of the k response categories how often the n
respondents selected the respective category on average when answering the m categorical items.
The APRP can be calculated as the average of the percentages of how often each category was
selected for all items by each respondent. It can be calculated as follows:
» for each response category, count how often the category was selected for each respondent.
» for each item set, count how many valid answers were given by each respondent.
» for each response category, calculate how often a category was selected on average.
» for each response category, scale these mean values to the percentage of valid responses.
» for all response categories, summarize the mean percentages of valid responses in a table.
These APRP give convenient aggregate (or rather compositional) measures for a set of categorial
items with identical response categories by using the information from all response categories.
They allow researchers and readers to:
» get a quick overview of how often response categories for variables in an item set were se-
lected on average (and to identify potential problems with unused response categories and
resulting problems with data quality),
» easily compare the use of response categories across different (sub)populations, and
» easily compare use of response categories across different item sets.
Quantitative data are described by their mean, median, and standard deviation. If appropriate, the
quartiles are specified in addition. These tables are either placed in the chapter or in its annex
(e.g., Table A 5.33).
In most tables, a reference value (across all countries, equally weighted) is provided to allow for
an easier identification of trends across countries. Depending on the context, this value is called
“All” in Figures or “Mean” in Tables and is referred to using “on average” in descriptions.
In general, confidence intervals are not provided for point estimates or associations in this report
because this would greatly reduce the readability of tables and figures which include 17 countries.
Due to the relatively large sample sizes, the confidence intervals would be very small and not very
informative.
4.5 Bivariate associations
Associations between two variables, either of HL measures with a correlate variable or between
correlate variables, were described by correlation coefficients and partly displayed graphically in
line charts to determine the linearity of the association. For associations between a categorical
variable and a quantitative one, line charts were used to compare across groups. Line charts allow
readers to easily discern patterns and common trends across countries. This visualization method
was chosen for clarity in the report, particularly about displaying the results for many countries in
the same figure. Each country was represented by the same color and line pattern in all figures
throughout this report.
The strength of the association between an ordinal variable and a quantitative one or between two
ordinal variables was estimated using Spearman correlation coefficients (ρ). The association be-
tween quantitative variables was estimated using Pearson correlation coefficients (r). If the distri-
bution of a quantitative variable was heavily skewed, the non-parametric Spearman ρ was used
instead.
The association between a categorical and a quantitative variable was measured using the corre-
lation ratio (η). In the case of a linear relationship for a dichotomous variable (e.g., gender), η is
equivalent to the Pearson correlation coefficient (point biserial). Again, if the distribution of a
quantitative variable was heavily skewed, the non-parametric Spearman ρ was used instead.
In general, confidence intervals were not provided for point estimates or associations in this report
because this would greatly reduce the readability of tables and figures including up to 17 coun-
tries. Tests for statistical significance or p-values are not provided for the correlation tables but
due to the relatively large sample sizes, the relevant values of coefficients will most probably be
significant. Statistical significance (as p < 0.01) was highlighted in regression models for ß coef-
ficient tables (cf. Section 4.6).
Identification of vulnerable or disadvantaged groups
Based on a similar procedure applied in the HLS-EU study, a set of potentially vulnerable or dis-
advantaged groups was defined and investigated using selected categories from seven variables.
Four socio-demographic or socio-economic indicators and three health or sickness behavior re-
lated indicators were selected:
» advanced age (> 76 years old),

» low education (ISCED code ≤ 1 on a scale from 0 to 8),
» low level in society/low social status (≤ 4 on a scale from 1 to 10),
» financial deprivation (≥ 50%, i.e., considerable, or severe financial deprivation),
» “very bad” or “bad” self-perceived health,
» one or more long-term illnesses or health problems,
» a high number of contacts with a GP/family doctor (≥ 6) within the last 12 months.
In contrast to the HLS-EU, the deviation of the mean score per group with respect to the surveyed
population is summarized in a table including the following information:
» the mean and standard deviation of the health literacy score concerned and the number of
respondents, for each country, and the mean for all countries;
» for each defined vulnerable group, the difference in the mean value of the health literacy
score from the overall country mean, and the number of respondents, for each country and
the mean for all countries.
4.6 Regression analyses
Regression analyses were used to answer the following two research questions:
1. to determine a social gradient of HL measures and to measure the relative strength of the
effects of the independent variables included, and
2. to check if there is an independent or direct effect of HL measures on selected potential
consequences of HL when potential confounders are controlled for.
The main goal of the regression analyses was to be as simple as possible when answering these
research questions and to facilitate the interpretation of the results by readers. More detailed and
complex analyses will be the subject of supplementary scientific articles on selected topics.
Two types of linear models were applied for this report:
1. a multivariable linear regression model with all variables being entered as quantitative data
(reported in the relevant chapters) and
2. a multivariable linear regression model with categorical variables encoded as dummy varia-
bles (provided in the annex of the chapters concerned).
The main text in the chapters covers the results for the regression models (β coefficients, R², the
total, and the valid numbers of observations) with all variables being entered as quantitative data.
Standardized β coefficients are also given for binary variables like gender. The unstandardized
coefficients and the results for the regression models with categorical variables encoded as
dummy variables can be found in the annexes but not in the main text.
Regression coefficients with a p-value of 0.01 or smaller are highlighted in bold. Given the large
sample size of the surveys and the large number of statistical tests being calculated, a threshold
(significance level) of 0.01 was used instead of the traditional 0.05. It should be noted that a
proper adjustment for multiple testing (e.g., the Bonferroni correction) would yield an even lower
threshold. The p-values and the highlighting of coefficients with p-values of 0.01 or lower should
be seen as heuristics to identify potentially differing coefficients (Goodman et al. 2019;
Krueger/Heck 2019).
The regression models were computed using the survey package (Lumley 2011) for R (R Core Team
2020a) to account for the complex sample design. For the models with all variables being entered
as quantitative data, standardized and unstandardized coefficients were computed. Only unstand-
ardized coefficients were computed for regression models with categorical variables encoded as
dummy variables since the coefficients of dummy variables cannot be standardized. R² is provided
as a measure of the variance explained by the regression model.
When the GEN-HL score is used as an outcome variable, the score (ranging from 0 to 100) was
entered as a continuous variable. Following the HL research tradition and the HLS-EU, HL measures
are treated as a gradient and not as a threshold affecting health (e.g., Wolf et al. 2010). Since the
GEN-HL score distribution is skewed for most countries, the use of a transformed score was also
investigated. Since this did not improve model performance but complicated the interpretation of
the regression parameter estimates, it was decided not to use any transformation procedure.
The outcome variables are generally interpreted as continuous, and simple linear regression mod-
els are calculated. The dichotomization of outcome variables to calculate logistic regression mod-
els was only used in the chapter on Vaccination HL where a dichotomous variable had been col-
lected for vaccination behavior. It should be noted that certain formal assumptions of linear re-
gression models, such as normally distributed residuals, may not be met in all the models pre-
sented in this report. This could be especially true for the assumption of linearity for models when
formally ordinal variables were entered as interval-scaled outcome variables since it cannot be
guaranteed that the true distances between the response categories are perfectly equally spaced
on a theoretical underlying continuum. However, there are numerous studies, particularly for self-
perceived health in health reporting and health literacy research, in which exactly this approach is
used, so this study fits into a long-standing research practice with this approach. For self-per-
ceived health, for example, (Griebler 2017) states (authors’ translation): “[S]elf-rated health is
treated as a continuous variable (Perruccio et al. 2010). It functions as a proxy for the underlying
measurement continuum (Jürges 2007; Leinonen et al. 2002) - a quite common method for ordinal
scaled variables with five or more expressions (Johnson/Creech 1983; Zumbo/Zimmermann
1993)”. In addition, respondents are often quite capable of reproducing their true attitudes given
on a continuous rating scale in more coarse 5- or 7-point Likert-style response formats
(Carifio/Perla 2007). While the uniformity of the analyses and, thus, the use of linear regression
models was important for the present International Report, for more detailed analyses and national
reports, the use of regression models better suited for ordinal outcome variables could be con-
sidered. The formal assumption of homoscedasticity is most likely not met when applying a simple
linear regression model to count data as outcome variables. This results in the models have a
suboptimal model fit (measured as R²), which is not a major problem since in research question
2, the only interest is in testing an effect of the HL measure(s). The suboptimal fit should only
have low impact on the relative importance of the predictor variables, though, which was verified
for some models. In the national reports and in more detailed analyses in follow-up scientific
publications, the use of regression models better suited for count data in outcome variables
should be considered.
Based on the existing literature, a core set of the following five socio-demographic and socio-
economic predictors and, in some models, three additional predictors were tested for their asso-
ciation with health literacy scores and with various other outcome variables, such as indicators of
healthy lifestyles, personal health, and the use of health care services. These outcome variables,
which are also referred to as the potential “consequences” of HL, are introduced and described in
detail in Chapters 7, 8, and 9 respectively as well as in Chapter 12 (Digital Health Literacy) and 13
(Vaccination Health Literacy).
Core socio-demographic and socio-economic predictors (in short, the core social determinants of
HL) are:
» gender,
» age,
» education,
» self-perceived level in society,
» financial deprivation/difficulties.
Additional predictors:
» migration background,
» long-term illnesses/health problems,
» training in a healthcare profession.
In the present report, all variables are entered in the models simultaneously. More detailed anal-
yses, possibly including path models and more elaborate regression models, will be covered in
dedicated publications on selected topics.
An important requirement for selecting a variable as a predictor is its fair measurement and in-
terpretability across all countries. This requirement cannot be assumed in the case of migration
due to the wide range of migration situations, policies, and activities in individual countries. In
addition, surveys in the languages of larger migrant groups only took place in Israel as well as in
specific additional studies in Germany and Norway. The HLS19 surveys also differ significantly con-
cerning the extent and representativeness (e.g., in education) of the respondents with migration
backgrounds. For this reason, special attention should be paid when interpreting the migration
coefficients from an international perspective.
In the surveys, age was recorded as years since birth, which was recoded to an ordinal variable
with seven levels. However, in the models, with all variables being entered as quantitative data,
age in years was used. The idea of entering age in years as an additional quadratic term was also
investigated, but this option was abandoned due to inconsistent and difficult-to-interpret results.
In the regression models with categorical variables encoded as dummy variables, age was, there-
for, entered as an ordinal variable, allowing a better handling of non-linear relationships with the
outcome variable. To better capture the various non-linear relationships formed between age and
health literacy, the use of these latter models is preferred for investigating this specific association
in detail.
Education was surveyed using the ISCED 2011 scale (Schneider 2013; UNECSCO 2012). The actual
implementation varied from country to country to accommodate different national education sys-
tems. The ISCED classification was used because it provides a roughly comparable classification
across the whole range of possible education levels, including tertiary education, for almost all
countries worldwide. The ISCED classification is often used in similar research and thus facilitates
the comparison of results. For better international comparability, the ISCED codes were recoded
to four levels:
3. lower secondary education or below (up to ISCED-2),
4. higher secondary education (ISCED-3),
5. post-secondary or short-cycle tertiary education (ISCED-4 and 5),
6. bachelor or higher (ISCED-6 to 8).
This four-level variable was used for tables, figures, and the regression models with categorical
variables encoded as dummy variables (see above). For the regression models and the calculation
of correlation coefficients presented in the main text, the raw ISCED 2011 code was entered as a
continuous variable. Due to differences in the sampling and data collection procedures, it may be
assumed that the coverage of people with lower levels of education differs across the various HLS19
surveys. This could possibly explain inconsistent results in the regression analyses with respect
to education across countries.
For the variable self-perceived level in society, the extreme categories (1+2 and 9+10) were com-
bined in some analyses to overcome potential low frequency effects in these categories. For the
regression analyses presented in the main text, this variable was entered as a continuous predic-
tor.
For the item set financial deprivation, a summary score of the following three items with the re-
sponse categories “very easy”, “easy”, “difficult”, and “very difficult” was calculated:
» C-DET9 How easy or difficult is it usually for you to afford medication, if needed?
» C-DET10 How easy or difficult is it usually for you to afford medical examinations and treat-
ments, if needed?
» C-DET11 How easy or difficult is it for you to pay all bills at the end of the month?
In line with the described calculation method of the GEN-HL score, the financial deprivation scores
were calculated as percentages (ranging from 0 to 100) for items with valid responses that were
answered with “very difficult” or “difficult”. The values of the financial deprivation score were as-
signed the following labels:
» 0=none,
» 33.33=some,
» 66.67=considerable,
» 100=severe.
This method simplifies the more elaborate method used in the HLS-EU study based on a principal
component analysis for each country. Since these two methods produce very highly correlated
scores, it was assumed that this extra step does not provide sufficient benefit to justify this addi-
tional complexity.
Migration is only included as a predictor in selected models such as determinants of health literacy
(Chapter 6), health care utilization as a consequence of General Health Literacy (Chapter 9), and
Digital Health Literacy (Chapter 11). A 4-point ordinal variable was used for a migration back-
ground with the following levels: “none”, “one parent was born abroad”, “both parents were born
abroad”, “born abroad”.
Long-term illness is only included as a predictor in selected models such as determinants of health
literacy (Chapter 6), health care utilization as a consequence of General Health Literacy (Chapter
9), and Digital Health Literacy (Chapter 11). The number of long-term illnesses or health problems
was entered as an ordinal variable with the levels “none”, “one”, or “more than one”; for one country
(Slovenia), “one or more” was used instead of the last two categories.
Training in a health profession is, in general, not considered a social determinant of health literacy.
Since it is potentially an important confounder, some regression models investigated its influence
on health literacy (in Chapter 6 and 10 to 14). Training in a health profession was entered as a
dichotomous yes/no variable into the regression models.
While some of the mentioned variables are weakly or moderately correlated, as shown in the rel-
evant chapters, the absolute Spearman ρ is generally below 0.4, so collinearity should be a minor
problem.
Stepwise regressions are not recommended since they lead to biased model fit estimates or p-
values (Heinze/Dunkler 2017). Even when applying stepwise regression, only full-model fits give
unbiased estimates. While a feature selection procedure can be helpful for data mining or machine
learning tasks, it is of limited use in the context of social science when each predictor is derived
from theoretical considerations or the established results of existing research. Another argument
against using stepwise regression in this International Report is that it could result in different
models for different countries, while we aimed at consistent models for all participating countries.
4.7 Validity analyses
In general, we follow the argumentation of (Osborne et al. 2021) that an instrument cannot be
validated as such but only for a specific survey sample. This is all the more true, when the instru-
ment is used in different languages. Therefore, we validated each survey or country separately.
Further uses of the instruments in follow-up studies will require their own validation analyses.
The construct validity of the instrument is ensured by selecting the best-fitting item for each cell
in the theoretically founded health literacy matrix and by applying appropriate statistical analyses.
In detail, the HLS19-Q12 (and, if appropriate, also the specific health literacy measures) was vali-
dated by Cronbach’s alpha for reliability, confirmatory factor analysis (CFA), by Rasch analyses for
unidimensionality, by correlations for the representativeness of the HLS19-Q12 of the long form
HLS19-Q47. With respect to concurrent convergent validity, the results of the chapters on the ex-
pected consequences of HL (Chapters 7 to 9) will provide additional evidence for the measure of
General HL and the individual chapters for specific aspects of HL.
The various surveys in the HLS19 study vary in methodology and in the language(s) used. Testing
the effects of the survey methodology is beyond the scope of the present study since an analysis
of such effects would require a specific research design. Item-score correlations, another popular
method to assess the psychometric quality of a measurement instrument, are provided for General
HL in Annex 5 (Chapter A 5.3).
Cronbach’s alphas
The internal consistency of a set of items was assessed using the Cronbach alpha coefficient
(Cronbach 1951). In the literature, a minimum value of 0.7 is recommended (Kline 2015). Lower
values may be acceptable since the coefficient is sensitive to the number of items included and
other circumstantial parameters. When calculating alpha coefficients using dichotomized varia-
bles, the value is expected to be lower than when using polytomous variables.
The alpha coefficients in Chapters 5 and 10 to 13 are calculated based on the dichotomized items
because the scores are also derived from dichotomized items. An alpha coefficient based on tet-
rachoric correlations would yield higher values. Given the heavily skewed distributions, it was ex-
pected that the tetrachoric correlations would overestimate the strength of the associations. An
ordinal alpha coefficient based on the polytomous items would yield higher values too. The
Cronbach alpha coefficients, based on the Pearson correlation of the dichotomized items, thus
represent a lower bound of possible alpha coefficients.
Confirmatory factor analyses
One way to assess the single-factor structure of an item set based on Classical Test Theory is
confirmatory factor analysis [CFA]. The model consists of the dichotomized items loading onto a
single latent variable. The CFA is conducted using the lavaan package (Roussel 2012) for R (R Core
Team 2020a). A WLSMV estimator with diagonally weighted least squares (DWLS) for model pa-
rameters is used (Beaujean 2014; Kline 2015; Rosseel 2021). The focus lies on whether the fit
indices suggest a sufficiently good model fit that indicates a single data-generating process gen-
erated the data. Since various fit indices have different advantages and disadvantages (Prudon
2015), a set of the six most commonly used fit indices is calculated.
The following target values are assumed as indications of a good fit:
» Standardized Root Mean Square Residual [SRMSR] ≤ 0.08 (Prudon 2015)

This directly compares the sample correlation matrix with the implied correlation matrix.
» Root Mean Square Error of Approximation [RMSEA] ≤ 0.06 (Prudon 2015)
The RMSEA builds on the Χ² test but penalizes free parameters.
» Tucker-Lewis Index [TLI] ≥ 0.95 (Prudon 2015)
This compares the Χ² of the implied matrix to that of the null model.
» Comparative Fit Index [CFI] ≥ 0.95 (Prudon 2015)
This compares the Χ² of the implied matrix to that of the null model but penalizes free pa-
rameters.
» Goodness of Fit Index [GFI] ≥ 0.95 (Beaujean 2014)
This compares the model of interest to a null model and can be interpreted similarly to R² in
regressions.
» Adjusted Goodness of Fit Index [AGFI] ≥ 0.9 (Beaujean 2014)
Like the GFI but it penalizes free parameters.
Rasch analyses
The psychometric properties of the HLS19-Q12 and the newly developed instruments applied in
the chapters about specific types of health literacy (cf. Chapters 10 to 14) were tested against the
partial credit parameterization of the polytomous unidimensional Rasch model (Masters 1982;
Rasch 1960). By testing data against Rasch models, we assess whether observed data sufficiently
meet the expectations of the theoretical Rasch model. The validity of instruments that intend to
measure latent traits or characteristics, such as health literacy, is increased by including items that
measure various aspects of the latent trait (Andrich/Marais 2019). However, to add scores from
single items into a total score for a set of items or a scale, the data must be sufficiently unidimen-
sional (Smith, 2002). The dimensionality of instruments applied in this report was assessed using
the combined principal component analysis (PCA) of residuals and the paired t-test procedure
(Smith 2002). We also assessed whether the content of pairs of items are too similar and collect
the same or overlapping information. We refer to this as response dependence or statistically
dependent items. Residual correlations between two items > 0.3 were used as possible indicators
of response dependence. If a set of items is sufficiently unidimensional and no questions are
statistically dependent, we can say that the instrument meets the assumption of local independ-
ence (Andrich/Marais 2019).
When testing data against Rasch models, we also assessed whether observed data sufficiently
meet the expectations of the theoretical Rasch model. If an item captures too much of something
other than what the item was intended to measure - that something other than health literacy
affecting how the respondents answer - we refer to this as an under-discriminating item
(Andrich/Marais 2019). By that we mean that the item does not manage to distinguish or discrim-
inate sufficiently well between people with low and high health literacy. Item fit was assessed by
examining χ² statistics and standardized residuals based on comparisons between observed and
expected values. Items with low χ² values and χ² probability values higher than a Bonferroni-
adjusted 5% were considered to have adequate fit. In addition, we used infit statistics to assess
item fit. Infit < 1 indicates an over-discriminating or over-fitting item, and infit > 1 indicates an
under-discriminating or under-fitting item. Infit values between 0.7-1.3 were considered to rep-
resent sufficient data-model fit.
We also tested whether the items worked in the same way for different groups of respondents,
investigating, for example, whether gender significantly affected the respondents’ answers to in-
dividual items. If factors such as age, gender, and education significantly influence the answers to
a certain item, we refer to this as “DIF” (differential item functioning). The items were examined
for DIF using the two-way analysis of variance (ANOVA) (Andrich/Marais 2019). In addition, we
assessed whether the response categories on the 4-point rating scales worked as intended, in
other words whether they are “ordered”.
Table 4.1:
Levels of person factors used for the analysis of differential item functioning (DIF)
Person factor Levels (categories)

1 2 3
Gender (CDET1) male female
Age (CDET2)
Dichotomized 18 to 45 years 46 years or older
agecat1 18 to 25 years 26 to 65 years 66 years or older
agecat2 18 to 45 years 46 to 75 years 76 years or older
Education (CDET6) ISCED 0–3 ISCED 4–8
Employment (CDET7) employed unemployed or retired
Paying bills (CDET11) easy difficult
Social level (CDET12) levels 1–4 levels 5–10
General health (CHSTAT1) good or fair bad
Predictive validity
For the predictive validity of the general and specific HL measures, see the results in Chapters 7
to 9 on the potential consequences of the General HL measure and in Chapters 11 to 15 on op-
tional packages for the specific HL measures.
Comparisons of the HLS19-Q12 short form with the

HLS19-Q16 and HLS19-Q47 long forms
Since the HLS19-Q12 is a short form of the HLS19-Q47 long form, another important criterion for
its validation is how well the HLS19-Q12, which uses just one item for each cell in the underlying
theoretical HL matrix, represents the scores of the HLS19-Q47. One criterion for this is the Pearson
correlation of the HLS19-Q12 short form with the HLS19-Q47 long form. Since only six countries
collected data for the HLS19-Q47, this kind of empirical equivalence could only be tested for that
subset of countries taking part in the HLS19. A high Pearson correlation coefficient is an indication
that the score of the Q12 short form can be used as a more economical substitute for the score
of the Q47 long form.
4.8 Discussion and conclusions
We tried to standardize data analyses and presentations of the results for all chapters to allow
comparisons between the different chapters and the participating countries. Due to the differences
in data collection, comparisons between individual countries are, nevertheless, only possible to a
limited degree and should be made with caution.
With respect to the HLS-EU predecessor study, several computational changes were adopted that,
in addition to the differences in the questionnaire (cf. Chapter 2) and data collection (cf. Chapter
3), limit the comparability of its results with the HLS19. Consequently, comparisons with results
from the HLS-EU study were only included in the present report in the “Discussion” sections.
Most importantly, the response categories of the HL items were interpreted as ordinal scales and
not interval scales, as was done in the HLS-EU and follow-up studies. This led to changes in the
calculation of the score as the percentage of items being answered as “easy” or “very easy” (ranging
from 0 to 100 instead of 0 to 50 as in the HLS-EU study). Hence the value of the HL score can
easily be interpreted as the percentage of health-relevant tasks experienced as “easy” or “very
easy”. To keep the definition of the four discrete levels of General HL based on the HLS19-Q12
measure comparable with the HLS-EU, the calculation of the four levels of HL takes not only the
score into account but also the proportions of responses in the extreme categories. Differences
between the score described above and the scores following the HLS-EU method were examined
but revealed only minor differences for key results.
Besides Cronbach’s alpha for internal consistency, CFA and Rasch analyses were administered for
the General HL (HLS19-Q12) and specific HL measures but not for the HLS19-Q47 and HLS19-Q16.
These analyses demonstrated that the General HL scale in the HLS19-Q12 is sufficiently unidimen-
sional.
The predictive validity of the HL measures was investigated partly with the same indicators of
correlates as in the HLS-EU, partly with somewhat modified indicators, and partly with indicators
of additional correlates.
Some of the indicators for relevant correlates of HL were measured or constructed differently than
in the HLS-EU, and indicators for additional correlates were included, which further limits compa-
rability between the two studies but allowed for additional analyses.
For potential determinants and a social gradient of the HL measures, the same variables were used
as in the HLS-EU with a few additional determinants. For bi-variate relations alongside Spearman
correlations, figures showing the form of the relationship are provided. For testing direct inde-
pendent effects, multivariable linear models were analyzed. In addition, multivariable linear mod-
els with the categorical variables encoded as dummy variables are provided in the annex. The
same holds true for testing the potential consequences of HL measures.
With respect to the HLS-EU study, we conclude that the HLS19 instrument, the data analyses, and
the presentation of results were improved, but these improvements limit the comparability with
the HLS-EU results.
4.9 References
Andrich, David; Marais, Ida (2019): A course in Rasch measurement theory: Measuring in the ed-
ucational, social and health sciences. Springer
Beaujean, A Alexander (2014): Latent variable modeling using R: A step-by-step guide. Routledge,
Carifio, James; Perla, Rocco J. (2007): Ten Common Misunderstandings, Misconceptions, Persistent
Myths and Urban Legends about Likert Scales and Likert Response Formats and their Anti-
dotes. In: Jounal of Social Science 3/3:106-116
Cronbach, Lee J. (1951): Coefficient alpha and the internal structure of tests. In: Psychometrika
16/3:297-334
Goodman, William M.; Spruill, Susan E.; Komaroff, Eugene (2019): A Proposed Hybrid Effect Size
Plus p-Value Criterion: Empirical Evidence Supporting its Use. In: The American Statistician
73/sup1:168-185
Griebler, Robert (2017): „Schule und Gesundheit“ Eine Studie zu den schulischen Determinanten
der Lehrergesundheit. Universität Wien, Wien
Heinze, Georg; Dunkler, Daniela (2017): Five myths about variable selection. In: Transpl Int
30/1:6-10
Johnson, David Richard; Creech, James C (1983): Ordinal measures in multiple indicator models:
A simulation study of categorization error. In: American Sociological Review 48/3:398-407
Jürges, Hendrik (2007): True health vs response styles: exploring cross‐country differences in self‐
reported health. In: Health Economics 16/2:163-178
Kline, Rex B (2015): Principles and practice of structural equation modeling. Guilford publications,
New York
Krueger, Joachim I.; Heck, Patrick R. (2019): Putting the P-Value in its Place. In: The American
Statistician 73/sup1:122-128
Leinonen, Raija; Heikkinen, Eino; Jylhä, Marja (2002): Changes in health, functional performance
and activity predict changes in self-rated health: a 10-year follow-up study in older people.
In: Archives of gerontology and geriatrics 35/1:79-92
Lumley, Thomas (2011): Complex surveys: a guide to analysis using R. Bd. 565. John Wiley & Sons
Masters, Geoff (1982): A Rasch model for partial credit scoring. In: Psychometrika 47/2:149-174
Osborne, Richard; Elsworth, Gerald; Hawkins, Melanie; Cheng, Christina; Elmer, Shandell (2021):
Measurement of health literacy: assumptions and potential consequences locally and globally.
Perruccio, Anthony V; Badley, Elizabeth M; Hogg-Johnson, Sheilah; Davis, Aileen M (2010): Char-
acterizing self-rated health during a period of changing health status. In: Social Science &
Medicine 71/9:1636-1643
Prudon, Peter (2015): Confirmatory factor analysis: a brief introduction and critique. In: Compre-
hensive Psychology 4/10:1-18
R Core Team (2020): R: A language and environment for statistical computing. R Foundation for
Statistical Computing, Vienna, Austria
Rasch, Georg (1960): Probabilistic Models for Some Intelligence and Attainment Tests. Expanded
Aufl., University of Chicago Press, Chicago
Rosseel, Yves (2021): The lavaan tutorial. Analysis, Department of Data, Belgium
Roussel, Yves (2012): lavaan: An R Package for Structural Equation Modeling. In: Journal of Statis-
tical Software 48/2:1-36.https://www.jstatsoft.org/v48/i02/
Schneider, Silke L. (2013): The International Standard Classification of Education 2011. In: Class
and Stratification Analysis.S. 365-379
Smith, Everett V (2002): Detecting and evaluating the impact of multidimenstionality using item
fit statistics and principal component analysis of residuals. In: Journal of applied measure-
ment: 3(2):205-31
UNECSCO (2012): International Standard Classification of Education. UNESCO Institute for Statis-
tics, Montreal, Quebec
Wolf, M. S.; Feinglass, J.; Thompson, J.; Baker, D. W. (2010): In search of 'low health literacy':
threshold vs. gradient effect of literacy on health status and mortality. In: Soc Sci Med
70/9:1335-1341
Zumbo, B. D.; Zimmermann, D. W (1993): Is the selection of statistical methods governed by level
of measurement? In: Canadian Psychology 34/:390-400
5 The HLS19-Q12 measure
Authors:
Thomas Link (ICC)
Tobias Alfers (ICC)
Nejc Berzelak (SI)
Oxana Drapkina (RU)
Robert Griebler (AT)
Christopher Le (NO)
Maria Lopatina (RU)
Marie Germund Nielsen (DK)
Jorge Oliveira (PT)
Sandra Peer (ICC)
Sanja Vrbovšek (SI)
Mitja Vrdelja (SI)
Karin Waldherr (ICC)
5.1 Distributions of the HLS19-Q12 items
The distributions of the responses to individual items are summarized in two ways: (1) by per-
centages of those who responded with “very difficult” or “difficult” per item (cf. Table 5.1, Fig-
ure 5.1) and (2) by Average Percentage Response Patterns (APRP) for each country (cf. Figure 5.2).
The frequency tables for each item by country are presented in Annex 5.1. The percentages of
respondents who responded with “very difficult” or “difficult” to the HLS19-Q16 and HLS19-Q47
measures can be found in Annex 5.2.
The overall percentage of respondents ticking “very difficult” or “difficult” varies between 8.1% and
43.0% for the HLS19-Q12 items (cf. Table 5.1) with item 4 “to act on advice from your doctor or
pharmacist” being the easiest item and item 3 “to judge the advantages and disadvantages of
different treatment options” being the most difficult. In general, the items in the HLS19-Q12 were
not rated as predominantly “very difficult” or “difficult”, with the sole exception of Germany, where
the tasks referred to by items 3, 8, and 5 were reported as “very difficult” or “difficult” by 56.1% to
71.2% of the respondents.
The item difficulties vary by country. The combined percentage of “very difficult” and “difficult”
responses ranges from 25.6% (SI) to 71.2% (DE) for the most challenging item 3 “to judge the
advantages and disadvantages of different treatment options”, and from 3.4% (PT) to 17.2% (CZ
and SK) for the least difficult item 4 “to act on advice from your doctor or pharmacist”. Neverthe-
less, there is a more or less common ranking by difficulty of the tasks across countries (cf. Table
5.1 and Figure 5.1).
Chapter 5 /The HLS19-Q12 measure 89

Table 5.1:
Percentages of respondents in each country who responded with “very difficult” or “difficult” to the HLS19-Q12 items (ordered by the mean for the row), for each
country and the mean for all countries (equally weighted)
AT BE BG CH CZ DE DK FR HU IE IL IT NO PT RU SI SK Mean
3 … to judge the advantages and disadvantages 36.9 49.9 46.7 46.3 46.9 71.2 45.5 39.9 32.0 35.6 45.1 46,5 43.2 33.5 38.3 25.6 58.4 43.0
of different treatment options?
8 … to decide how you can protect yourself from 25.9 51.9 51.0 44.9 31.8 61.3 39.7 40.5 49.3 38.3 43.5 47.0 43.3 26.8 27.6 34.2 43.5 39.7
illness using information from the mass media?
5 … to find information on how to handle mental 30.5 48.6 50.2 40.8 31.4 56.1 42.1 37.9 32.5 36.4 35.5 43,2 37.8 29.9 27.2 19.1 44.1 36.1
health problems?
12 … to make decisions to improve your health 11.9 37.1 39.3 26.0 30.7 30.7 32.5 27.3 26.4 14.5 34.8 32.0 22.1 13.4 26.6 16.9 42.1 25.6
and well-being?
2 … to understand information about what to do 11.5 35.0 30.7 17.9 20.6 37.2 14.2 21.4 18.8 24.7 30.8 34,4 20.9 11.3 26.7 10.0 27.9 22.7
in a medical emergency?
11 … to judge how your housing conditions may 9.5 42.1 21.5 22.9 23.9 49.5 26.6 25.4 13.0 18.3 31.6 32.0 16.2 8.7 17.2 9.6 26.5 22.2
affect your health and well-being?
7 … to judge if information on unhealthy habits. 11.8 28.9 31.6 11.9 16.5 35.8 11.6 16.9 11.1 15.9 24.7 23,2 16.6 10.5 18.1 10.2 26.4 17.8
such as smoking. low physical activity, or drink-
ing too much alcohol. is reliable?
6 … to understand information about recom- 10.2 32.7 40.8 21.5 20.5 21.0 15.6 16.4 9.7 19.2 24.8 30,6 12.0 17.1 9.3 11.8 23.7 17.2
mended health screenings or examinations?
10 … to understand advice concerning your 14.8 26.9 17.0 18.2 18.6 20.0 18.5 16.1 11.3 16.2 19.7 24,1 18.7 6.6 12.8 9.2 16.1 16.5
health from family or friends?
1 … to find out where to get professional help 7.3 23.6 25.3 7.4 17.9 20.7 12.8 15.4 23.8 20.4 18.3 31.0 9.7 13.8 12.7 9.7 24.8 16.2
when you are ill?
9 … to find information on healthy lifestyles such 6.1 20.7 16.9 10.2 9.6 8.3 7.2 9.8 9.7 10.0 7.8 16,4 11.0 7.1 9.7 7.0 13.1 9.9
as physical exercise. healthy food, or nutrition?
4 … to act on advice from your doctor or phar- 7.3 13.7 9.1 5.9 17.2 9.0 5.4 3.6 6.5 5.9 7.9 11,7 5.6 3.4 9.1 4.8 17.2 8.1
macist?
Mean 15.3 34.3 31.7 22.8 23.8 35.1 22.6 22.6 20.3 21.3 27.0 31 21.4 15.2 19.6 14.0 30.3 22.9
Figure 5.1:
Percentages of respondents who responded with “very difficult” or “difficult” to the HLS19-Q12
items (ordered by the overall mean), for each country
5.2 Average Percentage Response Patterns (APRP)
Average Percentage Response Patterns (cf. Section 4.4) are calculated to represent a summary of
a set of categorical variables (cf. Figure 5.2). In almost all countries, people most often responded
to the HLS19-Q12 items with “easy”. Overall, 55% of the respondents answered with “easy” (with
the country average ranging from 35% (IE) to 77% (PT)) and 21% with “very easy” (from 8% (PT) to
44% (IE)). Another 21% answered with “difficult” (from 12% (SI) to 29% (BE and DE)) and only 3%
with “very difficult” (2% for AT, CH, HU, NO, PT, RU, SI to 6% for DE) (Figure 5.2). Thus, the items
included in the HLS19-Q12 questionnaire are, on average, rather easy and therefore the measure
is more sensitive for lower grades of HL than for higher ones.

Figure 5.2:
Average Percentage Response Patterns (APRP) for the HLS19-Q12 item set, for each country and
the mean for all countries (equally weighted)
5.3 Internal consistency of the HLS19-Q12
The internal consistency or “test reliability” of the HLS19-Q12, which is also a measure of the scale’s
ability to distinguish between respondents with different sum scores, was estimated using
Cronbach’s alpha (cf. Subsection 4.7.1). In the literature, a minimum value of 0.7 is recommended
(Kline 2015).
The values range from 0.67 (AT) to 0.87 (PT). Except for Austria, the values are above the target
value of 0.7 (in Table 5.2). The internal consistency, thus, is acceptable for the given data of most
countries. For each country, the inter-item (Annex Tables A 5.15 to A 5.31) and the item-score
Spearman correlation coefficients (Table A 5.32) are available in Annexes 5.3 and 5.4.
Table 5.2:
Cronbach’s alpha for the HLS19-Q12, for each country and the mean for all countries (equally weighted)
Cronbach’s
0.67 0.82 0.78 0.72 0.78 0.73 0.75 0.81 0.76 0.72 0.80 0.85 0.73 0.87 0.86 0.82 0.81 0.78
alpha

5.4 Results of Confirmative Factor Analyses for the HLS19-
Q12
A confirmatory factor model with the twelve dichotomized HLS19-Q12 items loading onto a single
factor was estimated (see Subsection 4.8.2). Since various fit indices have different advantages
and disadvantages (Prudon 2015), the six most commonly used goodness-of-fit measures were
calculated.
The following target values are assumed as indications of a good model fit:
» Standardized Root Mean Square Residual [SRMSR] ≤ 0.08 (Prudon 2015)

» Root Mean Square Error of Approximation [RMSEA] ≤ 0.06 (Prudon 2015)
» Comparative Fit Index [CFI] ≥ 0.95 (Prudon 2015)
» Tucker-Lewis Index [TLI] ≥ 0.95 (Prudon 2015)
» Goodness of Fit Index [GFI] ≥ 0.95 (Beaujean 2014)
» Adjusted Goodness of Fit Index [AGFI] ≥ 0.9 (Beaujean 2014)
The fit indices indicate a good model fit, which means that the single factor confirmatory model
accounts sufficiently well for the correlation patterns among the HLS19-Q12 items (cf. Table 5.3).
The items for which the standardized parameter estimates (cf. Table 5-4) differ the most across
countries are (range ≥ 0.4):
» item 4 “to act on advice from your doctor or pharmacist”

» item 9 “to find information on healthy lifestyles such as physical exercise, healthy food, or
nutrition”
» item 10 “to understand advice concerning your health from family or friends”
As these are also among the easiest items (cf. Figure 5.1), this could cause these discrepancies in
the loadings.
Table 5.3:
Fit indices for the CFA for the HLS19-Q12, for each country and the mean for all countries (equally weighted)
SRMSR 0.07 0.08 0.07 0.07 0.05 0.07 0.06 0.05 0.07 0.06 0.06 0.05 0.07 0.05 0.05 0.04 0.06 0.06
RMSEA 0.03 0.05 0.04 0.03 0.03 0.04 0.03 0.02 0.03 0.03 0.03 0.04 0.04 0.02 0.04 0.02 0.04 0.03
CFI 0.97 0.98 0.99 0.98 0.99 0.97 0.98 1.00 0.98 0.97 0.99 0.99 0.97 1.00 0.99 1.00 0.99 0.98
TLI 0.96 0.97 0.98 0.97 0.99 0.96 0.98 0.99 0.98 0.96 0.99 0.99 0.96 1.00 0.99 1.00 0.98 0.98
GFI 0.99 0.98 0.98 0.99 0.99 0.98 0.99 0.99 0.99 0.99 0.99 0.99 0.99 1.00 0.99 1.00 0.99 0.99
AGFI 0.99 0.97 0.97 0.98 0.98 0.97 0.99 0.99 0.98 0.98 0.99 0.99 0.98 0.99 0.99 1.00 0.98 0.98
AGFI=Adjusted Goodness of Fit Index; CFI=Comparative Fit Index; GFI=Goodness of Fit Index; RMSEA=Root Mean Square Error of Approximation; SRMR=Standardized Root Mean Square Residual;
TLI=Tucker-Lewis Index

Table 5.4:
Standardized parameter estimates for the HLS19-Q12 score in a single-factor CFA, for each country and the mean for all countries (equally weighted)
Item no. AT BE BG CH CZ DE DK FR HU IE IL IT NO PT RU SI SK Mean

1 0.58 0.72 0.63 0.51 0.70 0.56 0.64 0.55 0.58 0.57 0.61 0.69 0.59 0.79 0.73 0.70 0.72 0.64
2 0.54 0.63 0.64 0.64 0.68 0.60 0.64 0.66 0.69 0.55 0.67 0.79 0.58 0.73 0.78 0.78 0.73 0.67
3 0.61 0.67 0.74 0.63 0.67 0.67 0.72 0.71 0.65 0.55 0.71 0.73 0.59 0.70 0.81 0.68 0.69 0.68
4 0.44 0.76 0.74 0.52 0.57 0.53 0.59 0.55 0.65 0.62 0.58 0.67 0.70 0.85 0.74 0.79 0.59 0.64
5 0.63 0.65 0.72 0.72 0.64 0.65 0.69 0.78 0.56 0.59 0.72 0.71 0.61 0.77 0.78 0.80 0.73 0.69
6 0.58 0.70 0.35 0.59 0.67 0.61 0.66 0.71 0.78 0.70 0.74 0.78 0.68 0.83 0.78 0.72 0.68 0.68
7 0.53 0.73 0.75 0.61 0.64 0.65 0.63 0.75 0.67 0.66 0.69 0.77 0.65 0.86 0.81 0.82 0.66 0.70
8 0.63 0.59 0.65 0.53 0.69 0.63 0.64 0.77 0.52 0.46 0.75 0.71 0.50 0.82 0.79 0.58 0.65 0.64
9 0.62 0.78 0.77 0.66 0.62 0.72 0.68 0.70 0.73 0.64 0.59 0.73 0.60 0.99 0.82 0.85 0.80 0.72
10 0.46 0.62 0.65 0.43 0.55 0.37 0.50 0.58 0.62 0.59 0.60 0.67 0.60 0.81 0.68 0.66 0.60 0.59
11 0.59 0.67 0.61 0.62 0.70 0.54 0.63 0.76 0.73 0.56 0.62 0.74 0.53 0.87 0.76 0.77 0.63 0.67
12 0.54 0.69 0.57 0.65 0.58 0.53 0.56 0.73 0.59 0.56 0.67 0.71 0.54 0.69 0.73 0.64 0.65 0.63
5.5 Rasch analyses
When testing data against the Partial Credit Model (PCM) (Masters 1982) for each country, the
HLS19-Q12 displays good overall data-model fit in Austria (CATI), Denmark, Germany, Israel
(CAWI), Italy (CAWI), Norway, Slovakia, and Switzerland (Table 5.5). Reducing the sample size to
n=360 (12 items × 3 thresholds × 10 persons per threshold) means that the HLS19-Q12 displays
acceptable overall data-model fit also in France, Hungary, Russia, and Slovenia. The HLS19-Q12
has an acceptable reliability index in each country (Table 5.5).
The HLS19-Q12 was somewhat “off target” as the items refer to tasks most people perceive as
manageable. The HLS19-Q12 was best targeted to the Belgian and German populations, as the
distributions of item difficulties matched the distribution of person proficiencies quite well.
The HLS19-Q12 items measure three health domains (health care, health promotion, and disease
prevention) and four cognitive domains (find, appraise, understand, and apply). These different
domains or aspects capture the complexity of the construct and increase the content validity of
the HLS19-Q12 scale, but they inevitably bring multidimensionality into the measure. Using a prin-
cipal component analysis of Rasch model residuals, two possible subscales or item subsets of the
HLS19-Q12 were identified empirically. Each respondent’s proficiency estimates, based on those
two subsets of items, were compared and the difference was tested using a dependent t-test.
Table 5.5 shows that the percentage of significant dependent t-tests are close to or below 10%
for each country (the column “Dim (%)”), varying between approximately 11% (BG, CAWI) and 5.5%
(NO). This means that relatively low proportions of respondents were assigned statistically signif-
icant different proficiency estimates based on the two empirically identified item subsets. The two
subsets therefore seem to measure “the same”, and we may conclude that the HLS19-Q12 is suf-
ficiently unidimensional and measuring one latent trait referred to as HL.
No evidence of response dependency or “too similar” items was observed, meaning that no pair of
items shared variance over and beyond the latent trait “health literacy”.
Most HLS19-Q12 items displayed acceptable data-model fit. However, item 8 “to decide how you
can protect yourself from illness using information from the mass media” discriminated somewhat
poorly between respondents with high and low HL (z-fit > 3.0 and/or infit > 1.2) in several coun-
tries (AT, BE, CH, HU, IE, NO, and SI). The response categories of the HLS19-Q12 items were ordered
and worked well, but item 4 “to act on advice from your doctor or pharmacist” displayed possibly
unordered thresholds in the BE, IE, and NO data (but was not significant in the NO data).
Several items displayed differential item functioning (DIF) even when sample size was reduced to
1,080, where the amended sample size of 1,080 is estimated as, using a “rule of thumb”, the
product of 30 respondents per 3 item thresholds for 12 items (see Table A2 in the Technical report
in (Guttersrud et al. 2021). For some items, DIF was still evident when reducing the sample size
to 720, where the amended sample size of 720 is estimated as the product of 20 respondents per

3 item thresholds for 12 items. One example is item 6 “to understand information about recom-
mended health screenings or examinations”, which displayed DIF for employment in some coun-
tries (BE, FR, and SI) as well as for respondent age in several countries (BE, CH, CZ, DK, FR, and SI).
The interpretation is that, despite the same level of health literacy, older respondents tend to
respond more often to item 6 with “easy” or “very easy” than younger respondents do. Young
people may be less familiar with health screenings and have a poorer understanding of information
about these examinations. Unemployed or retired people, who possibly have health problems
more often and therefore may be more exposed to this type of information, respond to item 6
with “easy” or “very easy” more often than employed respondents.
As the HLS19-Q12 measures health literacy in 17 countries, the items were translated into several
languages and applied in the context of different health systems. If some countries offer different
health screenings and/or concepts like “recommended” and/or “examination” are translated, in-
terpreted, or understood differently in different countries, people will respond differently depend-
ing on their country of residence. Therefore, people with the same level of health literacy may
respond differently to a specific item depending on their country of origin. We refer to this as
“differential item functioning” (DIF). Items displaying DIF for the variable “country” indicate that
these items measure differently across countries and that comparative analyses are invalid. Item
revisions based on HLS19 data may make comparative analyses possible in future studies. The
conclusion was that the HLS19-Q12 data seem to have acceptable quality within countries but that
comparative analyses between countries may have to be avoided.
Table 5.5:
Overall analyses for the HLS19-Q12, for each country and data collection method or “mode”
Country χ2, p Mode Meang Reliability Dim (%)
α PSI
Austria 72.1, .82 CATI 1.55h .84 .82 7.5
Austriaa 115.5, .01* CAWI 1.06h .86 .85 10.1
Belgium 109.4, .03* CAWI .62 .88 .88 7.7
Bulgariae2 141.2, .00** CAPI .70 .81 .79 6.7
Bulgariae2 238.2, .00** CAWI .90 .85 .83 11.0
Czech Republicb 110.3, .03* CATI 1.12h .82 .79 7.0
Czech Republic 112.8, .02* CAWI .83h .84 .84 6.4
Denmark 79.9, .61 CAWI 1.38 .86 .85 10.0
Francef 176.1, .00** CAWI 1.32 .89 .88 6.5
Germany 76.4, .71 PAPI .65 .80 .81 8.7
Hungaryf 137.0, .00** CATI 1.21 .84 .83 9.3
Ireland 113.5, .02* CATI 1.22 .82 .79 5.9
Israelc 111.2, .03* CATI 1.42h .85
Israel 98.5, .13 CAWI .95h .87 .87 8.1
Country χ2, p Mode Meang Reliability Dim (%)
α PSI
Italy 159.7, .00** CATI .71 .84 .81 9.1
Italy 66.2, .92 CAWI .81 .90 .88 8.0
Norway 91.5, .27 CATI 1.29 .84 .83 5.5
Portugalf 225.8, .00** CATI 1.26 .90 .82 5.8
Russiaf 135.3, .00** PAPI 1.11 .90 .87 7.0
Slovakia 81.2, .56 CAPI .88 .88 .88 9.1
Sloveniaf 145.2, .00** CAPI 1.67 .91 .88 7.3
Sloveniaf 201.6, .00** CAWI 1.85 .86 .84 6.9
Sloveniad - PAPI - - -
Switzerlande1 - CATI - - -
Switzerland 84.4, .47 CAWI 1.18 .84 .84 8.7
Note. PCM=Rasch Partial Credit Model, α=Cronbach’s alpha, PSI=Person Separation Index, *p < .05, **p < .01. The chi-
square test for overall data-model fit using the PCM was based on G=8 groups of respondents (df=7 for a single item and
df=84 for 12 items) and a reduced sample size with 20 persons for each of 36 thresholds n=720: χ2(df=84, n=720), where
the number of thresholds is 12 x (4-1) = 36.
aAustria applied CAPI and used CAWI in an additional comparable study.
e2Bulgaria applied CAPI (n=402) and CAWI (n=463) in small samples.
bThe Czech Republic applied CATI in a medium sample (n=532) with 8 extreme scorers.
cIsrael applied CATI in a small sample (n=311) with 25 extreme scorers.
dSlovenia applied PAPI in a minor sample (n=12) with no analysis reported.
e1Switzerland applied CATI in a minor sample (n=192) with no analysis reported.
fFrance, Hungary, Portugal, Russia, and Slovenia (CAWI and CAPI) have acceptable overall fit to the PCM when the sample
size is reduced to 10 persons per threshold (n=360) for the chi-square test. The generalized partial credit model [GPCM]
was estimated for each of these five countries.
gMean Rasch-based health literacy proficiency (using the PCM with a 4-point raw score).
hMean Rasch-based score when data for the two modes are merged to form a common point of zero. When analyzed
separately, the mean is 1.06 and 1.57 (Austria), .80 and 1.23 (Czech Republic), and .87 and 1.81 (Israel) for CAWI and CATI,
respectively. Only the two Austrian samples are comparable.
5.6 How well the short forms HLS19-Q12 and HLS19-Q16 rep-
resent each other and the long form HLS19-Q47 statisti-
cally
In the HLS-EU study, a predecessor of the current HLS19-Q47 was used. Based on the need for a
shorter, unidimensional scale with better psychometric properties, first the HLS-EU-Q16 and later
the HLS-EU-Q12 measure were developed as short forms of the HLS-EU-Q47. For the HLS19 the
three versions were revised slightly.
Six countries (BG, DE, IE, IT, NO, and SI) participating in the HLS19 used the HLS19-Q47. The HLS19-
Q12 and the HLS19-Q16 short forms can also be compared for the eight countries that used the
22 items set (BE, CZ, DK, HU, FR, IL, RU, and SK).

For the GEN-HL score, the Pearson correlation of the HLS19-Q12 short form and the HLS19-Q47
long form ranges between 0.898 (NO) and 0.949 (IT) (Table 5.6). The correlations of the HLS19-
Q16 scores with the HLS19-Q47 scores (Table 5.7) and between the HLS19-Q12 and HLS19-Q16
scores (Table 5.8) are in a similar range. Consequently, the HLS19-Q12 behaves in a sufficiently
equivalent manner to the HLS19-Q47 or the HLS19-Q16 so that it can be used as a short form
substitute, and the results of publications using the different measures can be compared to a
certain degree. In addition, the HLS19-Q12 has better psychometrical properties, since the HLS19-
Q47 was not proven to be a unidimensional scale by means of Rasch analyses. It should be noted
though that, because the HLS19-Q47 is not proven to be a unidimensional scale, it is not guaran-
teed that the HLS19-Q47 and the HLS19-Q12 strictly measure the same latent construct.
Table 5.6:
Pearson correlation of the HLS19-Q12 and HLS19-Q16 scores with the HLS19-Q47 scores for BG,
DE, IE, IT, NO, and SI and the mean for these countries
BG DE IE IT NO SI Mean
Q47 x Q12 GEN-HL .917 .919 .901 .949 .898 .927 .929
Q47 x Q16 GEN-HL .922 .926 .916 .956 .909 .940 .936
Table 5.7:
Pearson correlation of the HLS19-Q16 scores with the HLS19-Q12 scores for BE, BG, CZ, DE, DK, FR, HU, IE, IL, IT, NO, RU, SI, and SK, and the mean correlation
across these countries
BE BG CZ DE DK FR HU IE IL IT NO RU SI SK Mean
Q16 x Q12
.924 .906 .899 .889 .901 .930 .891 .879 .921 .934 .871 .938 .916 .929 .917
GEN-HL

5.7 Distributions of the HLS19-Q12 scores
The standardized score ranges from 0 to 100, with higher values referring to a higher level of
General HL (cf. Section 4.2). The distribution of the scores is negatively (left) skewed for all coun-
tries, with a considerable ceiling effect (Figure 5.3), which again indicates that the items asked
about tasks that many respondents found manageable. In most countries, the 75% quantile is close
or equal to the maximum value of 100, which indicates a ceiling effect (Table 5.8). This ceiling
effect does not affect the identification of respondents with low levels of health literacy, however.
Thus, the instrument is still sensitive for respondents with lower HL. This skewness does pose
problems for some statistical analyses. The distribution is approximately symmetric only for the
German data.
Figure 5.3:
Histograms of the HLS19-Q12 scores, for all countries
Table 5.8:
Means, standard deviations, and percentiles for the HLS19-Q12 score, by country
Mean SD 25th percentile Median 75th percentile

AT 84.8 16.1 75.0 90.9 100.0
BE 65.7 26.7 50.0 66.7 91.7
BG 68.5 23.1 54.5 66.7 90.0
CH 77.3 19.6 66.7 83.3 91.7
CZ 76.3 22.3 63.6 83.3 91.7
DE 64.9 21.9 50.0 66.7 83.3
DK 77.4 20.5 66.7 83.3 91.7
FR 77.5 22.9 58.3 83.3 100.0
HU 79.8 19.8 66.7 83.3 91.7
IE 78.8 19.5 66.7 83.3 91.7
IL 73.0 23.9 58.3 75.0 91.7
IT 69.1 27.4 50.0 75.0 91.7
NO 78.8 19.2 66.7 83.3 91.7
PT 84.8 20.5 80.0 91.7 100.0
RU 80.3 23.3 66.7 90.9 100.0
SI 86.0 19.1 83.3 91.7 100.0
SK 69.7 25.1 50.0 75.0 91.7
All 76.0 22.9 58.3 83.3 91.7
5.8 Differences in mean HLS19-Q12 scores between the coun-

try sample and selected vulnerable subpopulations
In the context of policy making, it is important to know for which vulnerable or disadvantaged
subpopulation the average level of HL is particularly low. Based on experiences with the HLS-EU
study, the following four socio-demographic or socio-economic indicators and three health or
sickness behavior-related indicators were selected (cf. Subsection 4.5.1):
» advanced age (>76 years old)

» low education (ISCED code ≤ 1 on a scale from 0 to 8)
» low level in society/low social status (≤ 4 on a scale from 1 to 10)
» financial deprivation (≥ 50%, i.e., considerable, or severe financial deprivation)
» “very bad” or “bad” self-perceived health
» one or more long-term illnesses or health problems
» a high number of contacts with a GP/family doctor (≥ 6) within the last 12 months.
On average, the differences between the GEN-HL score for the various vulnerable subpopulations
and the overall country mean vary between -2.6 and -13.8 score points. There is considerable
variation by country (Table 5.10).

The highest average difference between the “subpopulation means” and the country’s GEN-HL
mean score is found for the subpopulation with “poor self-perceived health”, which is -13.8 below
the total mean (62.2 vs. 76.0). The difference varies by country, from –5.2 (CZ) to -27.4 (PT). The
“financially deprived” subpopulation shows a difference of -8.2 points (ranging from +0.7 (BE) to
-14.4 (BG)). The third largest difference is found for the respondents reporting “low level in soci-
ety/low social status” (-7.9), with a variation across countries ranging from -2.0 (AT) to -17.9
(BG). The lowest average difference was found for respondents with one or more reported “long-
term illnesses or health problems” at -2.6 (varying from -0.3 (CZ) to -7.2 (PT)). Average differ-
ences for the other defined subgroups are -6.3 for “low education” (ranging from +0.6 (CZ) to -
21.7 (RU)), -4.5 for “six or more contacts with a GP/family doctor” within one year (ranging from
-06 (CZ) to -11.5 (BG, SK)), and -4.9 for respondents “aged 76 or older” (ranging from +6.6 (CZ)
to -24.4 (PT). Three of these vulnerable subpopulations had unexpected, positive differences from
the national country mean (i.e., better HL than the average population) in one to three countries.
Table 5.9:
Differences in mean HLS19-Q12 scores between the country sample and selected vulnerable subpopulations, for each country and the mean for all countries
(equally weighted)
AT BE BG CH CZ DE DK FR HU IE IL IT NO PT RU SI SK All
GEN-HL country mean 84.8 65.7 68.5 77.3 76.3 64.9 77.4 77.5 79.8 78.8 73.0 69.1 78.8 84.8 80.3 86.0 69.7 76.0
Aged 76 or older 3.4 3.3 -23.4 2.1 6.6 -6.1 3.6 - 0.0 -0.1 - -1.5 -0.6 -24.4 -19.2 -14.1 -22.4 -4.9
Education at ISCED - - -20.7 -8.2 0.6 - -2.6 - -3.0 -4.7 -3.4 -1.9 -3.2 -10.5 -21.7 -9.6 -20.4 -6.3
levels 0 or 1
Level in society less -2.0 -5.8 -17.9 -4.1 -5.6 -7.4 -7.9 -5.5 -3.7 -7.4 -9.1 -7.4 -4.6 -9.3 -13.2 -6.5 -16.7 -7.9
than or equal to 4
Considerable or se- -7.5 0.7 -14.3 -4.8 -7.3 -7.7 -10.7 -8.3 -7.0 -9.0 -7.1 -8.3 -12.9 -9.5 -9.5 -5.8 -10.6 -8.2
vere financial depriva-
tion
Bad or very bad self- -9.1 -8.7 -34.3 -11.3 -5.2 -12.9 -12.1 -9.5 -10.7 -13.1 -5.4 -9.3 -7.9 -27.4 -22.0 -16.1 -19.6 -13.8
perceived health
One or more long- -2.2 -0.4 -4.1 -2.0 -0.3 -1.6 -1.9 -1.7 -2.1 -1.4 -1.9 -2.7 -1.6 -7.2 -6.3 -3.7 -2.9 -2.6
term illnesses or
health problems
Limited by health -4.6 -1.8 -8.2 -3.6 -1.5 -2.5 -3.4 -3.1 -5.1 -4.1 -2.7 -5.8 -2.3 -12.4 -6.7 -5.6 -5.6 -4.6
problems
6 or more contacts -2.4 -2.7 -11.5 -4.6 -0.6 -4.3 -5.6 -3.2 -2.1 -4.7 -0.2 -2.8 -1.9 - -10.6 -3.6 -11.5 -4.5
with a GP/family doc-
tor
- Cells with less than 30 respondents were not reported.

5.9 Categorical levels for the HLS19-Q12
When reporting health literacy within certain subpopulations, it is sometimes advantageous to

assign respondents to categorical health literacy levels, as was done in the HLS-EU and other
studies. Since the calculation of the health literacy scores for the HLS19 deviates from the HLS-EU
methodology and because the score was computed differently and thus has a different meaning,
the calculation of such levels had to be adapted. To approximate the categorical levels based on
the HLS-EU methodology, the HLS19 enriched the information from the score value with infor-
mation about how often the extreme categories were selected. The terms used for the categorical
levels are identical to the HLS-EU study. For details of definitions of the cutting points, the cate-
gorical levels for the HLS19-Q12, and one possibly unexpected consequence, see Section 4.1.
Across all participating countries, about 40% of the respondents have a “sufficient” level of health
literacy and about 15% an “excellent” level. On the other hand, about 33% have a “problematic”
level of health literacy and 13% an “inadequate” level. The levels varied by country:
» “inadequate” General HL between 4% (AT) and 27% (BE),

» “problematic” General HL between 18% (SI) and 48% (DE),
» “sufficient” General HL between 23% (DE) and 65% (PT),
» “excellent” General HL between 5% (DE and PT) and 29% (IE).
For respondents at an “inadequate” level of HL, especially items 1, 2, 6, 7, 9, 11, and 12 were
“difficult” or “very difficult” more often than for respondents with better health literacy (Table
5.10). For respondents at a “problematic” level of HL, specifically items 3, 5, and 8 were “difficult”
or “very difficult” more often than for respondents with “sufficient” or “excellent” HL. These items
were also “difficult” or “very difficult” more often for respondents with “sufficient” HL than for
respondents with “excellent” HL (Table 5.10).
Respondents with an “inadequate” level of health literacy tend to say their health is worse than
other respondents (Figure 5.5), with the relationship between self-reported health and level of
health literacy being almost linear for most countries.
In line with the HLS-EU study, when the HL categorical levels “inadequate” and “problematic” are
combined as “limited health literacy”, the resulting variation ranges from 25% (SI) to 72% (DE), i.e.,
between one out of four (in SI) and three out of four (in DE) residents in participating countries
have limited General HL. Compared to the HLS-EU, with one out of three up to two out of three,
the variation between countries is even more pronounced, which could be the result of different
methodology and different countries being included in the two studies.
Figure 5.4:
Percentage of respondents by categorical level of General HL as measured by the HLS19-Q12, for
each country and the mean for all countries

Table 5.10:
Percentage of difficulty of each item for each level of General HL, for all countries (equally
weighted)
Inade- Problem-
Sufficient Excellent Total
quate atic
3 … to judge the advantages and disadvantages
88.42 68.42 19.90 5.67 42.44
of different treatment options?
8 … to decide how you can protect yourself from
84.84 60.50 18.44 8.79 39.24
illness using information from the mass media?
5 … to find information on how to handle mental
85.47 58.71 14.01 4.13 36.35
health problems?
12 … to make decisions to improve your health
73.53 38.11 8.62 2.56 25.67
and well-being?
2 … to understand information about what to do
70.47 33.59 5.77 1.90 22.57
in a medical emergency?
11 … to judge how your housing conditions may
70.03 32.63 5.44 1.37 21.97
affect your health and well-being?
6 … to understand information about recom-
62.04 24.91 3.85 0.93 17.76
mended health screenings or examinations?
7 … to judge if information on unhealthy habits,
such as smoking, low physical activity, or drinking 64.60 24.63 3.34 0.59 17.73
too much alcohol, is reliable?
10 … to understand advice concerning your
50.99 23.46 5.15 1.57 16.47
health from family or friends?
1 … to find out where to get professional help
54.94 22.83 4.02 1.01 16.26
when you are ill?
9 … to find information on healthy lifestyles such
44.24 11.51 1.53 0.31 10.08
as physical exercise, healthy food, or nutrition?
4 … to act on advice from your doctor or pharma-
32.94 9.64 1.54 0.46 8.06
cist?
Mean 65.21 34.08 7.63 2.44 22.88
Figure 5.5:
Average self-perceived health by level of General HL, for each country and the mean for all
countries
The score for the HLS19-Q12 measure was calculated for dichotomized item categories (“very easy”
or “easy”) to avoid assigning numbers to the ordinal response categories of the items constituting
the measure and was standardized from 0 to 100% for better comparability and interpretation of
the measure. The distribution of the score values is negatively skewed in all countries. In most
countries, this resulted in a ceiling effect. Future research should clarify the extent to which this
is due to the scale items being too easy, or to different survey modalities, or both. The ordering
of the items in relation to the percentage of respondents finding the task “difficult” or “very diffi-
cult” across the participating countries suggests that the scale is working adequately.
The Cronbach alpha coefficients demonstrate sufficient internal consistency. The confirmative fac-
tor analyses demonstrate a good model fit for a single latent variable model. The Rasch analyses
support the use of the HLS19-Q12 as a unidimensional measure for health literacy. However, the
Rasch analyses also suggest opportunities for improvement by modifying items in a future devel-
opment of the HLS19 questionnaire. Some items display DIF for different person factors, such as
age and gender, and for the variable “country”. The correlation of the HLS19-Q12 and HLS19-Q47

GEN-HL scores is sufficiently high to accept that the HLS19-Q12 short form can be used as an
equivalent for the HL-Q47 long form and that studies using the different forms can be compared.
However, due to the considerable methodological differences between the HLS19 and the HLS-EU,
the results of the two studies can only be compared to a certain degree.
In conclusion, the HLS19-Q12 is a psychometrically rather sound instrument for measuring com-
prehensive General HL in adult populations as intended in the HLS19.
5.11 References
Beaujean, A Alexander (2014): Latent variable modeling using R: A step-by-step guide. Routledge,
Guttersrud, Øystein; Le, Christopher; Pettersen, Kjell Sverre; Finbråten, Hanne Søberg (2021):
Rasch analyses of data collected in 17 countries. - A technical report to support decision-
making within the M-POHL consortium. https://m-pohl.net/Results
New York
Masters, Geoff (1982): A Rasch model for partial credit scoring. In: Psychometrika 47/2:149-174
Prudon, Peter (2015): Confirmatory factor analysis as a tool in research using questionnaires: a
critique. In: Comprehensive Psychology 4/:03. CP. 04.10
6 Determinants and a social gradient of General
Health Literacy measured by the HLS19-Q12
Authors:

Thomas Link (ICC)
Dominika Miksova (ICC)
Eva Maria Berens (DE)
Nejc Berzelak (SI)
Chiara Cadeddu (IT)
Paola De Castro (IT)
Jorge Oliveira (PT)
Youssoufa Ousseine (FR)
Luigi Palmieri (IT)
Aldo Rosano (IT)
Mitja Vrdelja (SI)
Chapter 6 / Determinants and social gradient of General health literacy measured by the HLS19-Q12 111
6.1 Background
To select determinants of HL in the HLS19, the HLS-EU conceptual model of HL (Sørensen et al.
2012) was used in its refined, more detailed, explicit causal version of the Vienna Model of Health
Literacy (see Figure 1.2, Chapter 1) (Pelikan/Ganahl 2017).
According to the Vienna Model, the personal HL of an individual is influenced by personal deter-
minants, like socio-demographic and socio-economic variables, such as gender, age, educational
level, self-assessed level in society, and financial deprivation (as used in HLS-EU publications) as
well as by situational determinants, such as characteristics of health (care) systems and regional
characteristics (e.g., rural/urban). In this chapter we focus on selected personal socio-demo-
graphic and socio-economic variables that were used in the HLS-EU study.
In the HLS-EU study it was demonstrated that there is a social gradient for health literacy by using
indicators for gender, age, educational level, self-assessed level in society, and financial depriva-
tion in the regression model (HLS-EU Consortium 2012; Sørensen et al. 2015), which was also
confirmed by further studies, e.g., by Duong et al. (2017); Schaeffer et al. (2017); Stormacq et al.
(2019). For all eight countries in the HLS-EU, the significant predictors for HL, ordered by the
strength of β coefficients in a linear regression model, which explained on average 17% of the
variance (varying across countries from 8% to 25%), were, from highest to lowest, financial depri-
vation (on average β=-.24), followed by self-assessed level in society (on average β=.14), educa-
tion (β=.13), age (β=-.09), and gender (β=.06) (HLS-EU Consortium 2012; Sørensen et al. 2015).
For this chapter, following the underlying generic causal model and the results of existing re-
search, the main research questions were:
» How strongly is General HL associated with socio-demographic, socio-economic and addi-

tional selected determinants?
» Is there a social gradient for HL?
Of course, with cross-sectional data, these questions can only be answered in a limited, explora-
tive way, but the underlying causal assumptions are very plausible for the independent variables
used in the specified regression model.
To answer the research questions, five core socio-demographic and socio-economic determinants
were used: gender and age for the former and education, level in society, and financial deprivation
for the latter. The following determinants, which were partly used in the HLS-EU and other studies,
were also investigated: migration background, long-term illness, being trained in a healthcare
profession, and partly status of employment (for status of employment, a figure with the distri-
butions and a table with the means of General HL are only provided in Annex 6). For all selected
determinants, a detailed overview of the measured indicators is provided in Annex 6, Table A 6.1,
including the wording of the items and their response categories used in the HLS19, the source of
the original item, and an indication of changes made compared to the item used in the predecessor
study, the HLS-EU.
In connection with results relating to each of the five, core socio-demographic and socio-
economic, determinants mentioned in the published literature, the following points can be high-
lighted:
Gender: published results are inconsistent. The HLS-EU study found on average a weak significant
β coefficient showing females to have higher health literacy scores than males, but this was not
significant in a few countries in the HLS-EU. In a systematic review of HL studies in Iran, six were
identified which showed that being a woman was associated with lower HL, and two studies found
that men had lower HL (Kamal et al. (2018). Men having lower HL was also demonstrated by
Almaleh et al. (2017), while Matsumoto/Nakayama (2017) found higher HL in women. However,
many studies did not find differences between women and men in HL (Garcia-Codina et al. 2019;
Jordan/Hoebel 2015; Tiller et al. 2015).
Age: published results are inconsistent. In two reviews, older age was associated with lower HL
(Berkman et al. 2011; Kamal et al. 2018). In the HLS-EU for age, β=-.09 was found for all countries
together, but the values were statistically significant for only five out of the eight countries (from
β=-.14 to β=-.16) (HLS-EU Consortium 2012). An increase in HL with age was shown by Tiller et
al. (2015), and higher HL in older populations was also found by Matsumoto/Nakayama (2017).
Education: a positive, significant association of HL with education was shown in the HLS-EU study
(β=.13 for all countries together), but it was only statistically significant for six out of the eight
countries (from β=.08 to β=.22) (HLS-EU Consortium 2012). A low educational level was found to
be a risk factor for low HL by van der Heide et al. (2013), Friis et al. (2016), Kamal et al. (2018),
Fleary/Ettienne (2019), and Svendsen et al. (2020).
Level in society (social status): a significant, positive association was shown with HL in the HLS-
EU study (β=.14 for all countries together), but it was only statistically significant for six out of
the eight countries (from β=.07 to β=.23) (HLS-EU Consortium 2012). A higher risk of limited HL
was found for persons with lower social status in van der Heide et al. (2013a), Berens et al. (2016),
Rikard et al. (2016), and Duong et al. (2017).
Financial deprivation: a significant, negative association with HL was demonstrated in the HLS-EU
study (β=-.24 for all countries together, ranging from -.07 to β=-.35) (HLS-EU Consortium 2012).
Financial deprivation was also associated with lower HL in studies by Levin-Zamir et al. (2016),
Palumbo et al. (2016), and Vogt et al. (2017). Furthermore, low economic status including low
income was found to be a risk factor for lower HL by Kamal et al. (2018).
In addition to these five socio-demographic and socio-economic indicators, the effects of three
further indicators were investigated, namely migrant status, long-term illness, and training in a
health profession.
6.2 Distribution of selected determinants, their associations,
and correlations with HL
Overview of the presentations of results
First, correlations between the five socio-demographic and socio-economic determinants plus
three further ones, and with General HL are presented for all countries together (with each country
weighted by 1,000) (Figure 6.1). Then distributions are presented for the selected determinants.
For the distributions of some of the determinants, it must be kept in mind that the data were
weighted for most countries by gender, age group, population density, regional administrative
units, and partly for education (for details of data weighting by country, see Section 2.4). The
distributions are followed by the associations of each determinant with General HL, with line charts
showing the relation of the mean HL values with selected determinants. Correlations of General
HL with selected determinants for each country are presented in Table 6.1 and are described in
the relevant sections for the single determinants.
Correlations between the five socio-demographic and socio-economic plus three further deter-
minants, and with General HL
As can be expected, the five socio-demographic and socio-economic determinants treated as core
determinants correlate with each other, the highest for level in society with financial deprivation
(ρ=-0.39) and with education (ρ=0.31), followed by education with financial deprivation (ρ=-
0.21) and with age (ρ=-0.14). Of these, only financial deprivation (ρ=-0.22) and level in society
(ρ=0.15) correlate to a considerable degree with General HL.
For the three additional determinants, migration background, trained in a healthcare profession,
health status, and long-term illness, included in the International Report, the highest correlations
are between long-term illness and age (ρ=0.34), with financial deprivation (ρ=0.17), and with
level in society (ρ=-0.14). Only long-term illness (ρ=-0.14) and training in a health profession
(ρ=-0.10) are correlated to General HL (Figure 6.1).
Figure 6.1:
Spearman correlations (ρ) between GEN-HL and selected determinants, for all countries (equally
weighted)*
* Values for the correlations with migration should be treated with caution as in five countries (BG, HU, IT, RU,
and SK) the percentage of respondents with a migration background was below 10%, and the HLS19 survey as
such was not adjusted to targeting migrants specifically (e.g., by offering translations of the instrument into
migrant languages).
Table 6.1:
Spearman correlations (ρ) between GEN-HL and selected determinants, for each country and for all countries (equally weighted)
Gender female 0.04 0.02 0.01 0 0.04 0.04 0.04 0 -0.03 0.06 0.03 0.02 0.08 -0.05 -0.03 0.03 0.01 0.02
Age in years -0.06 0.07 -0.21 0.04 0.14 -0.07 0.11 0.02 0.11 0.06 0.14 -0.06 0.01 -0.18 -0.23 -0.09 -0.19 -0.03
Education -0.01 0 0.2 0.04 -0.13 0.22 0.1 0 0.07 0.09 -0.03 0.05 0.08 0.24 0.09 0.14 0.23 0.03
Level in society -0.01 0.22 0.4 0.14 0.15 0.22 0.18 0.16 0.11 0.17 0.2 0.15 0.13 0.27 0.35 0.18 0.29 0.15
Financial depriva-
-0.19 0 -0.33 -0.16 -0.23 -0.21 -0.24 -0.19 -0.26 -0.28 -0.24 -0.26 -0.16 -0.33 -0.41 -0.25 -0.39 -0.22
tion
Migration* 0.07 -0.04 0.04 0.05 -0.02 -0.03 -0.02 -0.02 -0.03 0.01 0 -0.01 0 -0.01 0.04 -0.02 -0.03 0.01
No training in a
health 0.01 -0.05 -0.29 -0.09 -0.06 -0.18 -0.1 -0.08 -0.06 -0.04 -0.08 -0.09 -0.12 -0.12 -0.09 -0.09 -0.19 -0.10
profession
Long-term illness -0.11 -0.03 -0.24 -0.09 -0.03 -0.09 -0.09 -0.08 -0.08 -0.06 -0.04 -0.09 -0.09 -0.26 -0.28 -0.12 -0.17 -0.14
Education by 9 ISCED levels, from 0 (lowest) to 8 (highest level).

Level in society from 1=lowest level to 10=highest level in society.
Financial deprivation: 4 categories, from no deprivation (0) to severe deprivation (100).
Long-term illness: 3 categories: (1) none, (2) one, (3) more than one, except for SI where 2 categories were used (1) none, (2) one or more.
Migration background: 0=none, 1=one parent born abroad, 2=both parents born abroad, 3=born abroad.
* Values for the correlations with migration should be treated with caution as in five countries (BG, HU, IT, RU, and SK) the percentage of respondents with migration background was below 10%, and the HLS19 survey as such was not adjusted
to targeting migrants specifically (e.g., by offering translations of the instrument into migrant languages).
Socio-demographic determinants: Gender and age
Gender
Gender is evenly distributed with an equal or somewhat lower percentage of men (on average 48%)
for all participating countries (Figure 6.2). The lowest proportion of men is 45% (RU) and the high-
est 51% (NO).
Figure 6.2:
Percentage distribution of gender, for each country and for all countries (equally weighted)
With a mean correlation of ρ=0.02, gender is only very weakly associated with General HL. The
values of the correlation coefficients are low for all countries. Women have slightly higher General
HL in most countries, varying from ρ=0 (CH, FR) to 0.08 (NO), with the exception of HU, RU (ρ=-
0.03), and PT (ρ=-0.05), where men have somewhat higher General HL (Table 6.1 and Figure 6.3).
Figure 6.3
Means of GEN-HL scores by gender, for each country and for all countries (equally weighted)
Age
Age was measured in years but is grouped into seven categories (Figure 6.4). The relative size of
these categories (for overall countries 11% in 18-25 years, 17% in 26-35 years, 17% in 36-45
years, 18% in 46-55 years, 17% in 56-65 years, 13% in 66-75 years, 7% in 76 and more years)
varies somewhat by country (Figure 6.4). This is also reflected in the means and standard devia-
tions for age, with an overall mean of 48.1 years (varying from 45.6 (IL) to 51.6 (DK)) and an overall
standard deviation of 17.4 (varying from 15.7 (FR) to 18.6 (IT)) (Table 6.2).
The median age for all countries is 48 years (varying from 44 years (IL) to 53 years (DK)). The
overall value for all countries for the 25th percentile is 33 years (varying from 31 years (IL, NO) to
38 years (IT)) and the overall value for the 75th percentile is 62 years (varying from 59 years (FR,
PT) to 67 years (DK)) (Table 6.2).
In most participating countries, the association of General HL with age is nonlinear in different
ways (Figure 6.5). Therefore, the overall association of age with General HL for all countries is low
with a Spearman coefficient of ρ=-0.03 (Table 6.1). There are countries with an expected negative
correlation, ranging from ρ=-0.06 (AT) to ρ=-0.23 (RU) as well as countries with an unexpected
somewhat lower positive correlation, ranging from ρ=0.01(NO) to ρ=0.11 (DK, HU) (Table 6.1).
Figure 6.4:
Percentage distribution of age groups in seven categories, for each country and for all countries
(equally weighted)*
* FR did not include participants above 75 in the survey.
Table 6.2:
Means, standard deviations, and percentiles of age distributions, for each country and for all countries (equally weighted)
Mean 48.9 47.8 49.2 49 48.6 50.8 51.6 46.1 48.0 46.5 45.6 51.9 46.6 46.1 47.5 50.4 47.6 48.1
SD 17.8 16.1 17.6 18.3 17.1 18.5 18.2 15.7 17.5 17.0 16.7 18.6 18.5 16.7 17.3 18.2 18 17.4
25th per-
34.0 33.0 35.0 34.0 33.0 35.0 36.0 34.0 33.0 32.0 31.0 38.0 31.0 33.0 33.0 36.0 32.0 33.0
centile
50th per-
49.0 49.0 50.0 48.0 48.0 50.0 53.0 46.0 48.0 47.0 44.0 51.0 46.0 46.0 46.0 50.0 46.0 48.0
centile
75th per-
63.0 61.0 63.0 62.0 63.0 65.0 67.0 59.0 62.0 62.0 60.0 66.0 62.0 59.0 61.0 64.0 62.0 62.0
centile
Figure 6.5
Means of GEN-HL scores by age in seven groups, for each country and for all countries (equally
weighted)
Socio-economic determinants: Level in society, educa-

tion, and financial deprivation
Level in Society
Level in society was measured by a measure taken from the Eurobarometer which was also used
in the HLS-EU. This measure varies from 1 (=lowest self-assessed level in society) to 10 (=highest
level in society). It is approximately normally distributed, but its variation across countries is
considerable (Figure 6.6).
The mean self-assessed level in society for all countries together is 5.9 (varying from 5.1 (HU) to
6.5 (BE)) with a standard deviation of 1.6 (varying from 1.4 (PT) to 1.8 (BG, IL, RU)) (Table 6.3).
The correlation for General HL with level in society has a Spearman coefficient of ρ=0.15 on
average (vaying from -0.01 (AT) to 0.4 (BG)) and is the second strongest of all included
determinants (Table 6.1).
Chapter 6 / Determinants and a social gradient of General Health Literacy measured by the HLS19-Q12 121
Due to a few cases in the extreme answer categories 1 and 2 plus 9 and 10 (Figure 6.6), the answer
categories 1 and 2 (shown as 1-2), and the answer categories 9 and 10 (shown as 9-10) were
collated for Figure 6.7 on associations. The association is more or less linear for all countries but
on a somewhat different level of General HL. On average, respondents with a higher level in society
have better HL, except for IT, where only a low number of cases for level 9 and 10 exist so that
the mean value for this category may be not reliable (Figure 6.7).
Figure 6.6:
Percentage distribution of level in society, for each country and for all countries (equally
weighted)
Table 6.3:
Means, standard deviations, and percentiles of distribution of levels in society, for each country
and for all countries (equally weighted)
Mean 6.2 6.5 5.9 5.8 5.7 5.9 6.3 5.6 5.1 6.0 6.3 5.8 6.4 5.4 5.7 5.4 5.9 5.9
SD 1.5 1.5 1.8 1.7 1.6 1.6 1.7 1.5 1.5 1.7 1.8 1.5 1.6 1.4 1.8 1.6 1.7 1.6
25th percentile 5.0 6.0 5.0 5.0 5.0 5.0 5.0 5.0 4.0 5.0 5.0 5.0 5.0 5.0 5.0 5.0 5.0 5.0
50th percentile 6.0 7.0 6.0 6.0 6.0 6.0 7.0 6.0 5.0 6.0 6.0 6.0 7.0 5.0 6.0 5.0 6.0 6.0
75th percentile 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0 6.0 7.0 7.0 7.0 7.0 6.0 7.0 7.0 7.0 7.0
Figure 6.7:
Means of GEN-HL scores by level in society, for each country and for all countries (equally
weighted)
Education
Education was measured by ISCED, offering nine detailed categories (from 0 to 8), although some
participating countries already merged some categories when collecting data to better reflect the
educational system in their country.
To calculate correlations and regressions (see Sections 4.5 and 4.7 for details of the methods), the
categories were interpreted as ordinal or interval scales respectively and used in that way.
To show the distributions of categories by countries (Figure 6.8), the nine categories were com-
bined into four categories:
1. lower secondary education or below (up to ISCED-2)
2. higher secondary education (ISCED-3)
3. post-secondary or short-cycle tertiary education (ISCED-4 and 5)
4. bachelor or higher (ISCED-6 to 8)
The distribution of the four categories is 21% on average (varying from 3% (BE) to 51% (HU)) for
lower secondary education and below, 28% (from 9% (DK) to 48% (AT, DE)) for higher secondary
education, 18% (from 0% (CH) to 47% (RU)) for post-secondary or short cycle tertiary education,
and 33% (from 16% (HU) to 58% (BG)) for bachelor or higher, and thus varies considerably by coun-
try (Figure 6.8).
The mean value with all countries weighted equally (“All”), calculated by using the nine categories,
is 4.1 (varying from 3 (HU) to 5.3 (BG)), while for standard deviations it is 2.0 (varying from 1.3
(RU) to 2.1 (BG, PT, SK)) (Table 6.4). The median for all countries is 4.0 (from 2.0 (HU) to 6.0 (NO,
BG)), the value of the 25th percentiles for all countries is 3.0 (from 2.0 (CZ, HU, IE, IT, PT, and SI)
to 4.0 (BE, DK, FR, and RU)) and the value of the 75th percentiles for all countries is 6.0 (from 3.0
(CZ, HU, and IT) to 7.0 (BE, BG, and SK)) (Table 6.4).
Associations vary considerably by level and the format of the curves (Figure 6.9). The overall cor-
relation of General HL with education is ρ=0.03. For most countries, it is positive (varying from
ρ=0.04 (CH) to ρ=0.24 (PT)), while for three countries it is negative (ρ=-0.01 (AT), ρ=-0.03 (IL),
ρ=-0.13 (CZ), ρ=0 for BE and FR (Table 6.1).
Figure 6.8:
Percentage distribution of educational levels (four grouped categories), for each country and for
all countries (equally weighted)
Table 6.4:
Means, standard deviations, and percentiles of distribution of educational levels (nine ISCED
categories*), for each country and for all countries (equally weighted)
Mean 3.8 5.2 5.3 4.2 3.3 4.0 4.9 5.0 3.0 3.9 4.5 3.2 4.8 3.1 4.5 3.4 4.3 4.1
SD 1.7 1.9 2.1 1.9 1.9 1.7 1.6 1.5 1.8 2.0 1.6 1.9 1.9 2.1 1.3 2.0 2.1 2.0
25th percentile 3.0 4.0 3.0 3.0 2.0 3.0 4.0 4.0 2.0 2.0 3.0 2.0 3.0 2.0 4.0 2.0 3.0 3.0
50th percentile 3.0 4.0 6.0 3.0 3.0 3.0 5.0 5.0 2.0 4.0 4.0 3.0 6.0 3.0 4.0 3.0 3.0 4.0
75th percentile 5.0 7.0 7.0 6.0 3.0 6.0 6.0 6.0 3.0 6.0 6.0 3.0 6.0 6.0 6.0 5.0 7.0 6.0
* Nine ISCED categories: from 0=no formal education or below ISCED 1 to 8=doctoral or equivalent level
Figure 6.9:
Means of GEN-HL scores by education level (four grouped categories), for each country and for
all countries (equally weighted)
Financial deprivation
In the HLS19, the financial deprivation score was based on three items (for details on calculating
the score, see Section 4.6). Compared to the HLS-EU, the three items used in the HLS19 were
rephrased and for one item the categories were changed (D14 in the HLS-EU, corresponding to C-
DET11 in the HLS19). For a detailed description of the items in the HLS19 in comparison with the
HLS-EU, see Annex 6, Table 6.1.
The skewed and varying percentage distribution by country, with RU (followed by HU and SI) and
NO (followed by AT) being the poles, is presented in Figure 6.10.
There is a negative correlation between General HL and financial deprivation, with an average value
of ρ=-0.22, varying from ρ=-0.16 (CH, NO) to ρ=-0.41 (RU) and with ρ=0 for BE (Table 6.1).
Apart from BE, which deviates from all other countries in the category of severe financial depriva-
tion, the association is rather linear, but on a different level of General HL (Figure 6.11). (For NO,
where only a low number of cases exist for severe deprivation, the mean value for this category
may be unreliable) (Figure 6.10).
Figure 6.10.:
Percentage distribution of financial deprivation levels, for each country and for all countries
(equally weighted)
Figure 6.11:
Means of GEN-HL scores by financial deprivation level, for each country and for all countries
(equally weighted)
Further determinants: Migration background, long-term

illness, training in a healthcare profession, and status of
employment
Further determinants, which were partly used in the HLS-EU and other studies, were also explored.
The results are presented below in relation to migration background, long-term illness, training
in a healthcare profession, and partly for the status of employment.
Migration background
In the HLS19, all permanent residents living in private households were included and, thus, in prin-
ciple, all kinds of migrants, in contrast to the HLS-EU, where only EU citizens were included in the
definition of the population. However, it should be noted that the HLS19 was only implemented in
the national language(s) of participating countries and was not translated into migrant languages.
Therefore, and also partly due to the method of data collection, migrants may be underrepre-
sented. Only two countries (DE, NO) implemented a separate survey in parallel to the HLS19 which
addressed migrants with the HLS19 instruments in migrant languages, but these results are not
presented in the International Report.
While the HLS-EU only focused on parents’ migration status, in the HLS19 this concept was wid-
ened, with migration background being measured by three questions in combination with an index
of four categories: “none”, “one parent was born abroad”, “both parents were born abroad”, and
“born abroad”. Figure 6.12 presents the percentage distributions of these four categories. In most
countries, only a small proportion of respondents had a migration background: The lowest per-
centages on migration background were found in BG, HU, IT, RU, and SK. In contrast, for four
countries, more than a fifth of the respondents had some form of migration background, from
23% (IE), 24% (SI), and 44% (CH) to 57% (IL).
To examine the association of General HL by migration background, only countries with more than
10% of migrant respondents were included in the analyses (thereby excluding BG, HU, IT, RU, and
SK). For most countries, the association of migration status with General HL is not linear (Figure
6.13). The overall correlation (with all countries weighted equally) is ρ=0.01, resulting from partly
negative and partly positive values depending on the country (Table 6.1). Therefore, migration
status was not included in the main model but was included in an additional regression model
(Table 6.6).
Figure 6.12:
Percentage distribution of migration background, for each country and for all countries (equally
weighted)
Figure 6.13:
Means of GEN-HL scores by migration background, for each country and for all countries
(equally weighted)
* Data from BG, HU, IT, RU, and SK were excluded as the percentage of respondents with a migration back-
ground was below 10%.
Long term-illness
Long-term illness was not included when analyzing social gradients of HL but it can be expected
to be a relevant determinant of HL due to increased experience with the healthcare system. There-
fore, it was tested in an additional regression model but not included in the main model (Table
6.7).
Long-term illness was measured by an item from the Minimum European Health Module (MEHM),
asking respondents if they have any long-term illnesses or health problems that have lasted or
are expected to last for six months or more. Three answer options were offered: (1) “no”, (2) “yes,
one” or (3) “yes, more than one long-term illness or health problem”. (In one country (SI), answer
options (2) and (3) were merged). These three categories were interpreted as being ordinal.
Overall, 55% of the respondents had no long-term illness, varying from 40% (CZ) to 64% (AT), 28%
had one long-term illness (from 16% (DE) to 36% (FR)), and 18% (from 7% (AT) to 36% (DE)) had
more than one long-term illness (Figure 6.14). The association of General HL with the number of
long-term illnesses is negative and rather linear with only a few exceptions (Figure 6.15). The
overall correlation is ρ=-0.14, varying from ρ=-0.03 (BE, CZ) to ρ=-0.28 (RU) (Table 6.1). Re-
spondents with more than one chronic illness had lower HL than respondents with one or no
chronic illnesses.
Figure 6.14:
Percentage distribution of the number of long-term illnesses/health problems, for each country
* SI was not included in Figure 6.13 as it used a slightly different measure with only two categories for long-
term illnesses/health problems ((1) no and (2) one or more long-term illnesses/health problems). In SI 60% of
the respondents had no long-term illnesses/health problems and 40% had one or more long-term ill-
nesses/health problems.
Figure 6.15:
Means of GEN-HL scores by the number of long-term illnesses/health problems, for each
country and for all countries (equally weighted)
* SI was not included in Figure 6.14 as it used a slightly different measure with only two categories for long-
term illnesses/health problems ((1) none and (2) one or more). In SI the mean GEN-HL for (1) “none” was 88.4
and for (2) “yes, one or more” it was 82.3.
Training in a healthcare profession
There is no tradition of using “training in a healthcare profession” in HL research or as a determi-

nant of HL, but it was used in the HLS-EU and revealed interesting results. Therefore, it was in-
cluded in the HLS19 and tested as a possible determinant in a specific model in this chapter since
it could be argued that respondents trained as health professionals should have better HL (Table
6.8).
Training in a healthcare profession was measured by asking respondents: ‘Have you ever been
trained in a healthcare profession?’, a slightly rephrase of the one used in the HLS-EU.
On average, nearly one-fifth of the respondents had experienced training in a healthcare profes-
sion, but there was considerable variation from 7% (CZ) to 35% (BG) (Figure 6.16). For all countries
except AT, respondents with training in a healthcare profession had a higher mean value for Gen-
eral HL than those without but to a different degree and on a different level of HL (Figure 6.17).
The correlations between HL and having “training in a healthcare profession” are comparatively
high, with, on average, ρ=-0.10, varying from ρ=-0.04 (IE) to ρ=-0.29 (BG). The one exception
was AT (0.01) where respondents trained in a healthcare profession had lower HL. (Table 6.1).
Figure 6.16:
Percentage distribution of training in a healthcare profession, for each country and for all
countries (equally weighted)
Figure 6.17:
Mean of GEN-HL scores by training in a healthcare profession, for each country and for all
Status of employment
Due to its nominal categorization, just the distributions (Figure A 6.1) and means of General HL
for the status of employment categories (Table A 6.2) are presented in Annex 6 and no correla-
tions or regressions. On average, 51% of the respondents were employed (varying from 36% (IT)
to 60% (RU)) and 24% (from 15% (IL) to 31% (SI)) were retired; of the remainder, 7% were self-em-
ployed, 7% students or trainees, 5% unemployed, 3% did domestic work, and 2% were unable to
work due to long-standing health problems. General HL was on average highest for those in em-
ployment (78.7) and lowest for the group of respondents who were unable to work due to long-
standing health problems (71.4) (Annex 6, Table A 6.2).
6.3 Regression analyses
One of the research questions asked whether there is a social gradient for General HL and how
strong the included determinants are as predictors of General HL. To answer this question, mul-
tivariable linear regression models were calculated. R² is used as a measure of model performance.
Standardized coefficients (β) are used to compare the predictors’ relative importance for each
country (Table 6.5 to Table 6.8). The unstandardized coefficients (b) can be found in Annex 6,
along with the results of regression analyses of linear models with ordinal variables entered as
dummy variables (see Section 4.6 for methodological details).
Base model 1: Effects on HL of three measures of socio-

economic status, gender, and age
Model 1 - using five selected core socio-demographic and socio-economic determinants of HL

simultaneously - is the main model (and is also, in principle, comparable to the HLS-EU) to test
for a social gradient of HL (Table 6.5). Judged by the values of R², the social gradient defined by
the determinants included in this model is 7% on average. However, for individual countries, this
value varies considerably from 4% (BE, CH, and NO) to 25% (BG).
On average, financial deprivation is the predictor with the highest average β of -0.21 and is also
highest in each country, except for BE (where the relation with +0.05 is reversed), varying from
β=-0.15 (CH, DE, FR, and NO) to β=-0.32 (SK). The second highest predictor is level in society,
with β=0.1 on average (varying from 0.01 (AT, HU, and IT) to 0.26 (BG)).
For gender, those countries where being a woman predicts better General HL, β=0.04 on average,
varying from 0.01 (CH) to 0.09 (CZ). The exceptions are HU (β=-0.03) and PT (β=-0.07) (both
non-significant), where being a man predicts better General HL.
While the effects of the three determinants financial deprivation, level in society, and gender are
rather similar across all countries (with a few exceptions), the effects for age (β=-0.04 for all
countries) and education (β=-0.04 for all countries) are inconsistent across countries.
On average, a relevant social gradient for General HL was demonstrated for the combined five core
socio-demographic and socio-economic determinants, with the highest effects for financial
deprivation and level in society but differing considerably by country.
Table 6.5:
Model 1: Multivariable linear regression models of GEN-HL by five core social determinants (standardized coefficients (β) and R2), for each country and for all
Gender female 0.07 0.02 0.03 0.01 0.09 0.07 0.06 0.04 -0.03 0.06 0.07 0.05 0.07 -0.07 0.04 0.03 0.07 0.04
Age in years -0.08 0.03 -0.08 0 0.11 -0.1 0.07 0 0.11 0.06 0.09 -0.05 0 -0.15 -0.15 -0.13 -0.11 -0.04
Education -0.03 -0.06 0.11 -0.02 -0.14 0.13 0.03 -0.04 0.03 0.05 -0.09 0.01 0.04 0.08 0.03 0.03 0.04 -0.04
Level in society 0.01 0.2 0.26 0.08 0.13 0.09 0.1 0.14 0.01 0.11 0.14 0.01 0.08 0.11 0.16 0.09 0.16 0.10
Financial deprivation -0.21 0.05 -0.18 -0.15 -0.21 -0.15 -0.19 -0.15 -0.28 -0.24 -0.21 -0.27 -0.15 -0.18 -0.27 -0.2 -0.32 -0.21
R2 0.05 0.04 0.25 0.04 0.1 0.09 0.08 0.06 0.09 0.1 0.1 0.08 0.04 0.15 0.22 0.1 0.21 0.07
Valid count 2694 988 724 2020 1568 1847 3563 2003 1124 4301 1156 3248 2682 1168 5138 3187 1800
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Coefficients with p-values lower than 0.01 in bold.
Due to rounding the numbers to two significant decimals, zeros may represent a value in the range of -0.005 to +0.005.
Model 2: Effects on HL of migration background, three
measures of socio-economic status, gender, and age
In Model 2, in addition to the five core socio-demographic and socio-economic determinants,

migration background was included (Table 6.6). As in Model 1, R2, on average, explains just 7% of
the variation in General HL, with the same range (between 4% (BE, CH, and NO) and 25% (BG))
across countries. Migration on average has a low predicting value (β=0.03) for General HL. Still,
there are a few countries where migration background is a significant or relevant predictor of HL
and migrants having somewhat better HL: CH (β=0.06, but not improving R2), AT (β=0.05, im-
proving R2 somewhat), while at least in two countries (IL: β=-0.06, but not improving R2; DE: β=-
0.04, not improving R2), migrants seem to have slightly lower HL for the given data. Thus, on
average the same relevant social gradient for General HL was demonstrated for the combined five
socio-demographic and socio-economic determinants with migration background added, with the
highest effects for financial deprivation and level in society but differing considerably by country.
Migration background did not improve the variance explained by the model and had, on average,
no relevant effect on General HL.
Table 6.6:
Model 2: Multivariable linear regression models of GEN-HL by five core social determinants and migration background (standardized coefficients (β) and R2), for
each country and for all countries (equally weighted)
Gender female 0.07 0.02 0.03 0.01 0.09 0.08 0.06 0.05 -0.03 0.05 0.08 0.05 0.07 -0.07 0.04 0.03 0.07 0.04
Age in years -0.06 0.03 -0.08 0.01 0.11 -0.11 0.07 0 0.12 0.06 0.12 -0.05 0 -0.15 -0.15 -0.13 -0.11 -0.04
Education -0.03 -0.06 0.11 -0.02 -0.14 0.12 0.03 -0.05 0.03 0.05 -0.08 0.01 0.04 0.08 0.02 0.03 0.04 -0.04
Level in society 0.01 0.19 0.25 0.08 0.13 0.1 0.1 0.15 0.01 0.11 0.13 0.01 0.09 0.12 0.16 0.09 0.16 0.11
Migration background 0.05 -0.03 0 0.06 0.01 -0.04 0 -0.01 0.01 0.03 -0.06 0 0.03 0.01 0 0.02 -0.03 0.03
R2 0.06 0.04 0.25 0.04 0.1 0.09 0.08 0.06 0.09 0.1 0.1 0.08 0.04 0.15 0.21 0.1 0.21 0.07
Valid count 2689 985 724 2009 1563 1822 3563 1969 1122 4277 1154 3248 2675 1168 5012 3164 1794
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
In BG, HU, IT, RU, and SK, the percentage of respondents with migration background was below 10% so values on migration should be treated with caution.
Model 3: Effects on HL of long-term illness, three
measures of socio-economic status, gender, and age

long-term illnesses/health problems were included (Table 6.7). In this model explained R2 is 8%
on average, varying from 4% (BE, CH, and NO) to 25% (BG). Compared with Model 1, there are slight
increases of explained R² in three countries, namely AT (from 5% to 6%), PT (from 15% to 18%),
and RU (from 21% to 24%). Long-term illnesses/health problems have, on average, β=-0.10, which
is, after financial deprivation (β=-0.19), the second strongest effect in the model. In all countries,
respondents with at least one chronic disease still have lower General HL when other relevant
predictors are controlled for, varying from β=-0.02 (BE) to β=-0.20 (PT).
By including long-term illnesses/health problems, the average β of financial deprivation, level in

society, and age decreased slightly compared to Base Model 1 and Model 2 including migration.
Thus, on average, the same relevant social gradient was demonstrated for General HL for the
combined five socio-demographic and socio-economic determinants with long-term ill-
nesses/health problems added. The highest effects were for financial deprivation and level in so-
ciety, but they differed considerably by country. Long-term illnesses/health problems did not im-
prove the explained variance in the model much but had, on average, the second highest signifi-
cant effect on General HL.
Table 6.7:
Model 3: Multivariable linear regression models of GEN-HL by five core social determinants and long-term illness (standardized coefficients (β) and R2), for
Gender female 0.07 0.02 0.02 0.01 0.09 0.07 0.06 0.04 -0.02 0.06 0.07 0.05 0.08 -0.06 0.05 0.03 0.07 0.04
Age in years -0.06 0.03 -0.06 0.02 0.12 -0.08 0.09 0.02 0.13 0.07 0.11 -0.04 0.01 -0.08 -0.1 -0.11 -0.09 -0.01
Education -0.04 -0.06 0.1 -0.02 -0.14 0.13 0.03 -0.04 0.03 0.05 -0.09 0.01 0.04 0.08 0.04 0.03 0.03 -0.04
Level in society 0 0.19 0.24 0.08 0.12 0.09 0.1 0.14 0 0.1 0.13 0 0.08 0.1 0.14 0.09 0.16 0.09
Long-term illness -0.08 -0.02 -0.07 -0.06 -0.05 -0.03 -0.07 -0.06 -0.07 -0.04 -0.05 -0.04 -0.04 -0.2 -0.12 -0.07 -0.05 -0.10
R2 0.06 0.04 0.25 0.04 0.1 0.09 0.08 0.06 0.09 0.1 0.1 0.08 0.04 0.18 0.24 0.11 0.21 0.08
Valid count 2686 988 705 2017 1568 1821 3557 2003 1121 4291 1155 3185 2663 1163 4910 3183 1774
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Model 4: Effects on HL of training in a healthcare pro-
fession, three measures of socio-economic status, gen-
der, and age

training in a healthcare profession was included (Table 6.8). As in Models 1 and 2, in this model
just 7% of General HL is explained on average, varying between 4% (BE) and 27% (BG). Again, after
financial deprivation (β=-0.21) and level in society (β=0.1), training in a healthcare profession is
the third strongest predictor in the model with an average of β=-0.08. In all but one country (AT),
respondents with training in a healthcare profession have higher HL when other relevant predictors
are controlled. The strength of this effect varies from β=-0.02 (IE) to β=-0.16 (BG). Thus, on
average, the same relevant social gradient for General HL was demonstrated for the combined five
socio-demographic and socio-economic determinants plus training in a healthcare profession,
with the highest effects for financial deprivation, and level in society, but differing considerably
by country. Training in a healthcare profession did not improve the variance explained by the
model but it had on average the third strongest significant effect on General HL.
Table 6.8:
Model 4: Multivariable linear regression models of GEN-HL by five core social determinants and training in a healthcare profession (standardized coefficients
(β) and R2), for each country and for all countries (equally weighted)
Gender female 0.07 0.02 0 -0.01 0.09 0.05 0.04 0.04 -0.03 0.05 0.07 0.05 0.05 -0.07 0.03 0.03 0.05 0.03
Age in years -0.08 0.03 -0.06 0.01 0.10 -0.09 0.07 0.01 0.11 0.06 0.10 -0.05 0 -0.15 -0.14 -0.13 -0.11 -0.04
Education -0.04 -0.06 0.07 -0.03 -0.15 0.11 0.02 -0.05 0.03 0.05 -0.10 0.01 0.03 0.08 0.02 0.03 0.01 -0.05
Level in society 0.01 0.20 0.24 0.08 0.13 0.10 0.11 0.13 0 0.11 0.13 0 0.08 0.11 0.16 0.09 0.16 0.10
No training in a health profession 0 -0.04 -0.16 -0.07 -0.06 -0.15 -0.09 -0.08 -0.04 -0.02 -0.06 -0.04 -0.06 -0.05 -0.08 -0.04 -0.11 -0.08
R2 0.05 0.04 0.27 0.04 0.1 0.11 0.09 0.06 0.09 0.1 0.1 0.08 0.05 0.16 0.22 0.11 0.22 0.07
Valid count 2678 988 718 2020 1568 1842 3560 2003 1123 4295 1156 3248 2676 1168 5116 3184 1783
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
The focus of this chapter is on two research questions: (1) Is there a social gradient for General HL? and
(2) How strong are potential effects of selected determinants on General HL?
In summary, the HLS19 demonstrated a weak social gradient for General HL in a model combining the
five core socio-demographic and socio-economic determinants (gender, age, education, level in society,
and financial deprivation), with financial deprivation showing the strongest negative effects and level in
society the strongest positive effects. The magnitude of the explained variance and regression coeffi-
cients varied considerably across countries, though. Inclusion of migration background, long-term ill-
nesses/health problems or training in a healthcare profession did not improve model performance, but
long-term illnesses/health problems and training in a healthcare profession showed the third highest
effect on General HL, after financial deprivation and level in society.
Limitations: Due to the limited standardization of the HLS19 study design (mainly the type and time of
data collection), alongside overall trends and the range in variation in the measured values across coun-
tries, comparisons of the results between individual countries have not been described or interpreted in
the International Report; differences in the results of countries must be interpreted with caution. Fur-
thermore, due to the cross-sectional design of the study, the causal assumptions underlying the speci-
fied regression models may not be met.
Comparison with the results of the HLS-EU is further limited by changes in the measure of General HL,
by changes in the measures of selected determinants, by changes in the method of data collection, and
by different countries being included in the two studies.
In contrast to the HLS-EU study, although the same independent variables were used in principle, the
explained variance in the base model, i.e., the strength of the social gradient, is considerably lower at
7% vs. 17% in the HLS-EU study, but the range of variation for the explained variance across countries is,
from 4% (BE, CH, and NO) to 25% (BG), rather similar to the HLS-EU, which ranged from 8% (NL) to 29%
(GR). In both studies, financial deprivation (ß=-0.21 vs. ß=-.24 in the HLS-EU study) and level in society
(ß=+0.1 vs. ß=+0.14 in the HLS-EU study) have the strongest effects for the mean of all countries but
with somewhat smaller values for the HLS19. In contrast to the HLS-EU and other studies, education had,
on average, with ß=-0.04, much smaller but opposite effects across countries (positive effects for 11
countries, statistically significant for three countries; negative effects for six countries, statistically sig-
nificant for three countries). Age, too, had, on average, with ß=-0.04, much smaller and opposite effects
for countries compared to the HLS-EU (0 for three countries; significant negative effects for eight coun-
tries; positive effects for six countries, significant for five). For the female gender the effect was, on
average, small but statistically significant (ß=0.04): in 15 countries women had higher General HL than
men (statistically significant in 11 countries) and in two countries it was the opposite (but not statistically
significant).
The differences found are probably due mostly to methodological differences in the two studies and
cannot be attributed to real changes in HL over time in European countries.
Conclusions
In the HLS19, which involved more and partly different countries in the WHO European Region, a social
gradient for HL was demonstrated. This adds to the existing evidence that HL is a relevant factor for
public health policy in all participating countries.
Financial deprivation was demonstrated as the strongest predictor of General HL in all countries, followed
by level of society. There was a trend for higher General HL in women, but the results for age and edu-
cation were inconsistent across countries. Further research is needed to investigate these inconsistent
results.
6.5 References
Almaleh, R.; Helmy, Y.; Farhat, E.; Hasan, H.; Abdelhafez, A. (2017): Assessment of health literacy among
outpatient clinics attendees at Ain Shams University Hospitals, Egypt: a cross-sectional study. In:
Public Health 151/:137-145
Berens, Eva-Maria; Vogt, Dominique; Messer, Melanie; Hurrelmann, Klaus; Schaeffer, Doris (2016): Health
literacy among different age groups in Germany: results of a cross-sectional survey. In: BMC Public
Health 16/1:1-8
Berkman; Sheridan, Stacey L; Donahue, Katrina E; Halpern, David J; Crotty, Karen (2011): Low health
literacy and health outcomes: an updated systematic review. In: Annals of internal medicine
155/2:97-107
Duong, T. V.; Aringazina, Altyn; Baisunova, Gaukhar; Nurjanah; Pham, Thuc V.; Pham, Khue M.; Truong,
Tien Q.; Nguyen, Kien T.; Myint Oo, Win; Mohamad, Emma; Tin Su, Tin; Huang, Hsiao-Ling; Sørensen,
Kristine; Pelikan, Jürgen M.; Van den Broucke, Stephan; Wushou Chang, Peter (2017): Measuring
health literacy in Asia: Validation of the HLS-EU-Q47 survey tool in six Asian countries. In: Journal
of Epidemiology 27/2:80-86
Friis, Karina; Lasgaard, Mathias; Rowlands, Gillian; Osborne, Richard H; Maindal, Helle T (2016): Health
literacy mediates the relationship between educational attainment and health behavior: a Danish
population-based study. In: Journal of health communication 21/sup2:54-60
Garcia-Codina, O.; Juvinya-Canal, D.; Amil-Bujan, P.; Bertran-Noguer, C.; Gonzalez-Mestre, M. A.; Ma-
sachs-Fatjo, E.; Santaeugenia, S. J.; Magrinya-Rull, P.; Salto-Cerezuela, E. (2019): Determinants of
health literacy in the general population: results of the Catalan health survey. In: BMC public health
2019/19:1122
HLS-EU Consortium (2012): Comparative Report on Health Literacy in Eight EU Member States (Second
Extended and Revised Version, Date July 22th, 2014). The European Health Literacy Survey HLS-EU,
Vienna
Jordan, S.; Hoebel, J. (2015): [Health literacy of adults in Germany: Findings from the German Health
Update (GEDA) study]. In: Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz
58/9:942-950
Kamal, Seyed Hossein Mohaqeqi; Basakha, Mehdi; Sajjadi, Homeira (2018): The Prevalence and Risk Fac-
tors of Limited Health Literacy in Iran: A Systematic Review and Meta-Regression Analysis. In: J Adv
Med Biomed Res 26/118:1-8
Levin-Zamir, Diane; Baron-Epel, Orna B; Cohen, Vicki; Elhayany, Asher (2016): The association of health
literacy with health behavior, socioeconomic indicators, and self-assessed health from a national
adult survey in Israel. In: Journal of health communication 21/sup2:61-68
Matsumoto, Masayoshi; Nakayama, Kazuhiro (2017): Development of the health literacy on social deter-
minants of health questionnaire in Japanese adults. In: BMC Public Health 17/1:30
Palumbo, Rocco; Annarumma, Carmela; Adinolfi, Paola; Musella, Marco; Piscopo, Gabriella (2016): The
Italian Health Literacy Project: Insights from the assessment of health literacy skills in Italy. In: Health
Policy 120/9:1087-94
Pelikan, Jürgen M.; Ganahl, Kristin (2017): Die europäische Gesundheitskompetenz-Studie: Konzept, In-
strument und ausgewählte Ergebnisse. In: Health Literacy Forschungsstand und Perspektiven. Hg. v.
Schaeffer, Doris; Pelikan, Jürgen M. hogrefe, Bern. S. 93-S. 125
Rikard, RV; Thompson, Maxine S; McKinney, Julie; Beauchamp, Alison (2016): Examining health literacy
disparities in the United States: a third look at the National Assessment of Adult Literacy (NAAL). In:
BMC Public Health 16/1:1-11
Schaeffer, Doris; Vogt, D ominique; Berens, Eva-Maria; Messer, Melanie; Quenzel, Gudrun; Hurrelmann,
Klaus (2017): Health Literacy in Deutschland. In: Health Literacy Forschungsstand und Perspektiven.
Hg. v. Schaeffer, Doris; Pelikan, Jürgen M. hogrefe, Bern. S. 129-S. 144
Sørensen, K.; Van den Broucke, S.; Fullam, J.; Doyle, G.; Pelikan, J.; Slonska, Z.; Brand, H.; Consortium
Health Literacy Project, European (2012): Health literacy and public health: a systematic review and
integration of definitions and models. In: BMC Public Health 12/80:http://www.biomedcen-
tral.com/1471-2458/1412/1480
Sørensen, K.; Pelikan, J. M.; Rothlin, F.; Ganahl, K.; Slonska, Z.; Doyle, G.; Fullam, J.; Kondilis, B.; Agrafi-
otis, D.; Uiters, E.; Falcon, M.; Mensing, M.; Tchamov, K.; Broucke, S. V.; Brand, H.; HLS-EU Consor-
tium (2015): Health literacy in Europe: comparative results of the European health literacy survey
(HLS-EU). In: European journal of public health:1-6
Stormacq, Coraline; Van den Broucke, Stephan; Wosinski, Jacqueline (2019): Does health literacy mediate
the relationship between socioeconomic status and health disparities? Integrative review. In: Health
Promotion International 34/5:e1-e17
Svendsen, Majbritt Tang; Bak, Carsten Kronborg; Sørensen, Kristine; Pelikan, Jürgen; Riddersholm, Signe
Juul; Skals, Regitze Kuhr; Mortensen, Rikke Nørmark; Maindal, Helle Terkildsen; Bøggild, Henrik;
Nielsen, Gitte (2020): Associations of health literacy with socioeconomic position, health risk behav-
ior, and health status: a large national population-based survey among Danish adults. In: BMC Public
Health 20/:1-12
Tiller, Daniel; Herzog, Beatrice; Kluttig, Alexander; Haerting, Johannes (2015): Health literacy in an urban
elderly East-German population–results from the population-based CARLA study. In: BMC Public
Health 15/:883
van der Heide; Rademakers, Jany; Schipper, Maarten; Droomers, Mariël; Sørensen, Kristine; Uiters, Ellen
(2013): Health literacy of Dutch adults: a cross sectional survey. In: BMC Public Health 13/:179
van der Heide; Wang, J.; Droomers, M.; Spreeuwenberg, P.; Rademakers, J.; Uiters, E. (2013b): The rela-
tionship between health, education, and health literacy: results from the Dutch Adult Literacy and
Life Skills Survey. In: J Health Commun 18 Suppl 1/:172-84, DOI:
10.1080/10810730.10812013.10825668
Vogt, D.; Schaeffer, D.; Messer, M.; Berens, E. M.; Hurrelmann, K. (2017): Health literacy in old age: results
of a German cross-sectional study. In: Health Promot Int 1-9/:doi: 10.1093/heapro/dax1012
7 General Health Literacy as a predictor of
health behaviors and lifestyles
Authors:

Thomas Link (ICC)
Nejc Berzelak (SI)
Jorge Oliveira (PT)
Mitja Vrdelja (SI)
7.1 Background
Previous research
The association of HL with different indicators of health or lifestyle-related behaviors has been
researched and presented, especially for certain indicators, in international publications and in the
HLS-EU and its follow up studies; therefore, relevant research questions have been included in the
HLS19 as well. These research questions on identifying the relevance of HL for health and health
policy focus on establishing whether HL has a significant effect on these behaviors and not on
explaining the variance of these behaviors. The HLS19 is interested in the effects of HL and not in
these behaviors specifically.
Previous research showed that higher functional HL increases the likelihood of pursuing health-
promoting behaviors such as eating fruit and vegetables, being a non-smoker, and having a better
health perception, when controlling for socio-demographic and socio-economic data (Von Wagner
et al. 2007). Already the original HLS-EU study, and also some of its follow-up studies to a certain
extent, demonstrated significant Spearman correlations for HL with some behaviors. These corre-
lations were highest for physical activity (more physical activity/higher HL; ρ=-.19, variation
across countries from ρ=-.04 to ρ=-.21), followed by BMI (lower BMI/higher HL; ρ=-.07, variation
from ρ=.03 to ρ=-.13 but significant only in four (AT, DE, EL, and PL) out of the eight countries).
The picture was not so clear for alcohol consumption (the more “light” and the less “no” alcohol
consumption/higher HL; ρ=.07, significant only in 3 (ρ=.07 (PL), ρ=.10 (BG), ρ=.16 (EL)) out of
the eight countries) and for smoking behavior (ρ=-.01, not significant on average, inconsistent
for sign of correlation across countries, and significant only in four (ρ=.13 (BG), ρ=.09 (EL), ρ=-
.08 (DE/NRW), -.09 (IE)) out of the eight countries) (HLS-EU-Consortium 2012; Pelikan/Ganahl
2017b). Using multivariable linear regression analyses in models that explained on average 8%
(varying across countries from 1% to 10%) for physical activity, it was demonstrated that HL had
on average the highest significant effect with β=-.13 (varying across countries from β=-.06 to
β=-.18) (Pelikan/Ganahl 2017b).
Later studies also showed that individuals with adequate levels of self-reported HL were more
likely to be involved in physical activity (Fernandez et al. 2016) and better HL was also associated
with greater fruit and vegetable intake (Lim et al. 2017). Low levels of HL, in contrast, are associ-
ated with physical inactivity, unhealthy dietary habits, underweight and obesity, and daily smoking
(Aaby et al. 2017). Similar findings concerning the level of physical activity and body weight were
obtained by Svendsen et al. (2020), who used the HLS-EU-Q16 for measuring HL. Thus, there exist
consistent results for a positive effect of HL on physical activity and on dietary habits (fruit and
vegetable intake) as well as BMI in most studies, but the results for alcohol consumption and
smoking are inconsistent.
Chapter 7 / General health literacy as a predictor of health behaviors and lifestyles 149
Health behaviors and lifestyles variables in the HLS19 in comparison with the HLS-EU
An influence of health behaviors and lifestyles on HL was hypothesized by the HLS19 study, thereby
building on the HLS-EU study’s conceptual model of HL (Sørensen et al. 2012) and the refined
Vienna Model of Health Literacy (Pelikan/Ganahl 2017a) (see Figure 1.2, Chapter 1). The leading
research question in the HLS19 was whether there is a relevant and significant effect of General HL
on selected types of health behaviors and lifestyles (and not how variance in these behaviors can
best be explained.) However, in a cross-sectional study like the HLS19, this question can only be
answered in a limited, explorative way because the underlying causal assumptions cannot be
tested empirically with the existing set of data.
In the HLS19 four of the health behaviors and lifestyles variables which had already been used in
the HLS-EU (albeit with somewhat different indicators) were implemented. Additionally, fruit and
vegetable consumption was included as a further lifestyle variable. The precise wording of the
items on health behaviors and lifestyles used in the HLS19, the source of the original item, and
indications of changes made relating to the item used in the HLS-EU are shown in Annex 7, Table
A 7.1.
Summing up, the five variables used in the HLS19 were BMI, smoking behavior, alcohol consump-
tion, physical activity, and fruit and vegetable consumption. Apart from BMI, their measurement
was standardized as the number of days per week that respondents usually practice these be-
haviors, grouped as follows:
» never --> never

» less than one day per week & 1 day --> occasional use
» 2 days & 3 days --> light use
» 4 days & 5 days & 6 days & 7 days --> heavy use.
The data analyses and results are presented by:
» percentage distributions of health behaviors and lifestyles indicators (Annex 7),

» associations of health behaviors and lifestyles indicators with General HL (for all countries in
the main chapter and for each country in Annex 7),
» Spearman correlations (ρ) among indicators of health behaviors and lifestyles, with General
HL and selected socio-demographic and socio-economic determinants (for all countries in
the main chapter and for each country in Annex 7),
» regression analyses: R² and β coefficients of multivariable linear regression models are used
to compare the relative importance of the predictors included (results in the main chapter).
In Annex 7 results on regression models with unstandardized coefficients (b) are also pro-
vided.
7.2 Spearman correlations among indicators of health behav-
iors and lifestyles, with General HL and selected socio-
demographic and socio-economic determinants
Of the five indicators for health behaviors and lifestyles, physical activity and fruit and vegetable
consumption are highest positively correlated (ρ=0.23), followed by smoking behavior and alcohol
consumption (ρ=0.16) as well as alcohol consumption and physical activity (ρ=0.08). There were
somewhat smaller and negative correlations between smoking behavior and fruit and vegetable
consumption (ρ=-0.08) as well as between BMI and physical activity (ρ=-0.1), and BMI and fruit
and vegetable consumption (ρ=-0.05) (Figure 7.1).
Figure 7.1:
Spearman correlations (ρ) among indicators of health behaviors and lifestyles, with GEN-HL, and
selected socio-demographic and socio-economic determinants, for all countries (equally
weighted)*
* Values for the correlations with migration should be treated with caution as in five countries (BG, HU, IT, RU,
and SK) the percentage of respondents with a migration background was below 10%, and the HLS19 survey as
such was not adjusted to targeting migrants specifically (e.g., by offering translations of the instrument into
migrant languages).
On average, General HL correlates positively with “fruit and vegetable consumption” (0.11) and
“physical activity” (0.11), while just very weakly and negatively with “BMI” (-0.04), “alcohol con-
sumption” (-0.05), and “smoking behavior” (-0.03) (Figure 7.1). In fact, these indicators of health
behaviors and lifestyles mostly have a higher correlation with socio-demographic and socio-eco-
nomic indicators than with General HL (Figure 7.1).
7.3 General HL and BMI
For BMI on average, 2% of the respondents (from 1% (DK, PT, and SI) to 4% (BG, FR)) were classified
as underweight, 44% (from 33% (CZ) to 56% (CH)) as normal weight, 36% (29% (CH) to 44% (PT)) as
pre-obese, 13% (from 9% (CH) to 19% (CZ)) as obese class I, 3% (from 1% (PT) to 6% (BE, CZ, HU)),
as obese class II, and 2% (from 0% (SK) to 4% (DK) as obese class III. Thus, there is quite some
variation across countries (Annex 7, Figure A 7.1).
On average there is a rather linear, slight, negative relationship between BMI and General HL while
higher HL is associated with a decrease in the BMI group of obesity class I (Figure 7.2), but this
relationship is rather different for individual countries (Annex 7, Figure A 7.2).
Figure 7.2:
Percentage distribution of six categories of BMI by GEN-HL (10 groups from lowest HL to highest
HL), for all countries (equally weighted)
BMI is mainly correlated with socio-demographic and socio-economic factors (Annex 7, Table
A7.2), i.e.,
» age (ρ=0.25, varying from 0.1 (DK) to 0.42 (RU, SK)),

» female gender (ρ=-0.17, varying from -0.01 (IL) to -0.33 (BG)),
» education (ρ=-0.12, varying from -0.02 (BG, NO) to -0.22 (PT),
» financial deprivation (ρ=0.09 varying from 0.02 (CZ, NO) to 0.12 (CH)), and
» level in society (ρ=-0.07, varying from -0.01 (IE) to -0.13 (SI)).
In contrast, there is only a very weak association with General HL (ρ=-0.04, varying from +0.02
(BE, IL) to -0.12 (SK)), but also for migration (on average ρ=-0.01 but varying inconsistently with
positive or negative correlations for different countries) and for training as a healthcare profession
(on average ρ=0.05, mostly with low positive correlations varying from 0.02 (DK, IE) to 0.13 (BG),
but a few countries also have very low negative correlations). Therefore, migration and training as
a health professional were not used in the regression analyses (Annex 7, Table A 7.2).
Using a multivarible linear regression model including the five core socio-demographic and socio-economic indicators and General HL as predictors and BMI as
a dependent variable, on average just 5% of the variance is explained by the model (varying from 1% (IL) to 14% (RU)). With an average β=-0,01 (varying
inconsistently on a low level for countries), General HL is the predictor with the lowest β in the model, after level in society (β=-0.03), education (β=0.05),
financial deprivation (β=0.06), gender female (β=-0.11), and age (β=0.15) (Table 7.1). For the same model with unstandardized coefficients (b), see Annex Table
A 7.3.
In conclusion, there is, on average, no relevant and significant effect of General HL on BMI, and such an effect is only demonstrated for a few countries.
Table 7.1:
Multivariable linear regression models of BMI by GEN-HL and five core social determinants (standardized coefficients (β) and R2), for each country and for all
GEN-HL -0.06 0.02 0 -0.06 -0.04 -0.04 -0.06 -0.01 -0.01 -0.04 -0.01 -0.03 -0.02 0.06 0 0.01 -0.03 -0.01
Gender female -0.19 -0.05 -0.2 -0.21 -0.12 -0.18 -0.05 -0.1 -0.16 -0.11 0.06 -0.13 -0.14 -0.12 -0.02 -0.1 -0.24 -0.11
Age in years 0.14 0.18 0.09 0.15 0.26 0.2 0.05 0.2 0.17 0.11 0.08 0.16 0.13 0.15 0.37 0.1 0.37 0.15
Education -0.06 -0.1 -0.02 -0.06 -0.08 -0.05 -0.04 -0.05 -0.11 -0.04 -0.03 -0.02 -0.04 -0.12 -0.06 -0.07 -0.06 -0.05
Level in society -0.05 -0.06 -0.02 0.04 -0.04 -0.04 -0.05 -0.05 -0.02 0 -0.02 -0.04 -0.07 0 -0.04 -0.03 -0.01 -0.03
Financial deprivation 0.09 0.09 0.09 0.15 0.03 0.06 0.1 0.05 0.05 0.04 0.02 0.1 0.04 0.06 -0.03 0.03 -0.04 0.06
R2 0.08 0.06 0.08 0.09 0.11 0.09 0.04 0.07 0.07 0.03 0.01 0.06 0.05 0.08 0.14 0.04 0.2 0.05
Valid count 2598 973 723 2007 1459 1726 3459 1892 1111 4002 1103 3229 2572 1167 4829 3138 1747
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
BMI: used as continuous variable.
GEN-HL score: from 0=minimal HL to 100=maximal HL.
154 © GÖG 2021, HLS19 Draft International Report

7.4 General HL and smoking behavior
On average, taking the mean for all countries, 75% (from 59% (BG) to 86% (NO)) of respondents
were classified as never smoking, 4% (from 2% (DE, HU) to 7% (IT)) as occasional smokers, 2% (from
0% (PT) to 4% IT)) as light smokers, and 19% (from 10% (NO) to 34% (BG) as heavy smokers. Thus,
across countries there is considerable variation (Annex 7, Figure A 7.3).
On average, the association of the intensity of smoking behavior with General HL is very low, and
rather negative (Figure 7.3). Respondents with higher General HL smoke somewhat less heavily,
but the association differs widely across individual countries (Annex 7, Figure A 7.4).
Figure 7.3:
Percentage distribution of four categories of smoking behavior by GEN-HL (10 groups from
lowest HL to highest HL), for all countries (equally weighted)
On average with ρ=-0.03, General HL is very slightly correlated with smoking behavior, negatively
for most countries and varying from ρ=-0.01 (BG) to ρ=-0.08 (HU), but for a few countries posi-
tively to ρ=0.08 (PT) and for two countries just with ρ=0 (FR, IL, SI). Thus, smoking behavior has
a much lower correlation with General HL than with the socio-demographic and socio-economic
indicators (Annex 7, Table A 7.4).
Chapter 7 / General Health Literacy as a predictor of health behaviors and lifestyles 155
With, on average, R2 of 4% (varying across countries from 1% (BG, FR) to 14% (RU)), the model explains even less variance than the model for BMI. The predictive
value of General HL on average is β=0, due to inconsistent results across countries. Some countries show a negative β value from -0.01 (DK, HU, IE, and NO) to
-0.06 (AT, SK) and a few countries even have a positive β value up to 0.09 (PT) (Table 7.2). (For the same model with unstandardized coefficients (b), see Annex
Table A 7.5). Thus, no consistent and relevant effect of General HL on smoking behavior was determined for the countries on average, but there were opposite
and significant effects in a few countries.
Table 7.2:
Multivariable linear regression models of smoking behavior by GEN-HL and five core social determinants (standardized coefficients (β) and R2), for each country
GEN-HL -0.06 0 0.03 0.01 -0.03 -0.04 -0.01 0.02 -0.01 -0.01 0.04 0.08 -0.01 0.09 -0.04 0.04 -0.06 0
Gender female -0.12 -0.07 -0.04 -0.12 -0.13 -0.11 0 -0.03 -0.07 -0.04 -0.09 -0.08 -0.03 -0.19 -0.32 -0.08 -0.16 -0.1
Age in years -0.12 -0.11 -0.08 -0.11 -0.12 -0.14 -0.02 -0.05 -0.22 -0.17 -0.16 -0.14 0.05 -0.1 -0.16 -0.2 -0.13 -0.12
Education -0.1 -0.06 -0.01 -0.04 -0.19 -0.04 -0.06 -0.04 -0.09 -0.13 -0.18 -0.04 -0.11 0 -0.1 -0.12 -0.11 -0.08
Level in society 0.07 -0.07 0.02 -0.05 -0.02 -0.1 -0.02 0 -0.12 -0.07 0.02 0 -0.05 -0.08 -0.04 -0.01 -0.02 -0.03
Financial deprivation 0.05 0 0.05 0.13 0.07 0.02 0.12 0.1 0.06 0.13 0.05 0.13 0.12 0.04 0.03 0.08 -0.04 0.06
R2 0.05 0.03 0.01 0.06 0.06 0.05 0.03 0.01 0.08 0.07 0.07 0.05 0.04 0.06 0.14 0.06 0.06 0.04
Valid count 2691 988 715 2019 1567 1842 3539 2003 1124 4301 1155 3243 2680 1168 5081 3180 1791
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Smoking behavior: from (1) never to (4) heavy.

7.5 General HL and alcohol consumption
On average 29% (from 8% (BG) to 52% (RU)) of the respondents were classified as never consuming
alcohol, 41% (from 33% (IT) to 46% (NO, PT)) as occasional consumers, 18% (from 6% (PT, RU) to
27% (DE)) as light consumers, and 13% (from 1% (RU) and 5% (IL, NO) to 26% (BG)) as heavy con-
sumers. Thus, distribution varies considerably across countries (Annex 7, Figure A 7.5).
The relationship of the frequency of alcohol consumption and General HL is rather curvilinear, e.g.,
higher percentages of those who never consume alcohol are found in both the lowest and the
highest HL groups (Figure 7.4). The association for individual countries differs widely (Annex 7,
Figure A 7.6).
Figure 7.4:
Percentage distribution of four categories of alcohol consumption by GEN-HL (10 groups from
The correlation with General HL is, on average, with ρ=-0.05, the second lowest of all correlations
and varies inconsistently, i.e., for some countries negatively from ρ=-0.01 (HU) to ρ=-0.13 (IL)
and for others positively from ρ=0 (CH, PT) to ρ=+0.05 (BE, DK). Thus, the intensity of alcohol
consumption is much weaker related to General HL than with socio-demographic and socio-eco-
nomic indicators (Annex 7, Table A 7.6).
The regression model explains on average 8% of the variation in alcohol consumption (varying from 5% (IE, NO) to 16% (PT)). General HL is on average the predictor
with the second lowest β=-0.05 and varies inconsistently, for some countries with a negative β ranging from -0.01 (IE, SK) to -0.13 (IL) and for a few with a
positive β ranging from 0 (FR) to 0.09 (PT) (Table 7.3). For the same model with unstandardized coefficients (b), see Annex 7, Table A 7.7.
Thus, just a small significant effect of General HL on alcohol consumption was demonstrated for the average of countries, with expected significant effects for
four countries and one unexpected significant effect for one country.
Table 7.3:
Multivariable linear regression models of alcohol consumption by GEN-HL and five core social determinants (standardized coefficients (β) and R2), for each
GEN-HL -0.05 0.02 -0.02 0.01 -0.06 -0.06 -0.02 0 -0.03 -0.01 -0.13 0.01 -0.03 0.09 -0.07 -0.03 -0.01 -0.05
Gender female -0.25 -0.15 -0.2 -0.21 -0.33 -0.24 -0.1 -0.18 -0.35 -0.09 -0.16 -0.25 -0.13 -0.35 -0.24 -0.31 -0.3 -0.22
Age in years 0.08 0.16 0.15 0.21 -0.1 0.03 0.25 0.19 0.03 0.06 0 0.03 0.09 0.13 -0.12 0 -0.03 0.1
Education 0.08 0.14 0.12 0.12 0.09 0.05 0.1 0.05 0.04 0.19 0.12 0.07 0.13 -0.04 0.04 0.08 0.07 0.1
Level in society 0.05 0.13 0.03 0 0 0.04 0.11 0.06 -0.03 -0.01 -0.03 0 0.06 0.03 -0.05 0.01 0.04 0.03
Financial deprivation -0.05 -0.03 -0.07 -0.1 -0.06 -0.02 -0.04 -0.03 -0.08 -0.06 -0.06 -0.04 -0.03 0.03 -0.01 -0.02 -0.01 -0.07
R2 0.09 0.09 0.12 0.13 0.13 0.07 0.15 0.08 0.14 0.05 0.06 0.07 0.05 0.16 0.08 0.1 0.1 0.08
Valid count 2690 988 705 2019 1565 1818 3544 2003 1123 4301 1155 3239 2673 1168 5084 3169 1789
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Alcohol consumption: from (1) never to (4) heavy.

7.6 General HL and physical activity
On average 11% (from 2% (DK) to 28% (CZ)) of respondents were categorized as never performing
physical activity, 17% ( from 5% (IE) to 28% (PT)) as occasional performers, 26% (from 21% (HU, IE,
and PT) to 32% (DE)) as light performers, and 46% (from 22% (CZ) to 71% (IE)) as heavy performers.
Thus, there is considerable variation across countries in relation to the extent of physical activity
(Annex 7, Figure A 7.7).
On average, a positive, rather linear relationship can be observed for the extent of physical activity
and General HL. Respondents with higher General HL have a higher intensity of physical activity
(Figure 7.5). This was found in most countries, with just a few deviations (Annex 7, Figure A 7.8).
Figure 7.5:
Percentage distribution of four categories of physical activity by GEN-HL (10 groups from lowest
HL to highest HL), for all countries (equally weighted)
Compared to the other lifestyle indicators, there is, on average, a somewhat higher association of
physical activity with General HL, with ρ=0.11, varying from 0.04 (FR, HU) to 0.2 (SK). This
association is therefore the highest one, compared to the correlations with the socio-
demographic, socio-economic, and other indicators (Annex 7, Table A 7.8).
The regression model explains on average just 3% of the variance (varying from 1% (IE) to 9% (SK)). However, in this case, General HL is by far the predictor with
the highest ß (on average β=0.11, varying from 0.03 (FR) to 0.27 (BG)) (Table 7.4). (For the same model with unstandardized coefficients (b), see Annex 7, Table
A 7.9).
Thus, General HL was revealed to be a significant and relevant predictor of the extent of physical activity for most countries, with very few exceptions.
Table 7.4:
Multivariable linear regression models of physical activity by GEN-HL and five core social determinants (standardized coefficients (β) and R2), for each country
GEN-HL 0.1 0.11 0.27 0.04 0.09 0.09 0.1 0.03 0.08 0.08 0.04 0.12 0.09 0.07 0.14 0.18 0.19 0.11
Gender female 0.03 -0.03 -0.04 -0.05 -0.03 -0.07 0.04 -0.01 -0.1 -0.04 -0.01 -0.04 0.06 -0.03 -0.07 -0.05 -0.1 -0.03
Age in years -0.12 0 0.01 0.04 -0.13 -0.09 0.02 0.06 -0.07 0.03 0.04 -0.03 0.01 -0.11 -0.11 -0.05 -0.07 -0.02
Education 0.04 0.03 0.05 0 0.03 0.06 0.03 0.09 0.02 0.05 -0.01 0.06 0.01 0.08 0 0.02 0.04 0.06
Level in society 0 0.07 -0.09 0.01 0.07 0 0 0.05 0.02 0.06 0.13 0.04 0.02 0.12 0.06 0.05 0.11 0.05
Financial deprivation -0.03 -0.04 0.09 -0.1 -0.03 -0.05 -0.05 -0.05 0.04 0.02 -0.1 -0.05 -0.04 0 -0.03 0.05 0.02 -0.05
R2 0.03 0.02 0.07 0.02 0.04 0.03 0.02 0.02 0.02 0.01 0.04 0.04 0.02 0.07 0.06 0.04 0.09 0.03
Valid count 2685 988 698 2018 1566 1813 3543 2003 1122 4301 1155 3237 2673 1168 4873 3170 1784
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Physical activity: from (1) never to (4) heavy.

7.7 General HL and fruit and vegetable consumption
On average just 1% (from 0% (AT, BG, DE, NO, and PT) to 2% (FR, IL, and IT)) of respondents were
classified as never consuming fruit and vegetables, 6% (from 2% (IE; NO) to 11% (CZ, IL)) as occa-
sional consumers, 15% (from 8% (PT) to 24% (CZ)) as light consumers, and 78% (from 65% (CZ) to
88% (NO)) as heavy consumers. This distribution varies moderately across countries (Annex 7,
Figure A 7.9).
A positive, rather linear relationship between the frequency of fruit and vegetable consumption
and General HL was found on average. Respondents with higher HL consume more fruit and
vegetables (Figure 7.6). This trend was found im most of the participating countries, but with a
few irregularities for some countries (Annex 7, Figure A 7.10).
Figure 7.6:
Percentage distribution of four categories of fruit and vegetable consumption by GEN-HL (10
groups from lowest HL to highest HL), for all countries (equally weighted)
General HL on average is relatively highly associated with fruit and vegetable consumption, with a
mean ρ of 0.11, varying from 0.05 (IE) to 0.17 (RU, SK), except for ρ=-0.02 (PT). Fruit and vege-
table consumption is as highly correlated with General HL as with gender female (ρ=0.11), level
in society (ρ=0.11), and financial deprivation (ρ=-0.11) (Annex 7, Table A 7.10).
This multivariable linear regression model only explains 4% of the variance in fruit and vegetable consumption (varying from 4% (HU, IE, PT, and SI) to 9% (SK)).
However, General HL is, on average, with a value of β=0.09 (varying from -0.01 (PT) to 0.18 (SK)), the predictor with the second highest ß in the model, after
gender female (β=0.11) (Table 7.5). (For the same model with unstandardized coefficients (b), see Annex 7, Table A 7.12).
Thus, General HL was revealed to be a relevant and significant predictor of fruit and vegetable consumption for most countries, with a few exceptions.
Table 7.5:
Multivariable linear regression models of fruit and vegetable consumption by GEN-HL and five core social determinants (standardized coefficients (β) and R2),
for each country and for all countries (equally weighted)
GEN-HL 0.12 0.07 0.12 0.03 0.01 0.07 0.07 0.04 0.04 0.03 0.06 0.13 0.05 -0.01 0.12 0.1 0.18 0.09
Gender female 0.14 0.16 0.06 0.13 0.17 0.19 0.17 0.11 0.1 0.14 0.03 0.06 0.17 0.14 0.06 0.09 0.12 0.11
Age in years -0.07 0.09 0.06 0.11 0.1 0.05 0 0.16 0.12 -0.04 0.16 0.16 0.02 0.15 0.05 0.11 0.05 0.08
Education 0.04 0.12 0.1 0.09 0.06 0.07 0.14 0.15 0.07 0.11 0 0.06 0.11 0.01 0.12 0.02 0.07 0.06
Level in society 0 0.14 0.1 0.07 0.09 0.05 0.06 0.05 0.05 0.04 0.09 -0.03 0.04 0.04 0.08 0.11 0.1 0.06
Financial deprivation -0.05 -0.05 0.07 -0.12 -0.07 -0.06 -0.06 -0.08 -0.08 -0.03 -0.06 -0.05 -0.06 -0.04 -0.11 0.01 -0.02 -0.06
R2 0.05 0.08 0.05 0.07 0.05 0.06 0.08 0.07 0.04 0.04 0.06 0.05 0.06 0.04 0.07 0.04 0.09 0.04
Valid count 2692 988 716 2020 1568 1831 3555 2003 1123 4301 1156 3232 2679 1168 4884 3169 1790
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Fruit and vegetable consumption: from (1) never to (4) heavy.
Of the five potential health behavior and lifestyle indicators examined, two showed relevant and signif-
icant correlations and regression coefficients with General HL on average and for a majority of the par-
ticipating countries: physical activity (ρ=0.11, varying from 0.04 (FR, HU) to 0.20 (SK); on average
β=0.11, varying from 0.03 (FR) to 0.27 (BG)) and fruit and vegetable consumption (ρ=0.11, varying from
0.05 (IE) to 0.17 (RU, SK); on average β=0.09, varying from -0.01 (PT) to 0.18 (SK)).
For the other three indicators there is no or only a small relevant significant effect for General HL. For
BMI, on average, a significant effect is only demonstrated for very few countries. There is no consistent
and relevant effect of General HL on smoking behavior and alcohol consumption for the countries on
average, but opposite significant effects were found for a few countries. These results are, in principle,
consistent with the results highlighted from a few earlier studies (cf. Section 7.1).
Thus, the research questions could be answered positively only to a different and partly limited degree
by the HLS19 results, as in earlier studies, but the results confirmed that HL is a relevant determinant of
more physical activity and more fruit and vegetable consumption for all countries as well as of better BMI
and less alcohol consumption for some countries. It therefore matters for health policy.
Limitations: Due to only cross-sectional data being available, the causal assumptions underlying the
specified models could not be tested empirically. Due to the limited standardization of the HLS19 study
design (mainly the type and time of data collection), alongside the range in variation across countries,
comparisons of the results between individual countries have not been described or interpreted in the
International Report and must be interpreted with caution when undertaken selectively by countries
themselves. Comparison with the results of the HLS-EU is further limited by changes in the measure of
General HL, by changes in all measures of health behaviors and lifestyle indicators as well as in one of
the core determinants, by changes in the method of data collection, and by mostly different countries
included in the two studies. In comparison with the HLS-EU, just four indicators were measured and
measured differently for health behaviors and lifestyles which can be compared in principle: physical
activity (with the strongest significant bivariate association also in a multivariable linear regression
model); body-mass index (BMI); alcohol consumption (with significant slight associations); and smoking
behavior (with no or inconsistent associations). Thus, the HLS19 to a certain degree confirms the results
of the HLS-EU.
Comparison with other published results, as highlighted in the background section, is even more difficult
due to more differences in the methodology used, but the results found in earlier studies were also partly
confirmed by the HLS19 results.
In conclusion, as far as this is possible with data from a cross-sectional study design, the HLS19 enriched
the evidence for more countries in the WHO European Region that General HL is relevant for some health
behaviors and lifestyles indicators and, therefore, also relevant for health policy and practice in the WHO
European Region.
7.9 References
Aaby, A.; Friis, K.; Christensen, B.; Rowlands, G.; Maindal, H. T. (2017): Health literacy is associated with
health behaviour and self-reported health: A large population-based study in individuals with car-
diovascular disease. In: European journal of preventive cardiology 24/17:1880-1888
Fernandez, Dena M; Larson, Janet L; Zikmund-Fisher, Brian J (2016): Associations between health literacy
and preventive health behaviors among older adults: findings from the health and retirement study.
In: BMC public health 16/1:1-8
HLS-EU-Consortium (2012): Comparative Report on Health Literacy in Eight EU Member States. The Eu-
ropean Health Literacy Survey. The international Consortium of the HLS-EU Project, Online
Lim, Siew; Beauchamp, Alison; Dodson, Sarity; O’Hara, Jonathan; McPhee, Crystal; Fulton, Alexandra;
Wildey, Carol; Osborne, Richard H (2017): Health literacy and fruit and vegetable intake in rural Aus-
tralia. In: Public Health Nutrition 20/15:2680-2684
Pelikan, Jürgen M.; Ganahl, Kristin (2017a): Die europäische Gesundheitskompetenz-Studie: Konzept,
Instrument und ausgewählte Ergebnisse. In: Health Literacy Forschungsstand und Perspektiven. Hg.
v. Schaeffer, Doris; Pelikan, Jürgen M. hogrefe, Bern. S. 93-S. 125
Pelikan, Jürgen M.; Ganahl, Kristin (2017b): Measuring Health Literacy in General Populations: Primary
Findings from the HLS-EU Consortium´s Health Literacy Assessment Effort. In: IOS Press. Hg. v. Lo-
gan, G. D.; Siegel, Elliot R. S. 34-S. 59
Health Literacy Project, European (2012): Health literacy and public health: a systematic review and
integration of definitions and models. In: BMC Public Health 12/80:http://www.biomedcen-
tral.com/1471-2458/1412/1480
Svendsen, Majbritt Tang; Bak, Carsten Kronborg; Sørensen, Kristine; Pelikan, Jürgen; Riddersholm, Signe
Juul; Skals, Regitze Kuhr; Mortensen, Rikke Nørmark; Maindal, Helle Terkildsen; Bøggild, Henrik;
Nielsen, Gitte (2020): Associations of health literacy with socioeconomic position, health risk behav-
ior, and health status: a large national population-based survey among Danish adults. In: BMC Public
Health 20/:1-12
Von Wagner, Christian; Knight, Katherine; Steptoe, Andrew; Wardle, Jane (2007): Functional health liter-
acy and health-promoting behaviour in a national sample of British adults. In: Journal of Epidemiol-
ogy & Community Health 61/12:1086-1090
8 General Health Literacy as a predictor of health
status
Authors:
Thomas Link (ICC)
Nejc Berzelak (SI)
Jorge Oliveira (PT)
Mitja Vrdelja (SI)
Chapter 8 / General Health Literacy as a predictor of health status 165

8.1 Background
Previous research
The association of HL with different indicators of health status has been researched and presented in
international publications, especially in relation to certain indicators, as well as in the HLS-EU and its
follow up studies, so relevant research questions were included in the HLS19 as well. Previous empirical
research suggests that the main role of HL is in explaining variance in perceived health status. Evidence
from two systematic reviews supports the independent contribution of HL to perceived health status and
suggests a possible mediating role for HL (Berkman et al. 2011; Mantwill et al. 2015). Besides it being a
determinant, a limited contribution of HL as a moderator or mediator of self-perceived health status was
also demonstrated using the original HLS-EU data (Pelikan et al. 2018). A recent review highlighted a
mediating function in the relationship between socio-economic factors and health outcomes, health-
related behaviors, quality of life, and self-rated health status (Stormacq et al. 2019).
In the HLS-EU study, three indicators were taken from the Minimum European Health Module (MEHM) to
measure an individual’s health status, all of which showed significant correlations with HL: self-perceived
health (ρ=-.27 on average, varying by country from ρ=-.15 to ρ=-.33), number of long-term illnesses
or health problems (ρ=.16 on average, varying from ρ=.05 to ρ=.26), and the intensity of disease-
related limitations (ρ=.17 on average, from ρ=.08 to ρ=.32). In a multivariable linear regression model
with five socio-demographic and socio-economic core determinants, the NVS for functional HL, and the
HLS-EU-Q47 for comprehensive HL as independent variables of self-perceived health which, on average,
explained 27% of the variance (from 9% to 45%), the comprehensive HLS-EU-Q47 was the second highest
predictor of self-perceived health (β=.17 on average, varying from β=.09 to β=.21) after age (β=-.37)
(HLS-EU Consortium 2012; Pelikan/Ganahl 2017b; Sørensen et al. 2015).
In a cross-sectional survey in Israel using an instrument based on the HLS-EU-Q16, Levin-Zamir et al.
(2016) found that HL and age were the highest predictors of perceived health status among adults. A
recent nationwide study in Denmark showed that individuals with lower HL, as measured by the HLS-EU-
Q16, reported poorer health status and more compensation benefits for being on sick leave as a proxy
measure of health status (Svendsen et al. 2020).
Therefore, in the HLS-EU conceptual model of HL (Sørensen et al. 2012) and the refined Vienna Model of
Health Literacy (see Figure 1.2, Chapter 1) (Pelikan/Ganahl 2017a), also used in the HLS19, it was hy-
pothesized that HL will have direct and indirect effects on indicators of personal health status. Thus, in
the HLS19 the research question ‘Is there a relevant and significant effect of General HL on health status?’
was investigated but not how the variance in indicators of health status can best be explained. However,
in a cross-sectional study like the HLS19, the research question can only be answered in a limited, ex-
plorative way because the underlying causal assumptions cannot be tested empirically with the existing
set of data.
Health status variables in the HLS19 in comparison with the HLS-EU
To measure health status, the three items used in the HLS-EU were taken, but in the response category
of self-perceived health, an explanation for the wording “fair” (i.e., neither good nor bad) was added.
The precise wording of the items and response categories used as well as the source of the original items
are provided in Annex 8, Table A 8.1.
Summing up, the variables used in the HLS19 to explore health status and their categories are:
» self-perceived health, represented by five categories: very good-good-fair-bad-very bad,

» long-term illnesses/health problems, represented by three categories (none, one, more than one
long-term illness/health problem), except for one country (SI), where two categories were used (none
and one or more long-term illnesses/health problems).
» limitations due to health problems, represented by three categories: not limited, limited but not
severely, severely limited.
Data analyses and results in this chapter are presented by:
» percentage distributions of health status indicators (Annex 8),

» associations of health status indicators with General HL (for all countries in the main chapter and for
individual countries in Annex 8),
» Spearman correlations (ρ) among indicators of health status, with General HL and selected socio-
demographic and socio-economic determinants (for all countries in the main chapter and for indi-
vidual countries in Annex 8),
» regression analyses: adjusted R² and standardized β coefficients of multivariable linear regression
models are used to compare the relative importance of the predictors included (results in the main
chapter). Annex 8 provides the results of regression models with unstandardized coefficients (b).
8.2 Spearman correlations among indicators of health status with

General HL and selected socio-demographic and socio-eco-
nomic determinants
On average, the three health status indicators are highly correlated with ρ=0.5 for “self-perceived health”
and “long-term illnesses/health problems”, ρ=-0.47 for “self-perceived health” and “limitations due to
health problems”, and ρ=-0.52 for “long-term illnesses/health problems” and “limitations due to health
problems”. General HL correlates with “self-perceived health” (ρ=-0.21), “long-term illnesses/health
problems” (ρ=-0.14), and “limitations due to health problems” (ρ=0.18) (Figure 8.1).

Figure 8.1:
Spearman correlations (ρ) among indicators of health status, with GEN-HL, and selected socio-
demographic and socio-economic determinants, for all countries (equally weighted)*
* Values for the correlations with migration should be treated with caution as in five countries (BG, HU, IT, RU, and
SK), the percentage of respondents with a migration background was below 10%, and the HLS19 survey as such was
not adjusted to targeting migrants specifically (e.g., by offering translations of the instrument into migrant languages).
On average, there are also relatively high correlations of the three indicators with socio-demographic
and socio-economic indicators, especially with “age” and “financial deprivation”, and somewhat lower
also with “level in society” and “education” (Figure 8.1).
8.3 General HL and self-perceived health
On average in the participating countries, 19% of the respondents reported their self-perceived health
as “very good” (from 4% (RU) to 37% (AT)), 45% as “good” (from 37% (RU) to 54% (CH)), 30% as “fair” (from
15% (AT, NO) to 53% (RU)), 6% as “bad “ (from 2% (AT) to 9% (CZ)), and just 1% as “very bad” (from 0% (AT,
CH, IL, and PT) to 2% (CZ, HU)). Thus, there is considerable variation in self-perceived health status across
the participating countries (Annex 8, Figure A 8.1).
The relationship between self-perceived health and General HL is linear, on average. Better HL is asso-
ciated with better self-perceived health (Figure 8.2). This relationship can also be observed in most
countries although it varies somewhat by degree and by level of self-perceived health (Annex 8, Figure
A 8.2).
Figure 8.2:
Percentage distribution of five categories of self-perceived health by GEN-HL (10 groups from lowest
HL to highest HL), for all countries (equally weighted)
There is a correlation between self-perceived health and General HL (ρ=-0.21, varying from -0.07 (CZ)
to -0.38 (BG)). There are also correlations between self-perceived health and four of the five socio-
demographic and socio-economic determinants of General HL, namely age (ρ=0.3), financial deprivation
(ρ=0.27), level in society (ρ=-0.25), education (ρ=-0.16), and female gender (ρ=0.03). For migration
(ρ=-0.06) and training in a health profession (ρ=0.05), the correlation is very low (Annex 8, Table A 8.2).
To answer the research question as to whether a specific direct effect of General HL on self-perceived
health can be demonstrated, when other determinants correlating with self-perceived health and General
HL are controlled for, a multivariable linear regression model was used.

On average, this multivariable linear regression model (Table 8.1; for the model with unstandardized coefficients (b), see Annex 8, Table A 8.3) explains 21% of
the variance (varying from 11% (IE) to 38% (BG)). The predictor with the highest β is age (β=0.26, varying from β=0.08 (BE) to β=0.42 (SK)). The predictor with
the second highest ß is financial deprivation (β=0.16, varying from -0.03 (BE) to 0.22 (HU)). General HL (β=-0.15, varying from β=-0.07 (SK) to β=-0.22 (DK))
and level in society (β=-0.15, varying from -0.06 (RU) to -0.27 (BE)) are the predictors with the third highest β. Education has a much lower β (β=-0.05, varying
from 0.01 (IL) to -0.19 (BG)). With β=0.01 on average gender has no predictive value (varying inconsistently, partly positively and partly negatively across
countries).
Thus, it has been demonstrated that General HL is a relevant and significant predictor of self-perceived health in the participating countries, albeit varying in
strength across these countries.
Table 8.1:
Multivariable linear regression models of self-perceived health by GEN-HL and five core social determinants (standardized coefficients (β) and R2), for each
GEN-HL -0.18 -0.14 -0.2 -0.15 -0.08 -0.14 -0.22 -0.16 -0.12 -0.11 -0.13 -0.12 -0.14 -0.12 -0.14 -0.15 -0.07 -0.15
Gender female -0.01 0.05 0.01 -0.04 -0.03 -0.02 -0.05 -0.01 0.04 -0.02 -0.01 0 0.02 0.09 0.02 0.02 0.01 0.01
Age in years 0.23 0.08 0.26 0.22 0.35 0.41 0.11 0.24 0.31 0.11 0.31 0.24 0.18 0.35 0.36 0.36 0.42 0.26
Education -0.06 -0.08 -0.19 -0.03 -0.11 -0.03 -0.03 0.02 -0.05 -0.1 0.01 0.02 -0.09 -0.11 -0.05 -0.07 -0.05 -0.05
Level in society -0.11 -0.27 -0.11 -0.18 -0.13 -0.09 -0.14 -0.22 -0.12 -0.11 -0.16 -0.14 -0.17 -0.09 -0.06 -0.08 -0.11 -0.15
Financial deprivation 0.14 -0.03 0.13 0.15 0.16 0.13 0.17 0.1 0.22 0.16 0.12 0.16 0.15 0.16 0.17 0.18 0.16 0.16
R2 0.16 0.13 0.38 0.16 0.24 0.26 0.16 0.17 0.27 0.11 0.17 0.14 0.13 0.35 0.29 0.31 0.33 0.21
Valid count 2691 988 721 2019 1567 1845 3561 2003 1124 4301 1154 3240 2681 1168 5079 3184 1794
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Self-perceived health: from very good (1) to very bad (5).
8.4 General HL and long-term illnesses/health problems
On average, 55% of the respondents (from 40% (CZ) to 65% (PT)) had none, 28% (from 16% (DE) to
36% (FR)) had one, and 18% (from 7% (AT) to 36% (DE)) had more than one long-term illness/health
problem. The percentage distribution varies considerably by country (Annex 8, Figure A 8.3).
There is a negative, rather linear relationship between long-term illnesses/health problems and
General HL on average. Respondents with higher HL have fewer long-term illnesses/health prob-
lems (Figure 8.3). This relationship also holds true also for individual countries with few exceptions
(Annex 8, Figure A 8.4).
Figure 8.3:
Percentage distribution of three categories of long-term illnesses/health problems by GEN-HL
(10 groups from lowest HL to highest HL), for all countries (equally weighted)*
* SI was not included in Figure 8.3 as a slightly different measure was used with only two categories for long-
term illnesses/health problems: (1) no and (2) one or more long-term illnesses/health problems.

The average correlation between long-term illnesses/health problems and General HL is ρ=-0.14,
varying from -0.03 (BE, CZ) to -0.28 (RU). This means that General HL is the predictor with the
third highest ρ in the model (ρ=-0.14), along with level in society. Age (ρ=0.34) has the highest
and financial deprivation the second highest correlation (ρ=0.17) with long-term illnesses/health
problems (Annex 8, Table A 8.4).
This multivariable linear regression model (Table 8.2; for the model with unstandardized coeffi-
cients (b), see Annex 8, Table A 8.5) explains 15% of the variance on average for the number of
long-term illnesses (varying from 8% (IT, NO) to 37% (BG)). It explains somewhat less variation
than self-perceived health with 21%. With a β of -0.09 (varying from -0.02 (BE) to -0.19 (PT)),
General HL is, on average, the predictor with the third highest β in the model, after age (β=0.31)
and financial deprivation (β=0.11).
Thus, it has been demonstrated that General HL is a relevant and significant predictor of long-
term illnesses/health problems in some of the participating countries, albeit varying considerably
in strength across these countries.
Table 8.2:
Multivariable linear regression models of long-term illnesses/health problems by GEN-HL and five core social determinants (standardized coefficients (β) and
R2), for each country and for all countries (equally weighted)
GEN-HL -0.07 -0.02 -0.07 -0.06 -0.05 -0.03 -0.06 -0.06 -0.05 -0.04 -0.05 -0.04 -0.04 -0.19 -0.13 -0.07 -0.04 -0.09
Gender female 0.02 0.09 -0.03 -0.01 0.04 0.03 -0.01 -0.01 0.07 0.08 0.02 -0.02 0.07 0.07 0.07 0.05 0.04 0.04
Age in years 0.24 0.2 0.47 0.27 0.31 0.44 0.19 0.28 0.37 0.25 0.36 0.26 0.2 0.31 0.39 0.3 0.45 0.31
Education -0.07 -0.09 -0.15 -0.01 -0.05 0.03 -0.01 0.02 0.01 -0.07 -0.03 0.07 -0.07 0.02 0.03 -0.04 -0.01 -0.01
Level in society -0.09 -0.15 -0.12 -0.08 -0.05 -0.09 -0.07 -0.02 -0.05 -0.09 -0.09 -0.03 -0.07 -0.02 -0.09 -0.01 -0.01 -0.07
Financial deprivation 0.13 0.09 0 0.06 0.12 0.08 0.15 0.13 0.2 0.1 0.1 0.11 0.15 0.15 0.13 0.11 0.12 0.11
R2 0.13 0.1 0.37 0.09 0.13 0.23 0.09 0.1 0.25 0.12 0.16 0.08 0.08 0.23 0.28 0.15 0.27 0.15
Valid count 2686 988 705 2017 1568 1821 3557 2003 1121 4291 1155 3185 2663 1163 4910 3183 1774
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145

8.5 General HL and limitations due to health problems
The percentage distribution of the intensity of limitations due to health problems is, on average, rather
J-shaped with 10% (varying from 5% (HU, PT) to 20% (BE)) “severely limited”, 32% (from 19% (FR) to 55%
(BE)) “limited but not severely”, and 58% (from 25% (BE) to 75% (HU)) “not limited at all”, and there is
considerable variation by country (Annex 8, Figure A 8.5).
The relationship of the intensity of limitations due to health problems with General HL is positive and
rather linear on average: the higher the General HL, the higher the percentage of respondents who are
not limited at all by long-term illnesses/health problems (Figure 8.4). For individual countries this also
holds true, but with some irregularities for some countries (Annex 8, Figure A 8.6).
Figure 8.4:
Percentage distribution of three categories of limitations due to health problems by GEN-HL (10 groups
from lowest HL to highest HL), for all countries (equally weighted)
The correlation with General HL is, on average, ρ=0.18 (varying from 0.06 (IL) to 0.3 (PT)). Thus, General
HL is the predictor with the third highest ρ, after age (ρ=-0.21) and financial deprivation (ρ=-0.19).
Level in society (ρ=0.14) is the predictor with the fourth highest ρ, followed by education (ρ=0.11) (An-
nex 8, Table A 8.6).
The multivariable linear regression model (Table 8.3) explains, on average, 10% of the variance of limitations due to health problems (varying from 3% (IL) to 22%
(PT)). It explains somewhat less of the variance than self-perceived health (21%) or long-term illnesses/health problems (15%). On average, General HL (β=0.14,
varying from 0 (IL) to 0.21 (PT)) the predictor with the second highest β, after age (β=-0.18). Financial deprivation (β=-0.13) is the predictor with the third highest
β.
Thus, it has been demonstrated that General HL is a relevant and significant predictor of limitations due to health problems in most countries, albeit varying in
strength across these countries.
Table 8.3:
Multivariable linear regression models of limitations due to health problems by GEN-HL and five core social determinants (standardized coefficients (β) and R2),
GEN-HL 0.11 0.06 0.17 0.14 0.06 0.09 0.09 0.05 0.11 0.07 0 0.11 0.06 0.21 0.11 0.15 0.10 0.14
Gender female 0 -0.13 -0.02 -0.04 -0.03 -0.04 0 0.03 -0.07 -0.04 -0.02 -0.02 -0.12 -0.11 -0.01 -0.05 0 -0.04
Age in years -0.13 0.02 -0.19 -0.15 -0.2 -0.34 -0.14 -0.11 -0.27 0.04 -0.03 -0.12 -0.12 -0.17 -0.28 -0.25 -0.31 -0.18
Education 0.09 0.13 0.16 -0.02 0.1 0 0.02 -0.02 0.03 0.17 0.11 0.04 0.11 0.04 0.05 0.05 0.05 0.04
Level in society 0.13 0.15 0.15 0.14 0.04 0.07 0.11 0.02 0.03 0.09 -0.03 0.02 0.1 0.05 0.07 0.06 0.02 0.05
Financial deprivation -0.15 0.10 0.01 -0.1 -0.14 -0.13 -0.17 -0.15 -0.19 -0.17 -0.11 -0.15 -0.14 -0.17 -0.11 -0.13 -0.16 -0.13
R2 0.11 0.09 0.20 0.09 0.09 0.18 0.10 0.04 0.18 0.10 0.03 0.07 0.08 0.22 0.16 0.17 0.21 0.1
Valid count 2074 472 570 2014 1565 1726 3554 2003 1007 1576 1153 3114 2637 931 3866 3184 1655
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Limitations due to health problems: from severely limited (1) to not limited at all (3).

In summary, the three indicators of health are not only moderately correlated with General HL, but
General HL has also moderate independent direct effects on these indicators, when potentially
confounding independent variables are controlled for in a multivariable linear regression model.
On average, the correlation ρ, the standardized regression coefficient β, and explained variance
in the model R² is highest for self-perceived health (ρ=-0.21, β=-0.15, R²=21%), followed by
limitations due to health problems (ρ=0.18, β=0.14, R²=10%), and by long-term illnesses/health
problems (ρ=-0.14, β=-0.09, R²=15%). A general trend was demonstrated for all countries, but
for all three parameters there is also considerable variation across participating countries.
Limitations: Due to the limited standardization of the HLS19 study design (mainly the type and time
of data collection) alongside the range in variation across countries, comparisons of the results
between individual countries have not been described or interpreted in the International Report
and must be interpreted with caution when undertaken selectively by the countries themselves.
The results can be compared with the HLS-EU results only to a certain degree: Although the same
indicators were used for health status, General HL and one of the core determinants were meas-
ured differently, different methods of data collection were used, and mostly different countries
were included in the two multinational studies.
» In the HLS-EU, all three health status indicators showed significant associations with HL, with
a more pronounced one for self-perceived health (ρ=-0.27 on average vs. ρ=-0.21 in the
HLS19), followed by limitations due to health problems (ρ=0.17 vs. ρ=0.18 in the HLS19) and
the number of long-term illnesses/health problems (ρ=0.16 on average vs. ρ=-0.14 in the
HLS19).
» In the HLS-EU, in a multivariate linear regression model with five socio-demographic and so-
cio-economic determinants of self-perceived health and the NVS Test, HL was the predictor
with the second highest β on average (β=.17) following age (β=-.37).
Thus, in the HLS19 study the magnitude of most associations was, in general, slightly lower than
in the HLS-EU study, which is probably a result of methodological differences between the two
studies.
Conclusions
To the extent that this is possible with data from a cross-sectional study design, the HLS19 could
demonstrate for more countries in the WHO European Region that General HL has moderate, sig-
nificant effects on indicators of people’s health status and is therefore relevant for public health
policy in the WHO European Region as a critical determinant of health which is more easily modi-
fiable than other determinants of health.
8.7 References
Berkman; Sheridan, Stacey L; Donahue, Katrina E; Halpern, David J; Crotty, Karen (2011): Low
health literacy and health outcomes: an updated systematic review. In: Annals of internal med-
icine 155/2:97-107
Levin-Zamir, Diane; Baron-Epel, Orna B; Cohen, Vicki; Elhayany, Asher (2016): The association of
health literacy with health behavior, socioeconomic indicators, and self-assessed health from
a national adult survey in Israel. In: Journal of health communication 21/sup2:61-68
Mantwill, Sarah; Monestel-Umaña, Silvia; Schulz, Peter J (2015): The relationship between health
literacy and health disparities: a systematic review. In: PLoS One 10/12:e0145455
Pelikan, J. M.; Ganahl, K.; Roethlin, F. (2018): Health literacy as a determinant, mediator and/or
moderator of health: empirical models using the European Health Literacy Survey dataset. In:
Glob Health Promot:1757975918788300
spektiven. Hg. v. Schaeffer, Doris; Pelikan, Jürgen M. hogrefe, Bern. S. 93-S. 125
mary Findings from the HLS-EU Consortium´s Health Literacy Assessment Effort. In: IOS Press.
Hg. v. Logan, G. D.; Siegel, Elliot R. S. 34-S. 59
Stormacq, Coraline; Van den Broucke, Stephan; Wosinski, Jacqueline (2019): Does health literacy
mediate the relationship between socioeconomic status and health disparities? Integrative re-
view. In: Health Promotion International 34/5:e1-e17

Svendsen, Majbritt Tang; Bak, Carsten Kronborg; Sørensen, Kristine; Pelikan, Jürgen; Riddersholm,
Signe Juul; Skals, Regitze Kuhr; Mortensen, Rikke Nørmark; Maindal, Helle Terkildsen; Bøggild,
Henrik; Nielsen, Gitte (2020): Associations of health literacy with socioeconomic position,
health risk behavior, and health status: a large national population-based survey among Dan-
ish adults. In: BMC Public Health 20/:1-12
9 General Health Literacy as a predictor of
health care utilization
Authors:

Thomas Link (ICC)
Eva Maria Berens (DE)
Nejc Berzelak (SI)
Jorge Oliveira (PT)
Mitja Vrdelja (SI)
Chapter 9 / General Health Literacy as a predictor of health care utilization 179

9.1 Background
Previous research
In HL research, there is a tradition of analyzing the influence of HL on indicators relating to a

person’s or patient’s utilization of health care. The literature published on the relationship be-
tween HL and the utilization of health care services has revealed different results for different
types of health care services. Andrus/Roth (2002) found that low HL was associated with increased
levels of hospitalization and increased costs of medical treatment. Betz et al. (2008) found inad-
equate HL was associated with higher rates of emergency room attendance. Berkman et al. (2011)
published a systematic review on low HL and health outcomes. They showed that low HL was
consistently associated with more hospitalizations and greater use of emergency care. According
to Rasu et al. (2015), HL is inversely associated with health care utilization. In a review, Palumbo
(2017) found that limited HL predicts increased hospitalization rates and improper use of health
services. The French National Survey of Cancer Survivors also showed that limited HL is associated
with greater use of primary care, particularly with the utilization of general practitioners (Ousseine
et al. 2020). Friis et al. (2020) found that low HL for the general population predicted slightly more
visits to general practitioners and admissions to hospital as well as longer hospitalization periods
after 4 years of follow-up.
In the original HLS-EU study, indicators for four types of health services were investigated, for all
of which significant Spearman correlations were demonstrated with General HL: physicians’ visits
(ρ=-.11 on average, varying across countries from ρ=-.01 to ρ=-.19), emergency services (ρ=-
.06 on average, from ρ=-.01 to ρ=-.15), hospital services (ρ=-.06 on average, from -.03 to -.19),
and other health professionals (ρ=+.06 on average, with inconsistent results with opposite signs)
(HLS-EU Consortium 2012). For doctors’ visits, a multivariable regression model with five socio-
demographic and socio-economic determinants and General HL, which explained 13% of the var-
iance on average (varying across countries from 3% to 23%), found General HL to be the third
strongest predictor (significant at ρ=-.07 on average, significant for only three out of the eight
countries and ranging from ρ=-.07 to ρ=-.10) (Pelikan/Ganahl 2017b).
In later research using the HLS-EU measurements for HL, Vandenbosch et al. (2016) found that
low HL, as measured by the HLS-EU-Q16, is associated with greater use of healthcare services and
especially of more specialized services. Using the HLS-EU-Q47, Berens et al. (2018) found that
respondents with lower HL scores reported more frequent use of all four types of health services
included. However, multiple regression analysis showed a direct significant effect of HL only on
visits to doctors and other health professionals while no significant direct effect of HL on hospital
and emergency services use was found when socio-demographic and health-related factors were
controlled for.
Therefore, the HLS19 builds on the HLS-EU’s generic model of HL (Sørensen et al. 2012) and the
refined Vienna Model of Health Literacy (see Figure 1.2, Chapter 1) (Pelikan/Ganahl 2017a), which
hypothesize that HL has direct effects on indicators of health care utilization. Thus, the guiding
research question in the HLS19 for this chapter was whether there is a relevant, significant direct
effect of General HL on the utilization of selected types of health care services but not how the
variance in indicators relating to the utilization of health services can best be explained. However,
given the cross-sectional nature of this study, the research question can be answered only in a
limited, explorative way because the underlying causal assumptions cannot be tested empirically
with the existing set of data.
Health care utilization in the HLS19, and in comparison, with the HLS-EU
In the HLS-EU, health care utilization was measured by four indicators: use of emergency units,
hospitals, physicians, and other healthcare professionals (HLS-EU Consortium 2012), (Sorensen et
al. 2015). In comparison with the items and categories used in the HLS-EU, the items on health
care utilization in the HLS19 were rephrased. While one item was used to measure visits to “doctors”
in the HLS-EU (Q6.2), this rather broad and heterogeneous concept of “doctor” was divided into
“GP or family doctor” and “medical or surgical specialist” in the HLS19 and measured by two, more
specific items. Similarly, in the HLS-EU, one item was used to measure the utilization of “hospital
services”, and in the HLS19, two items were used relating to inpatient and day patient hospital
utilization separately.
Summing up, the variables used in the HLS19 for measuring health care utilization were:
» utilization of emergency services

» utilization of GPs/family doctors
» utilization of medical or surgical specialists
» utilization of inpatient hospital services
» utilization of day patient hospital services
Respondents were asked to provide the number of times they had utilized emergency services
within the previous 24 months and all other services within the previous 12 months. (For detailed
wording and response categories, see Table 9.1 in Annex 9.)
Data analyses and results are presented by:
» percentage distributions of health care utilization indicators (Annex 9),

» associations of health care utilization indicators with General HL (for all countries in the main
chapter and for individual countries in Annex 9),
» Spearman correlations (ρ) among indicators of health care utilization, with General HL and
selected socio-demographic and socio-economic determinants (for all countries in the main
chapter and for individual countries in Annex 9),
» regression analyses: R² and standardized coefficients (β) of multivariable regression models
are used to easily compare the relative importance of the predictors included, with the overall
results in the main chapter and the results of models with unstandardized regression coeffi-
cients (b) in Annex 9.

9.2 Spearman correlations among indicators of health care
utilization, with General HL and selected socio-demo-
graphic and socio-economic determinants
To give an idea of the relative strength of the relationships of the five indicators for health care
utilization with each other, with General HL, and with potential common determinants as poten-
tial confounders and these with each other, Spearman correlations for the mean of all participat-
ing countries are provided in Figure 9.1. (For the distribution of the socio-demographic and so-
cio-economic determinants see Chapter 6). Most of the correlations in Figure 9.1 are shown for
individual countries in the annex to this chapter. On average, the five health care utilization in-
dicators correlate with each other positively to a certain degree, from ρ=0.19 between the utili-
zation of emergency services and medical or surgical specialists to ρ=0.43 between the utiliza-
tion of GPs or family doctors and medical or surgical specialists.
Figure 9.1:
Spearman correlations (ρ) among indicators of health care utilization, with GEN-HL, and selected
socio-demographic and socio-economic determinants, for all countries (equally weighted)*
* Values for the correlations with migration should be treated with caution as in five countries (BG, HU, IT, RU, and SK) the
percentage of respondents with a migration background was below 10%, and the HLS19 survey as such was not adjusted to
targeting migrants specifically (e.g., by offering translations of the instrument into migrant languages).
General HL is negatively and weakly correlated with all five health care utilization indicators. The
strongest correlation was, on average, with GPs or family doctors (ρ=-0.12), emergency services
(ρ=-0.07), medical or surgical specialists (ρ=-0.07), hospitals as an inpatient (ρ=-0.05), and hos-
pitals as a day patient (ρ=-0.04). However, the correlations of the health care utilization indicators
with the five socio-demographic or socio-economic indicators are also rather weak on average,
ranging from ρ=0 to a maximum of ρ=0.2 (Figure 9.1).
9.3 General HL and the utilization of emergency services
Across all participating countries, on average, 77% of the respondents had never used an emer-
gency service in the previous 24 months, with 19% reporting 1-2 contacts (in figures indicated as
incidents), 3% 3-5 contacts, and 1% six or more contacts. The variation between countries is mod-
erate, with the lowest percentage for “never” used at 61% (PT) and the highest at 91% (BG) (cf.
Figure A 9.1 in Annex 9).
On average, there is a rather linear, negative relationship between the frequency of utilization of
emergency services with General HL – the higher the HL, the lower the use of emergency services
(Figure 9.2) – but this relationship is not so smooth for individual countries (Annex 9, Figure A
9.2).
The correlation of General HL with the number of emergency service contacts is, at ρ=-0.07 on
average (varying from ρ=-0.02 (CZ) to ρ=-0.16 (BG, PT)), the indicator with the second highest ρ
value. “Financial deprivation” (ρ=0.09) is the indicator with the highest ρ and “level in society”
(ρ=-0.06) is the indicator with the third highest ρ (Annex 9, Table A 9.2).

Figure 9.2:
Percentage distribution of four categories of utilization of emergency services by GEN-HL (10
A multivariable linear regression model with standardized β coefficients was specified to explain
the variation in the number of contacts with emergency services with the five socio-demographic
and socio-economic determinants and General HL as predictors (for models with unstandardized
b coefficients see Table A 9.3 in Annex 9).
The regression model explains, on average, only a small amount (2%) of the variation in emergency
services use, varying between 1% (CH, DK, IT, and SI) and 11% (BG). However, General HL is, on
average, the predictor with the second highest β=-0.06, (varying from β=-0.02 (DE, DK, and IT)
to β=-0.2 (BG)), after financial deprivation (β=0.07) (Table 9.1). (For the same model with un-
standardized coefficients (b) see Annex 9, Table A 9.3.)
Table 9.1:
Multivariable linear regression models of utilization of emergency services by GEN-HL and five core social determinants (standardized coefficients (β) and R2),
GEN-HL -0.11 -0.08 -0.2 -0.04 0.03 -0.02 -0.02 -0.03 -0.08 -0.07 -0.08 -0.02 -0.04 -0.11 -0.09 -0.05 -0.11 -0.06
Gender female 0.01 0.01 0.03 -0.02 -0.05 -0.02 0.01 -0.03 0.04 0 0.05 0 0.06 0.11 0.09 0 0.04 0.03
Age in years 0.05 -0.12 -0.1 -0.01 -0.21 0.15 0.01 -0.12 -0.01 -0.1 -0.05 0.02 0 -0.04 0.12 0.03 0.04 -0.02
Education -0.01 -0.08 -0.09 -0.01 -0.03 -0.01 -0.03 0.03 -0.06 -0.08 0.04 0.04 -0.05 0.03 -0.06 -0.01 -0.06 -0.03
Level in society -0.01 -0.09 -0.03 -0.05 0.01 -0.05 -0.03 0.02 0.02 -0.03 -0.05 0.01 0 0.05 -0.04 0.01 0.01 -0.03
Financial deprivation 0.09 -0.02 0.05 0.06 0.04 0.1 0.08 0.09 0.08 0.1 0.05 0.11 0.12 0.16 0.09 0.06 0.07 0.07
R2 0.03 0.04 0.11 0.01 0.04 0.04 0.01 0.02 0.02 0.03 0.02 0.01 0.02 0.05 0.07 0.01 0.04 0.02
Valid count 2681 970 724 2019 1556 1824 3556 2003 1124 4295 1150 2825 2674 1168 4990 3175 1781
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Utilization of emergency services: number of contacts in the last 24 months, from 0 to 6 or more contacts.

9.4 General HL and the utilization of GPs/family doctors
Among the participating countries, on average, 23% of the respondents had never used a GP/family
doctor in the previous 12 months, 41% had 1-2 contacts (in figures indicated as incidents), 23%
3-5 contacts, and 13% six or more contacts. There is quite some variation across countries, with
the lowest proportion of “never” at 10% (BE) and highest at 41% (PT, RU) (Annex 9, Figure A 9.3).
On average there is a slight, negative, rather linear relationship between the frequency of utiliza-
tion of GPs/family doctors and General HL. Respondents with higher General HL used this kind of
service less often (Figure 9.3). Associations between the utilization of GPs/family doctors and
General HL differ across countries (Annex 9, Figure A 9.4).
Figure 9.3:
Percentage distribution of four categories of utilization of GPs/family doctors by GEN-HL (10
186 © GÖG 2021, International Report of the European Health Literacy Population Survey 2019-2021 (HLS19) of M-POHL
The correlation with General HL is, at ρ=-0.12 (varying from ρ=+0.03 (CZ) to ρ=-0.2 (BG)), the
second highest of the socio-demographic or socio-economic indicators. Age (ρ=0.19) has the
highest correlation, and “financial deprivation” (ρ=0.09) is the predictor with the third highest ρ
in the model (Annex 9, Table A 9.4).
A multivariable linear regression model with standardized β coefficients was built to explain the
variation in the number of contacts with GPs/family doctors, with the five socio-demographic and
socio-economic determinants and General HL as predictors. (For a model with unstandardized b
coefficients, see Table A 9.5 in Annex 9).
This model for the utilization of GPs/family doctors explains somewhat more variation than that
for emergency services with an average of 6% (varying from 4% (DK, IL) to 15% (RU)). General HL
is, with an average of β=-0.09 (varying from β=0 (IL, IT) to β=-0.14 (BG)), the predictor with the
second highest ß of utilization of GPs/family doctors after age (β=0.18). Female gender (β=0.08)
and financial deprivation (β=0.05) are the predictors with the third and fourth highest ß (Table
9.3). (For the same model with unstandardized coefficients (b), see Annex 9, Table A 9.5.)

Table 9.2:
Multivariable linear regression models of utilization of GPs/family doctors by GEN-HL and five core social determinants (standardized coefficients (β) and R2),
GEN-HL -0.07 -0.03 -0.14 -0.08 0.01 -0.06 -0.05 -0.05 -0.02 -0.06 0 0 -0.04 -0.09 -0.09 -0.06 -0.04 -0.09
Gender female 0.1 0.08 0.03 0.02 -0.01 0.08 0.09 0.08 0.13 0.15 0.08 0.02 0.15 0.14 0.06 0.07 0.07 0.08
Age in years 0.22 0.16 0.16 0.18 0.17 0.31 0.09 0.19 0.26 0.14 0.04 0.23 0.08 0.13 0.29 0.17 0.23 0.18
Education 0 -0.18 -0.19 -0.03 -0.09 -0.02 -0.02 0 0 -0.02 -0.08 -0.02 -0.11 -0.03 0.01 0 -0.05 -0.02
Level in society 0.03 -0.06 -0.07 -0.07 -0.01 -0.05 -0.09 -0.01 0 -0.06 0.02 -0.05 0.01 0.01 -0.04 0.02 0.01 -0.01
Financial deprivation 0.09 -0.01 -0.13 0.08 0.12 0.09 0.06 0.09 0.09 0.1 0.14 0.07 0.12 0.06 0.11 0.11 0.18 0.05
R2 0.09 0.09 0.1 0.06 0.06 0.14 0.04 0.05 0.11 0.06 0.04 0.07 0.05 0.07 0.15 0.06 0.14 0.06
Valid count 2646 981 724 2019 1548 1794 3552 2003 1123 4286 1156 2931 2665 1168 4920 3177 1785
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Utilization of GPs/family doctors: number of contacts in the last 12 months, from 0 to 6 or more contacts.
9.5 General HL and the utilization of medical or surgical spe-
cialists
On average among the participating countries, 46% of respondents had never used a medical or
surgical specialist within the previous 12 months, 35% had 1-2 contacts (in figures indicated as
incidents), 13% 3-5 contacts, and 6% 6 or more contacts. The utilization of medical and surgical
services varies considerably by country, for example, “never” ranged from 26% (AT) to 72% (NO)
(Annex 9, Figure A 9-5).
The utilization of medical or surgical specialists (for the percentage distribution, see Annex 9,
Figure A 9.5) is, on average, slightly and rather linearly related to General HL. With better General
HL there is, on average, a slight decrease in the frequency of utilization of medical or surgical
specialists (Table 9.4). This kind of association can be found in most participating countries but
to a differing degree and a few deviations (Annex 9, Figure A 9.6).
General HL is, on average, negatively correlated with the utilization of medical or surgical special-
ists with ρ=-0.07 (varying from ρ=+0.03 (BE) to ρ=-0.16 (RU)) and thereby it is the predictor with
the fourth highest ρ value. The highest ρ value has age (ρ=0.14), followed by female gender
(ρ=0.1) and financial deprivation (0.08) (Annex 9, Table A 9.6).

Figure 9.4:
Percentage distribution of four categories of utilization of medical or surgical specialists by
GEN-HL (10 groups from lowest HL to highest HL), for all countries (equally weighted)
variation in the frequency of contacts with emergency services with the five socio-demographic
and socio-economic determinants and General HL as predictors. (For a model with unstandardized
b coefficients, see Annex 9, Table A 9-7).
The model including General HL and socio-demographic and socio-economic determinants

explains on average just 3% of the variation in the utilization of medical and surgical specialists
(varying between 1% (NO) and 12% (DE)). General HL is, at β=-0.05 (varying from β=+0.02 (BE) to
β=-0.1 (RU)), the predictor with the fourth highest ß for the utlization of medical and surgical
specialists, followed by age (β=0.12), female gender (β=0.09), and financial deprivation (β=0.07)
(Table 9.3). (For the same model with unstandardized coefficients (b), see Annex 9, Table A 9.7.)
Table 9.3:
Multivariable linear regression models of utilization of medical or surgical specialists by GEN-HL and five core social determinants (standardized coefficients (β)
and R2), for each country and for all countries (equally weighted)
GEN-HL -0.03 0.02 -0.04 -0.05 -0.03 -0.08 -0.05 -0.06 -0.02 -0.04 -0.07 -0.02 0 -0.02 -0.10 -0.01 -0.04 -0.05
Gender female 0.21 0.14 0.07 0.08 0.12 0.1 0.06 0.14 0.11 0.02 0.09 0.07 0.03 0.14 0.05 0.06 0.09 0.09
Age in years 0.11 0.16 -0.03 0.11 0.16 0.28 0.09 0.13 0.11 0.13 0.21 0.13 0.08 0.11 0.11 0.16 0.25 0.12
Education 0.1 -0.04 -0.11 0.11 0.06 0.06 0.04 0.1 0.14 0.03 0.09 0.14 0 0.17 0.09 0.08 0.02 0.03
Level in society 0 -0.09 -0.03 -0.07 -0.02 -0.04 0 0.04 -0.01 -0.06 -0.03 0.03 -0.01 0.03 -0.06 0 -0.01 -0.01
Financial deprivation 0.06 0.06 -0.09 0.12 0.06 0.08 0.08 0.09 0.16 0.04 0.08 0.05 0.06 0.04 0.06 0.07 0.06 0.07
R2 0.07 0.06 0.03 0.05 0.04 0.12 0.02 0.05 0.07 0.03 0.08 0.03 0.01 0.05 0.05 0.04 0.09 0.03
Valid count 2657 971 724 2017 1546 1817 3552 2003 1124 4292 1155 2932 2672 1168 4901 3172 1787
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Utilization of medical or surgical specialists: number of contacts in the last 12 months, from 0 to 6 or more contacts.

9.6 General HL and the utilization of inpatient hospital
services
On average among the participating countries, inpatient hospital services had never been used by
87% of the respondents within the previous 12 months, 11% had 1-2 contacts (in figures indicated
as incidents), and 1% either 3-5 or 6 or more contacts. The variation across countries is moderate
for “never”, ranging from 82% (DE) to 92% (IT, PT) (Annex 9, Figure A 9.7).
The utilization of inpatient hospital services (for the percentage distributions see Annex 9, Figure
A 9.7) has, on average, a very slight, rather linear, negative relation with General HL (Figure 9.5)
with very few countries deviating from this trend (Annex 9, Figure 9.8).
Figure 9.5:
Percentage distribution of four categories of utilization of inpatient hospital services by GEN-HL
(10 groups from lowest HL to highest HL), for all countries (equally weighted)
The correlation with General HL is, on average, negative and rather small at ρ=-0.05 (varying from
ρ=0.01 (CZ) to ρ=-0.14 (SK)) and was thus the predictor with the fourth highest ρ (together with
level in society) after age (ρ=0.11), education (ρ=-0.06), and financial deprivation (ρ=0.07) (An-
nex 9, Table A 9.8).
A multivariable linear regression model with standardized β coefficients was calculated to explain
the variation in the frequency of emergency service utilization with the five socio-demographic
and socio-economic determinants and General HL as predictors. (For a model with unstandardized
coefficients (b), see Table A 9.9 in Annex 9.)
The multivariable regression model explains, on average, just 2% of the variance in the use of
inpatient hospital services (varying from 0% (CZ) to 9% (SK)). At β=-0.04 (varying from β=+0.01
(NO) to β=-0.08 (BG)), General HL is, on average, the predictor with the third highest ß (together
with education, after age (β=0.09) and financial deprivation (β=0.06)) (Table 9.4). (For the same
model with unstandardized coefficients (b), see Annex 9, Table A 9.9.)

Table 9.4:
Multivariable linear regression models of utilization of inpatient hospital services by GEN-HL and five core social determinants (standardized coefficients (β)
GEN-HL -0.06 -0.01 -0.08 -0.05 0 -0.06 -0.01 0.03 -0.05 -0.05 -0.05 -0.01 0.01 -0.06 -0.06 -0.02 -0.06 -0.04
Gender female 0.03 0.04 0.01 0.02 0.02 0 0 -0.03 0.02 0 0.04 -0.05 0.05 0.05 0.02 -0.02 0.02 0.01
Age in years 0.08 0.06 0.1 0.11 0.04 0.2 0.1 -0.01 0.15 0.04 0.02 0.05 0.12 0.12 0.12 0.11 0.19 0.09
Education -0.04 -0.1 -0.21 -0.03 -0.02 -0.04 0 0.03 -0.06 -0.08 -0.05 0.01 -0.07 -0.01 -0.06 -0.02 -0.05 -0.04
Level in society -0.04 -0.04 -0.02 -0.08 0.02 -0.03 -0.01 0.05 0 -0.02 -0.01 0.06 0.04 0.05 0.02 -0.04 -0.03 -0.01
R2 0.02 0.02 0.07 0.03 0 0.06 0.02 0.01 0.04 0.02 0.01 0.01 0.03 0.02 0.03 0.03 0.09 0.02
Valid count 2679 968 724 2020 1564 1830 3561 2003 1124 4295 1147 2947 2681 1168 5043 3173 1787
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Utilization of inpatient hospital services: number of contacts in the last 12 months, from 0 to 6 or more contacts.
9.7 General HL and the utilization of day patient hospital ser-
vices
On average, among the participating countries, day patient hospital services had never been by
81% of respondents within the previous 12 months, with 14% reporting 1-2 contacts (in figures
indicated as incidents), 3% 3-5 contacts, and 2% 6 or more contacts. There is considerable varia-
tion across countries, for example, for “never” from 65% (DK) to 93% (PT) (Annex 9, Figure A 9.9).
On average there is no association between the frequency of day patient hospital service utilization
and General HL and it is otherwise somewhat irregular (Figure 9.6). There are also no associations
for individual countries, with just a few rather irregular cases (Annex 9, Figure A 9.10).
Figure 9.6:
Percentage distribution of four categories of utilization of day patient hospital services by GEN-
HL (10 groups from lowest HL to highest HL), for all countries (equally weighted)

The correlation with General HL is, on average, at ρ=-0.04 (varying from ρ=-0.01 (DE) to ρ=-0.11
(RU)), almost as weak as for the utilization of inpatient hospital services and thus it is the predictor
with the second highest ρ in the model, after age (ρ=0.08) (Annex 9, Table A 9.10).
variation in the frequency of use of day patient hospital services with the five socio-demographic
and socio-economic determinants and General HL as predictors (for a model with unstandardized
coefficients (b), see Table A 9.11 in Annex 9).
The model explains, on average, just 1% of the variation (from 1% (CZ, FR, HU, IL, and SI) to 4%
(BE, RU), which is even lower than for the utilization of inpatient hospital services. General HL is,
on average, at a value of β=-0.04 (varying from β=0 (DE, IL) to β=-0.07 (HU)), the predictor with
the second highest ß in the model, after age (β=0.08) (Table 9.5). (For the same model with un-
standardized coefficients (b), see Annex 9, Table A 9.11.)
Table 9.5:
Multivariable linear regression models of utilization of day patient hospital services by GEN-HL and five core social determinants (standardized coefficients (β)
GEN-HL -0.04 -0.02 -0.03 -0.02 -0.03 0 -0.01 -0.04 -0.07 -0.04 0 -0.03 -0.01 -0.05 -0.06 -0.03 -0.05 -0.04
Gender female 0.01 0.06 -0.11 -0.01 0 -0.02 0.05 -0.03 0.01 0.02 0 -0.05 0.06 0.02 0.07 -0.03 0.04 0.01
Age in years 0.11 0.11 0.02 0.12 -0.03 0.12 0.12 0.05 0.01 0.12 0.05 0.03 0.11 0.11 0.12 0.06 0.09 0.08
Education -0.03 -0.05 -0.14 0.04 -0.02 0.04 -0.02 0.05 0.01 0 -0.1 0 -0.03 -0.05 0.01 0.02 -0.03 0.02
Level in society 0.02 -0.11 0.01 -0.09 -0.01 0 -0.03 -0.01 -0.01 -0.04 0.01 0.06 -0.02 0.11 -0.05 -0.02 0 -0.01
R2 0.03 0.04 0.03 0.02 0.01 0.02 0.03 0.01 0.01 0.03 0.01 0.02 0.02 0.03 0.04 0.01 0.03 0.01
Valid count 2679 964 724 2018 1565 1830 3561 2003 1123 4299 1152 2957 2671 1168 5021 3173 1781
Total count 2967 1000 865 2502 1599 2143 3602 2003 1195 4487 1315 3500 2855 1247 5660 3360 2145
Utilization of day patient hospital services: number of contacts in the last 12 months, from 0 to 6 or more contacts.

Summarizing, the correlations of General HL with the utilization of selected health services were,
on average and as expected, negatively but rather weakly correlated with all five health care utili-
zation indicators. The strongest correlation was, on average, with GPs/family doctors (ρ=-0.12),
followed by emergency services (ρ=-0.07), medical or surgical specialists (ρ=-0.07), inpatient
hospital services (ρ=-0.05), and day patient hospital services (ρ=-0.04).
In multivariable linear regression models with five used socio-demographic and socio-economic
indicators which have been shown to be predictors of HL as potential confounders, the β values
for the potential effect of General HL on the utilization of health services were, on average, com-
pared to the other correlations somewhat lower for emergency services (on average β=-0.06,
significant for eight countries with β ranging from -0.05 to -0.20), GPs/family doctors (on average
β=– 0.09, significant for nine countries with β ranging from -0.05 to -0.14), medical and surgical
specialists (on average β=-0.05, significant only for four countries with β ranging from -0.05 to
-0.10), hospital inpatients (on average β=-0.04, only four countries significant with β ranging
from -0.05 to -0.06) and hospital day patients (on average β=-0.04, significant only for two
countries with β=-0.04 or -0.06). Thus, for the utilization of emergency services and GPs/family
doctors, there is a stronger and more significant trend for more countries that General HL nega-
tively affects the utilization of these health services, while for the other indicators significant ef-
fects were only demonstrated for a few countries.
Limitations: Due to the limited standardization of the HLS19 study design (mainly the type and time
of data collection), alongside overall trends and the range in variation across countries, compari-
sons of the results between individual countries have not been described or interpreted in the
International Report and must be interpreted with caution when undertaken selectively by the
countries themselves.
Comparison with the results of the HLS-EU is further limited by changes in the measure of General
HL, by fewer measures and changes in all measures of the utilization of health services as well as
in one of the core socio-demographic and socio-economic determinants, and by mostly different
countries included in the two studies.
The results of the HLS19 fit in rather well with already demonstrated trends, even if the values for
correlations, β coefficients, and explained variance are somewhat lower in the HLS19 compared to
the HLS-EU. Comparison with other publications is even more difficult due to methodological dif-
ferences in other published studies.
In conclusion, the HLS19 enriched the evidence for more countries in the WHO European Region
that General HL is relevant for at least for two, i.e., utilization of GPs/family doctors and emergency
services, out of the five selected indicators of the utilization of health services, as far as this is
possible to claim with data from a cross-sectional study design. In conclusion, HL is relevant for
health policy and practice in the WHO European Region.
9.9 References
Andrus, MR; Roth, MT (2002): Health literacy: a review. In: Pharmacotherapy 22/(3):282–302.
Berens, Eva-Maria; Vogt, Dominique; Ganahl, Kristin; Weishaar, Heide; Pelikan, Jürgen; Schaeffer,
Doris (2018): Health Literacy and Health Service Use in Germany. In: HLRP: Health Literacy
Research and Practice 2/2:e115-e122
Berkman, Nancy D.; Sheridan, Stacey L.; Donahue, Katrina E.; Halpern, David J.; Crotty, Karen
(2011): Low health literacy and health outcomes: an updated systematic review. In: Ann Intern
Med 155/2:97-107. doi:110.7326/0003-4819-7155-7322-201107190-201100005
Betz, Cecily L; Ruccione, Kathy; Meeske, Kathleen; Smith, Kathryn; Chang, Nancy (2008): Health
literacy: a pediatric nursing concern. In: Pediatric nursing 34/3:231
Friis, Karina; Pedersen, Marie Hauge; Aaby, Anna; Lasgaard, Mathias; Maindal, Helle Terkildsen
(2020): Impact of low health literacy on healthcare utilization in individuals with cardiovascular
disease, chronic obstructive pulmonary disease, diabetes and mental disorders. A Danish pop-
ulation-based 4-year follow-up study. In: European journal of public health 30/5:866-872
Ousseine, Youssoufa M; Bouhnik, Anne‐Déborah; Peretti‐Watel, Patrick; Sarradon‐Eck, Aline;

Memoli, Victoria; Bendiane, Marc‐Karim; Durand, Marie‐Anne; Mancini, Julien (2020): The im-
pact of health literacy on medico‐social follow‐up visits among French cancer survivors 5 years
after diagnosis: The national VICAN survey. In: Cancer medicine 9/12:4185-4196
Palumbo, Rocco (2017): Examining the impacts of health literacy on healthcare costs. An evidence
synthesis. In: Health services management research 30/4:197-212
spektiven. Hg. v. Schaeffer, Doris; Pelikan, Jürgen M. hogrefe, Bern. S. 93-S. 125
mary Findings from the HLS-EU Consortium´s Health Literacy Assessment Effort. In: IOS Press.
Hg. v. Logan, G. D.; Siegel, Elliot R. S. 34-S. 59
Rasu, Rafia S; Bawa, Walter Agbor; Suminski, Richard; Snella, Kathleen; Warady, Bradley (2015):
Health literacy impact on national healthcare utilization and expenditure. In: International
journal of health policy and management 4/11:747

literacy survey (HLS-EU). In: European journal of public health 2015/25.6:1053-1058.
doi:1010.1093/eurpub/ckv1043
Vandenbosch, Jessica; Van den Broucke, Stephan; Vancorenland, Sigrid; Avalosse, Hervé; Verniest,
Rebekka; Callens, Michael (2016): Health literacy and the use of healthcare services in Belgium.
In: Journal of epidemiology and community health 2016/0:1-7
10 Navigational Health Literacy
Authors:

Lennert Griese (DE)
Miguel Telo de Arriaga (PT)
Andreia Silva da Costa (PT)
Rita Francisco (PT)
Saskia Maria De Gani (CH)
Rebecca Jaks (CH)
Zdenek Kucera (CZ)
Diane Levin-Zamir (IL)
Thomas Link (ICC)
Peter Nowak (AT)
Carlota Ribeiro da Silva (PT)
Rajae Touzani (FR)
Sanja Vrbovsek (SI)
Mitja Vrdelja (SI)
Chapter 10 / Navigational Health Literacy 201

10.1 Background and development of instrument
Overview of the relevance, existing research, and

measures of Navigational Health Literacy
Already in 2001, the Institute of Medicine stated in their report Crossing the Quality Chasm: A
New Health System for the 21st Century that the situation in healthcare is “characterized by more
to know, more to manage, more to watch, more to do, and more people involved in doing it than
at any time in the nation’s history” (Institute of Medicine Committee on Quality of Health Care in
America 2001: 25). This description still holds true for many healthcare systems and in fact even
more strongly today than 20 years ago. Generally, positive efforts in science and technology have
led to a high degree of specialization in healthcare systems. At the same time, healthcare systems
have become more complex, and increasingly fragmented structures have led to coordination and
interaction challenges for health professionals and healthcare users alike. But especially for pa-
tients and users, these challenges can be demanding. Users are required to orientate themselves
within a large service landscape, to maneuver between and within various healthcare organizations
and to interact with a range of different health professions to plan and negotiate further health
care. However, meeting such demands is not always easy, and if not achieved, the consequences
for the individual (and for the healthcare system) are considerable. Fruitless searches, discontinu-
ities in health care, and, subsequently, uncertainties and burdens for patients are just some of the
consequences of failing healthcare navigation (Ørtenblad et al. 2018; Schaeffer 2017;
Snelgrove/Liossi 2013; Dow et al. 2012). To deal with the numerous challenges posed to patients
and users by healthcare systems as well as by their structures, norms, and functions, health liter-
acy, or more concretely, specific HL for navigating healthcare systems is needed.
Whereas the increasing complexity, fragmentation, and resulting problems have long been dis-
cussed and investigated in different countries (e.g. Ellen et al. 2018; SVR Economy 2017; WHO
2016; Hofmarcher et al. 2007; SVR 2007; Schaeffer 2004), the difficulties encountered by patients
and users when dealing with information on navigational issues in healthcare systems have rarely
been systematically considered. One exception is the work of Rima Rudd and colleagues
(Groene/Rudd 2011; Rudd/Anderson 2006; Rudd et al. 2004; Rudd 2004), which early drew at-
tention to the importance of HL in the context of navigation. As early as 2004, in a qualitative
exploratory study, Rudd showed how demanding navigation tasks within hospitals can be. Ac-
cording to Rudd (2004: 23), health organizations, i.e., hospitals, represent "literate environments"
which require literacy skills, e.g., reading and understanding signs and maps but also interactional
skills to receive assistance with directions to orientate oneself within and navigate these organi-
zations. In quantitative research, to our knowledge, the topic of Navigational HL was first ad-
dressed in the study Literacy and Health in America (Rudd et al. 2004), which built on a synthesis
of health-related data from the National Adult Literacy Survey (NALS) and the International Adult
Literacy Survey (IALS). In this study, literacy tasks related to rights and responsibilities, insurance
applications and other coverage plans, and informed consent for procedures and studies were
classified as one of five HL activities – entitled “Systems Navigation” – in the underlying Health
Activities Literacy Scale (HALS) (Rudd et al. 2004: 8).
In the following years, the topic was also addressed in further studies. These studies, however,
focused less on the competences and abilities needed to deal with information but more on those
directly required to navigate the healthcare system, also referred to as “navigation competencies”
(Gui et al. 2018: 6), which are usually considered an outcome of HL (e.g., Paasche-Orlow/Wolf,
(2007)). In this regard, there are quantitative studies relating HL to aspects which are linked to the
topic of navigation topic, such as delays in or foregoing needed care, difficulties in finding a pro-
vider, or in navigating and coordinating care for the elderly (Fields et al. 2018; Levy/Janke 2016)
but studies and measurements describing HL regarding the specific field of navigating healthcare
systems are still extremely rare.
One exception is the work by Osborne and colleagues: the authors conceptualized “navigating the
healthcare system” as one of nine subdimensions of HL (Osborne et al. 2013: 8) and developed a
corresponding subscale in the Health Literacy Questionnaire (HLQ) outlining the ability to find out
about services and support as well as to advocate in the healthcare system on one’s own behalf.
However, the navigation scale in the HLQ only partly reflects a complex definition of HL (especially
the steps of information processing it defines) on which a comprehensive understanding of HL
and the current study is based (HLS-EU Consortium 2012; Sørensen et al. 2012). To the best to
our knowledge, apart from the small number of works mentioned, there are no studies and meas-
urement tools on HL in the specific field of navigating healthcare systems.
Arguments for providing a new measure and the pro-

cedure for developing a measure for Navigational HL
Most findings connecting HL to navigation issues are based on a general assessment of health
literacy as described above. Data on HL displaying the specific information challenges faced when
navigating healthcare systems – conceptualized in this report as Navigational Health Literacy (Nav-
igational HL) – are missing. Due to the limited amount of research as well as the few attempts at
conceptualizing and operationalizing Navigational HL, the HLS19 aimed to develop and include a
new definition and associated instrument: the HLS19-NAV. 4
With the objective of conceptualizing Navigational HL in this study against the background of a
comprehensive understanding of HL, a definition of Navigational HL was developed during the
preparations for the HLS19. This definition was based on a scoping literature review of existing
definitions, concepts, and instruments in the field of navigation with a special focus on HL. It was
also related to the integrative definition of HL in the HLS-EU (HLS-EU Consortium 2012; Sørensen
et al. 2012). As a result, Navigational HL is defined as people’s knowledge, motivation and skills
to access, understand, appraise and apply the information and communication in various forms
This assumption is based on a scoping review of the literature. Its results and the subsequent steps in defining and concep-
tualizing Navigational HL as well as the process of instrument development can be found in detail in Griese et al. (2020).

necessary for navigating healthcare systems and services adequately to get the most suitable
health care for oneself or related persons. (Griese et al. 2020: 6)
With reference to the underlying model of HL (Sørensen et al. 2012) – in which HL is conceptualized
in the three domains of Health Care, Disease Prevention, and Health Promotion – in this study,
Navigational HL primarily focuses on the domain of health care. Although Navigational HL is also
required in the context of disease prevention and health promotion, and in other contexts as well,
such as rehabilitation or nursing care, it can be assumed that a large proportion of the navigational
requirements for patients will arise in the domain of health care.
In this regard Navigational HL is needed on three levels:
» a macro, systemic level (e.g., how is the health system organized, how does it function and
work?),
» a meso, organizational level (e.g., which service organization functions in which way, who is
the right contact person there, and what are the rules for using it?), and
» a micro, interactional level (e.g., how to interact with and communicate one’s own problems
to health professionals in such a way that a workable solution for making use of health ser-
vices can be jointly discussed and agreed upon). 5
At this point, it should also be emphasized that Navigational HL refers to the information require-
ments related to securing and shaping health care. Questions regarding aspects of treatment and
therapy (cures) are not considered here.
Like HL, Navigational HL can be understood as a relational concept (Parker 2009), i.e., it is related
to both personal abilities to access, understand, appraise, and apply information on navigational
issues (individual or personal Navigational HL) as well as the social, systemic, and contextual cir-
cumstances (organizational Navigational HL or responsiveness) in which information on the
healthcare system is provided and within which Navigational HL is acquired. Furthermore, the term
navigation is defined in this chapter as regarding navigation within a more topographical area (the
healthcare system, its organizations, and proceedings). However, the term has also been used in
many other contexts in the meantime, e.g., the navigation of digital environments and information
sources (Bittlingmayer et al. 2020; Levin-Zamir/Bertschi 2018). These aspects are further exam-
ined in the chapter on Digital HL (Chapter 12).
As for the development of the other optional packages, a working group on measuring Naviga-
tional HL was initiated, led by the first and second authors of this chapter. Representatives from
the HLS19 countries interested in developing and using the package Navigational HL were invited
to join the working group at an early stage of the preparations for the HLS19. In the end, seven
Since health literacy relating to communication with physicians in health care services (HL-COM) is treated as an autonomous
concept and measure in this study (Chapter 11), it was decided to cover the interactive/communicative level of Navigational
HL with just one item (HLS19-NAV12). Nevertheless, it is assumed that HL-COM is also important for patients to negotiate
health care and healthcare paths and is therefore a prerequisite for navigating healthcare systems and Navigational HL.
experts from Germany, Austria, Switzerland, Norway, Portugal, and the Czech Republic were in-
volved.
The detailed procedure for developing the instrument is shown in Figure 10.1. It is based on a
scoping review of the literature on existing definitions, concepts, and instruments on navigation
in healthcare systems with special regard to HL, developing a conceptual framework, formulating
the first items formation and item evaluation.
Figure 10.1:
Steps in the development of the HLS19-NAV (in accordance with Griese et al., 2020, p. 3)
Source: Griese et al., 2020, p. 3
The item formation step also included alignment with the HLS-Q47 items. Since item Q4 in the
HLS19-Q47 also refers to the navigation topic (“to find out where to get professional help when
you are ill”), it was decided not to include this item in the HLS19-NAV. Furthermore, the wording
of item Q35 in the HLS-Q47 in the subdimension of HL for health promotion (HP-HL) is roughly
reflected in item HLS19-NAV4 (“to understand information on ongoing health care reforms that
might affect your health care”). However, the wording was modified in such a way that it focuses
on reforms in the field of health care and is thus much narrower than item Q35 in the HLS-Q47,
which only refers to health in general. An overlap between these items can also be excluded be-
cause item Q35 in the HLS-Q47 is not part of the HLS19-Q12 measure used in this study.
To evaluate the items, the initial item pool was tested in four focus groups in relation to the clarity
and interpretation of the content. A panel of six experts/stakeholders was also asked how well
each item reflected the concept of Navigational HL. The Content Validity Index for Items (I-CVI)
and Scales (S-CVI) were applied to assess their content validity (Lynn 1986; Polit/Beck 2006). After
revising the items, the final instrument was field tested in 33 personal interviews in the German
pre-test, leading to slight adjustments based on the results and interviewers’ feedback in the

introduction part of the instrument. 6 The item evaluation took place in Germany. To ensure trans-
ferability to and practicability in other country contexts, the items were constantly translated back
and forth between German and English throughout the entire process to obtain feedback from the
other international experts participating in M-POHL. The methodological approach and the status
of the instrument were also presented and discussed at two M-POHL meetings. The agreed English
version of the final instrument was included in the HLS19 questionnaire and was integrated in the
national translation processes (Chapter 2).
The final instrument (HLS19-NAV) consists of 12 items mapping specific Navigational HL infor-
mation tasks on the system (macro), organization (meso), and interaction (micro) levels of the
healthcare system. Thus, Navigational HL is operationalized by asking for difficulties experienced
in relation to tasks on accessing, understanding, appraising, and applying information for navi-
gating the healthcare system. Like the HLS19-Q12, the HLS19-NAV uses the 4-point rating scale
response categories “very difficult” - “difficult” - “easy” - “very easy” (for detailed procedure see
Griese et al. 2020).
Objectives and research questions on Navigational HL
The overall objective was to develop and validate a new instrument for measuring Navigational HL
and to provide, for the first time, data on Navigational HL covering a set of different countries
participating in the HLS19 and to examine whether the new HL measure of Navigational HL adds
value to the existing measure of General HL.
It is hypothesized that Navigational HL is related to the amount of complexity involved in how

health care is provided in different healthcare systems and the associated requirements in infor-
mation processing. Furthermore, it was assumed that Navigational HL is distributed differently in
the population and, as was already shown in previous research on HL, is subject to a social gradi-
ent. In addition, the aim was to prove to what extent Navigational HL is linked to different health
outcomes, with the focus being on health care utilization and general health status outcomes.
6
Germany was the first country that was able to conduct the national field test, so that its results on the Navigational HL
measure could be included in the English version of the HLS19 questionnaire before other countries started their field test-
ing.
The specific research questions are:
» To what extent does the newly developed instrument constitute a scale for measuring Navi-
gational HL with acceptable psychometric properties?
» How is Navigational HL distributed over individual items and the score of its scale in the var-
ious countries participating in the HLS19?
» How is Navigational HL distributed in different subpopulations and which population groups
are particularly disadvantaged regarding Navigational HL?
» Is there a social gradient for Navigational HL and how strong are selected socio-economic
and socio-demographic predictors of Navigational HL?
» How does Navigational HL correlate with the other HL measures in the HLS19?
» Is there a significant association among Navigational HL, health care utilization, and general
health outcomes?
Countries participating in Navigational HL
The topic of Navigational HL was included in the HLS19 as an optional package. The package was
chosen and applied by eight countries: Austria (AT), Belgium (BE), Switzerland (CH), Czech Republic
(CZ), Germany (DE), France (FR), Portugal (PT), and Slovenia (SI). A detailed overview of the coun-
tries using the optional package, including the type and period of data collection as well as the
number of respondents can be found in Chapter 2 and Chapter 3 of this report.
10.2 Methods of analyses and results
To analyze and report on Navigational HL, the rated difficulties on the 12 HLS19-NAV items, their
Average Percentage Response Patterns (APRP) and measures of the HL-NAV (score) were used. The
calculation of these indicators is based on the same procedure described for the HLS19-Q12 (in
Sections 4.2 & 4.4).
In the visualization of the perceived difficulties at item level, the response categories "very difficult"
and "difficult" were combined for Figure 10.2. An overview of the results for all response categories
can be found in Annex 10.1 (Table A 10.1 to Table A 10.12).
To calculate the APRP (for more detail, see Section 4.4), how often each respondent selected one
of the four response categories was counted. Then, for each response category, it was calculated
how often a category was selected on average before the mean values were scaled to the percent-
age of valid responses. The APRP indicate the distribution of average percentages for the four
categories of all items in the HLS19-NAV.
Following the HLS19 procedure for calculating General HL, Navigational HL is also based on a count
of the dichotomized items by combining the categories “easy” and “very easy”. The resulting raw
score was standardized to the range of 0 to 100 and so the score indicates the percentage of valid

items that were answered with either “easy” or “very easy” by an individual respondent or, on
average, by a group of respondents. Scores were only computed for respondents who had an-
swered at least 80% of the 12 HLS19-NAV items.
Distributions of individual HLS19-NAV items by country
As Figure 10.2 demonstrates, there is some overlap in the ranking of the difficulty of Navigational
HL tasks across countries, with some deviations.
Figure 10.2:
Percentages of respondents who responded with “very difficult” or “difficult” to the HLS19-NAV
The percentages of the combined “difficult” or “very difficult” answers to the 12 HLS19-NAV items
range between 19.5% and 56.6% (cf. Annex 10.1, Table A 10.13). In all participant countries, item
HLS19-NAV9 “to understand how to get an appointment with a particular health service” was as-
sessed as being the easiest task (on average, with the countries weighted equally, only 19.5%
answered “difficult” or “very difficult”). On the other hand, on average, 56.6% answered “difficult”
or “very difficult” to item HLS19-NAV4 “to understand information on ongoing health care reforms
that might affect health care”, which thus was the most difficult task. Item HLS19-NAV8 “to judge
if a particular health service will meet the expectations and wishes on health care” was not much
easier (52.0%). Likewise, item HLS19-NAV5 “to find out about rights as a patient or user of the
healthcare system” proved to be challenging (51.6%), as did item HLS19-NAV3 “to judge to what
extent the health insurance covers a particular health service” (49.2%). Furthermore, item HLS19-
NAV7 “to find information on the quality of a particular health service” was also rated “difficult” or
“very difficult” by approximately half of all respondents (48.8%). In this regard it is striking that
large parts of the population face problems in seeking help for such navigation requirements.
Finally, 47.6% rated item HLS19-NAV10 “to find out about support options that may help to orien-
tate in the healthcare system” as being “difficult” or “very difficult”.
On average, 45% of the items were answered with "difficult" or "very "difficult" by all respondents
(Figure 10.3). This percentage varied from 33% (AT, SI) to 59% (DE).
Figure 10.3:
Average Percentage Response Patterns (APRP) for the response categories “very difficult”-
“difficult”-“easy”-“very easy” of the 12 HLS19-NAV items, for each country and the mean of all
Psychometric validity analyses
The newly developed HLS19-NAV was validated using both classical and modern test theory.
To test for unidimensionality in Confirmatory Factor Analysis (CFA), the 12 items were set to load
on a single factor using the lavaan package (Roussel 2012) for R (R Core Team 2020b). For details
on the procedure, see Subsection 4.7.2 in this report. To estimate the internal consistency of the
HLS19-NAV, the Cronbach alpha coefficient was computed for each country (Subsection 4.7.1). To

increase the validity of the scale, the two aspects of navigating on a systemic level and organiza-
tional level (Griese et al. 2020) were measured. 7 The two-factor model CFA indicates very slightly
improved model fit indices for some countries (Annex 10.2, Table A 10.14). The high latent cor-
relation between the two factors (r=0.84-0.96) indicated poor discriminant validity and, hence,
suggested a one-factor structure. When fitting the data against the unidimensional polytomous
partial credit Rasch model (Masters 1982; Rasch 1960) by using the RUMM2030plus software
(Andrich/Sheridan 2019), the available dependent t-test procedure revealed that the two aspects
introduced some multidimensionality into the scale. However, it was decided to treat the Naviga-
tional HL scale as unidimensional.
The results show (Table 10.1) that it is reasonable to include all items in a common index, even
though there are some limitations regarding the evaluation of the overall model. For three coun-
tries (BE, CH, and PT), the Root Mean Square Error of Approximation (RMSEA) was > 0.06. Never-
theless, RMSEA < 0.08 can be interpreted as acceptable (Browne/Cudeck 1993). Similar assump-
tions can be made for the Standardized Root Mean Square Residual (SRMR), where values of < 0.8
indicate an acceptable model fit. With a minimum of 0.97, the other fit indices indicate at least an
acceptable fit across all countries. The reliability values of the HLS19-NAV can be rated as good
(Table 10.3).
When testing for a multi-factor structure with an identical number of items in each scale, items HLS19-NAV12
and HLS19-NAV9 were excluded from analyses since they revealed limitations in the Rasch analyses for some
countries: System: HLS19-NAV1, 2, 3, 4, 5; Organization: HLS19-NAV6, 7, 8, 10, 11.
Table 10.1:
Fit indices for the HLS19-NAV single-factor CFA, for each country and the mean for all countries
(equally weighted)
AT BE CH CZ DE FR PT SI Mean
Standardized Root Mean Square

0.05 0.06 0.07 0.03 0.07 0.05 0.06 0.05 0.06
Residual
Root Mean Square Error of Ap-

0.05 0.07 0.07 0.02 0.06 0.06 0.07 0.05 0.05
proximation

0.05 0.06 0.06 0.01 0.06 0.05 0.06 0.05 0.06
proximation (CI lower bound)

0.05 0.07 0.07 0.03 0.07 0.06 0.08 0.06 0.06
proximation (CI upper bound)

0.48 0.00 0.00 1.00 0.00 0.01 0.00 0.06 0.19
proximation (p value)
Comparative Fit Index 0.99 0.99 0.99 1.00 0.98 1.00 1.00 0.99 0.99
Tucker-Lewis Index 0.99 0.99 0.98 1.00 0.97 0.99 0.99 0.99 0.99
Goodness of Fit Index 0.99 0.99 0.99 1.00 0.98 1.00 0.99 0.99 0.99
Adjusted Goodness of Fit Index 0.99 0.98 0.98 1.00 0.97 0.99 0.99 0.99 0.99
Standardized parameter estimates are shown in Table 10.2. Loadings are close to or above 0.70
for most items, meaning the theorized factor explained most of the items well (Knekta et al. 2019).
The loadings are highest between the factor and item HLS19-NAV10 “to find out about support
options that may help you to orientate yourself in the healthcare system” (mean: 0.86) and the
lowest loadings between the factor and item HLS19-NAV9 “to understand how to get an appoint-
ment with a particular health service” (mean: 0.68).

Table 10.2:
Standardized Parameter Estimates, for each country and the mean for all countries (equally
weighted)
HLS19-NAV1 0.74 0.80 0.74 0.78 0.73 0.79 0.85 0.80 0.78
HLS19-NAV2 0.76 0.82 0.77 0.81 0.70 0.72 0.83 0.84 0.78
HLS19-NAV3 0.72 0.75 0.73 0.76 0.61 0.84 0.80 0.84 0.76
HLS19-NAV4 0.78 0.87 0.82 0.80 0.81 0.90 0.93 0.85 0.84
HLS19-NAV5 0.85 0.86 0.87 0.84 0.80 0.85 0.87 0.86 0.85
HLS19-NAV6 0.76 0.79 0.79 0.76 0.58 0.86 0.86 0.80 0.78
HLS19-NAV7 0.81 0.88 0.83 0.77 0.70 0.89 0.93 0.87 0.84
HLS19-NAV8 0.83 0.90 0.85 0.82 0.65 0.93 0.94 0.85 0.85
HLS19-NAV9 0.58 0.61 0.62 0.82 0.65 0.57 0.82 0.77 0.68
HLS19-NAV10 0.84 0.84 0.83 0.85 0.83 0.89 0.89 0.87 0.86
HLS19-NAV11 0.77 0.79 0.75 0.79 0.65 0.85 0.89 0.79 0.78
HLS19-NAV12 0.71 0.77 0.78 0.72 0.61 0.91 0.88 0.71 0.76
Rasch analyses were administered to provide information on the overall data-model fit, targeting,
reliability, individual item data-model fit, the ordering of response categories, response depend-
ency, and the presence of differential item functioning (DIF).
Testing data against the unidimensional polytomous Rasch Partial Credit Model (PCM) (Masters
1982; Rasch 1960) for country-wise samples with 20 persons per threshold, good overall data-
model fit for the HLS19-NAV is observed in Austria (χ2: n=720, p > 0.05). In Switzerland, the Czech
Republic, and Germany, analyses display sufficient overall data-model fit (χ2: n=720, p > 0.01).
Reducing the sample size to n=360 or ten persons per threshold resulted in data collected in
Belgium, Portugal, and Slovenia displaying sufficient/good overall data-model fit, but not France.
For a well-targeted scale, the mean person location should be around zero, indicating that the
measure is neither too easy nor too hard (Tennant/Conaghan 2007). The mean person location
ranged between -0.31(DE PAPI) and 0.96 (SI CAWI). The scale was well-targeted for the following
populations: Belgium (CAWI mean=-0.07), the Czech Republic (CAWI mean=-0.15), France (CAWI
mean=0.11), Germany (PAPI mean=-0.31), Portugal (CATI mean=0.21), and Switzerland (CAWI
mean=0.04). In Austria (CATI mean=0.91) and Slovenia (CAWI mean=0.96), targeting could have
been somewhat better. Regarding the data-model fit at the item level, Infit (MNSQ) indicated poor
fit and under-discrimination for item HLS19-NAV9 in the Belgian (MNSQ=1.39) and French data
(MNSQ=1.54) as well as for item HLS19-NAV12 in the Czech (MNSQ=1.35) and Slovenian
(MNSQ=1.44) data (MNSQ >1.3 and significant χ2) (Yan/Mok 2012). As DIF analyses are sensitive
to sample size, only significant DIFs at a Bonferroni-adjusted 5% and amended sample size of
n=720 are reported. Item HLS19-NAV3 displayed DIF for employment status in data from Belgium
and Switzerland as well as for age in data from Switzerland and France. Furthermore, item HLS19-
NAV4 displayed DIF for age and item HLS19-NAV6 displayed DIF for difficulty with paying bills in
Portugal. Item HLS19-NAV7 displayed DIF for gender and age in the Czech Republic, for age in
France, and for education level and difficulties with paying bills in Switzerland. For item HLS19-
NAV8, the French data indicated DIF for age. Item HLS19-NAV9 displayed DIF for age and employ-
ment status in Austria. In Belgium, DIF was observed for item HLS19-NAV12 regarding paying bills.
It is striking that with the sample size of n=720, no items displayed DIF in the German and Slo-
venian data. Response dependency was observed between items HLS19-NAV7 and HLS19-NAV8 in
the Belgian (r=0.37), Portuguese (r=0.43), and Swiss (r=0.38) data. No signs of unordered re-
sponse categories were found, indicating that the 4-point response scale worked well.
In line with the reported Cronbach’s alpha (Tennant/Conaghan 2007), the Person Separation Index
(PSI) indicated high reliability.
Table 10.3:
Cronbach’s alpha and the Person Separation Index (PSI) for HLS19-NAV, for each country and the
mean of all countries (equally weighted)
Alpha 0.87 0.89 0.88 0.90 0.83 0.91 0.92 0.90 0.89
PSI 0.90 0.92 0.91 0.92 0.88 0.93 0.88 0.92 0.91
In all countries, the correlation between the Navigational HL score (HL-NAV) and the General HL
(GEN-HL) score was positive and at, on average, r=0.56 (varying from r=0.41 (BE) to r=0.63 (FR))
also of considerable size (Table 10.4).
Regarding the other specific HL measures used in the HLS19, a positive correlation was shown for
the HL-NAV and the long and short forms of the “HL relating to communication with physicians in
health care services measure” (HL-COM) in each country (r=0.47/r=0.43 on average, ranging from
0.49 (AT) to 0.36 (BE)). The same applies to HL-NAV and Digital HL (HL-DIGI) (r=0.55 on average,
ranging from 0.67 (FR) to 0.36 (BE)). A positive correlation was also observed between HL-NAV
and Vaccination HL (HL-VAC) with, on average, r=0.40, ranging from r=0.49 (SI) to r=0.26 (BE).
Therefore, it can be argued that the HL-NAV overlaps with the other HL measures used in the
HLS19, showing that it belongs to this family of HL measures, but that its use in the HLS19 is inde-
pendent enough from these to make a specific contribution to measuring HL.

Table 10.4:
Pearson correlation between HL-NAV and other HL scores used in the HLS19, for each country
and the mean for all countries (equally weighted)
HL-NAV and AT BE CH CZ DE FR PT SI Mean
GEN-HL 0.56 0.41 0.56 0.55 0.6 0.63 0.53 0.61 0.56
HL-COM-Q11 0.49 - - - 0.48 - - 0.45 0.47
HL-COM-Q6 0.46 0.36 - 0.45 0.45 0.44 - 0.44 0.43
HL-DIGI 0.57 0.36 0.52 0.57 0.59 0.67 0.54 - 0.55
HL-VAC 0.38 0.26 - 0.47 0.38 - 0.42 0.49 0.4
Distribution of the Navigational HL scores by country
Like the GEN-HL score, the HL-NAV score was also defined to range from 0-100, where 0 indicates
the lowest and 100 the highest possible level of Navigational HL. The mean score (Table 10.5)
indicates the percentage of valid items that were answered with either "easy" or "very easy" on
average by the respondents in individual countries or by selected subpopulation groups.
On average the mean score is 55.3, varying considerably from 41.6 (DE) to 67.4 (SI). The standard
deviation (SD) on average is 31.8 (varying from 28.2 (DE) to 34.1 (FR)). The distribution of the HL-
NAV does not indicate normal distribution but rather differing distribution patterns across coun-
tries and a strong ceiling effect for all countries, apart from DE, where the distribution is rather
right-skewed (Annex 10.4, Figure A 10.1).
Table 10.5:
Means, standard deviations, quartiles, for HL-NAV, for each country and the mean for all
HL-
NAV
Mean 66.8 48.6 52.9 50.7 41.6 50.4 64.2 67.4 55.3
SD 30.1 32.8 31.5 33.8 28.2 34.1 32.4 31.7 31.8
Median 75.0 41.7 50.0 50.0 41.7 41.7 66.7 75.0
25th
per- 41.7 16.7 25.0 20.0 16.7 16.7 64.2 41.7
centile
75th
per- 100.0 75.0 83.3 83.3 58.3 83.3 100.0 100.0
centile
Identification of specific vulnerable/disadvantaged
subpopulations
Like for General HL, it is also of interest to explore what disadvantaged or vulnerable
subpopulations have lower Navigational HL than the average population. The same eight
subgroups were investigated as for General HL (Table 10.6). The strongest deviations from general
population means were found for bad or very bad self-perceived health (on average -11.6, varying
from -4.4 (CZ) to -20.4 (PT)), followed by considerable or severe financial deprivation (on average
-9.9, varying from -1.5 (BE) to -14.4 (PT)), level in society/social status (less than or equal to 4)
(on average -9.0, varying from -1.9 (AT) to -15.9 (PT)), education (ISCED 0,1) (on average -5.6,
varying from 7.1 (CZ) to -14.2 (SI)), and limited by health problems (on average -5.4, varying from
-2.8 (CZ) to -13.4 (PT)), while the average deviations were lowest for age (76 or older) yet
inconsistent (on average -1.7, varying from +6.3 (CZ) to -14.5 (SI)), long-term illnesses/health
problems (one or more) (on average -3.4, varying from -0.4 (BE) to -8.6 (PT)), and utilization of
GPs/family doctors (6 or more contacts) (on average -3.4, varying from -1.0 (CZ) to -6.5 (DE)).
Table 10.6:
Deviation of Navigational HL mean scores for potentially vulnerable subpopulations relative to
the total mean score of the country, for each country and the mean for all countries (equally
weighted)
AT BE CH CZ DE FR PT SI All
HL-NAV country
66.8 48.6 52.9 50.7 41.6 50.4 64.2 67.4 55.3
mean
Aged 76 or older 4 - 4.2 6.3 -8.7 - - -14.5 -1.7
Education at ISCED
- - -4.5 7.1 - - -10.6 -14.2 -5.6
levels 0 or 1
Level in society less
-1.9 -11.7 -7.5 -6.5 -9.5 -7.7 -15.9 -11.6 -9
than or equal to 4
Considerable or se-
vere financial depri- -12 -1.5 -8.2 -13.9 -10.3 -8.7 -14.4 -10.5 -9.9
vation
Bad or very bad self-
-6.3 -10.6 -10.2 -4.4 -14.8 -6.3 -20.4 -19.6 -11.6
perceived health
One or more long-
term illnesses or -2.9 -0.4 -2.9 -0.6 -3.1 -3.3 -8.6 -5.5 -3.4
health problems
Limited by health
-6.2 -3.5 -4.6 -2.8 -3.9 -1.1 -13.4 -7.7 -5.4
problems
6 or more contacts
with a GP/family doc- -3 -2.2 -3.9 -1 -6.5 -3.3 - -4.2 -3.4
tor
- Cells with less than 30 respondents were not reported, as was the case in some countries for old age, low education,
and contacts with a GP/family doctor, were not reported.

Determinants of Navigational HL
Like in the HLS19 generally, a social gradient and core social determinants were also investigated
for Navigational HL. As hypothesized, core socio-demographic and socio-economic determinants
like gender, age, education, level in society, financial deprivation, and, additionally, migration
status and training in a health profession were investigated. Migration background was not in-
cluded in the regression analyses due to extremely low Spearman correlations with Navigational
HL across all countries. Training in a health profession, which is not a common social predictor of
HL, was just included as a Spearman correlation in Table 10.7.
In all countries, the correlations between Navigational HL and the hypothesized determinants are
rather weak, being highest on average for financial deprivation (ρ=-0.19, varying from ρ=-0.05
(BE) to ρ=-0.35 (PT)), level in society (ρ=0.15; varying from ρ=-0.01 (AT) to ρ=0.28 (PT)), and for
no training in a health profession (ρ=-0.09, varying from ρ=0.03 (AT) to ρ=-0.15 (DE). The cor-
relations with gender, education, age, and migration status are rather low, in contrast, partly due
to very different and inconsistent forms of associations of these potential determinants with Nav-
igational HL (see Figures A 10.2 to A 10.7 in Annex 10.7).
Table 10.7:
Spearman correlations between Navigational HL and selected determinants, for each country and
for all countries (equally weighted)
Gender female -0.03 -0.05 -0.04 -0.01 -0.03 -0.08 -0.02 -0.01 -0.04
Age in years -0.07 0 0.04 0 -0.09 -0.1 -0.13 -0.13 -0.06
Education 0 0.03 -0.03 -0.09 0.2 0 0.11 0.15 -0.02
Level in society -0.01 0.22 0.17 0.16 0.25 0.17 0.28 0.23 0.15
Financial deprivation -0.18 -0.05 -0.21 -0.27 -0.22 -0.16 -0.35 -0.3 -0.19
Migration background 0.07 0.05 0.02 0 0 0 0.01 0.02 0.03
No training in a
0.03 -0.08 -0.07 -0.13 -0.15 -0.11 -0.1 -0.14 -0.09
health profession

A multivariable linear regression model with the five socio-demographic and socio-economic var-
iables explained on average 6% of the variance (varying from 4% (AT) to 13% (PT)) (Table 10.8). The
strongest predictor is financial deprivation (β=-0.15, varying from -0.01 (BE) to -0.25 (CZ)), fol-
lowed by level in society (β=0.14, varying from 0 (AT) to 0.22 (BE)), age (β=-0.08, varying from
0.01 (CH) to -0.13 (FR)), education (β=-0.11, varying from 0.10 (DE) to -0.14 (CZ)), and gender
female (β=-0.02, varying inconsistently from +0.02 (CZ) to -0.07 (FR)). Thus, it can be concluded
that there is a social gradient for Navigational HL, differing considerably across countries.
Table 10.8:
Multivariable linear regression models of Navigational HL by five core social determinants
(standardized coefficients (β) and R2), for each country and for all countries (equally weighted)
Gender female -0.02 -0.05 -0.04 0.02 -0.01 -0.07 0 0.01 -0.02
Age in years -0.07 -0.02 0.01 -0.02 -0.09 -0.13 -0.10 -0.09 -0.08
Education -0.06 -0.03 -0.13 -0.14 0.10 -0.10 -0.08 -0.02 -0.11
Level in society 0 0.22 0.14 0.12 0.15 0.17 0.18 0.12 0.14
Financial deprivation -0.18 -0.01 -0.17 -0.25 -0.11 -0.09 -0.23 -0.23 -0.15
R2 0.04 0.05 0.07 0.1 0.09 0.06 0.13 0.11 0.06

Valid count 2587 988 1983 1523 1845 2003 1012 3160
Total count 2967 1000 2502 1599 2143 2003 1247 3360
With the inclusion of General HL in the model, the explained variance, on average, rises consider-
ably to 35% (varying from 19% (BE) to 43% (FR)) (Table 10-9). Now General HL is by far the strongest
predictor of Navigational HL (β=0.53, varying from 0.38 (BE) to 0.62 (FR)), which could be expected
due to the considerable correlation of the two measures. By adding General HL, the other predic-
tors are reduced, but level in society (β=0.09) and financial deprivation (β=-0.07) are still some-
what stronger on the overall level than education (β=-0.06), age (β=-0.05), and being female
(β=-0.04).

Table 10.9:
Multivariable linear regression models of Navigational HL by five core social determinants and
GEN-HL (standardized coefficients (β) and R2), for each country and for all countries (equally
weighted)
GEN-HL 0.54 0.38 0.52 0.51 0.56 0.62 0.48 0.56 0.53
Gender female -0.06 -0.06 -0.05 -0.02 -0.05 -0.09 0.02 -0.01 -0.04
Age in years -0.03 -0.03 0.01 -0.07 -0.03 -0.13 -0.05 -0.02 -0.05
Education -0.04 -0.01 -0.12 -0.07 0.03 -0.07 -0.13 -0.04 -0.06
Level in society 0 0.15 0.09 0.06 0.1 0.08 0.14 0.07 0.09
Financial deprivation -0.07 -0.03 -0.1 -0.14 -0.02 0 -0.13 -0.12 -0.07
R2 0.32 0.19 0.32 0.34 0.39 0.43 0.33 0.39 0.35

Valid count 2587 988 1983 1523 1845 2003 1012 3160
Total count 2967 1000 2502 1599 2143 2003 1247 3360

Consequences of Navigational HL
No specific potential consequences for Navigational HL were included in the HLS19. Therefore, to
test the relevance of Navigational HL for health-related outcomes, a few indicators were selected
for consequences which had mostly already been included in the HLS-EU and which had partly
been changed and added to as part of the HLS19. This was done by investigating Spearman corre-
lations (Table 10.10 and Table 10.13) and bi-variate associations (Annex 10.6, Figures A 10.8 to
A 10.16). Multiple linear regression models were used to test whether there is an independent,
direct effect of Navigational HL on selected indicators for health care utilization (Table 10.11 and
Table 10.12) and indicators for health status (Table 10.14 to Table 10.15) when potentially con-
founding factors are controlled for.
Health care utilization
Spearman correlations show a slight negative relationship between Navigational HL and different
indicators for health care utilization, i.e., with higher Navigational HL, somewhat less use is made
of health care services (Table 10.10). The correlation is strongest for GPs/family doctors (ρ=-0.12,
varying from ρ=0 (CZ) to ρ=-0.14 (DE)) and for medical or surgical specialists (ρ=-0.08, varying
from ρ=-0.01 (BE) to ρ=-0.10 (DE)), while the correlations are weaker for inpatient hospital ser-
vices (ρ=-0.01, varying from ρ=-0.01 (CH) to ρ=0.09 (FR)), day patient hospital services (ρ=-
0.01, varying from ρ=0 (CH, DE) to ρ=-0.05 (AT, CH)), and emergency services (ρ=-0.02, varying
from ρ=-0.01 (CH) to ρ=-0.05 (AT, DE, PT, and SI)).
Table 10.10:
Spearman correlations (ρ) of Navigational HL with five indicators for health care utilization, for
HL-NAV and… AT BE CH CZ DE FR PT SI All
Emergency ser-
vices -0.05 -0.04 -0.01 0.03 -0.05 0.01 -0.05 -0.05 -0.02
GPs/family
doctors -0.12 -0.04 -0.04 0 -0.14 -0.08 -0.04 -0.08 -0.12
Medical or sur-
gical specialists -0.07 -0.01 -0.06 -0.03 -0.1 -0.07 -0.06 -0.02 -0.08
Hospital as an
inpatient 0.01 0.01 -0.01 0.04 -0.06 0.09 0.04 -0.03 -0.01
Hospital as a
day patient -0.03 -0.02 0 -0.02 0 -0.01 0.06 -0.03 -0.01
Utilization of emergency services: number of contacts in the last 24 months, from 0 to 6 or more contacts.
Utilization of inpatient hospital services: number of contacts in the last 12 months, from 0 to 6 or more contacts.
Utilization of day patient hospital services: number of contacts in the last 12 months, from 0 to 6 or more contacts.
Multivariable linear regression models were calculated for the utilization of all five health services
as dependent variables and the Navigational HL score and five socio-demographic and socio-
economic determinants as independent variables. Only the regression models for the use of
GPs/family doctors and medical or surgical specialists are reported here since the models for the
other three indicators of health care utilization just explain 1% to 2% of the variance in total. The
multivariable regression models for the use of GPs/family doctors explain 7% of the variance (var-
ying from 5% (FR) to 15% (DE)) (Table 10.11) and for medical or surgical specialists 5% (varying
from 4% (FR) to 12% (DE)) (Table 10.12).
The values of β coefficients for Navigational HL in the models are the second strongest, but with
a slight β=-0.09 for GPs/family doctors (varying from β=0.03 (CZ) to -0.09 (DE), significant only
for two countries), and the fourth strongest with β=-0.06 for medical or surgical specialists
(varying from 0.01 (BE, SI) to -0.07 (DE), significant only for one country). Thus, when confounding
variables are controlled for, Navigational HL is only relevant for two indicators of health services
utilization and is only significant for two and one country respectively.

Table 10.11:
Multivariable linear regression models of utilization of GPs/family doctors by Navigational HL
and five social determinants (standardized coefficients (β) and R2), for each country and for all
HL-NAV -0.07 -0.02 -0.04 0.03 -0.09 -0.05 0.02 -0.03 -0.09
Gender female 0.1 0.08 0.03 0 0.07 0.08 0.16 0.08 0.07
Age in years 0.23 0.16 0.18 0.17 0.31 0.18 0.16 0.17 0.21
Education -0.01 -0.18 -0.03 -0.09 -0.02 0 -0.03 0 0.04
Level in society 0.03 -0.06 -0.08 -0.02 -0.04 -0.01 0 0.01 0
Financial deprivation 0.09 -0.01 0.08 0.12 0.09 0.09 0.08 0.11 0.06
R2 0.1 0.09 0.06 0.06 0.15 0.05 0.07 0.06 0.07

Valid count 2543 981 1982 1503 1792 2003 1012 3151
Total count 2967 1000 2502 1599 2143 2003 1247 3360

NAV-HL score: from 0=minimal HL to 100=maximal HL.
Table 10.12:
Multivariable linear regression models of utilization of medical or surgical specialists by
Navigational HL and five social determinants (standardized coefficients (β) and R2), for each
HL-NAV -0.04 0.01 -0.04 -0.01 -0.07 -0.03 -0.05 0.01 -0.06
Gender female 0.21 0.14 0.08 0.13 0.09 0.13 0.14 0.06 0.12
Age in years 0.11 0.16 0.11 0.17 0.28 0.12 0.1 0.16 0.15
Education 0.1 -0.04 0.11 0.06 0.06 0.1 0.18 0.08 0.07
Level in society 0 -0.09 -0.06 -0.03 -0.04 0.04 0.06 -0.01 0
Financial depri-
vation 0.05 0.06 0.13 0.06 0.09 0.1 0.06 0.07 0.03
R2 0.07 0.06 0.05 0.04 0.12 0.05 0.05 0.04 0.05
Valid count 2554 971 1980 1503 1815 2003 1012 3146
Total count 2967 1000 2502 1599 2143 2003 1247 3360

HL-NAV score: from 0=minimal HL to 100=maximal HL.
Indicators for health status
In comparison with the indicators for health care utilization, the correlations of Navigational HL
with indicators for health status are considerably stronger, being strongest on average for self-
perceived health (ρ=-0.19, varying from ρ=-0.13 (CZ, PT) to ρ=-0.24 (SI)), followed by limited in
activities due to health problems (ρ=0.16, varying from ρ=0.08 (CZ) to ρ=0.21 (PT)), and long-
term illnesses/health problems (ρ=-0.11, varying from ρ=-0.01 (BE) to ρ=-0.19 (PT)). On average,
associations are rather linear and continuous, but somewhat less consistent for individual coun-
tries (Annex 10, Figures A 10.14 to A 10.16).

Table 10.13:
Spearman correlations (ρ) of Navigational HL with three indicators for health status, for each
HL-NAV and… AT BE CH CZ DE FR PT SI ALL
Health in general -0.19 -0.16 -0.14 -0.13 -0.21 -0.15 -0.13 -0.24 -0.19
Long-term illnesses/
health problems -0.08 -0.01 -0.07 -0.02 -0.11 -0.08 -0.19 -0.15 -0.11
Limited in activities due
to health problems 0.11 0.14 0.12 0.08 0.15 0.01 0.21 0.19 0.16
Long-term illness: 3 categories: (1) none, (2) one, (3) more than one, except for SI where 2 categories were used (1)
none, (2) one or more.
The linear multivariable regression models testing the potential effects of Navigational HL on se-
lected indicators for health status explain on average much more variance in comparison with the
models for health care utilization. On average, the highest R2 is observed for self-perceived health
(the model explains 18% of the variance (varying from 12% (BE) to 32% (PT)) (Table 10.14), followed
by long-term illnesses/health problems with 12 % (variation from 8 % (CH) to 17 % (PT)) (Table 10-
15), and limited in activities due long-term illnesses/health problems with 9 % (variation from 4 %
(FR) to 18 % (DE)) (10.16).
The β values for Navigational HL are also comparably higher for self-perceived health (β=-0.13,
varying from -0.01 (PT) to -0.13 (AT, DE), significant for seven out of the eight countries), for
limitations due to health problems with an overall β of 0.11 (varying from -0.04 (FR) to 0.10 (BE,
PT), significant for five out of the eight countries), and less distinct for long-term illnesses/health
problems with an overall β of -0.07 (varying from +0.01 (BE, CZ) to -0,06 (SI), significant for only
two countries). Thus, Navigational HL has slight but significant potential effects on at least two
indicators of health status.
Table 10.14:
Multivariable linear regression models of self-perceived health (standardized coefficients (β) and
R2) by Navigational HL and five core social determinants, for each country and for all countries
(equally weighted)
HL-NAV -0.13 -0.1 -0.1 -0.07 -0.13 -0.06 -0.01 -0.12 -0.13
Gender female -0.02 0.04 -0.05 -0.03 -0.03 -0.02 0.09 0.02 0
Age in years 0.24 0.07 0.22 0.35 0.41 0.23 0.33 0.37 0.27
Education -0.06 -0.08 -0.04 -0.11 -0.03 0.02 -0.13 -0.08 -0.04
Level in society -0.11 -0.28 -0.18 -0.13 -0.08 -0.23 -0.11 -0.07 -0.16
Financial deprivation 0.16 -0.04 0.16 0.15 0.13 0.12 0.18 0.19 0.13
R2 0.15 0.12 0.15 0.25 0.26 0.15 0.32 0.3 0.18

Valid count 2584 988 1982 1523 1843 2003 1012 3157
Total count 2967 1000 2502 1599 2143 2003 1247 3360


Table 10.15:
Multivariable linear regression models of long-term illnesses/health problems by Navigational
HL and five core social determinants (standardized coefficients (β) and R2), for each country and
for all countries (equally weighted).
HL-NAV -0.04 0.01 -0.04 0.01 -0.05 -0.04 -0.1 -0.06 -0.07
Gender
0.03 0.06 -0.01 0.02 0.04 -0.01 0.04 0.05 0.03
female
Age in
0.24 0.23 0.26 0.27 0.41 0.29 0.31 0.3 0.29
years
Educa-
-0.08 -0.08 0.01 -0.02 0.03 0.01 0 -0.04 -0.02
tion
Level in
-0.07 -0.1 -0.08 -0.05 -0.06 -0.02 -0.01 -0.01 -0.04
society
Finan-
cial
0.13 0.07 0.05 0.11 0.08 0.11 0.16 0.11 0.09
depriva-
tion
R2 0.12 0.09 0.08 0.1 0.2 0.1 0.17 0.15 0.12

Valid
2579 988 1980 1523 1819 2003 1008 3156
count
Total
2967 1000 2502 1599 2143 2003 1247 3360
count

Long-term illness: 3 categories: (1) none, (2) one, (3) more than one, except for SI, where 2 categories were used (1) none,
(2) one or more.
Table 10.16:
Multivariable linear regression models of limited in activities due health problems (by
Navigational HL and five core social determinants standardized coefficients (β) and R2), for each
HL-NAV 0.07 0.1 0.09 0.03 0.08 -0.04 0.1 0.09 0.11
Gender female 0 -0.12 -0.04 -0.03 -0.03 0.03 -0.13 -0.04 -0.04
Age in years -0.13 0.03 -0.15 -0.19 -0.35 -0.11 -0.17 -0.26 -0.2
Education 0.09 0.13 -0.01 0.1 0 -0.03 0.05 0.05 0.03
Level in society 0.13 0.14 0.14 0.04 0.07 0.03 0.03 0.06 0.05
Financial deprivation -0.16 0.1 -0.11 -0.15 -0.13 -0.16 -0.2 -0.13 -0.12
R2 0.1 0.09 0.08 0.09 0.18 0.04 0.16 0.16 0.09

Valid count 1996 472 1977 1520 1724 2003 806 3157
Total count 2967 1000 2502 1599 2143 2003 1247 3360

With the inclusion of General HL in the regression models (Annex 10.8, Table A 10.16 to Table A
10.18), the explained variance for all countries rises only marginally from 18% to 20% (varying
from 13% (BE) to 33% (PT)) for general health and from 9% to 11% (varying from 5% (FR) to 19%
(PT)) for limited in activities due to health problems. No changes are observed for long-term ill-
nesses/health problems, with explained variance still at 12% (varying from 8% (CH) to 19% (PT)).
The values of β for Navigational HL are drastically reduced, showing that General HL is a much
stronger predictor than Navigational HL for the health status indicators under consideration. Now,
higher Navigational HL is linked to better self-perceived health with β increasing to -0.08 but
significant for only two countries, long-term illnesses/health problems but not significant for any
country, and limitations in activities due to health problems with β=-0.11 but significant for only
one country. Thus, Navigational HL only has small extra direct effects on indicators of health sta-
tus, in addition to the potential effects of General HL.
10.3 Discussion and Conclusion
The HLS19 introduced a new measurement instrument for Navigational HL based on an explicit
model and definition of Navigational HL (Griese et al. 2020) which is related to the model and
definition of the HLS-EU Consortium (Sørensen et al. 2012) and its somewhat revised operation-
alization in the HLS19. The instrument was applied in eight countries participating in the HLS19,
with a total of over 16,000 respondents. For these countries, the distributions of individual items

and the Navigational HL scores as well as the social gradients, determinants, and selected conse-
quences of Navigational HL were analyzed and demonstrated for the first time.
The newly developed HLS19-NAV instrument was extensively investigated psychometrically. The
results of analyses used (Cronbach’s alpha, CFA, and Rasch) indicate that the instrument mostly
proved to be satisfactory across the countries included in the survey. This demonstrates that the
HLS19-NAV is a suitable instrument to measure Navigational HL in different countries, languages,
and using different survey methods. Nevertheless, there is also room for some improvements
regarding the under-discrimination of individual items and DIF in some countries.
The Navigational HL topic was included as an optional package in the HLS19 because many
healthcare systems – as the literature suggests – are extremely complex and suffer from a high
level of disintegration, which in turn places great demands on users in terms of orientation, nav-
igation, and use that cannot be managed easily. This also applies to dealing with navigation-
related information. The findings confirm this and show that Navigational HL is low in adult resi-
dent populations and that dealing with the healthcare system and with information essential to
navigating it is difficult for a large proportion of potential users. This also becomes evident when
the findings are compared to those on General HL (and other specific health literacies in the HLS19),
showing Navigational HL to be lower than the other measured health literacies in the HLS19.
Regarding the different information tasks, the results of the survey show that information espe-
cially on the systemic level is experienced as being (very) difficult in many countries and that basic
knowledge and skills are therefore required to utilize information about the healthcare system, its
organization, and how it functions. Processing information on (political) changes, reforms within
the healthcare system, and patient rights, the latter being particularly important to enable more
autonomy and co-production, is also seen as being (very) difficult. In addition, there seems to be
a lack of sufficient support to overcome such challenges. All of this should be considered when
developing interventions to facilitate navigation of the healthcare system and improve the infor-
mation needed to do so.
This also applies to results on the organizational level where many experience it as being (very)
difficult to find information on quality-related issues. Orientation and navigation within healthcare
facilities are also experienced as being difficult, a finding that underscores the importance of
organizational HL (Pelikan 2019; Farmanova et al. 2018; Brach et al. 2012) and demonstrates the
need to make the healthcare system concerned more user-friendly and easily navigable, including
the immediate (literal and interactive) environments in which health care is sought
(Rudd/Anderson 2006; Rudd 2004).
On the interactive level 8 almost half of all respondents (46.5%) found it (very) difficult “to stand up
for yourself if your health care does not meet your needs”. The answers varied greatly among
countries, but the results indicate that changes in the patient’s role on the interactive level towards
more collaboration, informed decision making, and negotiation of health care based on one’s own
8 Only one item is included here, see Subsection 10.1.2.
preferences still pose difficulties that cannot be met adequately by patients and healthcare pro-
fessionals. As a result, communication with healthcare professionals is often one-sided. Thus,
replacing traditional (paternalistic) patterns of interaction with new ones remains a challenge
across countries, as was also demonstrated by the survey results on HL relating to communication
with physicians in health care services which show that respondents considered it particularly
important to be given more time to process information (HLS19-COM4) in a simplified language
that they could understand easily (HLS19-COM7) (see Chapter 11).
In addition, the findings reveal country-specific characteristics that are presumably based on the
structure of the healthcare system concerned. Therefore, improving Navigational HL also means
to including and further investigating country or context-specific challenges when developing in-
terventions to strengthen Navigational HL.
Furthermore, the results show that Navigational HL is distributed differently among various sub-
population groups. People with limited financial resources and low level in society/poor social
status have lower Navigational HL. Thus, a social gradient for Navigational HL has been demon-
strated that is more pronounced in some countries than in others. Less pronounced but also worth
mentioning is the fact that in some countries, an age gradient was shown that should be specifi-
cally considered in the production of navigation-related information. These results on Naviga-
tional HL, like those on General HL, underscore the importance of identifying disadvantaged
groups when developing interventions. Healthcare systems must become easier to use, especially
for these disadvantaged groups, through simple, transparent, clear, and user-friendly structures
and service models as well as through more targeted, group-specific information.
The HLS19 also shows that General HL can be interpreted as the strongest predictor of Navigational
HL, which suggests an overlap in basic competencies that are significant for both General HL and
Navigational HL. This is an important result since investments in General HL could also be bene-
ficial for Navigational HL and vice versa. Similar assumptions can be made for other health litera-
cies that have been examined, especially Communicative HL with physicians (Chapter 11) and
Digital HL (Chapter 12). On the one hand, the positive relationship between Navigational HL and
other HL measures was expected, since the measurement instruments were developed against the
background of a common understanding, definition, and operationalization of HL and the HLS19-
NAV is therefore part of a ‘family’ of new HL measurement tools; on the other hand, this points to
common interfaces between the concepts. Good communication skills in patients but also the
general circumstances created by physicians (e.g., enough time, opportunities for queries) obvi-
ously lead to a better understanding of information relevant for navigating healthcare systems.
Furthermore, much of the navigation-related information is available for users online. Such infor-
mation may be used for initial orientation in the healthcare system or when searching for a suitable
health service, access to it, and its modalities of utilization but also to clarify open questions on
navigational issues after consultations with health professionals. A hypothesis derived from the
results is that this is more successful when Digital HL is also high. Examining the relationship
between Navigational HL, Communicative HL with physicians, and Digital HL should be a topic of
further research.
Another important finding is that, like for General HL (Pelikan et al. 2018; Sørensen et al. 2015),
low Navigational HL is associated with implications for health status. The inclusion of General HL

in the regression models significantly reduced this effect, but some effect persists regarding self-
perceived health. Surprisingly, Navigational HL is less clearly associated with the use of health
care. On the basis of previous trends in HL research (Berens et al. 2018; Ownby et al. 2014;
Berkman et al. 2011), it was assumed that higher Navigational HL would be associated with lower
health care utilization. This was only partly true in a multivariable linear regression model for two
indicators of primary care utilization. In the future, however, more specific potential outcome
measures should be investigated for Navigational HL (e.g., indicators of misuse of the healthcare
system, problems with access or navigation).
Limitations
A comparative analysis of Navigational HL among participating countries is only possible to a

limited extent due to differences in the survey methods used. For example, in contrast to the other
countries, data collection in Germany took place before the Covid-19 pandemic; indications that
Navigational HL in Germany tended to improve slightly during the Covid-19 pandemic are pro-
vided in a complementary survey conducted as part of the national HL survey (Schaeffer et al.
2021).
The interactive level of Navigational HL is underrepresented in this Chapter because Communica-

tion with physicians in health services HL (part of the micro level of Navigational HL) was measured
separately in the HLS19. To enable the instrument to be used and tested as a stand-alone instru-
ment in the future, it would be desirable to elaborate on this level in an extended version.
Conclusions
The HLS19-NAV was developed, tested, and used for the first time in the HLS19 in eight different
countries. This added valuable information about dealing with information in the specific context
of navigating healthcare systems, but it also needs refining and testing. Regarding future meas-
urements of Navigational HL in an international context, it will be important to carefully review the
quality of translations into the national language(s) concerned to ensure comprehensibility for the
various population groups. In addition, it would be desirable to pay equal attention to the inter-
active or communicative level (micro level) in the future, which was underrepresented in this study.
A starting point here could be the items of the HLS19-COM. Further expansion and research are
required to test more specific potential consequences of Navigational HL.
Overall, the results confirm that navigating healthcare systems represents an “unfamiliar context”
for many respondents (Nutbeam 2009: 304), namely one that requires special knowledge and
special HL. In future, it will therefore be important to strengthen Navigational HL and General HL
at all levels of healthcare systems. The results provide a number of suggestions on how this can
be done (see Chapter 15 for recommendations). Implementation is of major importance since
Navigational HL is very low for considerable proportions of adult resident populations in many
countries.
10.4 References
Andrich, D.; Sheridan, B. (2019): RUMM2030 Plus [Computer software]. Rumm Laboratory Pty Ltd.
Berens, Eva-Maria; Vogt, Dominique; Ganahl, Kristin; Weishaar, Heide; Pelikan, Jürgen; Schaeffer,
Doris (2018): Health Literacy and Health Service Use in Germany. In: HLRP: Health Literacy
Research and Practice 2/2:e115-e122
Berkman; Sheridan, Stacey L; Donahue, Katrina E; Halpern, David J; Crotty, Karen (2011): Low
health literacy and health outcomes: an updated systematic review. In: Annals of internal me-
dicine 155/2:97-107
Bittlingmayer, Uwe H; Dadaczynski, Kevin; Sahrai, Diana; van den Broucke, Stephan; Okan, Orkan
(2020): Digitale Gesundheitskompetenz–Konzeptionelle Verortung, Erfassung und Förderung
mit Fokus auf Kinder und Jugendliche. In: Bundesgesundheitsblatt-Gesundheitsforschung-
Gesundheitsschutz 63/2:176–184
Brach, C.; Keller, D.; Hernandez, L. M.; Baur, C.; Parker, R.; Dreyer, B.; Schyve, P.; Lemerise, A. J.;
Schillinger, D. (2012): Ten attributes of health literate health care organizations. Institute of
Medicine., New York. Unveröffentlicht
Brown, Stephen L; Teufel, JAMES A; Birch, David A (2007): Early adolescents perceptions of health
and health literacy. In: Journal of School Health 77/1:7-15Dow, Clare M.; Roche, Patricia A.;
Ziebland, Sue (2012): Talk of frustration in the narratives of people with chronic pain. In:
Chronic Illness 8/3:176–191
Ellen, Moriah E.; Wilson, Michael G.; Vélez, Marcela; Shach, Ruth; Lavis, John N.; Grimshaw, Jeremy
M.; Moat, Kaelan A. (2018): Addressing overuse of health services in health systems: a critical
interpretive synthesis. In: Health research policy and systems 16/1:48
Farmanova, Elina; Bonneville, Luc; Bouchard, Louise (2018): Organizational Health Literacy: Review
of Theories, Frameworks, Guides, and Implementation Issues. In: Inquiry: a journal of medical
care organization, provision and financing 55/:46958018757848
Fields, Beth; Rodakowski, Juleen; James, A. Everette; Beach, Scott (2018): Caregiver health literacy
predicting healthcare communication and system navigation difficulty. In: Families, systems &
health : the journal of collaborative family healthcare 36/4:482–492
Griese, Lennert; Berens, Eva-Maria; Nowak, Peter; Pelikan, Jürgen M.; Schaeffer, Doris (2020):
Challenges in Navigating the Health Care System: Development of an Instrument Measuring
Navigation Health Literacy. In: International Journal of Environmental Research and Public
Health 17/16:5731

Groene, Raluca Oana; Rudd, Rima E. (2011): Results of a feasibility study to assess the health
literacy environment: navigation, written, and oral communication in 10 hospitals in Catalonia,
Spain. In: Journal of Communication in Healthcare 4/4:227–237
Gui, Xinning; Chen, Yunan; Pine, Kathleen H (2018): Navigating the Healthcare Service" Black Box"
Individual Competence and Fragmented System. In: Proceedings of the ACM on Human-Com-
puter Interaction 2/CSCW:1-26
Hofmarcher, M. M.; Oxley, H.; Rusticelli, E. (2007): Improved Health System Performance through
better Care Coordination: OECD Health Working Papers, No. 30. OECD Publishing. Unverö-
ffentlicht
Institute of Medicine Committee on Quality of Health Care in America (2001): Crossing the Quality
Chasm: A New Health System for the 21st Century. National Academies Press (US) Copyright
2001 by the National Academy of Sciences. All rights reserved., Washington (DC)
Knekta, Eva; Runyon, Christopher; Eddy, Sarah (2019): One Size Doesn’t Fit All: Using Factor Anal-
ysis to Gather Validity Evidence When Using Surveys in Your Research. In: CBE Life Sciences
Education 18/1:rm1. doi: 10.1187/cbe.1118-1104-0064.
Levin-Zamir, Diane; Bertschi, Isabella (2018): Media health literacy, eHealth literacy, and the role
of the social environment in context. In: International journal of environmental research and
public health 15/8:1643
Levy, Helen; Janke, Alex (2016): Health Literacy and Access to Care. In: Journal of health commu-
nication 21 Suppl 1/:43–50
Lynn, M. R. (1986): Determination and quantification of content validity. In: Nursing research
35/6:382–385
174
Nutbeam, D. (2009): Defining and measuring health literacy: what can we learn from literacy stud-
ies? In: International Journal of Public Health 54/5:303–305
Ørtenblad, Lisbeth; Meillier, Lucette; Jønsson, Alexandra R. (2018): Multi-morbidity: A patient per-
spective on navigating the health care system and everyday life. In: Chronic Illness 14/4:271–
282
Osborne, Richard H.; Batterham, Roy W.; Elsworth, Gerald R.; Hawkins, Melanie; Buchbinder,
Rachelle (2013): The grounded psychometric development and initial validation of the Health
Literacy Questionnaire (HLQ). In: BMC public health 13/:658
Ownby, Raymond L.; Acevedo, Amarilis; Jacobs, Robin J.; Caballero, Joshua; Waldrop-Valverde,
Drenna (2014): Quality of life, health status, and health service utilization related to a new
measure of health literacy: FLIGHT/VIDAS. In: Patient education and counseling 96/3:404–410
Paasche-Orlow, Michael K; Wolf, Michael S (2007): The causal pathways linking health literacy to
health outcomes. In: American journal of health behavior 31/1:19-26
Parker, R. M. (2009): Measuring Health Literacy: What? So what? Now what? In: Measures of health
literacy. Hg. v. Hernandez, Lyla M. Institute of Medicine of the National Academies, Washing-
ton, D.CS. S91–S98
Pelikan, Jürgen M. (2019): Health-literate healthcare organizations. In: International Handbook of

Health Literacy – Research, Practice and Policy across the Life-Span. Hg. v. Okan, O, Bauer, U,
Pinheiro, P, Levin-Zamir, D, Sørensen, K. Policy Press, 539-554
Pelikan, Jürgen M.; Ganahl, Kristin; Roethlin, Florian (2018): Health literacy as a determinant, me-
diator and/or moderator of health: Empirical models using the European Health Literacy Sur-
vey dataset. In: Global health promotion:1757975918788300
Polit, Denise F.; Beck, Cheryl Tatano (2006): The content validity index: are you sure you know
what's being reported? Critique and recommendations. In: Research in nursing & health
29/5:489–497
R Core Team (2020): R: A language and environment for statistical computing. R Foundation for
Statistical Computing, Vienna
Rasch, Georg (1960): Studies in mathematical psychology: I. Probabilistic models for some intel-
ligence and attainment tests.
tical Software 48/2:1-36.https://www.jstatsoft.org/v48/i02/
Rudd, R. E. (2004): Navigating Hospitals. In: Literacy Harvest 11/1:19–24
Rudd, R. E.; Anderson, J. E. (2006): The Health Literacy Environment of Hospitals and Health Cen-
ters. Department of Society, Human Development, & Health, Boston
Rudd, Rima; Kirsch, Irwin; Yamamoto, Kentaro (2004): Literacy and Health in America. Policy In-
formation Report. Educational Testing Service, Princeton, NJ

Schaeffer, Doris; Berens, Eva-Maria; Gille, Svea; Griese, Lennert; Klinger, Julia; de Sombre, Steffen;
Vogt, Dominique; Hurrelmann, Klaus (2021): Gesundheitskompetenz der Bevölkerung in
Deutschland vor und während der Corona Pandemie: Ergebnisse des HLS-GER 2. Interdiszip-
linäres Zentrum für Gesundheitskompetenzforschung (IZGK), Universität Bielefeld,
https://doi.org/10.4119/unibi/2950305
Schaeffer, Doris (2017): Chronische Krankheit und Health Literacy. In Schaeffer, D. & Pelikan, J.M.
(Eds.): Health literacy. Forschungsstand und Perspektiven. Bern: Hogrefe. 53–70.
Schaeffer, Doris (2004): Der Patient als Nutzer: Krankheitsbewältigung und Versorgungsnutzung
im Verlauf chronischer Krankheit. Bern: Huber.
Snelgrove, Sherrill; Liossi, Christina (2013): Living with chronic low back pain: a metasynthesis of
qualitative research. In: Chronic Illness 9/4:283–301
review and integration of definitions and models. In: BMC Public Health 12/80: http://www.bi-
SVR (2007): Kooperation und Verantwortung. Voraussetzungen einer zielorientierten Gesund-

heitsversorgung: Gutachten 2007. Sachverständigenrat zur Begutachtung der Entwicklung im
Gesundheitswesen, Berlin
SVR Economy (2017): Für eine zukunftsorientierte Wirtschaftspolitik: Jahresgutachten 2017/18.

Sachverständigenrat zur Begutachtung der gesamtwirtschaftlichen Entwicklung, Wiesbaden
Tennant, Alan; Conaghan, Philip G. (2007): The Rasch measurement model in rheumatology: what
is it and why use it? When should it be applied, and what should one look for in a Rasch paper?
In: Arthritis and rheumatism 57/8:1358–1362
WHO (2016): Shanghai Declaration on health promotion in the 2030 Agenda for Sustainable De-
velopment. World Health Organization, Shanghai, China
Yan, Zi; Mok, Magdalena Mo Ching (2012): Validating the coping scale for Chinese athletes using
multidimensional Rasch analysis. In: Psychology of Sport and Exercise 13/3:271–279
11 Communicative Health Literacy with physi-
cians in health care services
Authors:
Peter Nowak (AT)
Hanne S. Finbråten (NO)
Éva Bíró (HU)
Rana Charafeddine (BE)
Julien Mancini (FR)
Lennert Griese (DE)
Zdenek Kucera (CZ)
Thomas Link (AT)
Jürgen M. Pelikan (AT)
Mitja Vrdelja (SI)
Chapter 11 / Communicative Health Literacy with physicians in health care services 233
11.1 Background and development of instrument

measures of Communicative Health Literacy in health
care services
Communicative HL might be essential to participate actively in health communication with health

professionals, to obtain and understand information, and to use that information to manage
health. Health care services play a critical role in developing HL in the population. People in need
of health care are more open to health issues and therefore are in a “teachable moment” for health
education (Lawson/Flocke 2009). Currently there is a focus on health services being developed as
health literate organizations (Brach et al. 2012) to strengthen the HL of the population as well as
to promote patient-centered care, health promotion, and prevention (Palumbo 2021).
Typically, there are two ways to provide health-related information within the health services: 1)
health information provided through diverse media (apps, brochures, flyers, info-sheets, posters,
websites, videos, magazines, TV) and 2) (inter)personal communication between a health profes-
sional and a user of health care.
Communication in health care in the context of this optional package refers to physician 9-patient
communication within the healthcare system. This research focuses on face-to-face interactions
in co-presence, although communication within health care services also includes telephone and
video consultations. Communication is a core task for health professionals and patients when
establishing diagnoses, deciding on and implementing treatments, organizing adequate health
care, and maintaining good health.
People strongly rely on personal communication with health professionals (Chen et al. 2018). Thus,
health care communication and patient participation in health care has been recognized as a de-
cisive part of HL as well as a critical determinant of successful disease management and health
outcomes (van der Heide et al. 2018; Amalraj et al. 2009; Paasche-Orlow 2007).
The quality of communication between patients and health professionals – especially physicians –
is one of patients’ main concerns and the most important source of (dis-)satisfaction with health
care (Stahl/Nadj-Kittler 2013; Langewitz et al. 2002). Good communication in health care has been
shown to be associated with a wide range of improved health care outcomes and also contributes
to the workplace satisfaction of health professionals (Sator et al. 2015; Street et al. 2009;
Mead/Bower 2002). However, communication in almost all healthcare systems – as the findings of
the HLS-EU study revealed (HLS-EU Consortium 2012) – needs improvement on all levels.
9 The term ‘doctor’ was used in the questionnaire to ensure plain language. As the term ‘physician’ is used more in formal
discourse, this term is used in this chapter.
This focus on communication is becoming increasingly important due to changes in the patients’
role, such as the demand for more patient involvement and shared decision making (Elwyn et al.
2016; Meijers et al. 2019) as well as for formal informed consent. Health professionals are not
only required to communicate more comprehensively and in a patient-centered manner but must
also pay more attention to the provision of information, patient empowerment, and shared deci-
sion making to enable patients to become co-producers of their own health care (Elwyn et al.
2020; Palumbo/Lars Tummers 2016).
Especially for people with limited (health) literacy, personal oral communication is usually the main
source of information on their health and health care. However, patients with lower HL report on
worse communication with health professionals than those with higher HL (Castro et al. 2007;
Schillinger et al. 2004). Furthermore, patients with lower literacy have been shown to be less likely
to ask health professionals questions (Katz et al. 2007). Thus, the good communication skills of
health professionals could contribute to health equality.
Nutbeam (2000); Nutbeam (2008); Nutbeam, Don (2009) indicate the importance of interactive
processes by defining Communicative/Interactive HL as one of three relevant domains of HL and
linking it especially to the goal of greater autonomy and personal empowerment. Nutbeam (2000)
defines Communicative/Interactive literacy as: “more advanced cognitive and literacy skills which,
together with social skills, can be used to actively participate in everyday activities, to extract
information and derive meaning from different forms of communication, and to apply new infor-
mation to changing circumstances”. Nutbeam (2008) highlights the importance of developing spe-
cific measurements for these “oral literacy and social skills”. This part of the study aims to develop
such a measurement instrument focusing on communication with physicians in the specific con-
text of health care to be used in a general population-based survey.
Expanding Nutbeam’s understanding, the following definition of Communicative/Interactive HL in

health care is used in this study: Communicative HL refers to patients’ communicative and social
skills that enable them to actively engage in face-to-face encounters with healthcare profession-
als, to give and seek information, derive meaning from it, and apply this information in decision
making and in co-producing their health care. In the HLS19, attention was focused on physician-
patient communications in this regard.
To date research on measuring Communicative HL in health care has been quite limited
(Nouri/Rudd 2015). Roter et al. (2007) developed an Oral Literacy Demand Framework for genetic
counseling dialogues including three language elements: (1) use of technical terms, (2) general
complexity of language, and (3) structural characteristics of the dialogue. Building on Nutbeam’s
concepts, Ishikawa et al. (2008) focused on interactive HL as “Communicative HL”, highlighting the
importance of personal communication in HL. Rubin et al. (2011) used “the term ‘interactive health
literacy’ to denote patient/consumer’s propensity to exert individual agency in actively participat-
ing and seeking information in encounters with healthcare providers and information sources.” On
this basis, Rubin et al. (2011) developed the Measure of Interactive Health Literacy (MIHL) to code
the interactive skills of a consumer in telephone conversations. O’Hara et al. (2018) published the
concept of the “Conversational Health Literacy Assessment Tool (CHAT)” to provide a short action-
able survey tool for the clinical context to assess patients’ ability to interact with health profes-
sionals, but the ten questions developed for CHAT mainly focus on general health information-
seeking behavior and health promotion activities. Only one question focuses on interactive be-
havior.
Conversely, research on measuring the quality of communication processes between healthcare

providers (especially physicians) and patients in general is well established and has a longstanding
tradition mainly in communication science, linguistics, and medical psychology. The main con-
ceptual background for measuring the quality of communication in health care is the rise of “pa-
tient-centered medicine” (Balint 1969) over the last 50 years, with patient-centered communica-
tion being one of its most important dimensions (Zill et al. 2015). Very diverse measurement ap-
proaches have been developed (Ball et al. 2018; Brouwers et al. 2017; Zill et al. 2014; Eveleigh et
al. 2012; Epstein et al. 2005), that can be roughly categorized by:
» Observed aspects of the communication: linguistic and communicative behavior, self-re-

ported psychological and emotional experiences, perceptions and attitudes of patients
and/or doctors;
» Observer perspectives: the physician’s perspective, the patient’s perspective, and third-party
observers (raters, teachers/supervisors, co-workers) (Burt et al. 2018);
» Methodology of data generation and analysis: e.g., participant observation, recording inter-
actions, rating systems and checklists, interviews with doctors and/or patients, surveys for
patients and doctors;
» Purpose, context, setting, and population of measurement: basic research, assessment and
feedback in training, workplace-based assessments, quality management, intercultural com-
munication, etc.
Arguments for providing a new measure and the pro-

cedure for developing the measure
Only a few studies have tried to integrate HL instruments from these diverse research traditions
into the quality of communication in health care. One example is a French study (Ousseine et al.
2019), in which general measures of HL and numeracy were combined with a standard measure-
ment tool for Shared Decision Making (SDM). Ishikawa et al. (2008) included five questions about
Communicative HL in their instrument. However, the tasks included in these questions were not
specific to personal interaction with health professionals, such as physicians. The questions in-
cluded in Chinn/McCarthy (2013) do relate to this setting but this instrument lacks aspects of
active participation in communication, such as expressing one’s own preferences and being in-
volved in decision making. Hence, to our knowledge there is no comprehensive instrument that
captures all of the main tasks described by Nutbeam (2000), takes into account the important
aspects of communication in health care derived from the diverse research traditions mentioned
above, and considers the basic competencies of information processing according to the concep-
tual model of HL used by the HLS-EU Consortium (Sørensen et al. 2012) that is also the basic
measurement approach of the HLS19.
As a consequence, a conceptual framework was identified that integrates the most important as-
pects of communication in health care derived from these diverse research traditions, namely the
well-established “Calgary-Cambridge Guide to the Medical Interview” (C-CG) (Silverman et al.
2013). This framework has been developed over the last 25 years and integrates the results of
different research traditions to serve as a guide to teaching health professionals patient-centered
communication skills; it is also used as a framework for assessments (e.g. Iversen et al. 2020).
The C-CG describes 56 single communicative practices of a health professional in six main phases
of a routine interaction in health care. Within these six main phases, the communicative tasks of
patients can be identified that need to be considered in the conceptual framework for Communi-
cative HL (see Table 11.1). Relating to the definition of interactive HL (Nutbeam 2000), we also
identified “participating actively” as an additional overarching task that is relevant in all other
communicative tasks for patients (see Table 11.1).
Table 11.1:
Overview of the main communicative practices of health professionals in the Calgary-Cambridge
Guide to the Medical Interview (C-CG) and the main communicative tasks of patients, which
together constitute the Conceptual Framework for Communicative HL
C-CG main phases Patients’ communicative tasks

1. Initiating the session 1. Opening the session and giving initial information
2. Gathering information 2. Giving full information
Participating actively
3. Providing structure 3. Understanding and following the agenda
4. Expressing one’s own views/feelings and trusting the health profes-

4. Building relationship
sionals (e.g., physicians)
5. Understanding and decision making (extracting information and deriv-
5. Explanation and planning
ing meaning)
6. Closing the session 6. Final understanding, agreement (and application)
Based on this conceptual framework, steps to develop an instrument for measuring Communica-
tive HL were taken by a working group of representatives from the HLS19 countries led by the first
and second authors of this chapter. The detailed procedure for developing the instrument is shown
in Figure 11.1. As a general approach, the aim was to identify at least one question/item per main
communicative task to capture the most important challenges of HL in health care communication.
These questions were selected in a multistage process. In a first step the conceptual framework
developed for the existing HLS-Q47 instrument was applied (HLS-EU Consortium 2012). Six items
on communication were already included in the HLS-Q47 (Q5, Q8, Q9, Q10, Q13, and Q16), but
according to the C-CG, these items only measure aspects of two main phases of physician-patient
interactions (explanation and planning; closing the session). In particular, key patient information
tasks were not captured by the HLS-Q47, e.g., presenting their own concerns and preferences,
asking questions.
Therefore, as a second step a targeted literature search was carried out to identify questionnaires
and possible items on HL in the context of health care communication in English and German
literature. In total, 20 very diverse instruments (Bieber et al. 2010; Burt et al. 2014; CAHPS 2017;
Campbell et al. 2007; Chinn/McCarthy 2013; Ernstmann et al. 2017; ESTAT 2017; Farin et al. 2013;
Gibney/Moore 2018; HCAHPS 2019; Ishikawa et al. 2008; Makoul et al. 2007; Maly et al. 1998;
Myerholtz et al. 2010; National Institutes of Health 2018; Osborne et al. 2007; Picker Institute
Europe 2013; Smith et al. 2006; Sustersic et al. 2018; ten Klooster et al. 2012; van der Heide et
al. 2015; Waldherr et al. 2019; Zegers et al. 2020) were included in addition to the HLS-Q47. These
20 instruments contained 183 items all together on health care communication. The conceptual
framework (Table 11.1) was applied to map these 183 questions and select a preliminary set of
15 questions to measure Communicative HL. This set of items was discussed in several feedback
loops with the Communicative HL working group of the HLS19 and the International Coordination
Center. The understandability and importance of each item were also assessed in two focus groups
involving potential survey participants in Austria (using the German version). Based on these dis-
cussions and the piloting phase, the HLS19-COM-P instrument was developed to focus exclusively
on physician-patient communication and can be used in a longer form (11 items) and a shorter
form (6 items). Originally, the working group considered measuring Communicative HL in dialogue
with health professionals in general. However, during pretesting, the general term “health profes-
sionals” was not well accepted by the participants. Because their experiences differ depending on
the kind of health professional, the term was perceived to be too vague, which made it difficult to
form an opinion and respond to the items. In addition, the status of different health professions
varies widely across the participating countries but that of physicians is quite similar and compa-
rable. Therefore, the group decided to focus on physician-patient communication.
The HLS19-COM-P instrument measures all six main communicative phases of physician-patient
interactions according to the C-CG (Table 11.1) and can be used to analyze the dimensions of
Interactive/Communicative literacy in accordance with Nutbeam (2000) and the basic competen-
cies of information processing according to the conceptual model of HL developed by the HLS-EU
Consortium (Sørensen et al. 2012).
Table 11.2:
Final list of items for the HLS19-COM-P (the short form Q6 is on a grey background)
Patients’ communicative Proposed HLS19 items

tasks
On a scale from very easy to very difficult, how easy would you say it is for you …
1. Opening the session and Item 1 … to describe to your doctor your reasons for coming to the consultation?
giving initial information
Item 2 … to make your doctor listen to you without being interrupted?
2. Giving full information
Item 3 … to explain your health concerns to your doctor?
3. Understanding and fol- Item 4 … to get enough time in the consultation with your doctor?
lowing the agenda
4. Expressing one’s own Item 5 ... to express your personal views and preferences to your doctor?
views and trusting
Item 6 … to get the information you need from your doctor?
Item 7 … to understand the words used by your doctor?
5. Understanding and deci-
Item 8 … to ask your doctor questions in the consultation?
sion making
Item 9 ... to be involved in decisions about your health in dialogue with your
doctor?
6. Final understanding and Item 10 … to recall the information you get from your doctor?
agreement Item 11 … to use the information from your doctor to take care of your health?
Figure 11.1:
Steps in the development of the instrument to measure HL-COM
To ensure transferability to the health systems context of other countries, the items were trans-
lated multiple times back and forth between German and English to obtain feedback from the
other international experts participating in M-POHL. The methodological approach and the items
were also presented and discussed at two M-POHL meetings in March 2019 (Dublin) and August
2019 (Berlin). The agreed English version of the final instrument was included in the HLS19 ques-
tionnaire and made available to the participating countries. Responsibility for the country-specific
translation and integration in national pre-tests was assigned to the principal investigator of each
country using the HLS19-COM-P instruments.
Objectives and research questions on Communicative

HL
The underlying general hypotheses are that (1) better Communicative HL is a precondition for
successful communication between patients and physicians and, at the same time, (2) successful
communication in health care promotes Communicative HL itself and is also partly a precondition
for Navigational HL.
The seven research questions for the optional package “Communicative HL with physicians in
health care” in the HLS19 are:
1. To what extent do the newly developed HLS19-COM-P items constitute an instrument
with acceptable psychometric properties?
2. How is Communicative HL in health care distributed in various countries and different
subpopulations within each country?
3. What are experienced as the most difficult tasks in Communicative HL?
4. How is Communicative HL in health care related to General HL?
5. How is Communicative HL in health care related to Navigational HL?
6. What are the determinants of Communicative HL in health care, based on background
variables measured in the HLS19?
7. What are the consequences of Communicative HL in health care, independent of General
HL, based on relevant correlates measured in the HLS19?
Countries using the HLS19-COM-P
The HLS19-COM-P instrument was successfully applied by nine countries: Austria (AT), Belgium
(BE), Bulgaria (BG), Czech Republic (CZ), Germany (DE), Denmark (DK), France (FR), Hungary (HU),
and Slovenia (SI). An overview of the procedures in the countries using the optional package is
shown in Table 11.3.
Table 11.3:
Countries applying the optional package on Communicative HL
Country Short (SF)/Long Type of data collec- Period of data collection Number of re-
form (LF) tion spondents
AT LF CATI 16.03.2020-26.05.2020 2,967
BE SF CAWI 30.01.2020-28.02.2020 and 1,000

01.10.2020-26.10.2020
BG SF CAPI, CAWI 10.-24.11.2020 865
CZ SF CATI, CAWI 10.-24.11.2020 1,599
DE LF PAPI 13.12.2019-27.01.2020 2,143
DK SF CAWI 11.12.2020-05.02.2021 3,602
FR SF CAWI 27.05.2020-05.06.2020 and 2,003

08.01.2021-18.01.2021
HU SF CATI 02.12.2020-20.12.2020 1,205
SI LF CAPI, paper-and- 09.03.2020-15.03.2020 3,360

pencil, CAWI and 09.06.2020-10.08.2020
11.2 Methods and results
To analyze data and report on Communicative HL in the context of physician-patient interactions,

different indicators were constructed: the perceived difficulties of the individual items of the
HLS19-COM-P-Q11 and the HLS19-COM-P-Q6, the Average Percentage Response Patterns (APRP)
for HLS19-COM-P-Q11 and the HLS19-COM-P-Q6 (for more detail on the methods, see Section
4.4), and the Communicative HL (mean) scores for the long and short version. In the visualization
of the perceived difficulties at item level, the response categories "very difficult" and "difficult"
were combined.
Following the HLS19 procedure for calculating General HL (described in Section 4.2), the Commu-
nicative HL scores are based on a count of the dichotomized HLS19-COM-P items by combining
the categories “easy” and “very easy”. The resulting scores were standardized to the range of 0 to
100 and so the scores indicate the percentage of valid items that were answered with either “easy”
or “very easy” by an individual respondent or, on average, by a group of respondents. Scores were
only computed for respondents who had answered at least 80% of the HLS19-COM-P items.
Vulnerable/disadvantaged subpopulations were identified by comparing the mean score of groups

(e.g., specific age groups or people with a low education) with the overall mean of the population.
The procedures for the regression analyses performed for this chapter are described in detail in
Section 4.6.
The newly developed HLS19-COM-P-Q11 and the short version, HLS19-COM-P-Q6, were evaluated
using both classical and modern test theory. Confirmatory factor analysis (CFA) was performed
using the lavaan package (Rosseel 2012) for R (R Core Team 2020b). For both versions of the
instrument, a single factor model consisting of dichotomized items was assessed. To assess the
internal consistency (reliability) of the HLS19-COM-P, Cronbach’s alpha is reported for each coun-
try. Conducting Rasch modeling, data were tested against the unidimensional polytomous partial
credit Rasch model (PCM) (Masters 1982; Rasch 1960). Analyses at the overall level included those
for data-model fit, dimensionality, and targeting. At the item level, the analyses included evalua-
tion of item fit, response dependency, ordering of the response categories, and the presence of
differential item functioning (DIF). In addition, information on the reliability index, i.e., the Person
Separation Index (PSI), is provided. Rasch analysis was performed using the software
RUMM2030Plus (Andrich/Sheridan 2019). In addition, ConQuest5 (Adams et al. 2020) was used to
investigate infit. For the detailed procedure for the psychometric assessment of the instruments,
see Subsection 4.7.3.
Distributions of individual items by country
Figure 11.2 and Figure 11.3 show the percentages for the perceived difficulty (“very difficult” and
“difficult” responses) of each item on the HLS19-COM-P-Q11 and HLS19-COM-P-Q6 by country.
The proportion perceiving difficulties for the total sample ranges from 4.4% to 25.3% for the HLS19-
COM-P-Q11 items and from 9.2% to 26.2% for the HLS19-COM-P-Q6 items (for total and country
values, see Table A 11.12 in Annex 11.1). An overview of the results for all response categories
can be found in Annex 11.1 (Table A 11.1 to Table A 10.11). The items perceived to be the least
difficult using HLS19-COM-P-Q11 and HLS19-COM-P-Q6 for the total sample are “to describe to
your doctor your reasons for coming to the consultation” (4.4%) and “to explain your health con-
cerns to your doctor” (9.2%). The most difficult item is “to get enough time in the consultation with
your doctor”, where a quarter of the respondents perceived difficulties, whether in the HLS19-
COM-P-Q11 or HLS19-COM-P-Q6. This item had the highest difficulty in seven countries (AT, BE,
BG, DE, DK, HU, and SI) and was rated among the most difficult items in the other two countries
(CZ, FR) as well.
Figure 11.2:
Percentages of respondents who responded with “very difficult” or “difficult” to the HLS19-COM-
P-Q11 items (ordered by the overall mean), for each country
Figure 11.3:
Percentages of respondents who responded with “very difficult” or “difficult” to the HLS19-COM-
P-Q6 items (ordered by the overall mean), for each country
In CZ, where Communicative HL was measured using the HLS19-COM-P-Q6, item 9 “to be involved
in decisions about your health in dialogue with your doctor” was rated as being most difficult
(24.6% perceived this item as “difficult” or “very difficult”), whereas item 5 “to express your per-
sonal views and preferences to your doctor” was perceived as most difficult in FR (25.9% perceived
this item as “difficult” or “very difficult” measured using the HLS19-COM-P-Q6). Almost half (46.4%)
of the respondents from DE reported difficulties understanding the words used by physicians. As
the “difficulty order” varies somewhat between countries, the HLS19-COM instruments do not
measure Communicative HL invariantly across countries. Due to different cultural backgrounds
and different healthcare systems, respondents from different countries may have interpreted the
content of the items differently (items displaying differential item functioning, DIF, across coun-
tries). Hence, the scores obtained in different countries should be compared with caution. Differ-
ent data collection modes were also used in the different countries, which may also have affected
the response patterns.
Figure 11.4 and Figure 11.5 show the APRP for the HLS19-COM-P-Q11 and HLS19-COM-P-Q6 re-
spectively. These figures confirm the findings at item level, namely that on average the HLS19-
COM-P items were relatively easy, with about 10% (AT and SI) to just below 20% (BG, CZ, DK, and
FR) reporting, on average, difficulties with the Communicative HL tasks under consideration. In
Germany, a quarter of the respondents reported such difficulties. It is noteworthy that the pro-
portion of people reporting very difficult communication with their physician is very low (2%) in
the whole sample and in all countries.
Figure 11.4:
Average Percentage Response Patterns (APRP) for the item set of the HLS19-COM-P-Q11, for
each country and the mean of all countries (equally weighted)
Figure 11.5:
Average Percentage Response Patterns (APRP) for the item set of the HLS19-COM-P-Q6, for each
Psychometric validity analyses
The psychometric properties of the HLS19-COM-P-Q11 and the short version, HLS19-COM-P-Q6,
were assessed using CFA and Rasch analysis.
11.2.2.1 Confirmatory factor analysis
Both the HLS19-COM-P-Q11 and the HLS19-COM-P-Q6 obtained acceptable goodness-of-fit in-
dices (Table 11.4).
The standardized parameter estimates indicate that the theorized factor explained most of the
items well as the loadings are close to or above 0.7 for most items (Table 11.5). For the HLS19-
COM-P-Q11, items HL-COM3 (“to explain your health concerns to your doctor”) and HL-COM6
(“to get the information you need from your doctor”) had the highest loading (mean: 0.83),
whereas HL-COM7 (“to understand the words used by your doctor”) and HL-COM10 (“to recall
the information you get from your doctor”) had the lowest (mean: 0.67). For the HLS19-COM-P-
Q6 items, HL-COM5 (“to express your personal views and preferences to your doctor”) had the
highest loading (mean: 0.90), whereas item HL-COM10 again had the lowest (mean: 0.64).
Table 11.4:
Fit indices for the one-factor model (CFA) of the HLS19-COM-P-Q11 (left) and the HLS19-COM-P-Q6 (right), for each country
Q11 Q6 Ref.
Fit indices for the CFA AT DE SI AT BE BG CZ DE DK FR HU SI
SRMR 0.06 0.07 0.06 0.03 0.04 0.05 0.03 0.03 0.02 0.03 0.05 0.02 ≤ 0.08
RMSEA 0.02 0.05 0.04 0.00 0.02 0.04 0.01 0.03 0.01 0.02 0.02 0.01 ≤ 0.06
RMSEA; CI, lower bound 0.02 0.05 0.03 0.00 0.00 0.01 0.00 0.01 0.00 0.00 0.00 0.00
RMSEA; CI, upper bound 0.03 0.06 0.04 0.02 0.05 0.06 0.03 0.04 0.02 0.04 0.05 0.03
RMSEA; p-value 1.00 0.16 1.00 1.00 0.98 0.75 1.00 1.00 1.00 1.00 0.98 1.00
CFI 0.99 0.99 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 ≥ 0.95
TLI 0.99 0.98 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 ≥ 0.95
GFI 1.00 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 ≥ 0.95
AGFI 0.99 0.98 0.99 1.00 1.00 0.99 1.00 0.99 1.00 1.00 1.00 1.00 ≥ 0.9
AGFI=Adjusted Goodness of Fit Index; CFI=Comparative Fit Index; CI=Confidence interval; GFI=Goodness of Fit Index; Ref=reference values good fit; RMSEA=Root Mean Square Error of Approximation;
SRMR=Standardized Root Mean Square Residual; TLI=Tucker-Lewis Index
Chapter 11 / Communicative Health Literacy with Physicians 247

Table 11.5:
Standardized parameter estimates for the HLS19-COM-P-Q11 (left) and the HLS19-COM-P-Q6 (right), for each country and the mean for all countries (equally
weighted)
Q11 Q6
Item no. AT DE SI Mean AT BE BG CZ DE DK FR HU SI Mean
HL-COM1 0.77 0.79 0.82 0.79
HL-COM2 0.80 0.79 0.88 0.82
HL-COM3 0.78 0.78 0.93 0.83 0.74 0.69 0.81 0.81 0.72 0.84 0.79 0.90 0.92 0.80
HL-COM4 0.79 0.75 0.80 0.78 0.77 0.78 0.75 0.75 0.73 0.71 0.77 0.79 0.77 0.76
HL-COM5 0.82 0.77 0.88 0.82 0.86 0.91 0.92 0.91 0.81 0.94 0.88 0.92 0.91 0.90
HL-COM6 0.80 0.81 0.88 0.83
HL-COM7 0.57 0.68 0.75 0.67
HL-COM8 0.78 0.82 0.85 0.82 0.79 0.91 0.90 0.90 0.83 0.90 0.87 0.89 0.86 0.87
HL-COM9 0.80 0.76 0.87 0.81 0.80 0.86 0.89 0.89 0.77 0.86 0.89 0.85 0.88 0.85
HL-COM10 0.58 0.66 0.76 0.67 0.52 0.71 0.74 0.61 0.60 0.60 0.71 0.57 0.68 0.64
HL-COM11 0.59 0.71 0.76 0.69
11.2.2.2 Rasch analysis
Testing data against the unidimensional polytomous Rasch Partial Credit Model (PCM), the overall
data-model fit for the HLS19-COM-P-Q6 was sufficient for data collected in Austria (CATI) and
Germany (PAPI). The overall data-model fit for the HLS19-COM-P-Q6 was also acceptable for Bel-
gium (CAWI), Bulgaria (CAPI and CAWI), the Czech Republic (CAWI), Denmark (CAWI), Hungary
(CATI), and Slovenia (CAPI and CAWI) when the analysis was based on a reduced sample (n=180;
6 items x 3 thresholds x 10 respondents). The HLS19-COM-P-Q11 also displayed acceptable over-
all data-model fit in all countries when the sample size was reduced to n=330 (11 items x 3
thresholds x 10 respondents). Using the combined principal component analysis (PCA) of Rasch
model residuals and the dependent t-tests procedure, both the HLS19-COM-P-Q11 and the HLS19-
COM-P-Q6 were found to be sufficiently unidimensional. Targeting indicates that the respondents
have better Communicative HL than captured by the instrument, which increases the risk of ceiling
effects (the HLS19-COM-P-Q11 had a mean person location ranging from 1.38 (DE, PAPI) to 2.73
(SI, CAWI) and the HLS19-COM-P-Q6 had a mean person location ranging from 1.21 (DE, PAPI) to
2.47 (SI, CAWI).
The items in the HLS19-COM-P-Q11 displayed acceptable fit, whereas for HLS19-COM-P-Q6, item
10 under-discriminated in four countries (BE, CZ, DK, and HU). In addition, item 4 under-discrim-
inated in the Bulgarian (CAPI and CAWI) and Danish data (CAWI). Some items on the HLS19-COM-
P-Q11 displayed DIF for person factors, such as gender, age, and education, but there was no
consistent pattern across countries. Response dependency was observed between items 1 and 3
(r=0.35) in the German data. No signs of unordered response categories were found, indicating
that the 4-point response scale worked well. For more details about the results of the Rasch anal-
yses, see Guttersrud et al. (2021).
11.2.2.3 Reliability
Both the HLS19-COM-P-Q11 and the HLS19-COM-P-Q6 obtained acceptable values for internal
consistency and reliability (Table 11.6) in all countries.

Table 11.6:
Cronbach’s alpha and the Person Separation Index for the HLS19-COM-P-Q11 (left) and the HLS19-COM-P-Q6 (right), for each country and the mean for all
countries (equally weighted).
Q11 Q6
AT DE SI Mean AT BE BG CZ DE DK FR HU SI Mean
Cronbach’s alpha (based on

0.79 0.84 0.87 0.83 0.69 0.80 0.81 0.80 0.74 0.80 0.80 0.79 0.80 0.78
the Pearson correlation)
Person Separation Index 0.86 0.89 0.88 0.88 0.75 0.82 0.81 0.83 0.81 0.83 0.83 0.77 0.78 0.80
11.2.2.4 Correlations with other HL scores
The HLS19-COM-P-Q11 and the HLS19-COM-P-Q6 were found to be highly correlated (r=0.94
(AT), r=0.95 (DE), and r=0.96 (SI)) (Table 11.7). The HL-COM score correlated moderately with the
GEN-HL score in most countries as well as with the HL-NAV score and, somewhat lower, with
Digital HL and Vaccination HL. The association between the scores, as a measure of concurrent
discriminant validity, indicates important overlaps between the concepts and measures of the
HLS19 family of HL, but also indicates the independency of the specific HL measures.

Table 11.7:
Pearson correlations between the HLS19-COM-Q11 (left) and the HLS19-COM-P-Q6 (right) scores and general and other specific HL scores, for each country and
the mean for all countries (equally weighted)
Q11 Q6
AT DE SI Mean AT BE BG CZ DE DK FR HU SI Mean
GEN-HL 0.37 0.54 0.48 0.46 0.34 0.27 0.49 0.47 0.50 0.47 0.52 0.36 0.45 0.43
HL-COM-Q6 0.94 0.95 0.96 0.95 - - - - - - - - - -
HL-DIGI 0.36 0.42 - 0.39 0.32 0.20 - 0.36 0.39 0.31 0.38 0.24 - 0.31
HL-NAV 0.49 0.48 0.45 0.47 0.46 0.36 - 0.45 0.45 - 0.44 - 0.44 0.43
HL-VAC 0.25 0.36 0.34 0.32 0.22 0.25 0.23 0.36 0.34 - - 0.23 0.33 0.28
Distributions of scores
Table 11.8 reports the distribution of the Communicative HL scores by country. The score can
range from 0-100, with 100 representing the highest level of Communicative HL. The mean score
indicates the percentage of valid items that were answered on average by the countries or selected
population groups with either "easy" or "very easy".
The mean scores are quite high, with means of 90% or higher (AT and SI) and 80% or higher for all
other countries, except for DE, which has a mean just above 70%. For all countries, the distribu-
tions were rather left-skewed. An overview of the detailed distribution of the Communicative HL
score by country can be found in Figure A 11.1 in Annex 11.

Table 11.8:
Means, standard deviations, median and the 25th percentile for the HLS19-COM-P-Q11 (left) and the HLS19-COM-P-Q6 (right) scores, for each country and the
mean for all countries (equally weighted)
HL-COM score Q11 Q6

AT DE SI All AT BE BG CZ DE DK FR HU SI All
Mean 90.2 73.6 91.7 85.2 90.0 83.9 80.7 81.0 72.4 81.8 81.4 86.8 90.4 83.2
SD 16.6 26.4 17.9 22.3 18.4 25.8 27.7 27.4 28.8 26.2 27.2 23.0 20.5 25.8
Median 100 81.8 100 100 100 100 100 100 83.3 100 100 100 100 100
25th percentile 88.9 54.5 90.9 80.0 83.3 83.3 66.7 66.7 50.0 66.7 66.7 83.3 83.3 66.7
The 75th percentiles were 100 in all countries.
subpopulations
To better identify the subpopulations that could potentially be disadvantaged in terms of Com-
municative HL, the Communicative HL mean scores for a series of subpopulations were compared
to the mean Communicative HL score observed for the whole population. The potentially vulner-
able groups considered were: older people, those with the lowest educational level, with a low
level in society/low social status, financially deprived people, individuals with poorer self-per-
ceived health status, and those who make frequent use of GPs/family doctors.
The mean HLS19-COM-Q6 scores for the vulnerable subpopulations relative to the total mean
scores varied heterogeneously across the different groups. In most countries, people with a lower
socio-economic status (self-perceived level in society and financial deprivation) and poorer self-
perceived health had a lower Communicative HL score. Similar results were observed using the
long version of the scale (Table 11.9). The effect was particularly strong (most differences > 5
points) for level in society, financial deprivation, and bad or very bad self-perceived health. Only
in three countries (DE, HU, and SI) did older participants express more difficulties. In the opposite
direction, higher Communicative HL scores were observed among older participants in CZ and DK.
The relevant numbers of respondents for the selected vulnerable subpopulations are shown in
Table A 11.13 in Annex 11.3.

Table 11.9:
Differences between the total mean score and the score for potentially vulnerable subpopulations for the HLS19-COM-Q11 (left) and the HLS19-COM-Q6 (right),
for each country and the mean for all countries (equally weighted)
Q11 Q6
AT DE SI All AT BE BG CZ DE DK FR HU SI All
Total score 90.2 73.6 91.7 85.2 90 83.9 80.7 81 72.4 81.8 81.4 86.8 90.4 83.2
Aged 76 or older 0.2 -4.2 -6.8 -3.6 0.6 - - 6.8 -4.4 7.2 - -2.7 -7.5 -3.5
Education at ISCED
- - -4.7 -5.6 - - - -1.3 - 0.1 - -4.4 -5 -2.9
levels 0 or 1
Level in society less

-2.9 -8.9 -4.1 -5.3 -2.3 -11 -15.5 -6 -9.6 -7.4 -9.5 -5.9 -4.1 -7.9
than or equal to 4
Considerable or se-
vere financial depri- -5.1 -11.3 -3.9 -6.8 -4.4 -2.1 -14.2 -8.1 -11.8 -14.8 -9.5 -7.6 -3.9 -8.5
vation
Bad or very bad

self-perceived 0.5 -17.5 -12 -9.7 1.9 -12 -26.1 -3.4 -17.9 -16.6 -6.9 -4 -13.5 -10.9
health
One or more long-

term illnesses or -1.2 -2.5 -3.1 -2.2 -1.3 -0.6 -2.9 0.4 -2.6 -1.8 -0.8 -1.7 -3.6 -1.7
health problems
Limited by health
-2.6 -3.3 -4.5 -3.5 -2.8 -3.4 -5.2 -1.3 -3.4 -4.9 -3.1 -5.5 -4.8 -3.8
problems
6 or more contacts
with a GP/family 0.5 -6.5 -1.8 -2.6 0.7 0.2 -15.3 -0.6 -6.5 -5.2 -2.3 -1.2 -1.9 -3.6
doctor
- Cells with less than 30 respondents were not reported. Differences of more than 5 points are bold.
General and specific determinants of Communicative
HL
This section includes results on the association of Communicative HL with potential determinants
of Communicative HL, such as gender, age, education, level in society, and financial deprivation.
Table 11.10 shows the correlation coefficients between Communicative HL and the General HL
score as well as a series of HL determinants. The Communicative HL scores correlated moderately
positively with General HL, with Spearman coefficients ranging from 0.31 (BE; Q6) to 0.53 (DE;
Q11). The correlation coefficients between Communicative HL and the HL determinants were ra-
ther weak in all countries (ρ=0.30), while, on average, somewhat higher for financial deprivation
(ρ=-0.16) and level in society (ρ=+0.14) than for the other included indicators.

Table 11.10:
Spearman correlations (ρ) of the HLS19-COM-Q11 (left) and the HLS19-COM-Q6 (right) scores with General HL and HL determinants, for each country and the
mean for all countries (equally weighted)
Q11 Q6
HL-COM
and… AT DE SI All AT BE BG CZ DE DK FR HU SI All
GEN-HL 0.38 0.53 0.40 0.55 0.34 0.31 0.41 0.46 0.49 0.41 0.52 0.36 0.37 0.44
Gender fe-
-0.01 0.01 -0.05 -0.02 -0.01 -0.04 -0.1 -0.07 0 -0.04 -0.07 -0.09 -0.06 -0.05
male
Age in
-0.03 0.01 -0.02 -0.02 -0.02 0.23 -0.02 0.23 0.01 0.17 0.11 0.05 -0.05 0.07
years
Education 0.01 0.17 0.08 0.02 -0.02 -0.02 -0.09 -0.13 0.16 0.08 0 0.09 0.08 -0.02
Level in so-
0.04 0.23 0.13 0.09 0.02 0.21 0.27 0.09 0.23 0.16 0.19 0.15 0.12 0.14
ciety
Financial
-0.15 -0.23 -0.19 -0.1 -0.12 -0.1 -0.28 -0.2 -0.22 -0.24 -0.18 -0.23 -0.18 -0.16
deprivation
Migration
0.04 -0.05 0 0.04 0.04 -0.09 -0.03 0.01 -0.04 -0.04 -0.02 -0.02 -0.01 0
background
No training
in a health 0.03 -0.1 -0.04 -0.04 0.04 -0.01 -0.06 -0.03 -0.08 -0.03 -0.03 -0.04 -0.03 -0.03
profession

The results of the linear regression models are shown in Table 11.11 Model 1 (without General
HL) and Table 11.12 Model 2 (with General HL). Multivariable regression models include the sim-
ultaneous effect of gender, age, and three measures of social position (education, level in society,
and financial deprivation). In the absence of General HL, lower financial deprivation (except BE)
and higher self-perceived level in society (except AT) were statistically significant determinants
(at the 1%-level) of high Communicative HL with physicians in all countries. The individual effect
of gender was only a significant variable in SI, with females having lower levels of Communicative
HL. The independent effect of older age in the model was significantly associated with Communi-
cative HL in BE, BG, CZ, DK, and FR. Education was not a significant linear predictor in any country
(except for a negative effect in BG and CZ). The proposed model explains 5% of Communicative
HL variance, varying between 2% (AT) and 18% (BG).
With the inclusion of General HL in Model 2 (Table 11.12) lower financial deprivation was still a
statistically significant determinant of Communicative HL except in BE, FR, and SI. The independent
effect of an individual’s level in society reached statistical significance in only four countries (BE,
DE, FR, and HU). Female gender became a significant determinant in CZ, DK, FR, and SI. Age was
a statistically significant independent determinant in five countries (BE, BG, CZ, DK, and FR). Edu-
cation did not show any significant linear association with Communicative HL in this model except
in BG. When the data from all countries were considered, the proposed model explains 22% of the
Communicative HL variance, varying between 12% (AT) and 35% (BG).
In general, the models suggest that the level of Communicative HL is mostly explained by General
HL but with an independent effect of socio-economic status, mostly by self-perceived level in
society and with a smaller effect of financial deprivation, while education, surprisingly, had a lower
independent effect. Age had a smaller independent effect, while gender only had minimal effect.
The effect of higher General HL, higher socio-economic status, and higher age was related to a
higher level of Communicative HL. The results were similar for both the Q6 and the Q11.
When long-term illness was added to the model (Table A 11.14 in Annex 11.4), the effects and
the explained variance did not change much despite a negative significant association between
long-term illness and Communicative HL in four countries (BG, DE, DK, and SI).

Table 11.11 :
Model 1: Multivariable linear regression models of HL-COM by five core social determinants (standardized coefficients (β) and R2), for each country and for all
countries (equally weighted) (Results based on the HLS19-COM-Q11 score to the left and the HLS19-COM-Q6 score to the right).
Q11 Q6
HL-COM and… AT DE SI All AT BE BG CZ DE DK FR HU SI All
Gender female -0.02 0.03 -0.05 -0.02 -0.02 -0.05 -0.05 -0.03 0.02 -0.04 -0.04 -0.05 -0.06 -0.04
Age in years -0.04 -0.02 -0.03 -0.05 -0.03 0.17 0.05 0.17 -0.02 0.12 0.06 0.06 -0.04 0.05
Education -0.02 0.04 0.02 -0.03 -0.03 0.01 -0.17 -0.08 0.02 0.03 -0.04 0 0.01 -0.07
Level in society 0.04 0.13 0.07 0.09 0.03 0.16 0.17 0.09 0.14 0.06 0.17 0.1 0.06 0.14
Financial dep-
-0.14 -0.19 -0.14 -0.11 -0.12 -0.05 -0.29 -0.19 -0.18 -0.19 -0.12 -0.2 -0.13 -0.13
rivation
R2 0.03 0.08 0.05 0.03 0.02 0.07 0.18 0.09 0.08 0.08 0.07 0.07 0.04 0.05
Valid count 2677 1837 3177 2672 988 698 1562 1832 3557 2003 1102 3173
Total count 2967 2143 3360 2967 1000 865 1599 2143 3602 2003 1195 3360

Table 11.12:
Model 2: Multivariable linear regression models of HL-COM by GEN-HL and five core social determinants (standardized coefficients (β) and R2), for each
country and for all countries (equally weighted). (Results based on the HLS19-COM-Q11 score to the left and the HLS19-COM-Q6 score to the right).
Q11 Q6
HL-COM and… AT DE SI All AT BE BG CZ DE DK FR HU SI All
GEN-HL 0.35 0.49 0.45 0.55 0.32 0.24 0.45 0.42 0.46 0.43 0.49 0.31 0.42 0.42
Gender female -0.05 -0.01 -0.07 -0.04 -0.05 -0.06 -0.04 -0.06 -0.02 -0.06 -0.06 -0.04 -0.07 -0.06
Age in years -0.01 0.03 0.03 0.02 -0.01 0.17 0.09 0.13 0.02 0.08 0.06 0.02 0.01 0.06
Education -0.01 -0.02 0 -0.02 -0.02 0.02 -0.21 -0.02 -0.04 0.02 -0.02 -0.01 -0.01 -0.04
Level in society 0.04 0.08 0.03 0.05 0.03 0.11 0.06 0.03 0.09 0.02 0.10 0.10 0.03 0.09
Financial dep- -0.07 -0.11 -0.05 -0.04 -0.05 -0.06 -0.22 -0.10 -0.11 -0.11 -0.04 -0.12 -0.05 -0.07
rivation
R2 0.15 0.3 0.23 0.31 0.12 0.13 0.35 0.25 0.27 0.25 0.30 0.15 0.21 0.22
Valid count 2677 1837 3177 2672 988 698 1562 1832 3557 2003 1102 3173
Total count 2967 2143 3360 2967 1000 865 1599 2143 3602 2003 1195 3360


Communicative HL and self-perceived health
This section explores the association between Communicative HL and self-perceived health. Com-
pared to the other chapters, only this potential consequence was focused on because it was hy-
pothesized that self-perceived health could be directly impacted by Communicative HL. Of course,
there might be a direct association between Communicative HL and other behavioral indicators
(such as smoking or fruit and vegetable consumption), but only if physicians have discussed these
topics with their patients. Table A 11.5 in Annex 11.5 confirms the lack of strong and repeated
associations between Communicative HL and other indicators. Limitations in activity due to health
problems was the only additional indicator showing a correlation coefficient >0.10 with Commu-
nicative HL. However, this variable was, as expected, moderately to strongly correlated (ρ=-0.4 to
-0.63) with self-perceived health.
Figure 11.6 and Figure 11.7 show that participants with higher Communicative HL scores reported
better general health. The association between Communicative HL and general health by country
is displayed in Figure A 11.2 and Figure A 11.3 in Annex 11.5. The correlation between Commu-
nicative HL and self-perceived health (Table 11.13) was ρ=-0.23 (Q11) or ρ=-0.17 (Q6) when
considering the mean of all the countries (see Section 4.2 for information about how the score for
all countries is calculated). Except for a null correlation in CZ, it ranged between ρ=-0.13 (HU) and
ρ=-0.24 (DK).
In the multivariate regression models adjusted for gender, age, education, self-perceived level in
society, and financial deprivation (Model 1, Table 11.13), the association between Communicative
HL and better self-perceived health persisted in relation to the mean of all countries and in most
individual countries as well (except CZ and HU). After further adjustment for General HL (Model 2,
Table 11.13), the explained variance of self-perceived health (R²) increased only slightly and the
association between Communicative HL and self-perceived health decreased but was still statis-
tically significant (p<0.01) in relation to the mean of all countries and in most individual countries
(except CZ and HU (Q11) or FR and AT (Q6)). The full description of the model in Table A 11.16 in
Annex 11.5 highlights the fact that despite being statistically significance, the association between
self-perceived health and the Communicative HL score was, on average, weaker than the associ-
ation between self-perceived health and age, financial deprivation, self-perceived social level in
society, or General HL.
Figure 11.6:
Percentage distribution of five categories of self-perceived health by HL-COM Q11 (7 groups
from lowest HL to highest HL), for all countries (equally weighted)
Note: The group levels denote the number of items the respondents answered with “easy” or “very easy”.

Figure 11.7:
Percentage distribution of five categories of self-perceived health by HL-COM Q6 (7 values from
Note: The group levels denote the number of items the respondents answered with “easy” or “very easy”.
Table 11.13:
Spearman correlation coefficients and standardized coefficients (ß) and R2 for two linear regression models, for each country and for all countries (equally
weighted), illustrating the hypothesized association between HL-COM and self-perceived health (dependent variable)
Q11 Q6
Self-perceived health* AT DE SI All AT BE BG CZ DE DK FR HU SI All
Spearman correlation coefficient -0.18 -0.20 -0.20 -0.23 -0.15 -0.21 -0.21 0.00 -0.18 -0.24 -0.16 -0.13 -0.21 -0.17
Standardized beta coefficient (Model 1) -0.11 -0.17 -0.15 -0.15 -0.10 -0.17 -0.21 -0.06 -0.15 -0.20 -0.11 -0.02 -0.15 -0.13
Standardized beta coefficient (Model 2) -0.06 -0.14 -0.11 -0.11 -0.05 -0.14 -0.16 -0.03 -0.11 -0.13 -0.04 0.02 -0.11 -0.08
R² (Model 1) 0.14 0.27 0.32 0.24 0.14 0.14 0.35 0.24 0.26 0.15 0.16 0.26 0.31 0.22
R² (Model 2) 0.17 0.27 0.32 0.28 0.17 0.15 0.36 0.24 0.26 0.17 0.17 0.28 0.32 0.21
* Self-perceived health: from very good (1) to very bad (5), a higher number represents a worse condition.
Model 1: adjusted for gender, age, education, level in the society, and financial deprivation.
Model 2: same model also adjusted for General Health Literacy as an independent covariate.

11.3 Discussion and Conclusions
The results in this chapter are based on data obtained from a newly developed instrument, the
HLS19-COM-P-Q11 and its short version, the HLS19-COM-P-Q6. Previous instruments that were
meant to measure Communicative HL were either developed to measure certain communicative
tasks or outcomes or to capture only certain aspects of HL. As far as we know, no instrument
before integrated findings from communication research and HL research in one instrument.
Hence, there was a need for a new measure that covered the HL skills necessary for actively par-
ticipating in health communication with physicians within a consultation context. The HLS19-COM-
P instrument was developed based on a comprehensive theoretical framework that integrates
Nutbeam’s (2000) idea of Communicative HL, the basic competencies of information processing
according to the HL framework of the HLS-EU Consortium (Sørensen et al. 2012), and the main
communicative tasks of the Calgary-Cambridge Guide framework (Silverman et al. 2013).
The HLS19-COM-P instrument was well accepted in a huge sample (n>20,000 for the HLS19-COM-
P-Q6) in nine countries and seven languages with different formats of data collection (CAPI, CATI,
etc.). Both the HLS19-COM-P-Q11 and the HLS19-COM-P-Q6 displayed acceptable psychometric
properties at overall level and at item level alike. However, there is room for some improvements,
e.g., the wording of item 10, which tended to under-discriminate in the short version, and some
items displaying DIF for levels of person factors, for example gender, age, and education. The
HLS19-COM-P instrument has proven to map Communicative HL in research on a population level
and might have the potential to map Communicative HL in clinical settings as well. However, the
HLS19-COM-P should be validated for use in such settings in advance. The short version might
give a general overview of Communicative HL, while the long version provides more detailed in-
formation on individuals’ Communicative HL with physicians.
As expected, the short version correlated strongly with the long one and moderately with the
General HL instrument. Thus, the HLS19-COM-P instruments add valuable information about indi-
viduals’ Communicative HL with physicians. The HLS19-COM-P also correlated moderately with the
HLS19-NAV, indicating that Communicative HL in consultations with physicians is also important
for supporting the navigation tasks of health care users, although it still has to be regarded as a
distinctive construct.
A majority of the respondents found the Communicative HL tasks relatively easy. However, there
were still approximately 10% to 25% who found Communicative HL tasks difficult and might not
have sufficient Communicative HL proficiency to actively participate in communication with phy-
sicians. In general, to get enough time in the consultation with the physician and to express one’s
personal views and preferences to the physician were experienced as the most difficult tasks, while
explaining health concerns was the easiest. In consultations with patients with low Communicative
HL, physicians should set aside more time to be able to provide adapted information and to be
able to better facilitate active participation in the consultation. The difficulties to express one’s
personal views and preferences might indicate the need for more patient empowerment in shared
decision making (Elwyn et al. 2016; Meijers et al. 2019). As consultation time is the most important
difficulty in Communicative HL, it is essential that organizational contexts are developed to sup-
port patient-oriented communication in the framework of health literate health care organizations
(Palumbo 2021; Pelikan 2019; Working Group HPH & HLO 2019; Brach et al. 2012).
General HL was found to be the strongest predictor for Communicative HL. This is in accordance
with earlier research showing that patients with lower HL report worse communication with their
providers than those with higher HL (Castro et al. 2007; Schillinger et al. 2004) and are less active
in these communications (Katz et al. 2007). In most countries, participants with lower socio-eco-
nomic status (level in society and financial deprivation) and poorer health were found to have lower
HLS19-COM-P mean scores than those with higher socio-economic status and better health. In
most countries higher Communicative HL was associated with better general self-perceived health.
This would be in line with previous research that describes Communicative HL as a critical deter-
minant of successful disease management and health outcomes (van der Heide et al. 2018; Amalraj
et al. 2009; Paasche-Orlow/Wolf 2007). Conversely, there might also be the possibility that people
in good health assess their Communicative HL to be better because they have not had enough
problematic experiences with physicians.
Limitations
It should be acknowledged that the results and conclusions do have limitations due to the cross-
sectional design of the study and possibly due to the diversity of data collection modes used by
countries (and within countries), so that a comparative analysis between participating countries is
only possible to a limited extent.
The main limitation to the instrument is that it only measures Communicative HL between physi-
cians and patients. Healthcare systems are diverse in relation to the importance of physician-
patient interactions so generalizations on the quality of the communicative culture in health care
and its impact on health issues might be limited.
Finally, there might be a limitation relating to the ability to recall personal experiences of physi-
cian-patient interactions because of diverse factors like the time since the last interaction, the
frequency of interactions, individual dependence on health care, and cognitive skills, etc. Hence,
the Communicative HL results might be influenced by these factors.
From these results and their limitations, some conclusions for future research follow:
» to improve minor under-discriminating items,

» to investigate the link between gender, social gradient, dealing with chronic illness, and Com-
municative HL,
» to investigate the link between HLS19-NAV and HLS19-COM-P,
» to test the stand-alone use of the HLS19-COM-P instruments, specifically in the clinical context
as well,

» to adapt the HLS19-COM-P instruments as a measure for Communicative HL in relation to
other health professionals (nurses, physiotherapists, etc.). However, it is strongly recom-
mended that such an adaptation of the instrument should be thoroughly piloted and assessed
to ensure its validity and reliability.
11.4 References
Adams, R. J.; Wu, M. L.; Cloney, D.; Wilson, M. R. (2020): ACER ConQuest: Generalised Item Re-
sponse Modelling Software [Computer software]. Version 5. Australian Council for Educational
Research.
Amalraj, Sunil; Starkweather, Chelsea; Naeim, Arash (2009): Health literacy, communication, and
treatment decision-making in older cancer patients. In: Ethics and Social Welfare
23/4:http://www.physicianspractice.com
Andrich, D.; Sheridan, B. (2019): RUMM2030 Plus [Computer software]. Rumm Laboratory Pty Ltd.
Balint, Enid (1969): The possibilities of patient-centered medicine. In: The Journal of the Royal
College of General Practitioners 17/82:269
Ball, Lauren E; Barnes, Katelyn A; Crossland, Lisa; Nicholson, Caroline; Jackson, Claire (2018):
Questionnaires that measure the quality of relationships between patients and primary care
providers: a systematic review. In: BMC Health Services Research 18/1:1-12
Bieber, Christiane; Mueller, Knut G; Nicolai, Jennifer; Hartmann, Mechthild; Eich, Wolfgang (2010):
How does your doctor talk with you? Preliminary validation of a brief patient self-report ques-
tionnaire on the quality of physician-patient interaction. In: Journal of Clinical Psychology in
Medical Settings 17/2:125-136
Brach, C.; Keller, D.; Hernandez, L. M.; Baur, C.; Parker, R.; Dreyer, B.; Schyve, P.; Lemerise, A. J.;
Schillinger, D. (2012): Ten attributes of health literate health care organizations. Institute of
Medicine., New York. Unveröffentlicht
Brouwers, Marianne; Rasenberg, Ellemieke; van Weel, Chris; Laan, Roland; van Weel‐Baumgarten,
Evelyn (2017): Assessing patient‐centred communication in teaching: a systematic review of
instruments. In: Medical education 51/11:1103-1117
Burt, J.; Abel, G.; Elmore, N.; Newbould, J.; Davey, A.; Llanwarne, N.; Maramba, I.; Paddison, C.;
Benson, J.; Silverman, J.; Elliott, M. N.; Campbell, J.; Roland, M. (2018): Rating Communication
in GP Consultations: The Association Between Ratings Made by Patients and Trained Clinical
Raters. In: Med Care Res Rev 75/2:201-218
Burt, Jenni; Abel, Gary; Elmore, Natasha; Campbell, John; Roland, Martin; Benson, John; Silverman,
Jonathan (2014): Assessing communication quality of consultations in primary care: initial relia-
bility of the Global Consultation Rating Scale, based on the Calgary-Cambridge Guide to the Med-
ical Interview. In: BMJ Open 4/3:
CAHPS (2017): Patient Experience Measures from the CAHPS® Health Plan Survey 2109. AHRQ, Agency
for Healthcare Research and Quality, Rockville, MD.
Campbell, Craig; Lockyer, Jocelyn; Laidlaw, Toni; MacLeod, Heather (2007): Assessment of a matched‐
pair instrument to examine doctor− patient communication skills in practising doctors. In: Medical
education 41/2:123-129
Castro, Cesar M; Wilson, Clifford; Wang, Frances; Schillinger, Dean (2007): Babel babble: physicians'
use of unclarified medical jargon with patients. In: American journal of health behavior 31/1:85-
95
Chen, Xuewei; Hay, Jennifer L; Waters, Erika A; Kiviniemi, Marc T; Biddle, Caitlin; Schofield, Elizabeth;
Li, Yuelin; Kaphingst, Kimberly; Orom, Heather (2018): Health literacy and use and trust in health
information. In: Journal of health communication 23/8:724-734
Chinn, Deborah; McCarthy, Catherine (2013): All Aspects of Health Literacy Scale (AAHLS): developing
a tool to measure functional, communicative and critical health literacy in primary healthcare set-
tings. In: Patient education and counseling 90/2:247-253
Elwyn, Glyn; Frosch, Dominick L; Kobrin, Sarah (2016): Implementing shared decision-making: con-
sider all the consequences. In: Implementation Science 11/1:1-10
Elwyn, Glyn; Nelson, Eugene; Hager, Andreas; Price, Amy (2020): Coproduction: when users define
quality. In: BMJ Qual Saf 29/9:711-716
Epstein, Ronald M; Franks, Peter; Fiscella, Kevin; Shields, Cleveland G; Meldrum, Sean C; Kravitz, Rich-
ard L; Duberstein, Paul R (2005): Measuring patient-centered communication in patient–physician
consultations: theoretical and practical issues. In: Social Science & Medicine 61/7:1516-1528
Ernstmann, Nicole; Halbach, Sarah; Kowalski, Christoph; Pfaff, Holger; Ansmann, Lena (2017): Meas-
uring attributes of health literate health care organizations from the patients’ perspective: Devel-
opment and validation of a questionnaire to assess health literacy-sensitive communication (HL-
COM). In: Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 121/:58-63
ESTAT (2017): Voluntary module in EHIS wave 3: Patient Experience. Proposal. ESTAT
Eveleigh, Rhona M; Muskens, Esther; van Ravesteijn, Hiske; van Dijk, Inge; van Rijswijk, Eric; Lucassen,
Peter (2012): An overview of 19 instruments assessing the doctor-patient relationship: different
models or concepts are used. In: Journal of clinical epidemiology 65/1:10-15

Farin, Erik; Ullrich, Antje; Nagl, Michaela (2013): Health education literacy in patients with chronic
musculoskeletal diseases: development of a new questionnaire and sociodemographic predictors.
In: Health education research 28/6:1080-1091
Gibney, Sarah; Moore, Tara (2018): Older Patients' Views of Health Care Interactions in Ireland. In:
HLRP: Health Literacy Research and Practice 2/4:e180-e191
Guttersrud, Øystein; Le, Christopher; Pettersen, Kjell Sverre; Finbråten, Hanne Søberg (2021): Rasch
analyses of data collected in 17 countries. - A technical report to support decision-making within
the M-POHL consortium. https://m-pohl.net/Results
HCAHPS (2019): HCAHPS Survey. Centers for Medicare & Medicaid Services, Baltimore, MD [Online].
http://www.hcahpsonline.org/globalassets/hcahps/survey-instruments/mail/29-item-sur-
vey/updated-w-omb-date/2019_survey-instruments_english_mail-updateda.pdf [accessed:
25.5.2019]
HLS-EU Consortium (2012): Comparative Report on Health Literacy in Eight EU Member States (Second
Extended and Revised Version, Date July 22th, 2014). The European Health Literacy Survey HLS-
EU, Vienna
Ishikawa, Hirono; Takeuchi, Takeaki; Yano, Eiji (2008): Measuring functional, communicative, and crit-
ical health literacy among diabetic patients. In: Diabetes care 31/5:874-879
Iversen, Else Dalsgaard; Wolderslund, Maiken Overbeck; Kofoed, Poul-Erik; Gulbrandsen, Pål; Poulsen,
Helle; Cold, Søren; Ammentorp, Jette (2020): Codebook for rating clinical communication skills
based on the Calgary-Cambridge Guide. In: BMC medical education 20/1:1-9
Katz, Marra G; Jacobson, Terry A; Veledar, Emir; Kripalani, Sunil (2007): Patient literacy and question-
asking behavior during the medical encounter: a mixed-methods analysis. In: Journal of general
internal medicine 22/6:782-786
Langewitz, Wolf; Conen, Dieter; Nübling, Matthias; Weber, Heidemarie (2002): Kommunikation ist we-
sentlich-Defizite der Betreuung im Krankenhaus aus der Sicht von Patienten und Patientinnen. In:
PPmP-Psychotherapie Psychosomatik Medizinische Psychologie 52/08:348-354
Lawson, Peter J.; Flocke, Susan A. (2009): Teachable moments for health behavior change: a concept
analysis. In: Patient education and counseling 76/1:25-30
Makoul, Gregory; Krupat, Edward; Chang, Chih-Hung (2007): Measuring patient views of physician
communication skills: development and testing of the Communication Assessment Tool. In: Pa-
tient education and counseling 67/3:333-342
Maly, Rose C; Frank, Janet C; Marshall, Grant N; DiMatteo, M Robin; Reuben, David B (1998): Perceived
efficacy in patient‐physician interactions (PEPPI): validation of an instrument in older persons. In:
Journal of the American Geriatrics Society 46/7:889-894
Masters, Geoff N (1982): A Rasch model for partial credit scoring. In: Psychometrika 47/2:149-174
Mead, N.; Bower, P. (2002): Patient-centred consultations and outcomes in primary care: a review of
the literature. In: Patient Educ Couns 48/1:51-61
Meijers; Noordman, Janneke; Spreeuwenberg, Peter; Olde Hartman, Tim C; van Dulmen, Sandra (2019):
Shared decision-making in general practice: an observational study comparing 2007 with 2015.
In: Family practice 36/3:357-364
Myerholtz, Linda; Simons, Lynn; Felix, Sumi; Nguyen, Tuan; Brennan, Julie; Rivera-Tovar, Ana; Martin,
Pat; Hepworth, Jeri; Makoul, Gregory (2010): Using the Communication Assessment Tool in Family
Medicine Residency Programs. In: Family Medicine 42/8:567-573
National Institutes of Health (2018): Health Information. National Trends Survey
Nouri, Sarah S; Rudd, Rima E (2015): Health literacy in the “oral exchange”: An important element of
patient–provider communication. In: Patient education and counseling 98/5:565-571
education and communication strategies into the 21st century. In: Health Promotion International
15/3:259-267
67/12:2072-2078
Nutbeam, Don (2009): Defining and measuring health literacy: what can we learn from literacy studies?
Springer
O’Hara, Jonathan; Hawkins, Melanie; Batterham, Roy; Dodson, Sarity; Osborne, Richard H.; Beauchamp,
Alison (2018): Conceptualisation and development of the Conversational Health Literacy Assess-
ment Tool (CHAT). In: BMC Health Services Research 18/1:199
Osborne, Richard H; Elsworth, Gerald R; Whitfield, Kathryn (2007): The Health Education Impact Ques-
tionnaire (heiQ): an outcomes and evaluation measure for patient education and self-management
interventions for people with chronic conditions. In: Patient education and counseling 66/2:192-
201
Ousseine, Y. M.; Durand, M. A.; Bouhnik, A. D.; Smith, A.; Mancini, J. (2019): Multiple health literacy
dimensions are associated with physicians' efforts to achieve shared decision-making. In: Patient
Educ Couns 102/11:1949-1956
Paasche-Orlow, Michael K.; Wolf, Michael S. (2007): The causal pathways linking health literacy to
health outcomes. In: American journal of health behavior 31 Suppl 1/:19–26

Palumbo, Rocco (2021): Leveraging Organizational Health Literacy to Enhance Health Promotion and
Risk Prevention: A Narrative and Interpretive Literature Review. In: The Yale journal of biology and
medicine 94/1:115-128
Palumbo, Rocco; Lars Tummers, Prof Stephen Teo Pro Dr (2016): Contextualizing co-production of
health care: a systematic literature review. In: International Journal of Public Sector Management
29/1:72-90
Pelikan, Jürgen M. (2019): Health-literate healthcare organizations International. In: International

handbook of health literacy Research, practice and policy across the life-span. Hg. v. Okan, Orkan
et al. The Policy Press, BristolS. 539–554
Picker Institute Europe (2013): Inpatient Questionnaire. NHS: Care Quality Commission
R Core Team (2020): R: A language and environment for statistical computing. R Foundation for Sta-
tistical Computing, Vienna
Rasch, Georg (1960): Studies in mathematical psychology: I. Probabilistic models for some intelligence
and attainment tests.
Rosseel, Yves (2012): lavaan: An R Package for Structural Equation Modeling. In: Journal of Statistical
Software 48/2: Articles-1 - 36, https://www.jstatsoft.org/v48/i02/
Roter, Debra L.; Erby, Lori H.; Larson, Susan; Ellington, Lee (2007): Assessing oral literacy demand in
genetic counseling dialogue: Preliminary test of a conceptual framework. In: Social Science & Med-
icine 65/7:1442-1457
Rubin, D. L.; Parmer, J.; Freimuth, V.; Kaley, T.; Okundaye, M. (2011): Associations between older
adults' spoken interactive health literacy and selected health care and health communication out-
comes. In: J Health Commun 16 Suppl 3/:191-204
Sator, Marlene; Nowak, Peter; Menz, Florian (2015): Verbesserung der Gesprächsqualität in der Kran-
kenversorgung–Praxismodelle und Entwicklungsinitiativen. Gesundheit Österreich, Wien
Schillinger, Dean; Bindman, Andrew; Wang, Frances; Stewart, Anita; Piette, John (2004): Functional
health literacy and the quality of physician–patient communication among diabetes patients. In:
Patient education and counseling 52/3:315-323
Silverman, Jonathan; Kurtz, Suzanne; Draper, Juliet (2013): Skills for Communicating with Patients. 3
Aufl., Radcliffe Publishing Ltd, London
Smith, Meredith Y; Winkel, Gary; Egert, Jennifer; Diaz-Wionczek, Mariana; DuHamel, Katherine N
(2006): Patient-physician communication in the context of persistent pain: Validation of a modi-
fied version of the patients' perceived involvement in care scale. In: Journal of pain and symptom
management 32/1:71-81
Health Literacy Project, European (2012): Health literacy and public health: a systematic review
and integration of definitions and models. In: BMC Public Health 12/80: http://www.biomedcent-
ral.com/1471-2458/1412/1480
Stahl, Katja; Nadj-Kittler, Maria (2013): Zentrale Faktoren der Patienten- und Mitarbeiterzufriedenheit.
Picker Institut Deutschland gGmbH, Hamburg
Street, Richard L.; Makoul, Gregory; Arora, Neeraj K.; Epstein, Ronald M. (2009): How does communi-
cation heal? Pathways linking clinician–patient communication to health outcomes. In: Patient ed-
ucation and counseling 74/3:295-301. doi:210.1016/j.pec.2008.1011.1015
Sustersic, Mélanie; Gauchet, Aurélie; Kernou, Anaïs; Gibert, Charlotte; Foote, Alison; Vermorel, Céline;
Bosson, Jean-Luc (2018): A scale assessing doctor-patient communication in a context of acute
conditions based on a systematic review. In: PLoS One 13/2:e0192306
ten Klooster, PM; Oostveen, JCM; Oostveen, JCM; Zandbelt, LC; Taal, E; Drossaert, CHC; Harmsen, EJ;
van de Laar, MAFJ (2012): 'Further validation of the 5-item Perceived Efficacy in Patient-Physician
Interactions (PEPPI-5) scale in patients with osteoarthritis'. In: Patient education and counseling
87/1:125-133, https://doi.org/110.1016/j.pec.2001.1007.1017
van der Heide, I.; Heijmans, M.; Schuit, A. J.; Uiters, E.; Rademakers, J. (2015): Functional, interactive
and critical health literacy: Varying relationships with control over care and number of GP visits.
In: Patient Educ Couns 98/8:998-1004
van der Heide, Iris; Poureslami, Iraj; Mitic, Wayne; Shum, Jessica; Rootman, Irving; FitzGerald, J Mark
(2018): Health literacy in chronic disease management: a matter of interaction. In: Journal of cli-
nical epidemiology 102/:134-138
Waldherr et al. (2019): Anonyme Tablet-unterstützte standardisierte Befragung zur Gesprächsqualität

im Rahmen der Evaluation des Projektes „Sagen – Fragen – Verstehen“ (Patient/innen und Ange-
hörige). Wiener Neustadt: Ferdinand Porsche FernFH. Unveröffentlicht
Working Group HPH & HLO, International Working Group Health Promoting Hospitals and Health Lit-
erate Healthcare Organizations (2019): International Self-Assessment Tool Organizational Health
Literacy (Responsiveness) for Hospitals - SAT-OHL-Hos-v1.0-EN-international. WHO Collaborat-
ing Centre for Health Promotion in Hospitals and Healthcare (CC-HPH), Wien
Zegers, Carli A; Gonzales, Kelly; Smith, Lynette M; Pullen, Carol H; De Alba, Armando; Fiandt, Kathryn
(2020): The psychometric testing of the functional, communicative, and critical health literacy tool.
In: Patient education and counseling 103/11:2347-2352
Zill, Jördis M; Christalle, Eva; Müller, Evamaria; Härter, Martin; Dirmaier, Jörg; Scholl, Isabelle (2014):
Measurement of physician-patient communication—a systematic review. In: PLoS One
9/12:e112637

Zill, Jördis M; Scholl, Isabelle; Härter, Martin; Dirmaier, Jörg (2015): Which dimensions of patient-cen-
teredness matter? Results of a web-based expert delphi survey. In: PLoS One 10/11:e0141978
12 Digital Health Literacy
Authors:

Jürgen Pelikan (ICC)
Éva Bíró (HU)
Lucy Bruton (IE)
Saskia Maria De Gani (CH)
Sarah Gibney (IE)
Lennert Griese (DE)
Zuzana Klocháňová (SK)
Zdenek Kucera (CZ)
Thomas Link (ICC)
Julien Mancini (FR)
Christopher Le (NO)
Carlota Ribeiro da Silva (PT)
Kristine Sørensen (DK)
Mitja Vrdela (SI)
Chapter 12 / Digital Health Literacy 275

12.1 Background and development of the new instrument
The increasing availability and use of health-related digital/electronic resources such as electronic
health records, telehealth initiatives, digital health applications, interactive communication op-
tions with health care providers (e.g., for making appointments or reporting medical results)
places a growing demand on the population's skills to use these applications and resources. Health
care organizations and governments often encourage, initiate, and develop digital resources which
require cognitive domain skills, including understanding, appraising, and applying health infor-
mation, but also specific skills, such as skills for navigating digital health information. Research-
ers, practitioners, and policy makers should therefore acknowledge the importance of people’s
proficiency in using digital resources for managing disease and/or promoting people’s health.
The amount of information on disease management, prevention and health promotion that is
available and the number of channels that are used for disseminating this information have ac-
celerated in the process of digitalization. Due to the ubiquitous nature of digital communication,
commercial companies and individuals are also seeking the public's attention through digital
channels. As a result, more interest-driven, manipulative, or simply false information is circulated
to the public, requiring particularly critical and analytical skills from the public and individual users
(Paige et al. 2018).
Whether increased digitalization for health moves societies towards better health for all, leaving
no one behind or, alternatively, widens the digital divide between populations, remains a point of
discussion (Levin-Zamir/Bertschi 2018). To address this issue, a better understanding of the scope
and importance of Digital Health Literacy (DHL) is necessary. For that reason, a measure of DHL
was included as an option in the HLS19 study, as part of the “HLS family”, along with the other
specific literacies that were included in the national surveys.
In this chapter we will use the term Digital HL, in accordance with the concept of digital literacy
(American Library Association, 2013). This term is preferred to eHealth literacy by the authors.
While eHealth and Digital Health are often used interchangeably and are closely related, eHealth
is interpreted as focusing mainly on healthcare, while the term Digital Health is more inclusive and
relates also to mHealth (mobile), artificial intelligence and other emerging areas of innovation and
information technology (WHO, 2018).Overview on relevance, existing research, and measures of
Digital HL
The need to assess people’s use of electronic sources of health-related information was first rec-
ognized by Norman/Skinner (2006), who used the term eHealth literacy and defined it as “the
ability to seek, find, understand, appraise health information from electronic sources and apply
the knowledge gained to addressing or solving a health problem”. In accordance with this defini-
tion, they developed the self-report measure eHealth Literacy Scale (eHEALS), assessing skills to
find and evaluate health information on the Internet. However, the rapid development of digital
health resources, beyond information seeking from the Internet which was the only digital re-
source available when the above-mentioned definition was established, warrants new thinking
regarding definitions and measurement tool development. Subsequently, eHealth literacy has been
commonly assessed in health-related studies (Karnoe/Kayser 2015). Furthermore, eHealth Liter-
acy along with media health literacy has been conceptualized as the interplay of personal, situa-
tional, and contextual factors, and their interactions (Levin-Zamir/Bertschi 2018). A plethora of
studies have examined eHealth literacy and Digital HL respectively throughout the lifespan, from
childhood and adolescence (Levin-Zamir et al. 2011; Livingstone et al. 2017; Yang et al. 2017)
through adulthood and among the elderly (Choi/DiNitto 2013; Leung et al. 2007). Additionally,
Digital HL has been examined with respect to cultural transition (Levin-Zamir et al. 2017), to spe-
cific health conditions (Saha et al. 2017), and to specific health behaviors (Aharony/Goldman
2017), but studies on performance based eHealth literacy are still scarce (Neter/Brainin 2017; van
der Vaart et al. 2011). Empirically, General HL and eHealth literacy are different but related con-
cepts (Neter et al. 2015).
A recent study refers to Digital HL as the individual, social and technical competencies and re-
sources that are important for searching, finding, understanding, evaluating, and applying digi-
tally health information (Zeeb et al. 2021). According to Bittlingmayer et al. (2020) Digital HL refers
not only to navigation in digital space and use of digital resources and their evaluation (e.g., health
apps, social media, information sites on the Internet), but also to the significantly increased indi-
vidual "option spaces" that arise with digitalization and the related availability of health infor-
mation.
Yet, while eHEALS has been extensively used to assess the self-reported use of electronic sources
for health information, the past decades have offered many additional opportunities to promote
health through digital sources, thus widening the scope of the digital health skills that are needed
by individuals to maximize the potential for applying digital resources for health. To capture these
skills, a broader self-report scale, which also addresses the interactive task of adding content,
was developed, and validated in the Netherlands (DHLI; Van Der Vaart/Drossaert 2017). While it
well captures various aspects of Digital HL, to our knowledge at the time of the HLS19 study, this
scale has not been applied/field tested in large samples.
It should be acknowledged that during the COVID-19 pandemic, the need for learning about the
public’s potential response to digital resources, including the way in which these resources can
help mitigate the pandemic and promote health, significantly increased. Several surveys were
launched, including the cross-sectional international survey among university students in Ger-
many and other countries (Dadaczynski et al. 2021), using the Digital Health Literacy Instrument
(DHLI), based on 5 subscales adapted to the specific context of the COVID-19 pandemic. The
Digital Health Equity Framework was also developed during the COVID-19 pandemic, due to con-
cern over the potential of increased social disparities because of disparities in access and use of
digital health resources (Crawford/Serhal 2020).
In conclusion, valid and reliable information about people’s Digital HL can help health authorities
to discern the patient and public experience and difficulties in dealing health information from
digital sources, including the extent to which these resources are apparent, accessible, under-
stood, evaluated, and applied. For that reason, Digital HL was included as an optional package in
HLS19.

Rationale for providing a new measure and procedure
of developing the measure and specific correlates for
the measure in HLS19
While Digital HL has been assessed in few countries and in the European Region (Comission 2014;
Zrubka et al. 2020), these assessments mainly used the eHealth measure (eHEALS) or similar in-
struments. These studies have not looked at Digital HL in the broader societal context and not
provided relevant associations with other aspects of HL. Furthermore, there are no international
comparative studies on Digital HL. To our knowledge, apart from national studies conducted as
part of HLS19 and already published (Le et al. 2021; Schaeffer et al. 2021, and others), to date no
national HL survey has assessed and compared Digital HL to General HL. Also, even at a national
level, there are no studies that have examined the determinants of Digital HL and its contribution
to General HL or to health promoting behavior, healthy lifestyles, early detection of diseases, or
self-care in the case of long-term illness, and use of health services.
HLS19 offered an opportunity to develop and evaluate/validate a new measure for Digital HL, to
report Digital HL at the general adult population level, and to study its association with General
HL, with socio-demographic-economic and other determinants and with possible health-related
consequences and outcomes.
A working group including experts was established to develop the instrument and select potential
personal factors and covariates relevant for explaining variance in Digital HL.
Definition and the Concept of Digital HL in HLS19
The concept of Digital HL, adopted in HLS19, is based on the conceptual model, definition, com-
prehensive understanding, and operational matrix of General HL proposed by the HLS-EU consor-
tium (Sørensen et al. 2012), yet aligned with existing research on the scope and diversity of digital
health resources. The concept refers to the health-related use of updated digital technologies and
resources, such as social media, health apps, wearables, and personal health records/interaction
with healthcare providers.
The HLS19 concept of Digital HL for promoting health includes the ability to search for, access,
understand, appraise, validate, and apply online health information, the ability to formulate and
express questions, opinion, thoughts, or feelings when using digital devices. This concept relates
strongly to the frequency with which people use different health resources from digital sources
and resources such as online video consultations, digital personal health records, social media,
health related apps, etc. The HLS19 Digital HL optional package was developed with the purpose
of capturing these aspects of Digital HL.
The HLS19 Digital HL optional package measure
The HLS19 optional package on Digital HL includes the ability to search for, access, understand,
appraise, validate, and apply online health information (measured by eight items and referred to
as HL-DIGI), and the ability to clearly formulate questions, opinions, thoughts, or feelings when
interacting by typing/posting on a digital device (measured by two items), referred to as HL-DIGI-
INT. These ten items were aligned with the HLS19 template by asking how easy or difficult it is to
perform the tasks. The analysis of the two parts was conducted separately, as the psychometric
analyses identified them as separate dimensions. The instrument for measurement is heavily
based on the conclusions of a previous study on the development of the DHLI mentioned above
(Van Der Vaart/Drossaert 2017). That instrument was adjusted for current use in the HLS19 study
to ensure a better alignment with the health literacy model adopted by the rest of the HLS19 study,
applying the conclusions and recommendation of the authors of the previous study for measuring
on a population level. Redundancy on the topic of applying health information was eliminated, and
the dimension of understanding health information accessed digitally, which was not part of the
original measure, was added to align with the definition of DHL. Furthermore, the questions re-
garding privacy that appear in the DLHI tool were not included, as the authors themselves men-
tioned in the Discussion that this dimension might not be useful in future studies.
In the current study, a cluster of six additional items was added to collect information on the
frequency with which people use different digital sources and resources for promoting their health.
Included in the terms ‘digital sources and resources’ are websites, social media, digital devices
related to health or health care, or digital interaction with one’s health system (such as online
video consultations, digital personal health records, health related apps on mobile phone, etc.).
Objectives and research questions of the Digital HL

optional package
Considering Digital HL as an asset, the ultimate purpose of including measures of Digital HL in

the HLS19 survey was to provide evidence-informed recommendations that could drive health pro-
moting policy.
The research questions were:

1. What are the psychometric properties of the new eight-item HL-DIGI scale?
2. How are HL-DIGI score and the two items set on interactive digital devices (HL-DIGI-INT)
distributed in general populations and sub-populations?
3. How closely are Digital HL and General HL, measured by HLS19-Q12, related?
4. To what extent are the related traits of Digital HL, and General HL, measured by HLS19-
Q12, empirically different?
5. What are relevant determinants of Digital HL, based on the background variables availa-
ble in the HLS19? To what extent is there a social gradient for Digital HL?
6. How is the scope of self-reported usage of different digital health resources for promot-
ing health, related to Digital HL?

7. What are relevant consequences of Digital HL, based on health-related personal outcome
variable and covariates available in HLS19?
Countries participating in the Digital HL Optional Pack-

age
The following countries included the optional package on Digital HL in their national assessment
(in alphabetical order): Austria (AT), Belgium (BE), Czech Republic (CZ), Denmark (DK), France (FR),
Germany (DE), Hungary (HU), Ireland (IE), Israel (IL), Norway (NO), Portugal (PT), Slovakia (SK) and
Switzerland (CH), but for France this optional package was only included in the second wave of
data collection with 1,000 respondents. Analyses were based on 28,057 respondents, with country
specific sample sizes ranging from 1,000 to 3,602. As indicated in Chapter 2.4, there was a vari-
ation in data collection method (CAPI, CATI, CAWI, PAPI and mixed) among, but also within, coun-
tries as some countries applied different collection methods for different sub-populations. The
participation in a CAWI interview necessitates some familiarity with digital medias. CAWI interviews
were used exclusively in BE, DK, and FR. Parts of the surveys were done via CAWI in CH, CZ, and
IL.
12.2 Methods of analyses and results
The specific measures for the optional package Digital HL consist of three blocks:
1. Use of digital resources (6 items)
2. Digital health literacy (8 items)
3. Interaction with digital devices (2 items).
For the items on the use of digital media, respondents replied to the question “In a typical week,
how many days do you use the following digital resources for getting health related information”
for websites, social media, digital devices, mobile health apps, eHealth, or other digital resources
(see Annex 12.4). The response categories were: “Not relevant for me” or “Less than once per
week”, “1-3 days per week”, “4-6 days per week”, “Once a day”, or “More than once per day”. A
mean score (ranging from 1 “Not relevant or less than once per week” to 5 ”More than once per
day”) was calculated as a relative measure for the frequency of use of health-related digital re-
sources. The values of this mean score have no direct interpretation but can be used to order
respondents by the average frequency of use of digital resources. This item set was not included
in the questionnaire of the Norwegian survey, which is why Norway is missing in the respective
tables and figures.
To analyse and report on Digital HL, a scale measuring the skills related to seeking health infor-
mation digitally (HL–DIGI) was constructed. This scale consists of eight questions related to tasks
on how easy or difficult it is to search for, find, understand, and judge health information from
digital sources. The internal consistency and the unidimensionality of the scale were verified by
means of the Cronbach alpha coefficient, confirmative factor analyses, and Rasch analyses. The
HL–DIGI-HI scale data were tested up against the unidimensional Rasch partial credit model (PCM).
For each country, overall data-model fit, single item fit, the ordering of response categories, re-
sponse dependency, one-dimensionality, and differential item functioning (DIF) were evaluated.
As for the HLS19 General HL measure, the Digital HL scores are based on a count of the dichoto-
mized items, combining categories “easy” and “very easy” and “difficult” and “very difficult”, re-
spectively. The resulting score was standardized to the range of 0 to 100, as per the procedure of
HLS19. As such, the score indicates the percentage of items that are considered as “easy” or “very
easy” by a respondent. Scores were only computed for respondents who had answered at least
80% of the items.
For difficulty of interaction with Digital Devices (HL-DIGI-INT), two questions that focus on re-
ported difficulty of tasks for interacting with digital health resources were used. The correlation
between Digital HL with these combined to an index is considerable (r=0.48), but low enough to
justify two different measures.
For the visualization of the perceived difficulties at item level, the response categories "very diffi-
cult" and "difficult" were combined (Figure 12.1). Average Percentage Response Patterns (APRP)
(for details see chapter 4.4) (Figure 12.2) and the (mean) scores (Table 12.6) for HL–DIGI were
used to report on Digital HL. APRP indicate the average percentages for all eight items of the
measure, answered by each of the four response categories. The APRPs displays the percentages
with which the four response categories, on average, were used when responding to the eight
items by a specific population. For the calculation of the APRP for the HL–DIGI, the average number
of times (out of eight items) respondents used each response category while responding to the
eight items, was estimated.
Distributions of individual items of measuring Digital

HL
This section describes an overview of the valid responses for perceived difficulties (“very difficult”
and “difficult”) for the items of the HL-DIGI scale (Figure 12.1, Table 12.1) and the two-item set
on interactive digital devices for each country (Table 12.2) as well as the APRPs for each country
and for all countries (equally weighted) for the HL-DIGI scale and the two-item index (Figure 12.2,
Figure 12.3).

Figure 12.1:
Percentages of respondents who responded with “very difficult” or “difficult” to the HL-DIGI
items (ordered by the overall mean), for each country and the mean for all countries (equally
weighted)
Regarding the single items that comprise the HL-DIGI scale (Figure 12.1), ranking of combined
difficulty response categories of tasks across countries is rather similar with some exceptions. The
percentages of combined “difficult” and “very difficult” answers (Table 12.1), range (for all coun-
tries weighted equally) from 21.8% for the on average easiest item 1, “to use the proper words or
search query to find the information you are looking for” to 54.1% for perceived difficulty of the
on average most difficult item 4 “to judge whether the information is reliable”. For both items,
there is a considerable variance in difficulties across countries with the data from NO and DE as
minimum and maximum. For item 1 the perceived difficulty ranges from 9.5% (NO) to 38.5% (DE).
For item 4, the perceived difficulty ranges from 30.6% (NO) to 82.6% (DE). Percentage distributions
for all four categories, for each item, are provided in Annex 12.1 for each country and for the
mean of all countries (equally weighted) (Table A 12.1 to A 12.8, respective Tables A 12.23, and
A 12.24). For some countries, non-response rates for some items were markedly higher (35% (PT),
23% (IE), 20% (CZ), 20% (HU), 19% (NO), 17% (AT)) than for the other HL measures used in HLS19.
This is to some extent a result of how non-users were handled in these countries, some of which
used filter questions. As a result, the samples may not be representative of the full country sam-
ples, for example in Ireland, those missing have a higher mean age to the total sample and a lower
mean education level. Spearman correlations of the 8 items with each other are shown in Annex
12.2 (Tables A12.9 to A12.22), for individual countries and for all countries. Figure 12.2 shows
the APRP for the items of HL-DIGI scale. Nearly two-fifths of the mean of all countries report
difficulties, while this proportion ranged from 22% (NO) to 58% (DK).
Table 12.1:
Percentages of respondents in each country who responded with “very difficult” or “difficult” to the HL-DIGI items (ordered by the mean for All), for each
AT BE CH CZ DE DK FR HU IE IL NO PT SK All
4. to judge whether the information is reliable? 46.9 57.1 66.0 65.8 82.6 43.1 64.8 53.7 51.6 58.6 30.6 41.2 61.2 54.1
5. to judge whether the information is offered with com- 49.7 55.1 63.2 51.3 82.1 57.0 69.2 49.0 50.9 58.4 25.9 36.1 50.4 53.6
mercial interests?
8. to use the information to help solve a health problem? 31.7 51.6 52.7 47.3 61.9 34.4 51.4 23.4 44.1 46.2 31.4 35.5 40.8 42.0
7. to judge whether the information is applicable to you? 29.9 49.7 51.0 41.6 56.3 31.0 55.7 23.7 36.0 32.9 29.7 21.0 38.6 37.8
2. to find the exact information you are searching for? 23.7 42.6 40.5 39.9 58.9 31.3 41.7 24.1 38.7 31.6 22.8 28.6 37.8 35.0
3. to understand the information? 22.3 36.4 29.0 32.0 47.5 21.9 34.9 16.8 27.2 29.8 14.2 16.6 33.5 26.9
6. to visit different websites to check whether they pro- 22.2 36.3 28.7 23.1 37.9 26.2 22.1 22.7 24.7 22.9 7.6 17.7 31.5 25.0
vide similar information about a topic?
1. to use the proper words or search query to find the in- 15.1 32.3 23.6 23.7 38.5 17.8 29.3 15.4 29.9 18.0 9.5 10.8 24.0 21.8
formation you are looking for?

Figure 12.2:
Average Percentage Response Patterns (APRP) for the HL-DIGI scale, for each country and mean
of all countries (equally weighted)
Difficulty of interaction with digital devices
The ease or difficulty with which respondents estimated their ability to interact with digital devices
was assessed using a two-item set on interaction with digital devices. The ranking of difficulty of
tasks is similar across countries with one exception, where both items are equally difficult. On
average, item 1 "to clearly formulate your written message when communicating with a health
provider" was easier (on average 26.8% responded “difficult” or “very difficult”, varying from 9.6%
(PT) to 45.4% (SK)) than item 2 "to express your opinion, thoughts or feelings, ask a question in
writing on social media including online forums" (on average 35.6% responded “difficult” or “very
difficult” ranging from 14.0% (PT) to 50.6% (SK)).
Table 12.2:
Percentages of respondents in each country who responded with “very difficult” or “difficult” to the two items on interaction with digital devices (HL-DIGI-INT)
(ordered by the mean for the All), for each country and the mean for all countries (equally weighted)
2 express your opinion, thoughts, or feelings, ask 27.0 42.3 43.5 30.5 39.3 35.9 24.0 22.9 42.8 31.9 20.2 14.0 50.6 35.6
a question in writing on social media including
online forums?
1 clearly formulate your written message when 22.8 35.7 34.5 26.9 35.6 24.2 17.7 23.9 31.0 26.3 10.2 9.6 45.4 26.8
communicating with a health provider?

Figure 12.3 shows the APRP for the two items on interaction with digital devices. On average, more
than one-quarter of respondents (29% responded “difficult” or “very difficult”, varying by country
from 11% (PT) to 48% (SK)) reported difficulties with these two tasks, while 51% of the respondents
(varying from 35% (IE) to 78% (PT)) experienced the tasks as easy and 20% (varying from 10% (SK)
to 38% (NO)) as very easy.
Figure 12.3:
Average Percentage Response Patterns (APRP) for an index of the two items on interaction with
digital devices (HL-DIGI-INT), for each country and mean of all countries (equally weighted)
Validity and psychometric properties of the Digital HL
scale
Internal consistency or “test reliability” (i.e., the measure’s ability to distinguish between respond-
ents with different proficiency) was estimated by Cronbach’s alpha coefficient (cf. Chapter 4.7).
The Cronbach’s alphas for the HL-DIGI score are above 0.7 for each country (Table 12.3).
For the eight items of the HLS19-DIGI score, single-factor confirmatory factor model with dichot-
omized items loading on a single latent variable was estimated for each country. The fit indices
generally indicate an acceptable fit in all countries (Table 12.4). The standardized root mean
square residual (SRMSR) (assuming a 0.08 threshold value) and the lower bound of the confidence
interval of the root mean square error of approximation (with 0.05 as threshold) are slightly sub-
optimal for most countries hinting at some possible misfit between the observed data and the
single-factor model. Other fit indices (comparative fit index, the Tucker-Lewis index, the good-
ness of fit, and the adjusted goodness of fit index) generally indicate a sufficiently good fit be-
tween the observed covariance matrix and the model implied covariance matrix for all countries
(cf. Chapter 4.7.2 for the applied thresholds).

Table 12.3:
Cronbach’s alphas for the HL-DIGI scale, for each country and mean of all countries (equally weighted)
AT BE CH CZ DE DK FR HU IE IL NO PT SK Mean
HL-DIGI 0.81 0.86 0.85 0.82 0.83 0.86 0.86 0.79 0.79 0.83 0.77 0.83 0.87 0.83
Table 12.4:
Fit indices for the one-factor confirmatory factor model with the eight HL-DIGI items as indicators, for each country and mean of all countries (equally
weighted)
Standardized Root Mean Square Residual 0.07 0.12 0.08 0.07 0.07 0.09 0.06 0.10 0.06 0.08 0.07 0.07 0.07 0.08
Root Mean Square Error of Approximation 0.08 0.13 0.09 0.07 0.08 0.11 0.07 0.10 0.06 0.08 0.06 0.08 0.08 0.08
Root Mean Square Error of Approximation (p 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.06 0.00 0.05 0.00 0.00 0.01
Value)
Comparative Fit Index 0.98 0.98 0.99 0.99 0.99 0.98 0.99 0.96 0.99 0.99 0.98 0.99 0.99 0.98
Tucker-Lewis index 0.97 0.97 0.98 0.98 0.98 0.98 0.99 0.94 0.98 0.98 0.98 0.98 0.99 0.98
Goodness of Fit Index 0.98 0.97 0.99 0.98 0.99 0.98 0.99 0.97 0.99 0.99 0.99 0.99 0.99 0.98
Adjusted Goodness of Fit Index 0.97 0.95 0.98 0.97 0.98 0.97 0.98 0.94 0.98 0.97 0.98 0.98 0.98 0.97
A Rasch analysis was conducted to examine the unidimensionality of the scale (cf. Chapter 4.7.3).
When an amended sample size corresponding to 20 respondents for each of 24 thresholds
(n=480) was used, the overall data-model fit was sufficient (χ²statistic) for data collected in Aus-
tria (CATI), Germany, Ireland, Norway, and Switzerland. Acceptable fit for data collected in Belgium
and Czech Republic was also observed. With a reduced sample size of n=240 or 10 respondents
per threshold, the data collected in Denmark, Hungary, Israel, and Portugal displayed acceptable
overall data-model fit as well. Data from France and Slovakia was not included in the Rasch anal-
ysis because it was not available yet at the time of analysis.
According to a principal component analysis (PCA) of Rasch model residuals combined with de-
pendent t-tests to identify possible empirical subscales the HLS19-DIGI scale was sufficiently uni-
dimensional. The thresholds, and thus the response categories, were ordered and well-function-
ing. No significant statistical dependence between pairs of items were observed, which means that
no items are “too similar” and collect redundant information.
Item 5 “to judge whether health-related information is offered with commercial interests” tends
to discriminate somewhat poorly across most countries. Hence, this item does not strictly conform
to the latent trait underlying the scale. Some items display DIF for person factors, such as age and
gender, but there is no consistent pattern across countries. Furthermore, the HLS19-DIGI scale
does not measure invariantly across countries as the item location or “difficulty order” varies be-
tween countries (cf. Figure 12.1). This may be ascribed to DIF for country or language.
Table 12.5 displays the correlation coefficients for the HL-DIGI score with General HL (GEN-HL),
as well as with other specific HL scores measured in HLS19, such as Navigational HL (HL-NAV).
Overall, the strongest correlations are observed between Digital HL and General HL (0.53), and
between Digital HL and Navigational HL (HL-NAV) (0.55).

Table 12.5:
Pearson correlation (r) between HL-DIGI score and the specific scales measuring HL in HLS19, for each country and all countries (equally weighted)
HLS19-Q12 0.46 0.44 0.49 0.57 0.59 0.54 0.59 0.5 0.49 0.67 0.48 0.55 0.54 0.53
HL-COM-Q11 0.36 - - - 0.42 - - - - - - - - 0.39
HL-COM-Q6 0.32 0.2 - 0.36 0.39 0.31 0.38 0.24 - - - - - 0.31
HL-NAV 0.57 0.36 0.52 0.57 0.59 - 0.67 - - - - 0.54 - 0.55
HL-VAC 0.33 0.39 - 0.39 0.34 - - 0.45 0.33 - 0.38 0.41 - 0.38
Distributions of HL-DIGI score measures
Table12.6 reports the distribution of HL-DIGI scores and Figure 12.2 displays the distribution
visually. The score values range from 0 to 100. A higher value indicates a higher level of HL-DIGI.
The mean score over all countries (weighted equally) is 62.3, varying between 41.8 (DE) and 78.7
(NO). For most countries the distribution is left skewed with the 75% quantile starting at the max-
imum value, indicating a ceiling effect for the measure (Figure 12.4).
Table 12.6:
Means, standard deviations, medians, and quartiles of the HL-DIGI scores, for each country and
for all countries (weighted equally)

AT 70.1 29.2 50.0 75.0 100.0
BE 54.9 34.6 25.0 50.0 87.5
CH 55.7 33.0 25.0 62.5 87.5
CZ 59.3 31.4 37.5 62.5 87.5
DE 41.8 31.6 12.5 37.5 62.5
DK 67.2 32.2 37.5 75.0 100.0
FR 53.9 33.5 25.0 50.0 87.5
HU 71.8 26.9 62.5 75.0 100.0
IE 62.1 30.2 37.5 62.5 87.5
IL 62.7 31.0 37.5 62.5 87.5
NO 78.7 24.7 62.5 87.5 100.0
PT 74.0 28.7 57.1 85.7 100.0
SK 60.3 34.2 37.5 62.5 100.0
All 62.3 32.5 37.5 62.5 100.0

Figure 12.4:
Histograms of the distribution of the HL-DIGI scores, for each country

subpopulations
To identify the subpopulations that could potentially be disadvantaged in terms of the Digital HL,
the HL-DIGI mean scores were compared for selected potential vulnerable or disadvantaged sub-
populations to the mean HL-DIGI score observed for the whole population. The vulnerable sub-
populations being considered were older people, people with the lowest educational level, people
with a low self-reported social status/level in society, financially deprived people, people with
poorer health status, and people who most frequently use healthcare services. The results are
shown in Table 12.7.
On average, old age (-12.0) showed the strongest deviation from the whole population, followed
by (very) bad self-perceived health (-11.4), by strong financial deprivation (-8.7), by low education
(-7.8), for many contacts with GPs/family doctor (-6.9) low level in society (-6.8), while the two
healthcare-related indicators showed less deviation, for limited health problems (-4.7) and for
having long-term illnesses or health problems (-3.2). For most indicators variation between coun-
tries was considerable.
Table 12.7:
Differences in mean HLS19-DIGI score between the country sample and selected vulnerable subpopulations, for each country and the mean for all countries
(equally weighted)
HL-DIGI country mean 70.1 54.9 55.7 59.3 41.8 67.2 53.9 71.8 62.1 62.7 78.7 74 60.3 62.5
Aged 76 or older -9.7 -5.8 -16 -2 -23 -11.6 - -18.3 -11.5 -4.9 -8.4 1.7 -34.9 -12.0
Education at ISCED levels 0 or 1 -20.2 7.6 -10.6 1.5 -8.4 -24.1 6.2 -6 -11.1 -8.8 3.4 -9.9 -20.7 -7.8
Level in society less than or -2.2 -5.8 -5.9 -3.7 -8.2 -9.5 -4.3 -5.8 -4.7 -9.1 -4.3 -6 -19 -6.8
equal to 4
Considerable or severe financial -11.3 -2.4 -4.9 -8 -10.4 -9.6 -4.7 -9.6 -8.7 -7.3 -13.4 -8.7 -14.5 -8.7
deprivation
Bad or very bad self-perceived -8.3 0.4 -9.1 -4.4 -23 -8.9 -11.9 -9.8 -10 -9.2 -9.6 -21.5 -23.3 -11.4
health
One or more long-term illnesses -4.1 -1.4 -4.7 -1.2 -3.3 -2.7 -1.7 -4.7 -0.6 -3 -1.2 -8.2 -4.7 -3.2
or health problems
Limited by health problems -6.6 -3.3 -4.7 -3 -4.7 -5.2 -0.7 -7.2 -2.9 -1.7 -2.3 -9.3 -9.8 -4.7
6 or more contacts with a -4.6 -2.7 -7.7 -1 -8.9 -9.5 -3.3 -8.4 -6.1 -2.5 -3.2 -12.2 -19.2 -6.9
GP/family doctor
- … Cells with less than 30 respondents were not reported

Use of digital resources
To assess the use of specific digital resources of information, respondents were also asked to
state the number of days in a typical week they were using five specified selected digital resources:
websites, social media, digital devices related to health care, mobile health apps, digital services
of the respective national health system (for detailed wording of questions see Annex 12.4). Dis-
tributions vary by type of digital resources used, with on average higher use for websites, followed
by social media, digital interaction with the health system, a digital device related to health or
health care, a health app on a mobile phone, and other (Table 12.8). Distributions for individual
countries are provided in the Annex 12.5 (Tables A 12.25 to A 12.30), as well as associations for
these indicators of use of digital resources with Digital HL (Figure A 12.1 to A 12.6) as well as with
the index on use of digital resources (Figure A 12.7). On average there is a slight increase of the
mean score of HL-DIGI with an increase of each of the digital resources, but with differing and
partly not so consistent patterns across individual countries.
Table 12.8:
Percentage distributions of responses to use of digital resources, for all countries (equally
weighted)
Not relevant / DK Less than once 1-3 days 4-6 days Once a More than
Type of digital resources / Refusal per week per week per week day once per day
Websites 26.9 47.7 15.1 3.2 3.2 3.9
Social Media including online
42.6 39.3 9.0 2.6 3.1 3.4
forums
A digital device related to
49.8 24.7 7.1 4.6 7.0 6.8
health or health care
Health app on your mobile
52.6 24.8 7.2 3.9 7.0 4.5
phone
Digital interaction with your
48.3 43.9 4.7 1.2 1.1 0.9
health system
Other 70.7 24.2 2.2 1.0 0.9 0.9
A mean score was calculated to summarize the item set with a single score (see Section 12.2).
An interpretation of the use of digital resources and its association with other measures should
consider that in the following countries all or part of the data was collected by means of a web-
based questionnaire: Belgium, Switzerland, Czech Republic, Denmark, France, and Israel (cf. Sec-
tion 12.1.5). The mean score is, on average, 1.4, and varies across countries from 1.2 (CH, DE) to
1.8 (IL), its SD on average 0.6 varies from 0.4 (CH, DE) to 0.8 (IL) (Table 12.9).
Table 12.9:
Means, standard deviations, medians, and quartiles of the average digital resources use score,
AT 1.4 0.5 1.0 1.2 1.7

BE 1.4 0.7 1.0 1.2 1.6
CH 1.2 0.4 1.0 1.0 1.3
CZ 1.5 0.7 1.0 1.2 1.7
DE 1.2 0.4 1.0 1.0 1.2
DK 1.5 0.6 1.0 1.2 1.8
FR 1.3 0.6 1.0 1.0 1.3
HU 1.5 0.7 1.0 1.2 1.8
IE 1.4 0.5 1.0 1.3 1.7
IL 1.8 0.8 1.2 1.7 2.3
PT 1.3 0.5 1.0 1.0 1.4
SK 1.4 0.6 1.0 1.2 1.7
All 1.4 0.6 1.0 1.2 1.7
Determinants and social gradient of Digital HL
There is a legacy in HL research demonstrating a social gradient for HL and the relative predictive
value of selected potential determinants (see Chapter 6). In HLS19, as in HLS-EU, five socio-demo-
graphic and socio-economic factors were investigated as possible determinants, and some addi-
tional variables (migration status, employment status, and long-term illnesses). For Digital HL,
General HL was also considered as a possible determinant. But the most specific determinant for
Digital HL, available in HLS19, is use of digital resources. As a first step, Spearman correlation
coefficients were estimated (Figure 12.5 and Annex 12.7 Table A 12.31), and bivariate associations
of Digital HL with selected determinants were graphically explored (Annex 12. 8 Figures A 12.8 to
A 12.10).

Figure 12.5:
Spearman correlations (ρ) among HL-DIGI, GEN-HL, and selected socio-demographic and socio-
economic determinants, for all countries (equally weighted)
Highest correlated with HL-DIGI, was, on average, as already shown, General HL (ρ=0.57/0.58 10),
varying from ρ=0.45 (BE) to ρ=0.68 (IL)), next highest was financial deprivation (ρ=-0.19/-0.17);
from -0.09 (BE) to -0.37 (SK)). Graphical presentation for individual countries is provided in Annex
12.8 Figure A 12.10 which shows a rather linear patterns for most countries, i.e., the more severe
financial deprivation the lower HL-DIGI.
Over all countries (weighted equally), the Spearman correlation coefficient of HL-DIGI with the
score for use of digital resources (see above chapter 12.2.4) is ρ=0.15 (ranging from 0.04 (BE) to
0.31 (DE)) (see Annex 12.7 Table A 12.31). A graphical presentation for individual countries is
10
Differences in values of Figure 12.5 and Table A 12.31 are due to different samples: NO is not included in Table A 12.31 as
use of digital resources was not measured.
provided in Annex 12.6 Figure A 12.7. For most countries, HL-DIGI increases slightly with the
score for use of digital resources, while some countries show an inconclusive pattern.
No training in a health profession correlated with HL-DIGI with ρ=-0.14 for all countries (equally
weighted) (ranging from -0.03 (AT) to -0.18 (SK)). Level in society (ρ=0.13, ranging from 0 (AT)
to 0.25 (SK)) was next strongly related, followed by age in years (ρ=-0.11; from +0.03 to -0.25).
For all countries (weighted equally), HL-DIGI is slightly negatively correlated with age (ρ=-0.1, cf.
Annex 12.8 Figure A 12.8). For most countries the mean HL-DIGI score decreases with older age
group, but there are a few exceptions. Next with respect to the absolute size of the Spearman
correlation coefficient was education (ρ=0.07, from -0.03 (CZ) to 0.31 (SK)). In most countries,
the relationship between HL-DIGI and education is approximately linear in that the mean score of
DIGI-HI increases with educational level (Annex 12.8 Figure A 12.9). The Spearman correlation
coefficients of HL-DIGI with gender (ρ=0.01, varying from -0.07 (PT) to +0.07 (DK), and with
migration background (ρ=0.01, from -0.06 (IL) to 0.09 (CH) were very low for all countries (equally
weighted).
Regression model 1 with the five above mentioned social core determinants was investigated (Ta-
ble 12.10). Equivalent models were also analyzed for General HL and the other specific HL
measures in HLS19. Three further regression models were investigated. Regression model 2 in-
cludes additionally the score for use of digital resources (Table 12.11), regression model 3 in-
cludes additionally the General HL score (Annex Table A 12.32), and regression model 4 includes
additionally long-term illnesses/health problems (Annex 12.9 Table A 12.33).
Multivariable regression models for investigating determinants and social gradient of Digital HL
(HL-DIGI)
For investigating a social gradient of Digital HL, as measured by HL-DIGI, the above described five
social core determinants were used, which were also utilized in other chapters of this report (see
Chapter 4) and in HLS-EU.
Model 1 (Table 12.10), with five social determinants, explains 6% of the variance in Digital HL for
all countries weighted equally, varying from 2% (BE) to 23% (SK). The predictor with the, on average,
highest standardized coefficient for all countries weighted equally was financial deprivation with
ß=-0.15 (significant for eleven countries with a ß between -0.08 and -0.27), followed by age in
years (ß=-0.13, significant for six countries, with a ß between -0.15 and -0.26), and level in
society with ß=0.08 (significant for 10 countries, with ß between 0.05 and 0.13). Education and
gender female have, on average, a low ß and are significant only for 5, respectively 3, countries.
Thus, a social gradient for Digital HL was demonstrated, albeit with an inconsistent pattern across
countries.
Model 2 (Table 12.11), additionally including a measure for use of digital resources, explains 7%
(varying between 2% (BE) and 24% (SK)), of the variance in Digital HL in all countries (equally
weighted). This is just one percent more of the variance of digital HL than model 1. Again, financial
deprivation is the predictor with the highest absolute standardized regression coefficient (ß=-
0.14) for all countries (equally weighted), followed by age in years (ß=-0.12), and use of digital

resources (ß= 0.11, significant for ten countries, with ß ranging from 0.06 to 0.22). The regression
coefficient for level in society (ß=0.08) is significant for eight countries, education for just 4 coun-
tries, and gender female for one country. In summary, use of digital resources is a relevant pre-
dictor of Digital HL for almost all countries and partly mediates the effects of the five social pre-
dictors, whose direct effects are somewhat reduced, by including it into the model.
Model 3 (Annex 12.9 Table A 12.32) includes the General HL score in addition to the five social
predictors. General HL increases, not unexpectantly, the explained variance of Digital HL consid-
erably. R² is 33% for all countries weighted equally, varying from 21% (BE) to 46% (IL). The regres-
sion coefficient for General HL (ß=051 for all countries) is significant for all countries weighted
equally, varying between ß=0.44 (AT, BE) and ß=0.67 (IL). In Model 3, General HL is the strongest
predictor, followed by age in years (ß=-0.13, significant for 13 countries with ß between -0.04
and -0.2). The effects of financial deprivation (ß=-0.06 for all countries weighted equally, signif-
icant for four countries between ß -0.06 and -0.15), level of society (ß=0.03 for all countries
weighted equally, significant for two countries with ß between 0.05 and 0.07) and female gender
(ß=0 for all countries weighted equally, significant for just two countries) are considerably smaller.
The regression coefficient of education (ß=0.06 for all countries weighted equally, significant for
7 countries, with ß between 0.05 and 0.17) is larger. Thus, General HL is an important predictor
of Digital HL, partly mediating effects of the five social determinants.
Regression model 4 (Annex 12.9 Table A 12.33) includes long-term illnesses/health problems in
addition to the five social determinants but the explained variance of Digital HL is comparable to
model 1 (R²=0.06 for all countries weighted equally, varying between 2% (BE) and 22% (SK)). In this
model, long term illness is the fourth strongest predictor (ß=-0.05 for all countries weighted
equally, significant for two countries with ß between -0.10 and -0.16).
Table 12.10:
Model 1: Multivariable linear regression models of HL-DIGI by five social determinants (standardized coefficients (β) and R2), for each country and for all
Gender female 0.01 0.02 0 0.02 0.03 0.07 0 0.04 0.04 0.04 0.08 -0.07 0.02 0.03
Age in years -0.20 -0.05 -0.18 -0.03 -0.26 -0.17 -0.15 -0.06 0 -0.04 -0.02 -0.08 -0.16 -0.13
Education 0.05 0.01 0.02 -0.07 0.16 0.14 -0.01 0.01 0.07 0.01 0.11 0.04 0.18 0.03
Level in society 0.01 0.12 0.09 0.10 0.13 0.10 0.11 -0.01 0.05 0.13 0.06 -0.03 0.11 0.08
Financial deprivation -0.15 -0.07 -0.10 -0.16 -0.08 -0.09 -0.06 -0.27 -0.13 -0.14 -0.09 -0.17 -0.27 -0.15
R2 0.07 0.02 0.06 0.05 0.14 0.10 0.04 0.08 0.04 0.05 0.04 0.04 0.23 0.06
Valid Count 2253 988 1901 1249 1735 3537 1000 892 3516 1149 2272 760 1624
Total Count 2967 1000 2502 1599 2143 3602 1000 1195 4487 1315 2855 1247 2145
HL-DIGI score: from 0=minimal HL to 100=maximal HL.

Table 12.11:
Model 2: Multivariable linear regression models of HL-DIGI by five social determinants and use of digital resources (standardized coefficients (β) and R2), for
AT BE CH CZ DE DK FR HU IE IL PT SK All
Use of digital resources 0.07 0.06 0.08 0.08 0.22 0.06 0.09 -0.03 0.06 0.16 0.14 0.10 0.11
Gender female 0.01 0.02 -0.01 0.02 0.02 0.06 0.01 0.03 0.04 0.04 -0.06 0.02 0.02
Age in years -0.19 -0.04 -0.16 -0.02 -0.22 -0.16 -0.13 -0.06 0.01 -0.02 -0.07 -0.14 -0.12
Education 0.04 0.01 0.01 -0.06 0.15 0.14 -0.01 0.02 0.07 0 0.03 0.17 0.02
Level in society 0.01 0.12 0.09 0.09 0.09 0.10 0.09 0 0.04 0.11 -0.04 0.10 0.08
Financial deprivation -0.15 -0.07 -0.11 -0.17 -0.08 -0.1 -0.07 -0.26 -0.13 -0.15 -0.16 -0.27 -0.14
R2 0.07 0.02 0.06 0.05 0.19 0.1 0.05 0.08 0.04 0.07 0.06 0.24 0.07
Valid Count 2215 988 1899 1244 1698 3509 1000 871 3509 1139 750 1621
Total Count 2967 1000 2502 1599 2143 3602 1000 1195 4487 1315 1247 2145
Use of digital resources: from 0=minimal to 100=maximal use of digital resources
Health related consequences of Digital HL
For demonstrating the relevance of HL for health in HLS19, as in HLS-EU and earlier studies, indi-
cators for lifestyle, personal health and use of health care services were used (see Chapters 7-9).
For Digital HL, only indicators for health and utilization of health care services were hypothesized.
Concerning indicators for personal health, the strongest absolute Spearman correlation coefficient
with Digital HL was found for self-perceived health (ρ=-0.18 for all countries weighted equally,
on average, Figure 12.6) with ρ varying between -0.10 (IE) and -0.28 (DE) (see Annex 12.10 Table
A 12.34) and for limitation by long-term illnesses/health problems (ρ=0.18 for all countries
weighted equally). The Spearman correlation coefficient with long-term illnesses/health problems
(ρ=-0.11 for all countries weighted equally) is slightly smaller. Only the association with self-
perceived health was further investigated by regression models to demonstrate a potential direct
effect of Digital HL (see Table 12.12.)
Figure 12.6: Spearman correlations (ρ) among indicators of health status, with HL-DIGI, GEN-HL,
and selected socio-demographic and socio-economic determinants, for all countries (equally
weighted)

To determine if Digital HL has an additional effect on self-perceived health besides social deter-
minants, a regression analysis was performed with Digital HL, gender, age, education, level in
society and financial deprivation as predictor variables and self-perceived health as the outcome
variable (Table 12.12), This model explains 15% of the variance of self-perceived health for all
countries weighted equally (R² varying from 10% (IE) to 26% (SK)). Age (average ß=0.24, ranging
from 0.07 to 0.38, significant for all countries except BE) and self-perceived level in society (av-
erage ß=-.17, ranging from -0.29 to -0.09, significant for all countries) are among the strongest
predictors for all countries weighted equally. Digital HL significantly contributes to the regression
in nine out of the thirteen countries (average ß=-0.10, ranging from -0.05 to -0.14).
A second regression model (Annex Table A 12.37) with Digital HL, and General HL (which is cor-
related considerably with Digital HL and somewhat with self-perceived health - Figure 12.6), and
the five social determinants as predictor variables and self-perceived health as outcome variable,
was analyzed, to find out, if Digital HL has an additional effect on self-perceived health, besides
the already known effect of General HL (cf. Chapter 8.3). This model explains 17% of the variance
of self-perceived health for all countries weighted equally (R² varying from 11% (IE) to 26% (SK)).
Self-perceived level in society (ß=-0.16, ranging from -0.27 to -0.08) and General HL (ß=-0.15,
significant for nine countries with ß between -0.09 and -0.23) are among the strongest predictors,
for all countries weighted equally. For most countries, Digital HL (ß=-0.01, significant only for two
countries with ß ranging from -0.08 to -0.09) does not show an additional effect to General HL
on self-perceived health.
Table 12.12:
Model 1: Multivariable linear regression models of self-perceived health, by HL-DIGI score and five core social determinants (standardized coefficients (β) and
R2), for each country and for all countries (equally weighted)
HL-DIGI -0.11 -0.06 -0.08 -0.06 -0.14 -0.11 -0.12 -0.03 -0.05 -0.1 -0.07 -0.08 -0.09 -0.1
Gender female -0.03 0.04 -0.03 -0.02 -0.03 -0.05 -0.01 0.05 -0.03 -0.02 0.03 0.09 0.02 -0.01
Age in years 0.25 0.07 0.21 0.34 0.38 0.08 0.19 0.3 0.11 0.29 0.16 0.3 0.38 0.24
Education -0.06 -0.08 -0.04 -0.11 -0.03 -0.03 0.01 -0.05 -0.08 0.02 -0.09 -0.06 -0.03 -0.03
Level in society -0.1 -0.29 -0.18 -0.15 -0.09 -0.15 -0.23 -0.14 -0.12 -0.17 -0.2 -0.13 -0.1 -0.17
Financial deprivation 0.15 -0.04 0.16 0.16 0.15 0.2 0.13 0.2 0.2 0.13 0.16 0.17 0.14 0.14
R2 0.14 0.11 0.14 0.23 0.25 0.12 0.16 0.22 0.1 0.16 0.12 0.23 0.26 0.15
Valid Count 2251 988 1900 1249 1733 3535 1000 892 3516 1147 2271 760 1621
Total Count 2967 1000 2502 1599 2143 3602 1000 1195 4487 1315 2855 1247 2145
Due to rounding the numbers to two significant decimals. zeros may represent a value in the range of -0.005 to +0.005.
Education by 9 ISCED levels. from 0 (lowest) to 8 (highest level).
Financial deprivation: 4 categories. from no deprivation (0) to severe deprivation (100).

Figure 12.7:
Spearman correlations (ρ) among indicators of health care utilization, with HL-DIGI, GEN-HL, and
selected socio-demographic and socio-economic determinants, for all countries measuring
digital HL (equally weighted)
For indicators of use of healthcare services, the highest correlations of HL-DIGI were found for
utilization of GP/family doctors (ρ=-0.13, varying from -0.04 (BE, CZ) to -0.18 (DE, SK), and of
medical and surgical specialists (ρ=-0.11, varying from 0 (BE) to -0.12 (PT, SK) (Figure 12.7). For
the Spearman correlation coefficients by country see Annex 12.10 Table A 12.35 respectively A
12.36. Therefore, just the association with GP/family doctors (Table 12.13), was further investi-
gated by regression models.
Regression model 1 for Digital HL, gender, age, education, level in society and financial depriva-
tion as predictors for the number of contacts with GPs or family doctors (Table 12.13) explains on
average 6% of the variance (R² varying from 4% (DK, IL) to 14% (DE)). Age (average ß=018, signif-
icant for eleven countries with ß between 0.04 and 0.30), and female gender (average ß=0.09,
significant for ten countries, with ß between 0 and 0.19) are the most important predictors over
all countries weighted equally. On par with financial deprivation (average ß=0.08, significant for
ten countries with ß between -0.01 and 0.17), Digital HL is the third most important predictor of
contacts with GPs or family doctors (average ß=-0.08, significant for seven countries with ß be-
tween -0.01 and -0.09).
Regression model 2 for the number of contacts with GPs or family doctors, with also General HL
included as a predictor (Annex Table A 12.38), explains on average 7% of the variance (R² varying
from 4% (DK, IL) to 14% (DE)). After age in years (ß=0.18, significant for eleven countries with ß
between 0.08 and 0.30), gender female (ß=0.10, significant for ten countries, with ß between 0.07
and 0.19), and financial deprivation (ß=0.08, significant for ten countries with ß between 0.06 and
0.17), General HL (ß=-0.07) was just the fourth important predictor over all countries weighted
equally (p ≤ 0.01significant only for two countries, ranging from ß=-0.07 to -0.08). Digital HL
was the fifth important predictor over all countries weighted equally (ß=-0.05, significant for three
countries with ß between -0.07 and -0.11).

Table 12.13:
Model 1: Multivariable linear regression models of utilization of GPs/family doctors by HL-DIGI and five core social determinants (standardized β coefficients
HL-DIGI -0.1 -0.02 -0.05 -0.01 -0.08 -0.07 -0.06 -0.09 -0.05 -0.04 -0.06 -0.06 -0.09 -0.08
Gender female 0.09 0.08 0.03 0 0.09 0.1 0.07 0.14 0.17 0.09 0.16 0.19 0.07 0.09
Age in years 0.22 0.16 0.18 0.17 0.3 0.07 0.17 0.25 0.11 0.04 0.05 0.12 0.22 0.18
Education -0.01 -0.18 -0.03 -0.09 -0.02 -0.02 -0.01 -0.03 -0.02 -0.07 -0.09 -0.03 -0.03 -0.03
Level in society 0.05 -0.06 -0.08 -0.04 -0.05 -0.09 -0.02 0 -0.05 0.02 0 0 0.03 -0.01
Financial deprivation 0.1 -0.01 0.07 0.1 0.09 0.06 0.12 0.06 0.1 0.14 0.09 0.07 0.17 0.08
R2 0.1 0.09 0.06 0.06 0.14 0.04 0.06 0.11 0.06 0.04 0.05 0.07 0.13 0.06
Valid Count 2209 981 1901 1233 1689 3526 1000 891 3503 1149 2258 760 1615
Total Count 2967 1000 2502 1599 2143 3602 1000 1195 4487 1315 2855 1247 2145
Due to rounding the numbers to two significant decimals. zeros may represent a value in the range of -0.005 to +0.005.
Education by 9 ISCED levels. from 0 (lowest) to 8 (highest level).
Financial deprivation: 4 categories. from no deprivation (0) to severe deprivation (100).
12.3 Summary, Discussion and Conclusions
For measuring Digital HL in general adult populations, a newly developed scale for Digital HL and
additionally few specific correlates were introduced and examined in a sample of nearly 29,000
participants in 13 countries. Based on the conceptual model, definition and matrix of the HLS-EU
consortium, HLS19-DIGI measures the ability to find, understand, evaluate, and apply digitally
available health related information. Two items aimed at measuring difficulty of interaction with
digital sources. Also, frequency of using different kinds of digital resources was measured as a
potential determinant.
The HLS19-DIGI scale of eight items is sufficiently unidimensional. The thresholds, and thus the
response categories, were ordered and well-functioning. One item tends to discriminate some-
what poorly across most countries and some items display DIF for country or language, which
could limit international comparisons. In a later study, it should be investigated whether changes
to items like, e.g., 6 “to visit different websites to check whether they provide similar information
about a topic” and 5 “to judge whether the information is offered with commercial interests” could
improve the fit. The reliability of the scale, based on Cronbach’s alpha, is sufficiently high, with
values from 0.77-0.87. In several countries, the HLS19-DIGI score displays a ceiling effect, with
more than 25% scoring the highest value. The scores on HLS19-DIGI are correlated with General
HL (r=0.53), suggesting that both scales measure parts of the same constructs, but are independ-
ent enough to be treated as different scales. Concerning that, countries did not differ much. The
same holds true for Navigational HL (r=0.55).
The two items measuring difficulty of interacting with digital resources are highly correlated, and,
in some countries, do not differentiate well, suggesting that items should be changed. The two
measures are, on average, correlated (r=0.48), but this differs considerably between countries.
As far as frequency of use of digital sources of information is concerned, the response categories
should be further developed, as most participants did not use the digital sources within the last
week and by this did not differentiate participants.
On average, 8%, across countries from 2% to 18%, of the tasks of the HL-DIGI scale were answered
as “very difficult” and, on average, further 25%, from 20% to 40%, as “difficult”. Thus, depending
on country, from 22% to 58% of the tasks were experienced either as difficult or very difficult. Most
difficult was the task 5 “to judge whether the information is offered with commercial interests”
and 4 “to judge whether information is reliable”. Task 1 “to use the proper words or search query
to find the information you are looking for” or task 6 “to visit different websites to check whether
they provide similar information about a topic” were judged by fewer respondents to be difficult.
In a model of five social determinants, a social gradient for HLS19-DIGI was demonstrated for all
countries, but to a considerable different degree. Strongest social predictors of HLS19-DIGI, on
average, were financial deprivation, age, and level in society, although they were not consistently

significant with p ≤ 0.01 in all countries. Education and gender were significant just in few coun-
tries. In a model with additionally General HL included, it was shown to be a very strong predictor
of Digital HL for all countries, in a model with use of digital resources additionally included, it was
a rather strong predictor for most countries. Concerning vulnerable subpopulations, respondents
older than 75 years, people who perceived their health as bad or very bad, or with low education,
were the most vulnerable concerning Digital HL.
As far as potential consequences of Digital HL are concerned, it was shown that there were, on
average, relevant correlations with self-perceived health, limitations by long-term illnesses/health
problems. For self-perceived health in a model with the five social determinants, HLS19-DIGI had
a significant potential direct effect in 9 out of 13 countries, but just for two countries, when Gen-
eral HL was additionally included, which had a significant direct effect for nine countries. For health
care utilization, correlations were highest, on average, for GP/family doctors, and medical or sur-
gical specialists. In regression models for GPs/family doctors, including the five social determi-
nants, a potential direct effect of HLS19-DIGI was found for seven countries, but when General HL
was also included as a predictor, it had an effect only for three countries.
It should also be acknowledged that the results and conclusions do have limitations, mainly due
to differences in the data collection methods in HLS19 between countries, and within countries. The
method for data gathering in several countries (IL, BE, FR, CZ, DK) was (partly) through web-based
surveys which may possibly have biased the results in the direction of over-reporting for Digital
HL for these countries.
In conclusion, a compact new measure HLS19-DIGI has been validated for 12 national languages
(Arabic, Czech, Danish, Dutch, French, German, Hebrew, Hungarian, Italian, Norwegian, Portu-
guese, Russian, and Slovak) in 13 countries, with acceptable psychometric properties, but some
potential for improvement. The relevance of Digital HL was demonstrated by considerable pro-
portions of general adult populations having limited Digital HL, by a social gradient for Digital HL
and associations of Digital HL with indicators of health status and use of health services. To note,
difficult tasks for dealing with digital health-related information, and sub-populations with higher
proportions of limited HL were identified for prioritising health policy interventions.
12.4 References
Aharony, Noa; Goldman, Romina (2017): E-health literacy and the vaccination dilemma: an Israeli
perspective. In: Inf. Res. 22(2)
American Library Association. Office for Information Technology Policy. Digital Literacy Task Force.
(2013): Digital literacy, libraries, and public policy: Report of the Office for Information Tech-
nology Policy’s Digital Literacy Task Force. Retrieved from http:// www.districtdis-
patch.org/wp-content/ uploads/2013/01/2012_OITP_digInformation Litera-
cyitreport_1_22_13.pdf
Bittlingmayer, Uwe H; Dadaczynski, Kevin; Sahrai, Diana; van den Broucke, Stephan; Okan, Orkan
(2020): Digitale Gesundheitskompetenz–Konzeptionelle Verortung, Erfassung und Förderung
mit Fokus auf Kinder und Jugendliche. In: Bundesgesundheitsblatt-Gesundheitsforschung-
Gesundheitsschutz 63/2:176–184
Choi, Namkee G; DiNitto, Diana M (2013): The digital divide among low-income homebound older
adults: Internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. In:
Journal of medical Internet research 15/5:e93
European Comission (2014): Flash Eurobarometer 404: European citizens’ digital health literacy
Crawford, Allison; Serhal, Eva (2020): Digital health equity and COVID-19: the innovation curve
cannot reinforce the social gradient of health. In: Journal of medical Internet research
22/6:e19361
Dadaczynski, Kevin; Okan, Orkan; Messer, Melanie; Leung, Angela YM; Rosário, Rafaela; Darling-
ton, Emily; Rathmann, Katharina (2021): Digital health literacy and web-based information-
seeking behaviors of university students in Germany during the COVID-19 pandemic: cross-
sectional survey study. In: Journal of medical Internet research 23/1:e24097
Karnoe, Astrid; Kayser, Lars (2015): How is eHealth literacy measured and what do the measure-
ments tell us? A systematic review. In: Knowledge Management & E-Learning: An International
Journal 7/4:576-600
Le, Christopher; Finbråten, Hanne Søberg; Pettersen, Kjell Sverre; Joranger, Pål; Guttersrud,
Øystein (2021): Health Literacy in the Norwegian Population. part 1. Rapport IS-2959,
https://www.helsedirektoratet.no/rapporter/befolkningens-helsekompetanse
Leung, Angela; Ko, Peggy; Chan, Kin Sun; Chi, Iris; Chow, Nelson (2007): Searching health infor-
mation via the web: Hong Kong Chinese older adults' experience. In: Public Health Nursing
24/2:169-175
Levin-Zamir, Diane; Bertschi, Isabella (2018): Media health literacy, eHealth literacy, and the role
of the social environment in context. In: International journal of environmental research and
public health 15/8:1643
Levin-Zamir, Diane; Lemish, Dafna; Gofin, Rosa (2011): Media Health Literacy (MHL): development
and measurement of the concept among adolescents. In: Health education research 26/2:323-
335
Levin-Zamir, Diane; Leung, Angela Yee Man; Dodson, Sarity; Rowlands, Gillian (2017): Health lit-
eracy in selected populations: Individuals, families, and communities from the international
and cultural perspective. In: Information services & use 37/2:131-151

Livingstone, Sonia; Lemish, Dafna; Lim, Sun Sun; Bulger, Monica; Cabello, Patricio; Claro, Magda-
lena; Cabello-Hutt, Tania; Khalil, Joe; Kumpulainen, Kristiina; Nayar, Usha S (2017): Global
perspectives on children’s digital opportunities: An emerging research and policy agenda. In:
Pediatrics 140/Supplement 2:137-141
Neter, Efrat; Brainin, Esther (2017): Perceived and performed ehealth literacy: survey and simulated
performance test. In: JMIR human factors 4/1:e2
Neter, Efrat; Brainin, Esther; Baron-Epel, Oran (2015): The dimensionality of health literacy and
eHealth literacy. In: European Health Psychologist 17/6:275-280
Norman, C. D.; Skinner, H. A. (2006): eHEALS: The eHealth Literacy Scale. In: J Med Internet Res
8/4:e27
Paige, S. R.; Stellefson, M.; Krieger, J. L.; Anderson-Lewis, C.; Cheong, J.; Stopka, C. (2018): Pro-
posing a Transactional Model of eHealth Literacy: Concept Analysis. In: J Med Internet Res
20/10:e10175
Saha, Sarama; Riemenschneider, Henna; Müller, Gabriele; Levin-Zamir, Diane; Van den Broucke,
Stephan; Schwarz, Peter EH (2017): Comparative analysis of diabetes self-management edu-
cation programs in the European Union Member States. In: Primary care diabetes 11/6:529-
537
Schaeffer, Doris; Berens, Eva-Maria; Gille, Svea; Griese, Lennert; Klinger, Julia; de Sombre, Steffen;
Vogt, Dominique; Hurrelmann, Klaus (2021): Gesundheitskompetenz der Bevölkerung in
Deutschland vor und während der Corona Pandemie: Ergebnisse des HLS-GER 2. Interdiszip-
linäres Zentrum für Gesundheitskompetenzforschung (IZGK), Universität Bielefeld,
https://doi.org/10.4119/unibi/2950305
van der Vaart, R.; van Deursen, A. J.; Drossaert, C. H.; Taal, E.; van Dijk, J. A.; van de Laar, M. A.
(2011): Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Valida-
tion of a Dutch version of the eHEALS in two adult populations. In: J Med Internet Res 13/4:e86
van der Vaart, Rosalie; Drossaert, Constance (2017): Development of the digital health literacy
instrument: measuring a broad spectrum of health 1.0 and health 2.0 skills. In: Journal of
medical Internet research 19/1:e27
Yang, Shu-Ching; Luo, Yi-Fang; Chiang, Chia-Hsun (2017): The associations among individual
factors, eHealth literacy, and health-promoting lifestyles among college students. In: Journal
of medical Internet research 19/1:e15
Zeeb, H.; Pohlabeln, H.; Preising, A.; Schulz, B.; Naczinsky, A.: Dortmann, O.: & Kolpatzik, K.
(2021): Digital Health Literacy–results of a representative survey among adults in Germany.
European Journal of Public Health, 31(Supplement_3), ckab164-505.
Zrubka, Zsombor; Fernandes, Óscar Brito; Baji, Petra; Hajdu, Ottó; Kovacs, Levente; Kringos, Di-
onne; Klazinga, Niek; Gulácsi, László; Brodszky, Valentin; Rencz, Fanni (2020): Exploring
eHealth Literacy and Patient-Reported Experiences With Outpatient Care in the Hungarian
General Adult Population: Cross-Sectional Study. In: Journal of medical Internet research
22/8:e19013, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7448194/
WHO (2018): WHO guideline: recommendations on digital interventions for health system
strengthening. World Health Organization; 2019. Licence: CC BY-NC-SA 3.0 IGO, Geneva

13 Vaccination Health Literacy
Authors:
Chiara Cadeddu (IT)
Thomas Link (ICC)
Luca Regazzi (IT)
Christopher Le (NO)
Aldo Rosano (IT)
Luigi Palmieri (IT)
Mitja Vrdelja (SI)
13.1 Background and instrument

measures of Vaccination Health Literacy
Relevance
As a strategy of public health policy, immunization has proven to be effective in preventing dis-
ease, adverse health outcomes, and death (Andre et al. 2008; Ratzan 2011). The World Health
Organization (WHO) estimates that vaccinations save two to three million lives worldwide each
year (WHO 2017). Furthermore, high vaccination coverage rates contribute to the eradication of
communicable diseases (Centers for Disease Control and Prevention 1999).
Ensuring high vaccination coverage rates requires a high level of public and individual confidence
and trust in vaccination programs and the actors initiating these programs. However, research has
shown that confidence in vaccination has declined worldwide, leading to increased delays in or
refusal of vaccination despite the availability of vaccine services (WHO 2017). This contributes to
declining vaccination rates and to the re-emergence of communicable diseases, as the recent
measles outbreaks in Europe have revealed (WHO 2018).
A survey conducted by the Vaccine Confidence Project in 67 countries in 2016 (Larson et al. 2016)
shows that the population in the WHO European Region has the lowest confidence in the safety
and effectiveness of vaccines compared to other regions of the world. This result has been con-
firmed by a more recent comparison of 149 countries (de Figueiredo et al. 2020b). Vaccine hesi-
tancy is therefore a pressing public health issue, especially in Europe, and poses an increasing
challenge to health authorities, as the Corona pandemic is illustrating (Griebler et al. 2021). On
top of that, a Norwegian HL survey among five immigrant populations suggested that immigrants
are at even higher risk of vaccine hesitancy. Among immigrants, lower economic status and lan-
guage proficiency are associated with difficulties in dealing with and processing information about
vaccines (Le et al. 2021). In this context, an EU-wide study has indicated that public confidence in
vaccination correlates significantly with doctors’ confidence in vaccines: In countries where doc-
tors have a higher level of confidence in vaccinations, the population’s trust in vaccinations is also
higher (de Figueiredo et al. 2020). In addition, a systems approach to vaccines in individual coun-
tries has shown to facilitate high vaccination rates (Levin-Zamir 2020).
Credible and tailored information coupled with high HL among citizens could be a way out of the
“vaccination crisis” to regain people’s confidence in vaccinations. However, information on vac-
cination is often very complex, not always easy to understand, and difficult to access. In addition,
individuals nowadays are confronted with a wealth of biased and one-sided information that is
disseminated mainly through digital media (Lorini et al. 2018; Rowlands 2014). Although these
channels can also be used by vaccination advocates, the internet and social media mainly provide
a stage for vaccination opponents to spread misinformation, disinformation, and skepticism (Dubé
et al. 2013; Puri et al. 2020). A high level of population HL is therefore a prerequisite for assessing
Chapter 13 / Vaccination Health Literacy 313

the trustworthiness and quality of information as well as for dealing competently with false and
misleading information (Lorini et al. 2018; Wang et al. 2018).
Therefore, a specific optional package (OP) on Vaccination Health Literacy and relevant explanatory
variables was included in the HLS19 study, even before the Covid-19 pandemic.
Definition
As no adequate definition of Vaccination HL was found in the literature, the definition of General
HL was adopted and applied to vaccination. Following the HLS-EU Consortium’s definition
(Sørensen et al. 2012), Vaccination HL may refer to individuals’ knowledge, motivation, and skills
to find, understand, and evaluate immunization-related information in order to make adequate
immunization decisions (see also Zhang et al. 2020). Understood as a relational concept (Parker,
Ruth 2009), HL does not only emerge from personal skills and abilities but also in conjunction
with the availability, comprehensibility, accessibility and practicability of health-related infor-
mation, communications, and offers, and this also applies to Vaccination HL (Ratzan 2011).
For a better understanding of the concept behind Vaccination HL, it is necessary to distinguish
Vaccination HL from other relevant determinants of vaccination behavior. Research on determi-
nants of (non-)vaccination has shown that decisions for or against immunization are driven by
individual and collective experiences and beliefs, knowledge, situational/contextual conditions
(vaccination information, doctors’ attitudes and knowledge, anti- or pro-vaccination lobbies, etc.),
and the vaccination itself (Dubé et al. 2013; Larson et al. 2015; MacDonald 2015; Thomson et al.
2016). Although vaccination hesitancy proves to be both context- and vaccine-specific, some
general factors have been identified that influence the decision (not) to get vaccinated. The most
important of these factors have been summarized in the “7Cs model” by Geiger et al. (2021):
» Confidence - (lack of) trust in the effectiveness and safety of vaccines, in the healthcare pro-
viders who deliver them, and in committees/individuals who recommend and approve vac-
cines,
» Complacency - (low) awareness of the risks of vaccine-preventable diseases and of the need
for vaccination,
» Constraints - (low) structural or psychological barriers that make vaccination difficult (availa-
bility, affordability, accessibility, comprehensibility of information, effort, etc.),
» Calculation – the degree to which the personal costs and benefits of vaccination are weighed
up,
» Collective responsibility - (low) sense of responsibility towards the community to contribute
to the reduction in contagious and infectious diseases by getting vaccinated to protect others
(e.g., children or sick people),
» Compliance – the degree to which social monitoring and the sanctioning of people who are
not vaccinated are supported,
» Conspiracy – the degree of belief in conspiracy theories and fake news about vaccinations.
The 7Cs model, which emphasizes individual determinants, integrates various categorizations of
vaccination determinants, such as the 5As model, the 3Cs model, the 5Cs model, and the SAGE
model on determinants of vaccine hesitancy (Betsch et al. 2018; Dubé et al. 2013; Larson et al.
2015; Thomson et al. 2016). HL and Vaccination HL seem to relate to the 7Cs model and should
therefore be investigated in conjunction with its constituents.
A brief literature review
A recent systematic review (Lorini et al. 2018) identified nine studies that examine the relationship
between HL and vaccination behavior and attitudes towards vaccination. The nine studies cover
the elderly, adults, and the parents of young children; vaccination against hepatitis B, diphtheria-
tetanus-pertussis, mumps-measles-rubella, polio, influenza, pneumococcus, rotavirus, and hu-
man papillomavirus (HPV); and use different instruments to assess participants’ HL. Six of the nine
studies assessed HL using measures of functional HL (self-reported or performance-based: HL
scale of the NAAL, NVS, Chew’s SBSQ, S-TOFHLA). Three of the studies created and used an ad-
hoc instrument for vaccine-related HL but did not report on the psychometric properties of the
measure (Aharon et al. 2017; Johri et al. 2015; Lee et al. 2015). One study included an instrument
for functional Vaccination HL (Johri et al. 2015).
The nine studies reported partly contradictory results. However, the results indicated a positive
correlation between HL or Vaccination HL and vaccination attitudes and behavior. Higher HL was
associated with a more positive attitude towards vaccination and a higher uptake of vaccinations.
Additional studies underline this result (Albright/Allen 2018; Castro-Sánchez et al. 2018; Griebler
et al. 2021; Kitur et al. 2021; Montagni et al. 2021; Wisk et al. 2014; Zhang et al. 2020).
HL and childhood vaccinations
In an Israeli study, a negative association appeared between parents’ HL and their willingness to
vaccinate their children (Aharon et al. 2017), which means that parents with higher functional,
communicative, and critical HL have a higher risk of refusing to vaccinate their children. This at-
titude was confirmed by a Spanish study (not included in Lorini et al. 2018), showing that women
with high HL were more likely to decline immunization against influenza for their children (Castro-
Sánchez et al. 2018).
An experimental study in the Netherlands on parents’ preferences for vaccinating their children
against rotavirus (Veldwijk et al. 2015) reported that parents with high HL are more concerned
about the effectiveness of the vaccine and the incidence of possible severe side effects, while less
proficient parents are more often interested in the duration of vaccine protection. Most parents
were willing to vaccinate their children against rotavirus if the vaccination is included in the na-
tional vaccination program. However, less health literate parents may choose to vaccinate their
children even if the vaccination is only available on the open market.
A positive association for mothers’ HL and their children’s full diphtheria-tetanus-pertussis im-
munization was observed in an Indian study (Johri et al. 2015), while an American cohort study
found no association between mothers’ HL and their children’s general vaccination status (Pati et
al. 2011).

HL and vaccination among young people
Many young people are unvaccinated, even though they are particularly at risk of getting infectious
diseases. In a study from the USA (not included in Lorini et al. 2018), greater knowledge of HPV
was associated with HL while there was no association between HL and vaccination status in rela-
tion to HPV among college students (Albright/Allen 2018). The HPV and its ease of sexual trans-
mission is the reason why it is highly prevalent in the young, sexually active population, for in-
stance in a large proportion of female adolescents/young people. However, another study from
the USA on disparities in HPV vaccine awareness among parents of preadolescents and adolescents
(also not included in Lorini et al. 2018) revealed that only parents of adolescents who are charac-
terized with factors associated with higher HL are more likely to be aware of HPV vaccines (Wisk
et al. 2014).
Studying college-aged women, Lee et al. (2015) observed a positive association between HL and
their willingness to have an HPV vaccination. A similar conclusion was reached by another Amer-
ican study (not included in Lorini et al. 2018) showing a positive association between HL and HPV
vaccine acceptance in university students (Kitur et al. 2021).
In a review study from Canada about HPV vaccine uptake among Canadian youth, lack of
knowledge about vaccines as well as attitudes that are affected by disinformation or not supported
by relevant scientific evidence were found to be among the barriers to uptake of the HPV vaccina-
tion. (Scott/Batty 2016).
HL and vaccination among adults and the elderly
Three American studies looked at the relationship between HL and vaccination in the elderly. White
et al. (2008) observed that the correlation between HL and influenza vaccination uptake changed
from negative to positive as age increased and found no association between HL and pneumonia
vaccination uptake in adults 65 and older. Studying elderly people, Bennett et al. (2009) confirmed
a positive association between HL and the willingness to have an influenza vaccination. According
to an Austrian study (not included in Lorini et al. 2018), higher vaccination-related HL was asso-
ciated with a greater willingness to be vaccinated against Covid-19 (Griebler et al. 2021). This was
confirmed by a French study (also not included in Lorini et al. 2018) showing that acceptance of a
Covid-19 vaccination was associated with a better ability to detect fake news and higher HL
(Montagni et al. 2021). A population-based HL survey from Norway suggested that people asso-
ciated with low economic status, low oral and written language proficiency, and an immigrant
background are less likely to access vaccine information or understand what vaccines they need
and why (Le et al. 2021).
HL and vaccination-related HL
A study conducted in Hong Kong among people aged 65 or over (not included in Lorini et al. 2018)
observed a significant correlation between HL and their definition of vaccination-related HL (Zhang
et al. 2020). HL and Vaccination HL were assessed with the Chinese version of the HLS-EU-Q47
(HLS-Asia-Q). A negative association for vaccination-related HL and age was indicated (the older
the person, the lower their HL) and so was the association for vaccination-related HL and the
number of chronic health complaints and diseases (the more health complaints/diseases, the
lower the HL).
Conclusion
Overall, it seems that “relevant” vaccinations are positively correlated with HL (influenza/old peo-
ple, HPV/young females). Perhaps people with high HL identify vaccinations that are relevant to
them and “skip” others.
Overall objective and research questions
The overall objective was to provide a jointly developed and validated, internationally applicable
measure of and data on Vaccination HL in different countries, contributing to an understanding of
the role of HL in the debate on vaccination hesitancy and supporting relevant policy action.
The following research questions were formulated:

1. To what extent do the four items on Vaccination HL empirically constitute a unidimensional
scale with sufficient internal consistency that measures a different latent trait to the HLS19-
Q12 for General HL?
2. How are the scores of the individual Vaccination HL items and the composite Vaccination HL
score distributed in general adult populations?
3. How is the Vaccination HL score distributed in different vulnerable or disadvantaged subpop-
ulations?
4. What are the most influential determinants of Vaccination HL based on the socio-demographic
and socio-economic variables collected in the HLS19?
5. How are the scores on Vaccination HL related to other relevant specific determinants of vac-
cination behavior (vaccination confidence, knowledge about the risks of vaccination, and per-
ceived risks of developing a disease if not vaccinated)?
6. How is Vaccination HL related to vaccination behavior and how does Vaccination HL influence
vaccination behavior, controlling for General HL and other determinants of reported vaccina-
tion behavior (vaccination confidence, knowledge about the risks of vaccination, and perceived
risks of developing a disease if not vaccinated)?
Measuring vaccination-related HL and vaccination-

specific background variables in the HLS19
With no specific definition or conceptual model of Vaccination HL, there are currently very few
survey instruments available to assess context-independent, overall Vaccination HL, i.e., not fo-
cused on a particular disease. Only the Vaccine Health Literacy Scale (Aharon et al. 2017), which

captures Nutbeam (2000) definition of HL, assesses context-independent Vaccination HL, and the
scale has been field tested in Italy (Biasio et al. 2020).
In another study, Zhang et al. (2020) selected the four vaccination-related items (items 19, 22,
26, and 29) from the 15 items in the disease prevention item bundle of the HLS-EU-Q47 (Sørensen
et al. 2013), and assessed Vaccination HL in line with the current understanding of HL by the HLS-
EU Consortium (Sørensen et al. 2012). However, one of these four items is context dependent
(item 29 refers to vaccination against influenza), and one item refers to both vaccination and
health screenings (item 19). Therefore, the HLS19 study and a related Austrian study on SARS-
CoV-2- and Covid-19-related HL (Griebler et al. 2021) used a revised version of the latter item
by discarding the screening facet. The four Vaccination HL items are aligned with the definition of
Vaccination HL by measuring the process dimensions of finding, understanding, judging, and ap-
plying vaccination information to make better immunization decisions and were already fully
aligned with the HLS19 methodology by asking “how easy or difficult it is” to perform tasks related
to vaccination information. Data were collected using the HLS19 4-point rating scale “very easy” –
“easy” – “difficult” - very difficult”. To validate the measure and to analyze the potential conse-
quences of Vaccination HL, additional specific correlates were assessed.
In addition to the four Vaccination HL items, the HLS19 optional package on Vaccination HL in-
cluded
» one item on personal vaccination behavior during the previous five-year period,
» four items referring to personal confidence in vaccinations (Confidence),
» three items on myths about the possible risks of getting vaccinated (Calculation/Conspiracy),
» one item on the risk of getting a disease for which a vaccine exists (Complacency).
While the latter three aspects represent other relevant specific vaccination determinants, vaccina-
tion behavior is the main expected study outcome. A more detailed description of these measure-
ment instruments can be found in Annex 13 (Tables A 13.1 to A 13.14).
In line with previous EU surveys (de Figueiredo et al. 2020; Larson et al. 2018), the “Confidence in
vaccination” aspect was assessed by using the shortened 4-item version of the Vaccine Confidence
Index (Larson et al. 2016; Larson et al. 2015), which measures confidence in vaccinations in terms
of their importance, safety, and effectiveness. The items ask respondents to rate how strongly
they agree (“strongly agree” – “agree” – “disagree” – “strongly disagree”) that vaccinations are im-
portant to protect themselves and their children, that vaccinations are safe, that vaccinations are
effective, and that vaccinations are compatible with their religious beliefs. In the HLS19, this item
block was supplemented by an additional item on the importance of vaccinations (to prevent the
spread of diseases). In the HLS19 the item asking about the compatibility of vaccinations with re-
ligious convictions was discarded from the “confidence” item set, since it does not elicit confidence
in vaccinations but reflects religious attitudes. This has also been confirmed by factor analyses.
The four items used were combined into an index reflecting the respondents’ confidence in vac-
cinations.
The one item on vaccination behavior (“Have you, your children, or has someone in your family
had any vaccinations in the last five years?”) and the three items on perceived vaccination risks (“Is
it true or false that vaccines overload and weaken the immune system / can cause the diseases
against which they protect / often produce serious side effects) were taken from the Eurobarom-
eter survey “Europeans’ attitudes towards vaccination” (EU 2019). The response options for the
“vaccination behavior” item were adapted slightly, changing the response categories from “yes,
yourself, yes, your children, yes, someone else, and no” to “yes” and “no”, and the items on vac-
cination risks were combined into an index that reflects the respondents’ knowledge about those
risks.
The item on disease risk was taken from a Swiss survey on vaccination (Schulz et al. 2019). In the
HLS19, the risk of disease was not asked in connection with specific diseases but only overall.
The HLS19 Working Group and Optional Package on Vaccination HL was initiated and developed by
three HLS19 countries (AT, BE, and IE). The results on the distribution of the individual items can
be found in Annex 13 (Tables A 13.1 to A 13.14).
Countries implementing the optional package on vac-

cination-related HL in their HLS19 national assessment
Seven countries implemented the complete optional package on Vaccination HL, and four addi-
tional countries collected data on at least the four Vaccination HL items and general background
variables. Hence, 11 of the 17 countries participating in the HLS19 collected data on the four Vac-
cination HL items. In total, data on Vaccination HL are available for just over 25,000 respondents
in Europe.
Table 13.1 below reports the data collection method(s) or “mode(s)” used by each country, such
as computer-assisted telephone interviewing (CATI), computer-assisted web interviewing (CAWI),
and computer/paper-assisted personal interviewing (PAPI/CAPI), the period of the survey, and the
size of the samples concerned.

Table 13.1:
Countries applying the optional package on Vaccination HL or only the measure on Vaccination
HL
Country HL-VAC items Type of data collec- Period of data collection Number of re-
only/full option spondents
tional package
HL-VAC
AT OP HL-VAC CATI 16.03.2020-26.05.2020 2,967
BE OP HL-VAC CAWI 30.01.2020-28.02.2020 and 1,000

01.10.2020-26.10.2020
BG HL-VAC items CAPI, CAWI 10.-24.11.2020 865

only
CZ OP HL-VAC CATI, CAWI 10.-24.11.2020 1,599
DE HL-VAC items PAPI 13.12.2019-27.01.2020 2,143

only
HU OP HL-VAC CATI 02.12.2020-20.12.2020 1,195
IE OP HL-VAC CATI 24.07.2020-07.12.2020 4,487
IT HL-VAC items CATI, CAWI 8.04.2021-08.05.2021 3,500

only
NO HL-VAC items CATI 04.04.2020-13.05.2020 2,855

only
PT OP HL-VAC CATI 10.12.2020–13.01.2021 1,247
SI OP HL-VAC CAPI, CAWI 09.03.2020-15.03.2020 3,360

and 09.06.2020-10.08.2020
13.2 Methods and results
Rasch modeling, confirmatory factor analysis, and reliability indices
The latent trait Vaccination HL was measured using the 4-item Vaccination HL scale with a 4-point
Likert scale. As the scale collects polytomous categorical data at the ordinal measurement level,
the data were tested against the partial credit parameterization of the unidimensional Rasch model
(Masters 1982). Overall data-model fit, single item data-model fit, ordering of the response cat-
egories, differential item functioning (DIF), response dependency, and dimensionality were evalu-
ated country by country. Analyses of dimensionality were also used to determine whether the
Vaccination HL scale collected additional information “over and above” the HLS19-Q12 scale (see
Section 5.5).
Applying confirmatory factor analysis (CFA) within the structural equation modeling (SEM) frame-
work, dimensionality was assessed by estimating a single-factor (Vaccination HL items and HLS19-
Q12 items loading onto one factor) and a two-factor (Vaccination HL items and HLS19-Q12 items
loading onto separate factors) measurement model by using a diagonally weighted least squares
(DWLS) estimator (Beaujean 2014; Kline 2015; Roussel 2012). To assess how well the model fits
to the data, the following goodness-of-fit (GOF) indices were selected and used (with the target
values in parentheses):
» Standardized Root Mean Square Residual (SRMR ≤ 0.08), which is the averaged sum of the
squared elements of the residual correlation matrix. The SRMR value typically increases with
the number of indicators i.e., by the model degrees of freedom (df),
» Root Mean Square Error of Approximation (RMSEA ≤ 0.06), which attempts to correct the
tendency of the sample-size dependent chi-square statistics to reject models when the
sample size is large; it includes a “parsimony correction” by taking into account the model
df,
» Goodness of Fit Index (GFI ≥ 0.95), which estimates the proportion of covariances in the
sample data matrix explained by the model,
» Adjusted Goodness of Fit Index (AGFI ≥ 0.90), which is the adjusted GFI taking into account
the degrees of freedom of a model relative to the number of variables,
» Comparative Fit Index (CFI ≥ 0.95), which measures the relative improvement in fit going
from the baseline model, which assumes zero covariance among the indicators, to the cor-
relation structure identified by the postulated model,
» Tucker-Lewis Index (TLI ≥ 0.95), which is a CFI corrected for parsimony taking into account
the degrees of freedom of the postulated model and the baseline model.
Internal consistency or “test reliability” – the measurement scale’s ability to distinguish between
respondents with different Vaccination HL scores, or degrees of Vaccination Literacy, was esti-
mated by the Cronbach alpha coefficient and equivalent coefficients, such as the Person Separation
Index” (PSI).
The HLS19 procedures for reporting results
For analyzing and reporting Vaccination HL, the difficulty of each Vaccination HL item (i.e., the
percentage of the responses “very difficult” or “difficult” combined), the Average Percentage Re-
sponse Patterns (APRP) and the Vaccination HL mean score were estimated. For each item, the
proportion of respondents ticking each of the four response categories is available in Annex 13
(Tables A 13.11 to A 13.14).
To calculate the APRP (as described in Section 4.4), the average number of times respondents used
each response category while responding to the four Vaccination HL items was estimated. The
APRP displays the percentages in which the four response categories were used, on average, by
respondents across all Vaccination HL items.

To create a Vaccination HL score, following the HLS19 procedure (described in Section 4.2), the
four Vaccination HL items were dichotomized by merging the two response categories “very diffi-
cult” (1) and “difficult” (2) (values 1 and 2 were recoded to “zero”) and the two response categories
“easy” (3) and “very easy” (4) (values 3 and 4 were recoded to 1). For each respondent, the recoded
values were added to a sum score, and the sum score was standardized to 0–100 according to the
HLS19 procedure. Thereby, the score indicates the percentage of items the respondents ticked as
“easy” or “very easy”. Vaccination HL scores were computed only for respondents with a complete
data set for the four Vaccination HL items. A higher score means higher Vaccination HL.
Responses to individual items, the Average Percentage Response Patterns (APRP), and the Vac-
cination HL score were explored for each of the 11 countries.
Vulnerable/disadvantaged subpopulations were identified by comparing the mean score of pop-

ulation groups using t-tests to compare pairs of mean scores.
Regression models
The effects or “regression parameters” of determinants of Vaccination HL, such as gender, age,
education, self-perceived level in society, financial deprivation, and training in a health profession,
were estimated (as described in Section 4.6) using a multivariable linear regression model for each
country and for all countries (equally weighted).
The effect of Vaccination HL on “vaccination behavior” was estimated using a single-level logistic
regression model for each country. The variables General HL, socio-demographic and socio-eco-
nomic background, training in a health profession (for the wording and distribution of these var-
iables, see Chapter 6), and relevant vaccination variables were used as control variables.
Responses at item level by country
Figure 13.1 and Table A 13.15 (in Annex 13.1) show the combined proportion of responses in the
“difficult” or “very difficult” response categories for the four Vaccination HL items.
Item 26 “to judge which vaccination you or your family may need” has the highest difficulty in five
out of the 11 countries (BE, BG, CZ, DE, and PT) and is rated among the most difficult items in four
other countries (AT, HU, IE, and IT). The difficulty of this item varies from around 21% (AT and HU)
to 54.0% (BG).
The item 22 “to understanding why you and your family may need vaccination” has the lowest
difficulty in eight countries (AT, CZ, DE, HU, IE, NO, PT, and SI) and is rated among the least difficult
items in two countries (BE and BG). The difficulty of this item varies from 5.2% (PT) to 36.8% (BG).
Finding vaccination information (item 19) is the most difficult Vaccination HL item in only two
countries (IT and NO) and among the most difficult Vaccination HL items in another two countries
(BE and BG). The percentage of difficulties finding vaccination information varies by country be-
tween 15.7% to 49.4%, with four countries (BE, BG, DE, and IT) exceeding the 30 percent level.
Applying vaccination information (item 29) is the most difficult Vaccination HL item in SI and in
AT. The percentage of difficulties in applying vaccination information varies between countries
from 8.8% (PT) to 36.0% (BG), ranging from about 20% to 30% in most countries.
Figure 13.1:
Percentages of respondents who responded with “very difficult” or “difficult” to the HLS19-VAC
The item IDs are based on the HLS19-Q47, which this item set was derived from.
Across countries, 4.4% of respondents rated all four Vaccination HL items as “very difficult” and a
quarter of respondents ticked either “very difficult” or “difficult” for all four items (cf. Figure 13.2).
The percentage of “very difficult” or “difficult” responses varied between countries from 17.6% (HU)
to 44.5% (BG), apart from PT (12.3%; see Figure 13.2).

Figure 13.2:
Average Percentage Response Patterns (APRP) across all Vaccination HL items, for each country
and the mean for all countries (equally weighted)
Psychometric properties of the Vaccination HL scale
Rasch modeling
Applying a 4-point rating scale, three “thresholds'' were estimated for each of the four Vaccination
HL items. Using an “amend sample size” corresponding to 30 respondents for each of the 12
thresholds (n=360), the overall data-model fit to the Rasch PCM was sufficient for Vaccination HL
scale data collected in CZ, DE, HU, IE, and IT. The overall data-model fit was acceptable for data
collected in AT, BE, NO, and SI. The data collected in BG and PT displayed rather poor overall data-
model fit.
The response categories of the four Vaccination HL items were ordered, and no response depend-
ency between items was observed. As the Vaccination HL items displayed DIF for country/lan-
guage, i.e. the item location or “difficulty order” varied somewhat between countries, the Vaccina-
tion HL scale did not measure invariantly across countries (cf. Figure 13.1). Items also displayed
DIF for different person factors, such as age and gender, but there was no consistent overall pat-
tern across countries.
Item 29, which explicitly refers to vaccination against influenza and is therefore “context depend-
ent”, taps into an additional influenza-related latent trait and therefore tends to discriminate
somewhat poorly in several countries.
As a joint analysis of the vaccination item set and the HLS19-Q12 item set did not lead to a unidi-
mensional structure when equating these two item subsets (by using dependent t-tests), the vac-
cination scale and the HLS19-Q12 scale seem to empirically measure different latent traits.
Confirmatory factor analysis
Using a two-factor confirmatory factor model, the latent correlation between the Vaccination HL
factor and the HLS19-Q12 factor was sufficiently low in 10 out of the 11 countries and suggests
that discriminant validity is likely to exist between the two scales. Except for IT (0.92), all latent
correlations were 0.85 or less, with the lowest observed value (0.57) in the BE data. A sufficiently
low latent correlation or “strong discriminant validity” means that the two scales Vaccination HL
and the HLS19-Q12 seem to measure somewhat different but related traits or competencies.
Table A 13.29 and Table A 13.30 in Annex 13 report GOF indices for the two-factor model and
the corresponding single-factor measurement model respectively. In the single-factor model, the
Vaccination HL items and the HLS19-Q12 items load on one common latent factor. Ignoring the
poor GOF indices for the single-factor model (Table A 13.30) for the Belgian (BE) and Czech (CZ)
data, the overall trend is that the two-factor model (Table A 13.29) and the single-factor model
(Table A 13.30) display good fit. One exception is the high SRMR values > .08 for some countries
when the single-factor model is applied (Table A 13.30). Due to modeling a large number of var-
iables (4+12 items) the model df is relatively high, and a large SRMR (not corrected for parsimony)
and a small RMSEA (which compensates for the effect of model complexity) is expected, as noted
by Savalei (2012). Given the size of the SRMR relative to the RMSEA, the single-factor model seems
to be “too parsimonious” compared to the two-factor model. A country-wise comparison of the
two-factor model to the single-factor model using the Scaled Chi-Squared Difference Test
(Satorra/Bentler 2001) strengthened the hypothesis that the two-factor model did significantly
better in recreating the observed covariance matrix. Therefore, the Vaccination HL scale seems to
measure a different latent trait than the HLS19-Q12 scale.
Table 13.2 reports the selected GOF indices for the hypothesized model where the four Vaccination
HL items load onto a single Vaccination HL factor. According to the country-wise CFAs, this model
recreated the observed covariance matrix quite well for the data collected in all eleven countries,
with a somewhat large RMSEA for the Bulgarian data (the correlation of the individual items per
country can be seen in the tables in Section 13.2 of the Annex of this Chapter).

Table 13.3 reports the standardized item loading for each of the four Vaccination HL items when
loading onto the single Vaccination HL factor. Except for the Bulgarian and Italian data, item 29
has the lowest standardized loading of the four Vaccination HL items at less than 0.7. As the
squared standardized item loading (0.7x0.7=0.49 = 49%) can be interpreted as “variance ex-
plained”, it follows that the latent factor Vaccination HL explains less than 50% of the variance in
the responses to item 29 in most countries.
Table 13.2:
Fit indices for the one-factor confirmatory factor model with the four Vaccination HL items as
indicators, for each country
AT BE BG CZ DE HU IE IT NO PT SI
SRMR 0.02 0.01 0.05 0.03 0.01 0.01 0.04 0.01 0.02 0.03 0.02
RMSEA (CI 0.01 0.00 0.04 0.00 0.00 0.00 0.04 0.00 0.00 0.00 0.01
lower bound)
RMSEA 0.03 0.00 0.08 0.04 0.00 0.00 0.06 0.00 0.01 0.00 0.03
RMSEA (CI 0.05 0.06 0.12 0.07 0.03 0.04 0.08 0.03 0.04 0.05 0.05
upper bound)
RMSEA (p 0.91 0.88 0.12 0.69 1.00 0.97 0.19 1.00 0.99 0.94 0.94
value)
CFI 1.00 1.00 0.99 1.00 1.00 1.00 0.99 1.00 1.00 1.00 1.00
TLI 1.00 1.00 0.97 1.00 1.00 1.00 0.98 1.00 1.00 1.00 1.00
GFI 1.00 1.00 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
AGFI 1.00 1.00 0.96 0.99 1.00 1.00 0.98 1.00 1.00 1.00 1.00
AGFI=Adjusted Goodness of Fit Index; CFI=Comparative Fit Index; CI=Confidence interval; GFI=Goodness of Fit Index;
RMSEA=Root Mean Square Error of Approximation; SRMR=Standardized Root Mean Square Residual; TLI=Tucker-Lewis
Index
Table 13.3:
Standardized factor loadings when the four dichotomized Vaccination HL items load onto a
single factor, for each country
AT BE BG CZ DE HU IE IT NO PT SI
19 … to find infor-
mation on recom-
mended vaccinations 0.83 0.91 0.62 0.81 0.78 0.74 0.78 0.78 0.75 0.81 0.80
for you or your fam-
ily?
22 … to understand
why you or your fam-
0.95 0.94 0.81 0.87 0.86 0.91 0.81 0.85 0.79 0.82 0.83
ily may need vaccina-
tions?
26 … to judge which
vaccinations you or 0.93 0.95 0.90 0.92 0.91 0.88 0.83 0.83 0.92 0.96 0.93
your family may need?
29 … to decide if you
should have a flu vac- 0.54 0.79 0.66 0.66 0.53 0.65 0.60 0.79 0.56 0.36 0.65
cination?
The item IDs are based on the HLS19-Q47, which this item set was derived from.
In line with the result from Rasch modeling, context-dependent item 29, therefore, probably has
a specific variance component owing to an unmodeled “influenza” factor not included in the sin-
gle-factor CFA for the 4-item Vaccination HL scale. Specific variance is displayed as part of an
indicator’s unique variance: the more unique the variance the less common the variance, and,
accordingly, a lower standardized factor loading.
Reliability
The internal consistency of the Vaccination HL scale was, on average, 0.72, ranging from 0.60 (PT)
and 0.67 (NO) to 0.85 (BE). For most countries the test reliability of the Vaccination HL scale was
acceptable (cf. Table 13.4) and above the minimum value of 0.70 recommended in the literature
(Kline 2015).
Table 13.4:
Cronbach’s alphas for the 4-item Vaccination HL scale, for each country and the mean for all
AT BE BG CZ DE HU IE IT NO PT SI All
Cronbach
0.75 0.85 0.70 0.75 0.71 0.71 0.68 0.75 0.67 0.60 0.73 0.72
’s alphas

Distributions of the Vaccination HL score by country
The distribution of the Vaccination HL standardized score is negatively skewed across all coun-
tries. This ceiling effect makes it evident that the Vaccination HL scale could have been better
targeted to respondents’ Vaccination Literacy, meaning that “more difficult” items should be de-
veloped and added to the scale. More items would also strengthen the test reliability of the scale.
Table 13.5 shows the average Vaccination HL score for each country. The mean score varies be-
tween 57.6 points (BG) and 87.0 points (PT).
Table 13.5:
Distribution of the Vaccination HL score, for each country and the mean for all countries (equally
weighted)
25th percentile Median Mean 75th percentile SD

AT 75 100 81.1 100 29.7
BE 25 100 67.6 100 38.8
BG 25 50 57.6 100 35.6
CZ 50 100 75.4 100 32.2
DE 50 75 67.3 100 34.2
HU 75 100 82.4 100 27.4
IE 50 100 75.5 100 30.5
IT 50 75 71.6 100 34.2
NO 75 100 78.3 100 28.6
PT 75 100 87.0 100 22.5
SI 50 100 76.6 100 30.7
All 50 100 74.5 100 32.6
Vulnerable/disadvantaged subpopulations
Defining three age categories (18–35, 36–65, and 66+) and using t-tests to compare pairs of
mean scores, significantly different mean Vaccination HL standardized scores were observed for
the age groups in eight out of the 11 countries. In four countries (BE, CZ, IT, and NO), 18–35 year
olds and 36–65 year olds had the lowest Vaccination HL, with no significant difference between
these two groups. In IE, 18–35 year olds, in AT, 36–65 year olds, and in PT and SI, 66+ year olds
scored, on average, significantly lowest. In DE, it was both the young and the elderly who had the
lowest Vaccination HL, without differing significantly from each other (cf. Table 13.6). A significant
age gradient in terms of Vaccination HL was only observed in the Irish data.
Differences in mean scores between the three educational categories defined (low, middle, and
high), were observed for six out of the 11 countries. In HU, PT, and SI, the “low” education group
had the lowest Vaccination HL score. In BG, DE, and NO, only the “high” education group differed
significantly from the other two groups (Table 13.6). A significant “educational gradient” with re-
spect to Vaccination HL was observed only for the SI data.
Based on the variable for level in society, using ten response categories 1–10, three groups of
status were defined (low 1–4, medium 5–7, and high 8–10). Significantly different scores were
observed in all countries except AT (Table 13.6), with a significant gradient in Vaccination HL by
level in society for five out of these ten countries (BG, DE, IT, PT, and SI).
A “financial deprivation” score was estimated based on three items aligned with the HLS19 meth-
odology by asking how easy or difficult it is usually to afford medication (C-DET9) or medical
examinations and treatments (C-DET10), and to pay all bills at the end of the month (C-DET11).
These are referred to as the three “areas” of financial deprivation. In all countries the financially
deprived subpopulations with “some to severe” deprivation, had a lower Vaccination HL compared
to the non-deprived subpopulations (Table 13.6). A gradient of Vaccination HL with increasing
financial deprivation is observed for six countries (HU, IE, IT, NO, PT, and SI), but not for five. This
could be because examinations, treatments, and medicines are financed to a greater extent by
health insurance and the national health system in some countries than in others.
For eight countries (AT, BG, HU, IE, IT, NO, PT, and SI) people with “average to very poor” self-
perceived health status reported, on average, lower Vaccination HL than people with “very good”
or “good” health status did (Table 13.6). For eight countries (most of which are consistent with the
results on self-assessed health status), people with health limitations in everyday life had a lower
Vaccination HL than people with no health limitations. Only in three countries were people with
chronic diseases characterized by a lower Vaccination HL than people without chronic diseases,
but the national study results from Austria indicate that maybe it is not chronic diseases per se
that lead to a difference, but the question of whether people can cope with their chronic diseases
or not (Griebler et al. 2021).

Table 13.6:
Mean Vaccination HL standardized scores for potentially vulnerable/disadvantaged subpopulations, for each country and all country (equally weighted)
Gender: male 81.3 66.4 59.9 75.4 66.4 83.1 73.4 71.8 76.9 87.9 77.0 74.3
Gender: female 81.0 68.8 55.5 75.3 68.1 81.8 77.4 71.5 79.7 86.3 76.3 74.6
Age: 18–35 85.4 66.0 62.2 75.0 62.8 82.8 68.9 72.0 76.8 88.8 75.7 74.4
Age: 36–65 77.3 64.4 59.6 74.0 70.7 82.4 75.8 69.8 77.9 87.1 78.2 74.1
Age: 66+ 84.9 80.8 46.4 79.3 64.4 81.7 84.9 74.9 82.2 81.7 74.0 75.3
Education: ISCED 0–2 (low) 74.5 87.5 41.2 77.7 58.8 74.3 75.2 70.4 70.9 82.0 67.3 73.1
Education: ISCED 3–4 (middle) 81.8 69.8 46.0 76.3 63.2 83.8 74.2 71.0 74.1 89.1 76.9 74.9
Education: ISCED 5–8 (high) 79.9 66.9 63.1 72.0 73.0 84.9 76.5 74.0 81.0 87.2 80.8 74.3
Level in society: 0–4 (low) 78.4 61.1 47.0 72.3 58.7 76.8 69.2 62.2 69.8 81.4 69.3 68.3
Level in society: 5–7 (middle) 81.1 65.1 60.1 75.0 67.7 84.6 75.5 73.5 78.3 88.0 78.1 75.4
Level in society: 8–10 (high) 81.8 76.8 67.3 81.6 75.9 84.1 79.6 74.8 81.8 96.8 86.2 79.0
Financial deprivation: none 83.4 71.8 64.5 80.9 71.0 90.0 80.8 79.7 79.7 90.7 84.2 79.3
Financial deprivation: some to severe 73.4 63.2 49.9 68.9 60.7 77.3 67.5 65.1 67.8 82.1 70.9 67.9
Self-perceived health: very good or good 82.2 69.2 64.6 76.7 68.2 85.1 75.7 77.0 79.1 89.0 79.6 77.0
Self-perceived health: average to very poor 76.0 65.3 46.1 73.5 65.7 79.0 74.8 67.1 75.0 83.5 70.4 69.7
Chronic disease or health problems: one or more 79.8 70.0 53.7 75.7 68.2 80.6 78.0 70.3 77.9 84.0 73.5 73.0
Chronic disease or health problems: none 81.9 65.4 61.8 74.9 66.7 83.6 74.1 73.3 78.4 88.6 78.6 75.8
Limited by health problems: some to severe 78.0 68.8 46.7 74.2 65.9 77.2 77.3 65.9 76.1 81.5 71.0 69.9
Limited by health problems: not 82.3 73.8 64.7 76.7 69.1 83.9 79.1 75.6 79.4 88.5 79.8 78.6
Values in bold highlight vulnerable/disadvantaged subpopulations.
Determinants of Vaccination HL
For most HL measures, social gradients have been demonstrated and the relative strength of the
effects of different social determinants have been described (cf. Chapter 6). Therefore, this is
tested here for Vaccination HL as well using the same potential determinants as for General HL in
Chapter 6.
Model 1 (Table 13.7) presents standardized ß coefficients, p-values, and R2 for linear regression
models on Vaccination HL with gender, age, education, self-perceived level in society, and finan-
cial deprivation as explanatory variables, for each of the 11 countries. It explains on average
only 4% (varying from 3% (AT, BE) to 9% (BG)) of the variance of Vaccination HL; thus, there is
only a weak social gradient, which differs considerably between countries. For all countries ex-
cept BG, financial deprivation is the strongest predictor of Vaccination HL (inversely associated)
with ß=-0.17 on average, varying by country from -0.08 (BE) to -0.27 (HU). Self-perceived level
in society (positively associated) appears to be another important predictor of Vaccination HL
with a standardized ß coefficient of ß=0.05 on average being significantly different from zero in
seven out of the 11 countries (BE, BG, CZ, DE, IE, NO, and PT), ranging from 0.03 (IE) to 0.13
(PT).The standardized ß coefficient of education level is positive and significantly different from
zero in five countries (BG, DE, IE, NO, and SI), ranging from 0.03 (IE) to 0.18 (BG), and negative in
the model for CZ (-0.10).The effect of being “female” is significantly different from zero in only
three out of the11 countries (DE, IE, and NO), where females have a higher predicted Vaccination
HL score than males. In IE, IT, and NO, the predicted Vaccination HL score increases with age,
while the score decreases with age in PT.
Model 2 (cf. Table A 13.32 in Annex 13.6) shows a regression model where the variable “trained
in a health profession” has been added. This model does not explain more variance for Vaccination
HL than Model 1, but there is an “effect” of being “trained in a health profession” which is signifi-
cantly different from zero (with p ≤ 0.01) in four out of the eleven countries (BG, IE, NO, and SI).
This means that in these countries, the predicted Vaccination HL score is higher for those who are
trained in a health profession compared to those who are not when all other explanatory variables
are held constant.
In Model 3 (cf. Table A 13.33 in Annex 13.6)) where General HL is added, which is a latent variable
measured with standard error. General HL turns out to be the strongest predictor of Vaccination
HL for all countries, with a standardized ß of 0.51 for all countries (weighted equally), varying from
0.39 for HU to 0.70 for IT. This result illustrates that General HL and Vaccination HL measure
different aspects of HL (as has already been shown with the Rasch analysis and CFA) and that there
seems to be a basic skill for dealing with health information that is not topic specific. The Spear-
man correlation between Vaccination HL and selected determinants is shown in Annex 13.5.

Table 13.7:
Model 1: Multivariable linear regression models of Vaccination HL by five core social determinants (standardized coefficients (β) and R2), for each country and
for all countries (equally weighted)
Gender female 0.01 0.02 -0.05 0.03 0.05 -0.01 0.06 0.02 0.04 0.00 0.00 0.02
Age in years -0.02 0.06 -0.05 0.04 0.00 0.03 0.15 0.05 0.04 -0.08 0.03 0.01
Education -0.03 -0.03 0.18 -0.10 0.09 0.01 0.03 0.03 0.12 -0.07 0.06 -0.04
Level in society 0.01 0.12 0.12 0.06 0.07 0.02 0.03 0.02 0.08 0.13 0.03 0.05
Financial deprivation -0.16 -0.08 -0.04 -0.21 -0.11 -0.27 -0.20 -0.24 -0.09 -0.13 -0.21 -0.17
R2 0.03 0.03 0.09 0.06 0.04 0.08 0.08 0.06 0.04 0.06 0.06 0.04
Valid count 2598 988 642 1529 1822 1043 4144 3110 2506 933 3107
Total count 2967 1000 865 1599 2143 1195 4487 3500 2855 1247 3360
Coefficients with p-values lower than 0.01 in bold and coefficients with p-values lower than 0.05 in italics.
Consequences of Vaccination HL
Analyses of the consequences of low Vaccination HL, here expressed in terms of vaccination be-
havior, can be carried out for seven out of the 11 countries. In addition to the determinants of HL
considered in Chapter 6, selected specific correlates of vaccinations are considered (confidence in
vaccinations, knowledge about risks, risk perception of getting a disease if not vaccinated).
Table 13.8 displays the correlations between Vaccination HL and selected vaccination correlates.
In most countries Vaccination HL is significantly positively correlated with the selected variables,
except for PT. The higher the Vaccination HL, the higher the confidence in vaccinations, the more
realistic the knowledge about the risks of vaccines, and the better the risk assessment of devel-
oping a specific disease if not vaccinated.
The strongest correlations are found for confidence in vaccinations, followed by a realistic assess-
ment of vaccination risks, aspects that are influenced by vaccination knowledge and Vaccination
HL. The weakest correlation is observed between Vaccination HL and the perception of risks of
developing a disease if not vaccinated. Apart from PT, this pattern applies to all countries (cf.
Table 13.8).
Table 13.8:
Correlations between the Vaccination HL score and selected vaccination correlates considered in
the HLS19 optional package on vaccination, for each country and for all countries (equally
weighted)
Spearman correlation
between HL-VAC score AT BE CZ HU IE PT SI All
and …
VAC confidence (score,
0.24 0.30 0.26 0.25 0.32 0.04 0.25 0.23
low to high)
VAC knowledge risks
0.22 0.22 0.16 0.22 0.22 0.15 0.18 0.21
(score, low to high)
VAC risk perception of
getting a disease if not
0.16 0.08 0.10 0.07 0.11 0.04 0.14 0.12
vaccinated (single item,
low to high)
Using a multivariable logistic regression model (not shown) with the dichotomous variable “vac-
cination behavior” as the dependent variable and the Vaccination HL score as explanatory varia-
bles, the odds of “someone in the family being vaccinated in the last five years” increases for five
out of the seven countries (AT, BE, CZ, IE, and SI) when the predictor Vaccination HL score in-
creases. This pattern does not change when socio-demographic and socio-economic variables as
well as training in a health profession are considered in an extended model (cf. Table 13.9). In
Table A 13.34 a model including General HL is shown.

A model that also includes vaccination-specific variables shows (using the Baron and Kenny ap-
proach (Baron/Kenny 1986)) that the relationship between Vaccination HL and vaccination behav-
ior is mediated by confidence in vaccinations, risk knowledge, and risk perception, entirely in AT
and partially in the other countries (cf. Table 13.10). The model highlights that vaccination infor-
mation management skills are important to build confidence in vaccinations and to gain a realistic
assessment of the risks associated with vaccinations. In addition, a high Vaccination HL helps to
assess the risks of developing a disease if not vaccinated. In Table 13.35 a model including General
HL is shown.
Table 13.9:
Odds ratios of a multivariable logistic regression model on being vaccinated within the last five years (vaccination behavior) with HL-VAC, gender, age,
education, level in society, financial deprivation, and trained in a health profession as determinants, for each country and for all countries (equally weighted)
AT BE CZ HU IE PT SI All
Intercept 2.93 2.83 0.59 0.23 0.74 7.06 1.11 1.22
HL-VAC 1.30 1.19 1.35 1.13 1.37 1.03 1.26 1.26
Gender female 1.39 1.05 1.22 1.03 1.36 1.58 1.16 1.19
Age in years 0.98 0.99 1.00 1.00 1.00 1.02 0.99 1.00
Education 1.14 1.07 1.13 1.15 1.12 1.09 1.14 1.10
Level in society 1.01 1.08 1.04 1.06 0.99 0.9 1.04 1.05
Financial deprivation 0.91 0.93 0.99 1.05 0.99 0.78 0.91 0.92
No training in a health profession 0.63 0.52 0.83 1.14 0.75 0.93 0.73 0.73
Pseudo R² 0.05 0.03 0.04 0.02 0.04 0.03 0.06 0.04

Valid count 2487 988 1522 1026 4105 912 3041
Total count 2967 1000 1599 1195 4487 1247 3360
HL-VAC score: from 0=minimal HL to 100=maximal HL.

Table 13.10:
Odds ratios of a multivariable logistic regression model on being vaccinated within the last five years (vaccination behavior) with Vaccination HL, gender, age,
education, level in society, financial deprivation, trained in a health profession, and specific vaccination correlates as determinants, for each country and for all
AT BE CZ HU IE PT SI All
Intercept 0.08 0.58 0.09 0.01 0.30 0.57 0.14 0.14
HL-VAC 1.07 1.10 1,21 1.00 1.24 0.96 1.10 1.13
Gender female 1.60 1.13 1.17 1.22 1.25 1.43 1.19 1.22
Age in years 0.98 0.99 0.99 1.00 1.00 1.02 0.98 0.99
Education 1.18 1.03 1.10 1.15 1.06 1.07 1.10 1.06
Level in society 0.97 1.07 1.04 1.09 0.99 0.94 1.04 1.04
Financial deprivation 0.99 0.95 1.03 1.23 1.09 0.82 0.91 0.97
No training in a health profession 0.56 0.49 0.63 1.2 0.64 0.78 0.78 0.70
VAC confidence (score low to high) 1.29 1.16 1.22 1.12 1.10 1.19 1.13 1.14
VAC knowledge risks (score low to high) 1.29 1.18 1.18 1.42 1.33 1.28 1.31 1.36
VAC risk perception of getting a disease if
not vaccinated (score low to high) 1.57 1.12 1.33 1.38 1.04 1.18 1.42 1.33
Pseudo R² 0.19 0.06 0.11 0.08 0.07 0.05 0.12 0.10

Valid count 1924 988 1438 751 3746 706 2460
Total count 2967 1000 1599 1195 4487 1247 3360
HL-VAC score: from 0=minimal HL to 100=maximal HL.
The results on Vaccination HL presented in this chapter are based on a 4-item set of indicators
already used in the HLS-EU-Q47, with one item slightly adapted. Thus, the Vaccination HL ques-
tionnaire is based on the HL definition developed by Sørensen et al. (2012) and not on a specific
definition of Vaccination HL and covers an aspect of HL that is not captured in the HLS19-Q12. The
4-item scale showed acceptable psychometric properties with some DIF for individual countries,
which has been considered by reporting the results for each country. There was also some DIF for
person factors but not consistently across countries, which can cause a problem in regressions
and can affect regression parameters; this has been considered by looking at association patterns
across countries and less at the magnitude of regression parameters. Overall, the instrument for
measuring Vaccination HL should thus be extended with a focus on more specific Vaccination HL
tasks, ideally based on a specific Vaccination HL concept and definition. There are also indications
that the different survey methods used (CAPI/PAPI, CATI, CAWI, and combinations) have an influ-
ence on the results. For example, computer-assisted telephone interviews seem to lead to better
results than other survey methods. Therefore, when comparing countries, the survey method must
also be considered. To gain more understanding in this respect, further research is needed.
The main results of our research highlight a considerable issue in dealing with vaccination infor-
mation in all but one country (PT).
General HL was shown to be a strong predictor for Vaccination HL; likewise, Vaccination HL ap-
pears to be a relevant determinant of vaccination behavior in all countries (except for PT) also
considering General HL, albeit partly mediated by other relevant vaccination determinants such as
vaccination confidence, knowledge about the risks of vaccination, and perceived risks of develop-
ing a disease if not vaccinated. These results are consistent with previous studies (Castro-Sánchez
et al. 2018; Kitur et al. 2021; Montagni et al. 2021).
The fact that the most difficult Vaccination HL task appears to be the evaluation of vaccination
information is relevant on several points. The first is the communication issue, which should be
addressed by the media and the scientific community, including the way they interact. The scien-
tific community is responsible for conveying evidence-based information in non-technical lan-
guage that is understandable for the general population. Relevant support for this action could be
provided by journalists and the media in general, which should avoid giving personal interpreta-
tions of scientific results and should create a concrete alliance with the scientific community to
reduce misinformation and hesitancy as much as possible. We are living in an infodemic; thus, the
fight against misinformation and disinformation should be a priority for Europe, as well as for the
rest of the world, in the present but also in the future. This is not just a matter for the scientific
community and scientific bodies but should be the goal of a coordinated approach followed by
governments at various levels (national, regional, and local) in partnership with the scientific com-
munity.
Chapter 13 / Vaccination health literacy 337

A clear demonstration of the dangerous consequences of misinformation is represented by current
“facts” related to the Covid-19 vaccination, which have led to a rise in vaccine hesitancy levels
and, consequently, to lower coverage rates. This situation should be better considered individually
for each country and by comparing countries presenting similarities in vaccination strategies. But
in most countries, the results appeared to be the same as those described above, involving strug-
gles to improve strategies. This is relevant, as each country differs in characteristics related to
routine vaccinations as well, mainly because of the category of healthcare system in place (e.g.,
welfare-based, public insurance-based, etc.).
Another important point to be stressed when comparing countries’ results is the time when the
survey was administered. In most countries it took place during the pandemic but before the
availability of Covid-19 vaccines and all the events related to communication in this specific field.
In other countries data collection was carried out after “V-day” in December 2020. Even if the
questionnaire used did not include Covid-19 vaccination questions, a possible source of bias re-
lated to this situation should be considered.
A further point is that the results show that Vaccination HL is lower for people with lower socio-
economic status, while age and gender are not associated with the level of Vaccination HL in most
of the countries. This seems interesting and is aligned with the highly debated topic of inequalities
related to different socio-economic groups, which is well known and vigorously debated in the
prevention field. This topic should soon be discussed and solved by several stakeholders, not only
by those working in the public health area, but also by institutions and governments, remembering
that reducing inequalities is one of the Sustainable Development Goals to be reached before 2030
as defined in the UN Agenda 2030 (United Nations 2015).
Finally, our results show that being “trained in a health profession” has a negative effect on Vac-
cination HL in six out of the eleven countries considered. This means that the predicted Vaccina-
tion HL score is lower for those who are trained compared to those who are not trained, when
other explanatory variables are set to zero. This result is particularly interesting as it suggests
that in some countries health professionals, despite their training, show lower literacy in vaccina-
tion-related matters, highlighting the need for further training and research on this topic.
In conclusion, Vaccination HL has become a topic of interest more than ever due to the current
pandemic and Covid-19 vaccinations. Problems still exist, underlining the fact that dealing with
vaccination information is not easy at all for populations in all but one country (PT). Evaluating
vaccination information is the most difficult Vaccination HL task in almost all countries and, con-
sidering the spread of misinformation in the current infodemic, it is an important requirement for
dealing with vaccination-related information. Action should be undertaken by the scientific com-
munity and at a policy level to raise Vaccination HL with appropriate and effective interventions at
community level. No one should be left behind in the process, with the aim of reducing, as much
as possible, the inequalities which are associated with lower Vaccination HL, especially concerning
socio-economic status.
13.4 References
Aharon, Anat Amit; Nehama, Haim; Rishpon, Shmuel; Baron-Epel, Orna (2017): Parents with high
levels of communicative and critical health literacy are less likely to vaccinate their children.
In: Patient education and counseling 100/4:768-775
Albright, Amy E; Allen, Rebecca S (2018): HPV misconceptions among college students: The role
of health literacy. In: Journal of community health 43/6:1192-1200
Andre, Francis E; Booy, Robert; Bock, Hans L; Clemens, John; Datta, Sibnarayan K; John, Thekkekara
J; Lee, Bee W; Lolekha, S; Peltola, Heikki; Ruff, TA (2008): Vaccination greatly reduces disease,
disability, death and inequity worldwide. In: Bulletin of the World health organization 86/:140-
146
Beaujean, A Alexander (2014): Latent variable modeling using R: A step-by-step guide. Routledge
Bennett, I. M.; Chen, J.; Soroui, J. S.; White, S. (2009): The contribution of health literacy to dispar-
ities in self-rated health status and preventive health behaviors in older adults. In: Ann Fam
Med 7/3:204-211
Betsch, Cornelia; Schmid, Philipp; Heinemeier, Dorothee; Korn, Lars; Holtmann, Cindy; Böhm, Rob-
ert (2018): Beyond confidence: Development of a measure assessing the 5C psychological
antecedents of vaccination. In: PLoS One 13/12:e0208601
Biasio, LR; Giambi, C; Fadda, G; Lorini, C; Bonaccorsi, G; D’Ancona, F (2020): Validation of an Italian
tool to assess vaccine literacy in adulthood vaccination: a pilot study. In: Ann Ig 32/3:205-
222
Castro-Sánchez, Enrique; Vila-Candel, Rafael; Soriano-Vidal, Francisco J; Navarro-Illana, Esther;

Díez-Domingo, Javier (2018): Influence of health literacy on acceptance of influenza and per-
tussis vaccinations: A cross-sectional study among Spanish pregnant women. In: BMJ Open
8/7:e022132
de Figueiredo, Alexandre; Karafillakis, Emilie; Larson, Heidi (2020a): State of vaccine confidence
in the EU + UK 2020. Luxembourg: Publications Office of the European Union, Luxembourg
de Figueiredo, Alexandre; Simas, Clarissa; Karafillakis, Emilie; Paterson, Pauline; Larson, Heidi J
(2020b): Mapping global trends in vaccine confidence and investigating barriers to vaccine
uptake: a large-scale retrospective temporal modelling study. In: The Lancet 396/10255:898-
908
Dubé, Eve; Laberge, Caroline; Guay, Maryse; Bramadat, Paul; Roy, Réal; Bettinger, Julie A (2013):
Vaccine hesitancy: an overview. In: Human Vaccines & Immunotherapeutics 9/8:1763-1773

EU (2019): Special Eurobarometer 488. Europeans’ attitudes towards vaccination. European Union,
Geneva
Geiger, Mattis; Rees, Franziska; Lilleholt, Lau; P., Santana Ana; Zettler, Ingo; Wilhelm, Oliver;
Betsch, Cornelia; Böhm, Robert (2021): Measuring the 7Cs of Vaccination Readiness. In: Euro-
pean Journal of Psychological Assessment 0/0:1-9, https://doi.org/10.1027/1015-
5759/a000663
Griebler, Robert; Straßmayr, Christa; Dietscher, Christina; Flaschberger, Edith; Nowak, Peter
(2021): Bereitschaft zur Corona-Schutzimpfung und Gesundheitskompetenz. Factsheet. Ge-
sundheit Österreich, Wien
Johri, Mira; Subramanian, SV; Sylvestre, Marie-Pierre; Dudeja, Sakshi; Chandra, Dinesh; Koné,
Georges K; Sharma, Jitendar K; Pahwa, Smriti (2015): Association between maternal health
literacy and child vaccination in India: a cross-sectional study. In: J Epidemiol Community
Health 69/9:849-857
Kitur, Harriet; Horowitz, Alice M; Beck, Kenneth; Wang, Min Qi (2021): HPV Knowledge, Vaccine
Status, and Health Literacy Among University Students. In: Journal of Cancer Education:1-8,
https://doi.org/10.1007/s13187-13021-01997-13181
New York
Larson, Heidi; de Figueiredo, Alexandre; Karafillakis, Emilie; Rawal, Mahesh (2018): State of vaccine
confidence in the EU 2018. Luxembourg, European Union
Larson, Heidi; de Figueiredo, Alexandre; Xiahong, Zhao; Schulz, William S; Verger, Pierre; Johnston,
Iain G; Cook, Alex R; Jones, Nick S (2016): The state of vaccine confidence 2016: global in-
sights through a 67-country survey. In: EBioMedicine 12/:295-301
Larson, Heidi J; Jarrett, Caitlin; Schulz, William S; Chaudhuri, Mohuya; Zhou, Yuqing; Dube, Eve;
Schuster, Melanie; MacDonald, Noni E; Wilson, Rose (2015): Measuring vaccine hesitancy: the
development of a survey tool. In: Vaccine 33/34:4165-4175
Le, Christopher; Finbråten, Hanne Søberg; Pettersen, Kjell Sverre; Joranger, Pål; Guttersrud,
Øystein (2021): Health Literacy in Five Immigrant Populations in Norway: Pakistan, Poland,
Somalia, Turkey, and Vietnam. English summary. In Helsekompetansen i fem innvandrerpop-
ulasjoner i Norge: Pakistan, Polen, Somalia, Tyrkia og Vietnam. Befolkningens helsekompet-
anse, del II. The Norwegian Directorate of Health
Lee, Hee Yun; Kwon, Melissa; Vang, Suzanne; DeWolfe, Jessica; Kim, Nam Keol; Lee, Do Kyung;
Yeung, Miriam (2015): Disparities in human papillomavirus vaccine literacy and vaccine com-
pletion among Asian American Pacific Islander undergraduates: Implications for cancer health
equity. In: Journal of American College Health 63/5:316-323
Levin-Zamir, Diane (2020): Communication, Health Literacy and a Systems Approach for Mitigating
the COVID-19 Pandemic: The Case for Massive Vaccine Roll-out in Israel. In: Journal of health
communication 25/10:816-818
Lorini, Chiara; Santomauro, Francesca; Donzellini, Martina; Capecchi, Leonardo; Bechini, Angela;
Boccalini, Sara; Bonanni, Paolo; Bonaccorsi, Guglielmo (2018): Health literacy and vaccination:
A systematic review. In: Human Vaccines & Immunotherapeutics 14/2:478-488
MacDonald, Noni E (2015): Vaccine hesitancy: Definition, scope and determinants. In: Vaccine
33/34:4161-4164
174
Montagni, Ilaria; Ouazzani-Touhami, Kevin; Mebarki, A; Texier, N; Schück, S; Tzourio, Christophe

(2021): Acceptance of a Covid-19 vaccine is associated with ability to detect fake news and
health literacy. In: Journal of public health (Oxford, England) Published online 2021 Mar 9:doi:
10.1093/pubmed/fdab1028
education and communication strategies into the 21st century. In: Health Promotion Interna-
tional 15/3:259-267
Parker, Ruth (2009): Measures of Health Literacy. Workshop Summary: What? So What? Now What?
The National Academies Press, Washington
Pati, Susmita; Feemster, Kristen A; Mohamad, Zeinab; Fiks, Alex; Grundmeier, Robert; Cnaan, Avi-
tal (2011): Maternal health literacy and late initiation of immunizations among an inner-city
birth cohort. In: Maternal and child health journal 15/3:386-394
Puri, Neha; Coomes, Eric A; Haghbayan, Hourmazd; Gunaratne, Keith (2020): Social media and
vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases.
In: Human Vaccines & Immunotherapeutics 16/11:2586-2593
Ratzan, Scott C (2011): Vaccine literacy: a new shot for advancing health. In: J Health Com-
mun16:229-3
tical Software 48/2:1-36. https://www.jstatsoft.org/v48/i02/
Rowlands, Gillian (2014): Health literacy. In: Human Vaccines & Immunotherapeutics 10/7:2130-
2135
Satorra, A; Bentler, P (2001): Is used as a chi-square test statistic with degrees of freedom equal
to the difference on the number of independent parameters estimated under the models AdO

and Adl. As in the case of the goodness-of-fit test, it is of interest to scale the statistic. In:
Psychometrika 66/:507-514
Savalei, Victoria (2012): The Relationship Between Root Mean Square Error of Approximation and
Model Misspecification in Confirmatory Factor Analysis Models. In: Educational and Psycholo-
gical Measurement 72/6:910-932
Schulz, Peter J.; Hartung, Uwe; Cafaro, Teresa (2019): Vertiefungsstudie zur Gesundheitskompe-
tenz betreffend Impfungen in der Schweizer Bevölkerung. Istitute of Communication and
Health, Lugano
Scott, Katlyn; Batty, Mary Lou (2016): HPV vaccine uptake among Canadian youth and the role of
the nurse practitioner. In: Journal of community health 41/1:197-205
Thomson, Angus; Robinson, Karis; Vallée-Tourangeau, Gaëlle (2016): The 5As: A practical taxon-
omy for the determinants of vaccine uptake. In: Vaccine 34/8:1018-1024
United Nations (2015): Transforming our world: The 2030 agenda for sustainable development.
United Nations
Veldwijk, J.; van der Heide, I.; Rademakers, J.; Schuit, A. J.; de Wit, G. A.; Uiters, E.; Lambooij, M. S.
(2015): Preferences for Vaccination: Does Health Literacy Make a Difference? In: Med Decis
Making 35/8:948-958
Wang, Xiaomin; Zhou, Xudong; Leesa, Lin; Mantwill, Sarah (2018): The effect of vaccine literacy on
parental trust and intention to vaccinate after a major vaccine scandal. In: Journal of health
communication 23/5:413-421
White, Sheida; Chen, Jing; Atchison, Ruth (2008): Relationship of preventive health practices and
health literacy: a national study. In: American journal of health behavior 32/3:227-242
WHO (2017): Report of the SAGE Working Group on Vaccine Hesitancy. World Health Organization,
Geneva
WHO (2018): What is the Evidence on Existing Policies and Linked Activities and Their Effectiveness
for Improving Health Literacy at National Regional and Organizational Levels in the WHO Eu-
ropean Region? World Health Organization
Wisk, Lauren E.; Allchin, Adelyn; Witt, Whitney P. (2014): Disparities in human papillomavirus vac-
cine awareness among U.S. parents of preadolescents and adolescents. In: Sexually transmit-
ted diseases 41/2:117-122
Zhang, F.; Or, P. P.; Chung, J. W. (2020): The effects of health literacy in influenza vaccination
competencies among community-dwelling older adults in Hong Kong. In: BMC Geriatr
2020/20:103

14 Health literacy and health-related quality of
life as a mediator for health costs
Authors:
Gerardine Doyle (IE)

Gill Rowlands (UK)
Pål Joranger (NO)
Christopher Le (NO)
Hanne S. Finbråten (NO)
Maria Lopatina (RU)
Josefin Wångdahl (SE)
Thomas Link (ICC)
14.1 Introduction
It has been shown that limited health literacy (HL) results in poorer health and suboptimal use of
health system resources (Berkman et al. 2011; Howard et al. 2005; Vandenbosch et al. 2016). The
Organization for Economic Co-operation and Development (OECD) has estimated that healthcare
systems could save 3-5% of the annual health budget by improving HL of citizens (Eichler et al.
2009). Other countries have similarly assessed the economic burden on their healthcare systems
due to low levels of HL in the population. For example, Canada estimates a cost of $8 billion per
annum and in the United States (US) an annual cost of $73 billion has been estimated (Kickbusch
et al. 2013). Furthermore, people with limited HL may have more difficulties in living a fulfilling
life (Kickbusch et al. 2013) and the association between HL and health-related quality of life may
therefore be relevant for the costs of health care (Zheng et al. 2018)
The literature is sparse on both the link between limited HL and increased economic costs as the
prior literature focuses on the links between low HL and an increased risk of hospitalisation (Baker
et al. 2002) which has been investigated in Chapter 9 of this report, and on the link between HL
and quality of life (Zheng et al. 2018). Furthermore, most prior research focuses on functional HL,
which can be defined as “the basic skills in reading and writing that are necessary to function
effectively in everyday situations” (Nutbeam 2008) and using functional HL measures. There is a
need for research centred around a comprehensive understanding of General HL, which has been
defined by the HLS-EU consortium as something that “is linked to literacy and entails people's
knowledge, motivation and competences to access, understand, appraise, and apply health infor-
mation in order to make judgments and take decisions in everyday life concerning healthcare,
disease prevention and health promotion to maintain or improve quality of life during the life
course” ((Sørensen et al. 2012).
The purpose of this chapter is twofold. First to explore whether there is a relationship between
General HL (as measured by HLS19-Q12), and health-related quality of life (as measured by EQ-
5D-5L), which has cost and economic implications for health services and for society at large.
Secondly, the chapter is concerned with understanding the relationship, if any, between HL and
work absenteeism.
The structure of this chapter is as follows. The prior research on HL and health care utilization is
explored followed by the literature that examines the association between HL and health-related
quality of life (HRQoL). To the best of the authors’ knowledge no prior studies have examined the
relationship between HL and work absenteeism. The research questions and methods adopted are
then outlined. The results follow and are presented for the relationship between General HL (as
measured by HLS19-Q12), and health-related quality of life (as measured by EQ-5D-5L). Results
of the relationship between HL and work absenteeism are also presented. The findings are then
discussed and provide evidence to inform future policy along with suggestions for further re-
search. These findings result in a call for investment in and design of HL interventions as a disease
prevention strategy at local, national and regional levels which may lead to significant benefits to
Chapter 14 / Health literacy and health-related quality of life as a mediator for health costs 345
citizens for their quality of life whilst simultaneously making more effective use of scarce resources
and expensive health services.
Previous research on HL and health care utilization
Typically studies on health care utilization focussing on functional HL have been conducted in the
US or Asia, and have focussed on a specific population group e. g. people with a particular disease
or in a certain age category rather than the general population (Palumbo 2017; Vandenbosch et
al. 2016). In addition, for most studies centred around the health care costs associated with HL,
the indicators have been measured using self-reporting instruments (Vandenbosch et al. 2016).
Only a few studies on HL and HL-related interventions have considered the cost implications of
limited HL levels (Eichler et al. 2009).
Many studies claim that limited HL contributes to increased health and medical care costs, based
on identified links between limited HL and non-optimal health service utilization alongside nega-
tive health outcomes in terms of unnecessary ill health (Berkman et al. 2011; Howard et al. 2005;
Vandenbosch et al. 2016). This in turn is deemed to contribute to unnecessarily high costs in
health care at both the health systems level and at societal level. However, few studies have ex-
plored the financial costs associated with limited HL at either the population level or at the indi-
vidual level.
Functional HL and health care utilization
Common indicators for estimating the costs of limited functional HL centre on measures of health
care utilization (Palumbo 2017). The most frequently used indicators are number of hospitaliza-
tions, the use of emergency care and the use of general practitioner services (GPs). Associations
between indicators based on the number of hospitalizations and the use of doctor services and
functional HL have been found in most studies and seem to be consistent (Palumbo 2017). Asso-
ciations between functional HL and emergency care are less consistent.
Another group of indicators are individuals' abilities to understand and use health information and
instructions correctly, as well as their ability to communicate about health matters. Associations
have been found between limited functional HL and less understanding of one's medical condition,
impaired ability to interpret labels and health messages, and poorer medication adherence
(Palumbo 2017). Other studies have used health status and health care needs as indicators for
functional HL costs (Palumbo 2017).
Few studies have investigated the financial costs of limited functional HL. Various health care
costs, most often related to health care visits and the use of emergency care, have been calculated
either by modelling costs with assumptions of such care costs in general (Eichler et al. 2009;
Herman/Jackson 2010; Howard et al. 2005), by using direct costs from medical records and ad-
ministrative claims data (Herndon et al. 2011; Vann Jr et al. 2013), or by combining these two
methods (Eichler et al. 2009) Few studies contain both functional HL level and health care costs at
the individual level ). The review by Eichler et al. (2009) found that only six studies met all of the
authors’ inclusion criteria, and none of them explored the cost-effectiveness of interventions in
the health care system or at the patient level. The included studies were all from the US, except
for Spycher (2006), which combines Swiss functional HL prevalence data with US economic data to
produce an estimate for the cost of limited functional HL in Switzerland. However, the US and
European health care systems are very different, making the use of US data and modelling as-
sumptions problematic for a Swiss context (Drummond et al. 2005). A further limitation of these
studies is that they focus solely on costs to the health system ignoring the costs to the patient or
individual.
Limited HL is also likely to incur considerable additional costs to society through work absenteeism
(frequent absence of an employee from work) and presenteeism (showing up for work when one
is ill) (Garrow 2016). HL is likely to be associated with high levels of absenteeism and presenteeism
through the known associations between HL and chronic illness.
General HL and health care utilization
In the past five years, studies have focused on associations between General (i.e. comprehensive)
HL and health care costs in different countries across Europe (Berens et al. 2018; Friis et al. 2020;
Ousseine et al. 2020; Sørensen et al. 2015; Vandenbosch et al. 2016). Some of those were focused
on the general adult population (Berens et al. 2018; Sørensen et al. 2015; Vandenbosch et al.
2016). They have examined costs for health care, using the same or similar indicators as the
studies that focused on costs for functional HL. Associations between General HL and the number
of GP visits (Friis et al. 2020; Ousseine et al. 2020; Sørensen et al. 2015; Vandenbosch et al. 2016),
hospitalizations (Friis et al. 2020; Sørensen et al. 2015) or longer stay in hospital (Vandenbosch
et al. 2016) have been demonstrated. Associations between General HL and the use of various
forms of emergency care are more inconsistent (Berens et al. 2018; Friis et al. 2020; Vandenbosch
et al. 2016). However, the type of doctor's visits, estimation of emergency health care costs and
hospital costs, as well as the type of study population varies between these studies, which may
explain the inconsistent results. No study, to the best of the authors’ knowledge, has explored
the cost implications of limited General HL at the health system and the individual level.
Interventions that improve HL (McCaffery et al. 2016; Morony et al. 2018; Muscat et al. 2016) exist,
however such studies have not examined the effect of investment in General HL around the costs
and benefits of such interventions. Increasingly, governments are undertaking health economic
assessments to aid in decisions about funding health interventions; for example, in the UK the
National Institute for Clinical Excellence (NICE) determines whether interventions provide sufficient
value for money to merit health service funding. This presupposes the calculation of Quality Ad-
justed Life Years (QALY) as a ‘common currency’ enabling the costs and benefits of different health
interventions to be compared (Guide to the methods of technology appraisal, 2013).
14.1.1 HL and health-related quality of life
Health Related Quality of life (HRQoL) denotes the impact of health on a person’s ability to live a
fulfilling life, defined by the World Health Organisation (WHO) as an individual's perception of their
position in life in the context of the culture and value systems in which they live and in relation to
their goals, expectations, standards and concerns
(https://www.who.int/tools/whoqol). HRQoL thus represents a broad concept of physical, psy-
chological and social functioning and well-being including both positive and negative aspects
((Karimi/Brazier 2016)).
HRQoL is measured with a variety of instruments such as the Short Form 36 (SF-36) (Ware et al.,
1993), for results of one item of this in HLS19 see Chapter 8. Special interest has focused on the
EQ-5D-5L, which combines evaluation of the individual subject’s health state based on five as-
pects with a set of provided values or weights for each of the health states, based on the prefer-
ences of the general population in a country or region. In this way, EQ-5D-5L measures the value
that is attached on having specific health conditions as an indication of QALY (EuroQoL).
The association between HL and HRQoL has been examined in a number of studies, recently re-
viewed by Zheng et al (2018) who found 23 studies including 12,303 participants. Measurements
of HRQoL were based on different instruments, most often SF-36 in nine studies and EQ-5D-5L
in six studies. Measurement of HL was based on a number of different instruments, mostly REALM
and TOFHLA both of which measure functional HL. No studies have yet examined the association
between HLS19-Q12 which measures General HL, and EQ-5D-5L.
Nineteen of the 23 studies were combined using meta-analyses, finding a meta-estimate of the
correlation between HL and HRQoL of 0.35 (0.25-0.44). Since this review a number of other studies
have been published, mostly on patient groups, and two studies have measured General HL
(Jovanić et al. 2018; Ozkaraman et al. 2019) using HLS-EU-Q and HRQoL using respectively SF-36
and a cancer-specific instrument for HRQoL measurement.
In summary, more knowledge about the cost implications of General HL for the individual, as well
as at the health system and societal levels, is needed. Such knowledge could lead to a better
understanding amongst policymakers and health care professionals that General HL is important
and something that should be considered in health care delivery and in understanding its impact
on society. Similarly, the possible association between limited HL and low quality of life highlights
the need to focus on improving HL as a disease prevention strategy and a consequent means to
raise the quality of life for the wider population.
HLS19 presented an opportunity to collect high quality cost and economic data as well as examining
the association with QALY. The optional package ‘cost and economics of low HL’ enabled a first
exploration of societal and individual-level costs of low HL in participating WHO European member
states. A further opportunity was the collection of health economic data to enable the utility of
investment in HL be calculated using the EQ-5D-5L measure, which is the European standard
health economic measure (https://euroqol.org).
Of the countries participating in the HL survey, three (DK, IE and NO) collected the health-related
quality of life data. Results from these three countries are presented in this chapter.
14.2 Overall objective and research questions
The overall objective was to explore whether there is a correlation between General HL (as meas-
ured by HLS19-Q12 as an independent variable), and the dependent variables of health-related
quality of life (as measured by EQ-5D-5L) and absenteeism due to health problems.
The following research questions were formulated:

1. To what extent does General HL (HLS19-Q12) have implications for health-related quality
of life as measured by the EQ-5D-5L instrument?
2. To what extent does General HL have implications on absenteeism from work due to
health problems?
14.3 Methods
For the cost and health economic analyses, the main objective was to determine the extent to
which General HL (GEN-HL), as measured by HLS19-Q12 correlates with “health-related quality of
life” as measured by the EQ-5D-5L instrument. In health economic evaluations, a freely available
English syntax file for transforming the responses to the EQ-5D-5L questionnaires into an EQ-
5D-5L estimate for individual respondents was used. EQ-5D has a scale where 0 represents
“death” and 1 represents “full health”.
14.3.1 The statistical analyses
In regression analyses with EQ-5D-5L as the dependent outcome variable, Ordinary Least Squares
(OLS) estimation with “robust” estimates of variance to account for violations of homoscedasticity,
which is recommended by e.g., Pullenayegum et al. (2010), were used. Using Stata 16, respondents
with missing data were “excluded listwise”. For the robust estimation the standard errors the Hu-
ber-White sandwich estimators were used.
The analysis of «number of days absenteeism per year due to health problems» is based on the
two-step model (“Two-part model”) where the first step is based on “probit” and the second step
on General Linear Modelling (GLM) with gamma “distribution family”, log link function and “robust”
estimation of variance. The choice is based on recommendations in Deb et al. (2017). In this anal-
ysis, only respondents who had answered that they were employed were used.
The procedures for all analyses were based in part on the method described in Thoresen et al.
(2012) and assume that all relevant variables are entered simultaneously into the analysis and the
statistical analysis is then performed. The variable with the highest p-value (i.e., the least signif-
icance) is removed and the analysis is run again. This is repeated until only significant variables
remain, except for the measurement scale for General HL, which is retained in the model regard-
less. Next, variables that were removed earlier in the elimination process are entered and retested
to see if they become significant in the new combination of independent variables. Interaction and
polynomial transformations (quadratic elements) were included in the analyses. The variables em-
ployed for this were entered into the model from the start of the elimination process as described
above. These are shown in Table 14.1 below. In instances where the significance levels of the
variables were inconclusive, the explanatory power of the different models was relied upon. The
models remaining after the process described above are presented in Table 14.2. This procedure
is based on Thoresen et al. (2012)
The average of the dependent variables was estimated based on the regression equations. In these
estimations and for the estimations that form the basis for the figures in this chapter, the margins
command in Stata was used. The estimation methods followed Cameron/Trivedi (2010) and Deb
et al. (2017).
14.3.2 The variables included
Table 14.1 provides an overview of the variables included in the analyses of the EQ-5D-5L variable
and days absent from work due to poor health. The original variable for education was an ordinal
variable based on the ISCED system. This variable was converted to number of years of education.
The conversion was done as followed: ISCED level 0, 1, 2, 3, 4, 5, 6, 7, and 8 were converted to 0,
7, 10, 13, 14, 16, 18, and 21 years, respectively. Due to the pandemic some people were “tempo-
rarily unemployed” in some countries. There is a possibility that this has been registered somewhat
differently for the various countries. Therefore, in some countries, this category may be imprecise
due to the pandemic.
Table 14.1:
Independent variables used to explain variance in the dependent outcome variables
Independent variables Variables included Values

EQ-5D-5L score Days of absence
General HL (GEN-HL) X X 0-100
Control variables
Age (years) X X
Male X X 0, 1
Education (years) X X 0-21
Trained in a health care profession X X 0, 1
(THP)
Student X 0, 1
Unemployed X 0, 1
Retired X 0, 1
Interactions
Education x GEN-HL X X
Education x Age X X
THP GEN-HL X X
Gender x THP X X
Gender x Unemployed X
Polynomial transformations
Age squared X X
Education squared X X
GEN-HL squared X X
14.1 Results
14.1.1 HL and Health-related quality of life
In this section the results concerning the association between HL and health-related quality of life
(EQ-5D-5L) in the Irish, Danish, and Norwegian data is presented. Several variables were used as
potential confounders.
A significant association between General HL (GEN-HL) and health-related quality of life (EQ-5D-
5L) can be seen for all three countries (Table 14.2). For all countries, age, gender, education and
being unemployed explain variances in EQ-5D-5L. The explanatory power of the models for the
three countries varies from 6.3% to 7.9%.
Table 14.2:
Multiple linear regression with health-related quality of life (EQ-5D-5L) as the dependent
variable (unstandardized coefficients (b) and R2), for Denmark (DK), Ireland (IE) and Norway (NO).
Variance is estimated by using robust estimation. Cells are empty because the variables were not
part of the regression models.
Variables DK IE NO
General HL (GEN-HL) .0015 *** .0010 *** -.0020 **

Age .0021 * -.0005 *** -.0006 ***
Male .0135 ** .0158 *** .0284 ***
Education .0177 *** .0063 *** .0069 ***

Training in a health care profession (THP) -.0934 **
Student .0266 * .0363 ***
Unemployed -.0607 ** -.0398 *** -.0487 *

Retired -.0134 *
GEN-HL squared .00002 ***
GEN-HL x THP .0009 *

Education x Age -.0002 **
Constant .5274 *** .7555 *** .8495 ***
𝑅𝑅 2 6.7% 7.9% 6.3%

Valid Count 3 566 4 440 2 785
Total Count 3 602 4 487 2 885
*: p<.05, **: p<.01, ***: p<.001
Figure 14.1 below shows a positive relation between GEN-HL and health-related quality of life
(EQ-5D-5L) for Ireland and Denmark. For the Irish data the difference between a Q12 score of 10
(EQ-5D-5L = 0.822) and 100 (EQ-5D-5L = 0.930) is 0.108 EQ-5D-5L score. For Denmark the
same difference is 0.135 EQ-5D-5L score (0.910 – 0.775).
Figure 14.1:
The relationship between GEN-HL on the x-axis, and health-related quality of life (EQ-5D-5L)
on the y-axis for DK, IE, NO. The graphs are predictive margin plots. They are based on the
models presented in Table 14.2, and all the other variables in the models are kept constant on
their average.
For Norway, the same relation is observed when the GEN-HL score is from 50 to 100, 94% of
respondents (2,655 respondents) are within this interval. The difference between a respondent
with GEN-HL score of 50 (EQ-5D-5L = 0.889) and a respondent with 100 (EQ-5D-5L = 0.939) is
an EQ-5D-5L score of 0.05. However, an opposite association was found for Norwegian respond-
ents with GEN-HL scores between 0 and 50. Here, the EQ-5D-5L score decreased as the GEN-HL
score increased. This is unexpected and contradicts an earlier study from Norway (Le et al. 2021b)
where another GEN-HL score was used, and where the results were similar to Ireland and Denmark
above. Of note is that the levels of uncertainty, as indicated by the 95% confidence intervals, are
very high in this part of the graph, reflecting the low numbers of participants (see Figure A 14.1
in the Annex of this chapter), indicating that this finding should be interpreted with caution. The
uncertainty is high, partly because only 6% (179 respondents) of the sample has a GEN-HL score
less than 50.
Figure 14.2:
The relationship between GEN-HL on the x-axis, and health-related quality of life (EQ-5-5LD)
on the y-axis, shown for different levels of education in Denmark. The graphs are predictive
margin plots. They are based on the Danish model presented in Table 14.2. The estimates are
computed keeping all the other variables in the model constant at their average. The vertical
lines represent the 95% confidence intervals for the point estimates
.95
Health-related quality of life (EQ-5D)
.9
.85
.8
.75
.7
.65
0 10 20 30 40 50 60 70 80 90 100
General Health Literacy Score (Q12)
Educ. = 7 year Educ. = 18 year
For all three countries, it was found that a higher educational level is associated with increased
health-related quality of life (Table 14.2). Figure 14.2 shows this for Denmark, and the EQ-5D-5L
score for people with seven years of education is lower than for people with 18 years of education
(master's degree or equivalent). The difference in education between these groups is estimated to
be a difference in the EQ-5D-5L score of 0.081. This shows the isolated effect of education when
all the other variables shown in Table 1 are kept constant. Respondents with both low HL and low
education had particularly low EQ-5D-5L scores. The difference between having a GEN-HL score
equal to 10 and seven years of education versus having a GEN-HL equal to 100 and 18 years of
education, respectively, is a difference of 0.216 in the EQ-5D-5L score.
Figure 14.3:
The relationship between GEN-HL on the x-axis, and health-related quality of life (EQ-5D) on
the y-axis, shown for females and males in Ireland. The graphs are predictive margin plots. They
are based on the Irish model presented in Table 14.2 and all the other variables in the model are
kept constant at their average. The vertical lines represent the 95% confidence intervals for the
point estimates.
Figure 14.3 shows, that at a certain level of HL, females have a lower EQ-5D-5L score than males,
and that the difference is 0.0135 EQ-5D-5L score. For Norway and Ireland, the difference is 0.0284
and 0.0158 EQ-5D-5L score, respectively (see Table 14.2).
Figure 14.4:
The relationship between GEN-HL on the x-axis, and health-related quality of life (EQ-5D-5L)
on the y-axis, shown for unemployed and employed individuals in Ireland. The graphs are
predictive margin plots. They are based on the Irish model presented in Table 14.2, where all the
other variables in the model are kept constant at their average. The vertical lines represent the
95% confidence intervals for the point
Figure 14.4 shows that unemployed persons have a lower EQ-5D-5L score than persons who are
employed (working, retired etc.). The differences are 0.040, 0.06 and 0.049 EQ-5D-5L score re-
spectively for Ireland, Denmark, and Norway. For the Irish respondents, being unemployed and
having limited HL appears to have additive adverse associations with health-related quality of life.
This finding was not seen in the Danish or Norwegian data.
14.1.2 HL and absenteeism
The results from the analyses for Ireland, Denmark, and Norway of the relationship between ab-
sence from work and HL are presented, using several other variables as control variables.
Table 14.3:
Multiple linear regression of absence from work due to health problems as the dependent
variable (unstandardized coefficients (b) and R2), for Denmark (DK), Ireland (IE) and Norway (NO).
Cells are empty because the variables were not part of the regression models.
Variables DK IE NO
Step 1: How do the variables associate with whether the respondents have zero days of absence versus one or more days
of absence (probit model)?
General HL (GEN-HL) -.0016 .0163 .0008
Age .0117 -.0154*** -.0122***
Male -.2242** -.2640*** -.3377***
Education .0035 -.0407
Training in a healthcare profession (THP) .2507** .1951* 1.4401*
GEN-HL squared -.0001*
Age squared -.0003
Education squared .0028
GEN-HL X Education -.00002
GEN-HL X THP -.0023
Education X THP -.0766*
Gender X THP -.0113
Constant .5672 .1863 .9056
Step 2: For those respondents who have one or more days of absence, how do the variables covariate with the number
days of absence (GLM)?
General comprehensive HL (GEN-HL) -.0048 .0645** -.0121**
Age -.0802* .0078 .0132*
Male .0952 -.1520 -.3818**
Education .1109 .1753*
Training in a healthcare profession (THP) .4860** .2079 -2.6203*
GEN-HL squared -.0002
Age squared .0009*
Education squared -.0091**
GEN-HL X Education -.0024*
GEN-HL X THP .0317***
Education X THP -.0027
Gender X THP -.4865*
Constant 4.5109*** -.7902 2.9403***
𝑅𝑅 2 2.46 % 2.76 % 2.84 %
Valid Count 1596 2401 1570
Total Count 1670 2467 1622
*: p<.05, **: p<.01, ***: p<.001
The two-part model provides two kinds of estimates. The first step of the model provides esti-
mates for the variables which are associated with either zero days of absence from work or one or
more days of absence. The second step focuses only on the respondents who had one or more
days of absence from work. For these respondents, the analysis explores which independent var-
iables covariate with the number of days of absence (step 2). Table 14.3 shows that in step 1
General HL is not significantly (p>.05)) associated with absence from work for any of the countries.
In step two General HL is significant for Ireland and Norway. The explanatory power of the models
lies between 2.46% to 2.84 % for the three countries.
Figure 14.5:
The relationship between GEN-HL on the x-axis, and days of absence per year due to poor
health on the y-axis. The graphs are predictive margin plots. They are based on the country
models presented in Table 14.3, and all the other variables in the model are kept constant on
their average
For Norway, the increased General HL is associated with decreasing absence from work due to
health problems. For the Norwegian respondents, it can be seen, that the difference between a
GEN-HL score of 10 (days absent = 19.0) and a score of 100 (days absent = 10.2) is about 8.8
days. For Denmark, the same negative association between HL and absenteeism is found, but
this association is not significant for any of the steps. The p-values are 0.34 and 0.12 in step 1
and 2, respectively.
In Ireland the days of absence from work due to health problems increases when GEN-HL score
increases from 0 to about 70, but absenteeism decreases thereafter. 73.1% (1,797 respondents)
of the respondents have a GEN-HL score between 70 and 100.
From Figures A 14.4 - A 14.6 in the Annex the uncertainty for these three graphs is very high in
some places.
14.2 Discussion and Conclusions
14.2.1 Main Findings
Our findings answer the two research questions posed:

1. To what extent does General HL (HLS19-Q12) have implications for health-related quality
of life as measured by the EQ-5D-5L instrument?
2. To what extent does General HL have implications on absenteeism from work due to
health problems?
Research Question 1
The analysis of Health-Related Quality of Life (HRQoL) as measured by EQ-5D-5L, in the three
countries that collected these data, shows a significant association between General HL and
health-related quality of life. As HL increases so does HRQoL. The reverse direction of association
for Norwegian respondents with HL scores below 50 is unexpected; the numbers of respondents
in this category is low, as reflected by the large 95% confidence intervals (Figure A 14.1 - Annex),
meaning that this finding should be interpreted with caution.
The magnitude of the association between General HL and HRQoL is larger than that for the asso-
ciation between education and HRQoL highlighting the importance of HL interventions to improve
General HL levels for adult populations. The associations observed between HRQoL and General
HL and other social determinants of health appears to be additive for education level, gender and
employment status.
For comparison, the magnitude of the differences seen should be considered as fairly large, as
e.g. Yabroff et al (2004) found that the difference in health-related quality of life between cancer
patients and the control group without cancer was 0.06 measured with the "Health Activities and
Limitation Index (HALex)". This index is based in part on EQ-5D-5L and uses a similar scale from
0-1.
As reported in Chapter 9, General HL is associated with health service utilization; lower HL is

associated with more contacts with General Practitioners and higher emergency service utilization
significantly for some countries. Of note is that, for both HRQoL and health service utilization, the
extent of the associations, and whether they reach statistical significance, varies between coun-
tries. This is likely to reflect the importance of national context on the impact of limited General
HL on health and health service use.
Research Question 2
For both Norway and Denmark, a negative association between HL and absenteeism can be ob-
served, as General HL increases there is a decrease in absenteeism from work due to health prob-
lems. For Ireland when General HL score increases from 0 to about 70, absenteeism increase how-
ever decreases thereafter, noting that 73.1% (1,797 respondents) of the respondents have a Gen-
eral HL score between 70% - 100%. These findings suggest that further research and analysis of
the HLS19 international data is required to more fully understand the complexities surrounding the
association between HL and absenteeism from the workplace.
Strengths of the investigation of HL and health-related

quality of life as a mediator for health costs
This is the first European study to explore and measure the relationship between General HL (as
measured by HLS19-Q12) and health-related quality of life (as measured by EQ-5D-5L) which have
consequential cost and economic implications for the health services, for individuals and for so-
ciety. This is therefore the first generation of data to directly measure General HL alongside
health-related quality of life and days absent from work due to poor health. The sample size within
this study is large across the three countries where the health-related quality of life data were
collected for EQ-5D-5L. Unlike prior studies of the health economic implications of HL all the data
were directly measured, with no data inferred.
Limitations of the investigation of HL and health-re-
lated quality of life as a mediator for health costs
The analyses in this chapter examines the strength of covariance between the measurement scale
for General HL (independent variable) and the dependent variable EQ-5D-5L and absence from
work due to poor health. Cross-sectional studies are typically limited in their ability to analyse
cause-and-effect factors. More suitable designs to offer indication of this would be longitudinal
designs. The fact that there is an association is not a sufficient basis for concluding that a cause-
and-effect factor is present, however it represents one of multiple factors that should be present
to consider whether one factor might be the cause of another, e. g. that increased HL gives rise to
increased health-related quality of life.
It is important to note also that data were collected using different modes of collection Computer
Assisted Telephone Interviewing (CATI (Ireland and Norway) and Computer Assisted Web Interview
(CAWI) (Denmark).
14.2.2 Implications for future research
As a result of these findings, it is reasonable to hypothesise that General HL impacts on health-

related quality of life as demonstrated here for EQ-5D-5L and absenteeism and in Chapter 8 for
other indicators and results in higher health service utilization as shown in Chapter 9. These hy-
potheses should be investigated using research approaches that can demonstrate causality, such
as longitudinal studies.
The findings presented also make the case for estimations of the financial impact of limited Gen-
eral HL. Combining the findings on the associations between limited General HL with national
health care costs would enable countries to see the actual cost of limited General HL in terms of
health service costs. Further studies could also include measures capturing wider personal and
societal costs such as the associations between General HL and, for example, absenteeism and
presenteeism.
EQ-5D-5L is a useful measure to use alongside the General HL measure, HLS19-Q12, and therefore
could be used to assess change in HRQoL with HL interventions. Such studies on HL interventions
where data are collected before and after an intervention will enable more granular cost data to
be gathered at the patient and individual level rather than only at the population level.
The results of this study demonstrate an association between General HL and health service utili-
sation in Chapter 9 and General HL and health-related quality of life and absenteeism in this
chapter and in Chapter 8, thereby providing evidence to support a call for research studies to be
designed to understand causality and to obtain an in-depth understanding of the drivers of pre-
ventable health care costs to health care systems and to individuals. This is critical in light of
limited resources in the context of escalating expenditure on health care globally in tandem with
ageing populations.
14.2.3 Implications for policy
The findings of the HLS19 study are sufficiently strong for national and local governments to rec-
ognise the importance of General HL for the health and well-being of their citizens and in the
utilization of health services. Investment in HL interventions as a disease prevention strategy at
local, national, and regional levels may lead to significant benefits to citizens’ quality of life and
more effective use of scarce resources and expensive health services. For Ireland, being unem-
ployed and having limited HL appears to have additive adverse associations with health-related
quality of life. This warrants further investigation to understand why this occurs in Ireland and not
in Denmark or Norway. This finding suggests a need for HL interventions designed specifically for
unemployed persons.
14.2.4 Conclusion
To the knowledge of the authors, this is the first exploration at individual respondent level, be-
tween General HL and HRQoL as measured by the EQ-5D-5L and of absenteeism from work due
to poor health. The findings show a significant relationship between General HL and HRQoL, that
appears to be augmented by other social determinants of health such as education, gender and
employment. The findings also show, in most instances, a negative association between HL and
absenteeism, as General HL increases there is a decrease in absenteeism from work due to health
problems.
The findings in Chapter 9 also confirm those of the previous European Health Literacy study (HLS-
EU) that limited General HL is associated with higher health care utilization.
Taken together, these findings indicate the likely impact of limited General HL on peoples’ health-
related quality of life, absence from work and on health care and societal costs. Further research
should be undertaken to examine the issue of causality. The findings of this study are, however,
sufficiently strong for policy makers to take steps to actively incorporate HL and HL interventions
into local, national, and regional policies.
14.3 References
Baker, D. W.; Gazmararian, J. A.; Williams, M. V.; Scott, T.; Parker, R. M.; Green, D.; Ren, J.; Peel, J.
(2002): Functional health literacy and the risk of hospital admission among Medicare managed
care enrollees. In: Am J Public Health 92/8:1278-1283
Berens, Eva-Maria; Vogt, Dominique; Ganahl, Kristin; Weishaar, Heide; Pelikan, Jürgen; Schaeffer, Doris
(2018): Health literacy and health service use in Germany. In: HLRP: Health Literacy Research and
Practice 2/2:e115-e122
Berkman, Nancy D; Sheridan, Stacey L; Donahue, Katrina E; Halpern, David J; Crotty, Karen (2011): Low
health literacy and health outcomes: an updated systematic review. In: Annals of internal medicine
155/2:97-107
Cameron, Adrian Colin; Trivedi, Pravin K (2010): Microeconometrics using stata. Bd. 2. Stata press
College Station, TX,
Deb, P.; Norton, E C.; Manning, W G. (2017): Health econometrics using Stata (Vol. 3). Stata Press
College Station, Texas, US.
Drummond, M.; Manca, A.; Sculpher, M. (2005): Increasing the generalizability of economic evalua-
tions: recommendations for the design, analysis, and reporting of studies. In: Int J Technol Assess
Health Care 21/2:165-171Deb, P.; Norton, E C.; Manning, W G. (2017): Health econometrics using
Stata (Vol. 3). Stata Press College Station, Texas, US.
Eichler, Klaus; Wieser, Simon; Brügger, Urs (2009): The costs of limited health literacy: a systematic
review. In: International Journal of Public Health 54/5:313
Karimi, M.; Brazier, J. (2016): Health, Health-Related Quality of Life, and Quality of Life: What is the
Difference? In: Pharmacoeconomics 34/7:645-649
Le, Christopher; Finbråten, Hanne Søberg; Pettersen, Kjell Sverre; Joranger, Pål; Guttersrud, Øystein
(2021): Health Literacy in the Norwegian Population. part 1. Rapport IS-2959,
67/12:2072-2078
Spycher, Stefan (2006): Ökonomische Aspekte der Gesundheitskompetenzen: Konzeptpapier. BASS

AG, Bern
Thoresen, M; Gjessing, HK; Veierød, MB; Lydersen, S; Laake, P (2012): Mixed models. In: Medical sta-
tistics in clinical and epidemiological research 1/:231-258
Eichler, Klaus; Wieser, Simon; Brügger, Urs (2009): The costs of limited health literacy: a systematic
review. In: International Journal of Public Health 54/5:313
Friis, Karina; Pedersen, Marie Hauge; Aaby, Anna; Lasgaard, Mathias; Maindal, Helle Terkildsen (2020):
Impact of low health literacy on healthcare utilization in individuals with cardiovascular disease,
chronic obstructive pulmonary disease, diabetes and mental disorders. A Danish population-
based 4-year follow-up study. In: European journal of public health 30/5:866-872
Garrow, V. (2016): Presenteeism: A review of current thinking. Institute for Employment Studies,
Brighton, UK
Herman, A.; Jackson, P. (2010): Empowering low-income parents with skills to reduce excess pediatric
emergency room and clinic visits through a tailored low literacy training intervention. In: J Health
Commun 15/8:895-910
Herndon, Jill Boylston; Chaney, Michelle; Carden, Donna (2011): Health literacy and emergency de-
partment outcomes: a systematic review. In: Ann Emerg Med 57/4:334-345
Howard, David H; Gazmararian, Julie; Parker, Ruth M (2005): The impact of low health literacy on the
medical costs of Medicare managed care enrollees. In: The American journal of medicine
118/4:371-377
Jovanić, Marija; Zdravković, Marija; Stanisavljević, Dejana; Jović Vraneš, Aleksandra (2018): Exploring
the Importance of Health Literacy for the Quality of Life in Patients with Heart Failure. In: Int J
Environ Res Public Health 15/8:1761
Karimi, M.; Brazier, J. (2016): Health, Health-Related Quality of Life, and Quality of Life: What is the
Difference? In: Pharmacoeconomics 34/7:645-649
Kickbusch, Ilona; Pelikan, Jürgen M.; Apfel, Franklin; Tsouros, Agis D. (2013): Health literacy: The solid
facts. World Health Organization (WHO) Regional Office for Europe, Copenhagen
Le, Christopher; Finbråten, Hanne Søberg; Pettersen, Kjell Sverre; Joranger, Pål; Guttersrud, Øystein
(2021): Health Literacy in the Norwegian Population. part 1. Rapport IS-2959,
McCaffery, K. J.; Morony, S.; Muscat, D. M.; Smith, S. K.; Shepherd, H. L.; Dhillon, H. M.; Hayen, A.;
Luxford, K.; Meshreky, W.; Comings, J.; Nutbeam, D. (2016): Evaluation of an Australian health
literacy training program for socially disadvantaged adults attending basic education classes:
study protocol for a cluster randomised controlled trial. In: BMC Public Health 16/:454
Morony, S.; Lamph, E.; Muscat, D.; Nutbeam, D.; Dhillon, H. M.; Shepherd, H.; Smith, S.; Khan, A.;
Osborne, J.; Meshreky, W.; Luxford, K.; Hayen, A.; McCaffery, K. J. (2018): Improving health literacy
through adult basic education in Australia. In: Health Promot Int 33/5:867-877
Muscat, D. M.; Smith, S.; Dhillon, H. M.; Morony, S.; Davis, E. L.; Luxford, K.; Shepherd, H. L.; Hayen,
A.; Comings, J.; Nutbeam, D.; McCaffery, K. (2016): Incorporating health literacy in education for
socially disadvantaged adults: an Australian feasibility study. In: Int J Equity Health 15/:84
Ousseine, Youssoufa M; Bouhnik, Anne‐Déborah; Peretti‐Watel, Patrick; Sarradon‐Eck, Aline; Memoli,

Victoria; Bendiane, Marc‐Karim; Durand, Marie‐Anne; Mancini, Julien (2020): The impact of health
literacy on medico‐social follow‐up visits among French cancer survivors 5 years after diagnosis:
The national VICAN survey. In: Cancer medicine 9/12:4185-4196
Ozkaraman, A.; Uzgor, F.; Dugum, O.; Peker, S. (2019): The Effect of Health Literacy on Self-Efficacy
and Quality of Life among Turkish Cancer Patients. In: J Pak Med Assoc 69/7:995-999
Palumbo, Rocco (2017): Examining the impacts of health literacy on healthcare costs. An evidence
synthesis. In: Health services management research 30/4:197-212
Pullenayegum, E. M.; Tarride, J. E.; Xie, F.; Goeree, R.; Gerstein, H. C.; O'Reilly, D. (2010): Analysis of
health utility data when some subjects attain the upper bound of 1: are Tobit and CLAD models
appropriate? In: Value Health 13/4:487-494
Health Literacy Project, European (2012): Health literacy and public health: a systematic review
and integration of definitions and models. In: BMC Public Health 12/80:http://www.biomedcen-
tral.com/1471-2458/1412/1480
Sørensen, Kristine; Pelikan, Jürgen M; Röthlin, Florian; Ganahl, Kristin; Slonska, Zofia; Doyle, Gerardine;
Fullam, James; Kondilis, Barbara; Agrafiotis, Demosthenes; Uiters, Ellen (2015): Health literacy in
Europe: comparative results of the European health literacy survey (HLS-EU). In: European journal
of public health 25/6:1053-1058. doi: https://doi.org/1010.1093/eurpub/ckv1043
Spycher, Stefan (2006): Ökonomische Aspekte der Gesundheitskompetenzen: Konzeptpapier. BASS

AG, Bern
Thoresen, M; Gjessing, HK; Veierød, MB; Lydersen, S; Laake, P (2012): Mixed models. In: Medical sta-
tistics in clinical and epidemiological research 1/:231-258
Vandenbosch, Jessica; Van den Broucke, Stephan; Vancorenland, Sigrid; Avalosse, Hervé; Verniest, Re-
bekka; Callens, Michael (2016): Health literacy and the use of healthcare services in Belgium. In:
Journal of epidemiology and community health 70/10:1032-1038
Vann Jr, WF; Divaris, K; Gizlice, Z; Baker, AD; Lee, JY (2013): Caregivers’ health literacy and their young
children’s oral-health–related expenditures. In: Journal of dental research 92/7:55-62
Zheng, M.; Jin, H.; Shi, N.; Duan, C.; Wang, D.; Yu, X.; Li, X. (2018): The relationship between health
literacy and quality of life: a systematic review and meta-analysis. In: Health Qual Life Outcomes
16/1:201
15 Recommendations for policy, practice, and
research
Authors:
Christina Dietscher (AT, M-POHL)

Jürgen M. Pelikan (ICC, M-POHL)
Peter Nowak (ICC)
Paola De Castro (IT)
Oxana Drapkina (RU)
Karin Gasser (CH)
Lennert Griese (DE)
Christopher Le (NO)
Luigi Palmieri (IT)
Kristine Sørensen (DK)
Josefin Wangdahl (SE)
In the HLS19, the integrated, comprehensive definition of health literacy as defined by the HLS-EU
Consortium was used: “HL is linked to literacy and encompasses people’s knowledge, motivation
and competencies to access, understand, appraise and apply information to form judgments and
take decisions in terms of healthcare, disease prevention and health promotion to improve quality
of life during the life course” (HLS-EU-Consortium 2012; Sørensen et al. 2012)
This definition and the refined study design of the HLS-EU study guided the HLS19 survey with its
17 participating countries (AT, BE, BG, CH, CZ, DE, DK, FR, HU, IE, IL, IT, NO, PT, RU, SI, and SK)
from most parts of the WHO European Region. The empirical results of this international survey
are the basis for the following recommendations. Furthermore, actions for potential interventions
to improve HL are proposed for planning health policy to deal with the identified problems and
potentials for improvements.
The recommendations address policy and decision makers, practitioners, and researchers at the
level of the WHO European Region as well as at national or regional levels. They are divided into
three sections and structured as follows:
a) General HL: Section 15.1 covers overall recommendations and proposed actions that relate
to General HL and the specific facets of HL measured in the survey.
b) Specific HLs: Section 15.2 highlights recommendations and proposed actions that apply to
the specific HLs measured within the HLS19: Navigational HL, Communicative HL with physi-
cians, Digital HL, and Vaccination HL.
c) Research: Section 15.3 summarizes recommendations and proposed actions for further re-
search on population HL for adults at national levels, especially for a next round of the HLS.
While the data, resulting recommendations, and proposed actions building on the HLS19 are of
specific relevance to the 17 participating countries and the WHO European Region, they may also
serve as a basis for other countries and regions, especially in relation to those findings that are
universal in nature. They will be highlighted in more detail below.
However, to select the areas where interventions are needed most and have the greatest potential
to address HL, countries will need country-specific results and data. Therefore, countries are en-
couraged to invest in undertaking regular national HL surveys, following the example of the Health
Behavior in School-Aged Children (HBSC) study (http://www.hbsc.org/) (see Recommendation 1
below).
While the data from the HLS19 survey provide evidence for areas where interventions are needed,
they do not measure or estimate the feasibility, effectiveness, efficiency, or cost-effectiveness of
potential interventions. For that, more research is needed, which is beyond the scope of the re-
sources of the HLS19. Therefore, many of the proposed and potential actions in this chapter are
primarily based on the criterion of plausibility.
The recommendations and proposed actions consider the fact that HL can be addressed by inter-
ventions aimed at improving personal HL and/or by enhancing the HL responsiveness of situa-
tions, settings, organizations, or systems, or by a combination of both.
Improvements in the personal HL of populations usually provide learning opportunities. These

work best for children in schools, (health) professionals, and chronically ill patients (or people with
chronicity). Yet enhancing these opportunities will only produce results in terms of competencies,
skills, and knowledge when participants are motivated and have the capability to learn. Further-
more, as far as knowledge is concerned, information resources need to be updated when new
information becomes available. For health and disease, this is continuously the case at rather short
intervals. In addition, people tend to forget information that they do not use on a regular basis.
Therefore, interventions addressing the HL responsiveness of organizations, situations, or sys-
tems are also needed and partly preferred.
Improvements to the HL responsiveness of situations in which people must decide and act focus
on the availability of up-to-date, understandable, reliable, and actionable health-related infor-
mation on the organizational and systemic levels. This kind of intervention has the potential to
Chapter 15 / Recommendations for policy, practice, and research 367

reach large numbers of people and facilitate the accessibility, understandability, appraisability,
and applicability of health-related information for many at once and can be up-dated regularly
more easily.
Applying the results of the HLS19, the decision whether to focus on personal or situational HL or
on both will depend on the specific context and problem as well as on the availability of tested,
feasible, and acceptable interventions, coupled with evidence or the plausibility of their effective-
ness.
To make best use of these recommendations and proposed actions, they need to be adapted and
refined as a function of national or subnational HL levels and results as well as of existing policy
and practice contexts. Ideally this should be based on prior mapping. When planning concrete
interventions, it will often be preferable to start pilot interventions first and to evaluate these
before investing in further systematic rollouts.
15.1 Recommendations and proposed actions relating to

General HL
Regarding the General HL of populations, the HLS19 study supports the results of the first com-
parative European Health Literacy Survey (HLS-EU) and other studies that have highlighted the
relevance of HL for people’s health and healthcare outcomes as well as for health policy.
The findings confirm the hypothesis that lower levels of HL are associated with indicators of per-
sonal health and the outcomes of healthcare systems: lower HL is associated with more unhealthy
lifestyles (lower fruit and vegetable consumption; less physical activity) (see Chapter 7), with lower
health status (poorer self-perceived health, more long-term illnesses/health problems, more lim-
itations due to health problems) (see Chapter 8), and with more frequent utilization of healthcare
services (emergency services, GPs/family doctors) (see Chapter 9), and therefore also with lower
quality of life (compare Chapter 14).
This is even more relevant because a considerable proportion of the adult resident population in
each country that participated in the HLS19 has low levels of HL and therefore has a higher prob-
ability of suboptimal personal health and healthcare outcomes. Across the countries that partici-
pated in the HLS19, between 25% and 72% of the respondents had limited HL, in other words they
experienced considerable difficulties in completing tasks related to the management of health-
related information and communication. In relation to these difficulties, a social gradient (i.e.,
inequalities in HL status are related to inequalities in social status/level in society) was demon-
strated, but to a different degree, for the participating countries.
HL, therefore, must be considered a critical determinant of health (Shanghai Declaration 2016)
and healthcare utilization. Moreover, HL can be a modifiable determinant of health, which makes
it feasible to address and improve low HL with the help of interventions addressing systems or
organizations as well as groups of persons.
Recommendation 1: Health policy should include an investment in longitudinal studies, measuring
and monitoring population HL, following the example of the WHO’s Health Behavior in School-
Aged Children (HBSC) study, and should implement interventions to improve HL.
Actions proposed for possible interventions in this respect – from simple awareness-raising to
creating impact – include:
» Agenda setting for HL policy, practice, and research with relevant decision makers, funders,
managers, and stakeholders (professionals, clients) from different sectors and settings in
society
» by policy briefs, conferences, workshops, newsletters, websites,
» by linking HL to other health policy priorities (such as patient safety, patient orien-
tation, patient’s self-management, noncommunicable diseases, Sustainable Devel-
opment Goals) in the sense of HL mainstreaming,
» by including HL in national (public) health goals and by developing specific action
plans for HL.
» Capacity building for HL policy, practice, and research (providing leadership and governance,
strengthening organizational infrastructures, securing specific and sufficient resources (fi-
nances, high-quality materials, and tools), developing knowledge, training the health work-
force, strengthening partnerships).
» Legal regulations for HL practice: defining HL as a responsibility of health systems, defining
quality standards for the HL capacity of (healthcare) organizations, (health) professionals,
and publicly available information as well as defining incentives for good HL practice.
» Investing in specific programs on improving HL practice in different sectors of society
(health care, education, work, mass media, politics) based on prior mapping.
» Investing in regular, standardized, internationally comparable monitoring and research of
HL.
Recommendations and proposed actions concerning

specific population groups most affected by limited
General HL
While low HL is a widespread problem in the population of all participating countries, some pop-
ulation groups are more affected by the consequences of lower HL than others. Ordered by the
extent of deviation in HL from the general population, an increased probability of low HL and of
the consequences which lower levels of HL represent was observed among people with low self-
perceived health (very bad or bad health), those experiencing financial deprivation, those having
a self-perceived low level in society (categories 1-4), and those with a lower level of education
(ISCED 1 or 2) (in some countries). In addition, frequent utilization of a GP or family doctor (6+
contacts per year), older age (76 or older) (although not in all countries), and long-term illnesses
or health problems (1+) are associated with lower HL as well.

Recommendation 2: While efforts to improve HL, especially when applied on system and organi-
zational levels, are beneficial to the population at large, interventions should be specifically tar-
geted at at-risk groups for low HL to reduce the health gap between groups.
The following proposed actions for potential interventions addressing target groups should be
considered:
Systems-level interventions
» Provide regulations and incentives to support the implementation of HL-supportive infor-

mation and communication strategies, including services especially for vulnerable groups
(for example, by making compliance with HL-related criteria a condition for receiving public
funding and by including criteria in quality schemes).
» Monitor the HL responsiveness of relevant services for vulnerable groups, for example in na-
tional quality reports.
Organization-level interventions
» Adopt a universal precautions approach to HL/organizational HL/HL sensitivity approach, for

example by including HL in quality management.
» Train staff in HL-friendly communication (clear language, using techniques like teach-back
or chunk-and-check or norm-critical relevant and pedagogical images).
» Prioritize target groups in most need of support. Typically, these will include people with
lower general education, lower perceived social status, those suffering from financial depri-
vation, and older people. A risk of low HL may also exist for people with migration back-
grounds, especially when they are not familiar with the local language.
» Provide quality assured information in different languages and provide interpreting services.
» Improve the availability, accessibility, understandability, appraisability, and applicability of
high-quality information in major formats (written, audio-visual, digital) on selected topics
for selected target groups by involving members of the target groups in defining problems
and selecting topics as well as in developing and testing materials.
Individual-level interventions
» Provide specific training courses and material in clear language to support the HL of vulner-
able groups (using chunk-and-check or norm-critical relevant and pedagogical images).
» For people with very low HL, provide personal assistance/case management to compensate
for low HL.
» Use relevant contact points and influencers from selected target groups (including health
professionals and other key staff) to reach out to them.
Recommendations and proposed actions addressing
limited HL in specific aspects and domains of General
HL
HL refers to the skills and abilities related to accessing, understanding, appraising, and applying
health-related information, and is a necessary precondition for adequate health-related decisions
and actions in the domains of health promotion, disease prevention, and health care/treatment.
A conceptual matrix consisting of 12 subdomains was used to operationalize the HLS19 question-
naire based on its predecessor, the HLS-EU survey questionnaire (see Chapter 1 and Chapter 3).
Each of the subdomains in the matrix was measured with at least one specific item in the HLS19
survey. Because of the different interests and prior experience with HL measurement in the 17
countries that participated in the survey, three versions of the questionnaire were used: a 12-item
scale (HLS19-Q12) with only one item per subdomain, selected on the basis of the Rasch scalability
principle; a 16-item scale (HLS19-Q16) to allow comparison with the older, short form of the HLS-
EU; and a 47-item scale (HLS19-Q47). The HLS19-Q47 contains all the items in the HLS19-Q12,
while to obtain scores for the HLS19-Q16, ten additional items had to be administered during data
collection in addition to the 12 items on the HLS19-Q12. Results of the HLS19-Q12 are available
for all 17 countries that participated in the HLS19; the HLS19-Q47 was used by six countries, and
the HLS19-Q16 is available for 14 countries. While using versions of different length limits the
international comparability of the relevance of specific items, some overall recommendations con-
cerning aspects and domains of HL can still be made:
Independently of the domain (health promotion, disease prevention, healthcare), the four aspects
of HL represent a logical hierarchy for processing health-related information.
Recommendation 3: Interventions to improve HL should focus on all four aspects of processing

health-related information: accessing, understanding, appraising, and applying information to
form judgements and make decisions conducive to health, within the domains of healthcare, dis-
ease prevention, and health promotion.
To provide effective health information and communication on any theme, it is necessary to ensure
that all aspects of HL are sufficiently supported by interventions.
» First, ensure the availability of and access to relevant, high quality health information espe-
cially through those channels that are typically used by the target group(s) addressed, for
example by at-risk groups for low HL (including peer communities and expert peers, neigh-
borhoods, places of worship, social media).
» Second, enable the understandability of information for the selected target groups (consider
culturally appropriate content, language levels, availability in diverse languages, the infor-
mation needs of illiterates and people with impaired senses).
» Third, enable the appraisability of information by making sure that information sources are
given or, preferably, that information that is publicly available is quality assured and that

people understand well why a certain piece of information is of relevance to them (for exam-
ple by using the “ask me 3” approach).
» Fourth, ensure that the information provided is applicable by being clear and detailed
enough to guide concrete decisions and actions (for example, by breaking information down
into steps).
In the meantime, a wealth of “how-to” books on these aspects are available. See, for example,
DeWalt et al. (2010); Rudd/Anderson (2010); Rudd, Rima E./Anderson, Jennie E. (2006).
Recommendations and proposed actions addressing

concrete tasks of General HL
Each of the aspects and domains of HL implies a multitude of tasks involving health-related in-
formation and communication. The tasks that were assessed in the HLS19 allowed us to identify
difficulties concerning HL experienced by larger groups of the general population or by specific
(vulnerable) target subpopulations.
While country-specific analysis is recommended as a basis to select tasks to address specific dif-
ficulties and develop interventions to improve them, the following table provides an overview of
the tasks experienced as being difficult by more than 20% of the respondents in the international
sample (see Table 15.1):
Table 15.1:
Average percentage shares for “very difficult” and “difficult” answers to the different HL-related
tasks*
Ordered by task, item no. in the Q47, the HLS19 instrument from which the percentages are taken
(Q47, Q16, or Q12), and the percentage of “very difficult” and “difficult” answers in the task
Health care/treatment (10)* Disease prevention (9)* Health promotion (12)*
» Find information on the » Find information on how to » Find information about

treatment of illnesses that handle mental health prob- changes in laws that af-
concern you (CORE-HL2: Q47 lems (CORE-HL18: Q47 fect health (CORE-HL35,
26.1%; Q16 23.3%) 36.7%; Q16 36.4%; Q12 Q47 54.3%)
» Find information on what to 36.1%) » Find out how the neigh-
do in a medical emergency » Find information on recom- borhood could become
Accessing/
(CORE-HL3: Q47 24.3%) mended vaccinations (CORE- more health friendly
finding (7)*
HL19, Q47 27.9%) (CORE-HL34, Q47 50.4%)
» Find information about
how to promote health at
work, at school, in the
neighborhood (CORE-
HL36, Q47 38.7%)
Under- » Understand the instruction » » Understand information
standing leaflets that come with your on food packaging
(5)* medicine (CORE-HL6, Q47 (CORE-HL38, Q47 31.8%)
27.1%) » Understand information
» Understand information in the mass media on
about what to do in a medical how to improve health
emergency (CORE-HL7, Q47 (CORE-HL39, Q47 28.4%;
24.3%; Q12 22.7%) Q16 26.7%)
» Understand information
on how to keep your
mind healthy (CORE-
HL40, Q47 26.3%)
Apprais- » Judge the reliability of infor- » Judge the reliability of infor- » Judge how the neighbor-
ing/judg- mation on illness in the mass mation on health risks in the hood may affect health
ing (9)* media (CORE-HL12, Q47 55%) mass media (CORE-HL28, and well-being (CORE-
» Judge the need for a second Q47 49.4%; Q16 47.5%) HL41, Q47 34.7%)
opinion (CORE-HL11, Q16 » Judge which vaccinations you » Judge how housing af-
44.1%; Q47 42%) or your family need (CORE- fects health and well-be-
» Judge the advantages and HL26, Q47 30.1%) ing (CORE-HL42, Q47
disadvantages of treatment » Judge which health screen- 22.8%; Q12 22.2%)
options (CORE-HL10, Q47 ings or examinations you
41.4%; Q12 43%) should have (CORE-HL27,
Q47 29.3%)
» Judge when to go to the doc-
tor for a check-up (CORE-
HL25, Q47 21.7%)

Ordered by task, item no. in the Q47, the HLS19 instrument from which the percentages are taken
(Q47, Q16, or Q12) and the percentage of “very difficult” and “difficult answers” in the task
Health care/treatment (10)* Disease prevention (9)* Health promotion (12)*
Applying » Use information from the » Decide how to protect your- » Take part in community
(8)* doctor to make decisions self from illness using infor- activities that improve
about illness (CORE-HL13, mation in the mass media health and well-being
Q16 22%) (CORE-HL31, Q47 43%; Q16 (CORE-HL47, Q47 38.1%)
40.8%; Q12 39.7%) » Influence living condi-
» Decide how to protect your- tions affecting health and
self from illness using infor- well-being (CORE-HL46,
mation from family or friends Q47 32.4%)
(CORE-HL31, Q47 31.2%) » Join a sports club or ex-
» Decide on whether to have ercise group (CORE-
the flu vaccination (CORE- HL45, Q47 27.5%)
HL29, Q47 27%) » Make decisions to im-
prove health and well-
being (CORE-HL44, Q12
25.6%; Q47 22.1%)
* The numbers in brackets in the column and row headers refer to the number of very difficult and difficult tasks relating to the HL domain
or aspect.
Recommendation 4: When planning interventions related to specific, concrete HL tasks, the tasks
that are experienced as being more difficult by study participants should be prioritized.
The subsections below (15.1.3.1 and 15.1.3.2) provide exemplary proposals of interventions that
can be considered for difficult tasks related to two topics that are not sufficiently covered in other
sections of this chapter, namely HL in relation to the mass media and HL regarding mental health.
15.1.3.1 Improving HL through high quality, appropriate

health information in the mass media
As shown by the HLS19 results, participants across all countries especially experienced consider-
able difficulties in judging the trustworthiness of health-related information in the media as well
as in understanding and using information conveyed through the media. These findings may be
related to the tendency of mass media to draw the public’s attention by making information sen-
sational. In relation to health, this can either contribute to unrealistically high hopes for new cures
or in unjustified fears for specific health risks or diseases.
Table 15.2:
Tasks in relation to the mass media that were experienced as “difficult” or “very difficult” by
more than 25% of respondents
Tasks
» Judge the reliability of information on illness in the mass media (CORE-HL12: Q47 55%)
» Judge the reliability of information on health risks in the mass media (CORE-HL28: Q47 49.4; Q16 47.5%)
» Decide how to protect yourself from illness using information in the mass media (CORE-HL31: Q47 43%; Q16 40.8%;
Q12 39.7%)
» Understand information in the mass media on how to improve health (CORE-HL39: Q47 28.4%; Q16 26.7%)
Since the mass media, by definition, reach everybody and thus permeate the whole of society, it
is very difficult to try and improve people’s ability to understand, appraise, and apply information
from the media through interventions targeted at individuals, and efforts to do so would be inef-
fective.
Recommendation 5: The quality of health information in the mass media should be improved and
guaranteed on systemic and organizational levels.
Therefore, interventions on systemic and organizational levels would make more sense.
Systems-level actions proposed for potential interventions:
» Aim at a national media strategy to improve the quality of information either by specific reg-
ulations or based on voluntary improvements in the quality of health information, including
illicit and overt health messages.
» A national media strategy may involve incentives like a media prize for trustworthy health
information.
» Since the ability to judge information from the mass media is strongly related to general ed-
ucation, media health literacy (MHL), and critical media literacy, it may make sense to in-
clude this topic in school curricula.
» In addition to media HL, it might be worthwhile investing in a national health information
portal covering aspects of health care, disease prevention, and health promotion, as well as
in a strategy for making the portal known both to relevant expert stakeholders and the gen-
eral public (see also Subsection 16.2.3).
» It may also make sense to establish quality standards for health information that should be
followed by those developing, commissioning, or distributing publicly available health infor-
mation (see also Subsection16.2.3).
Organizational-level proposals (for media organizations):
» Have a policy on timely, relevant, and transparent health information.

» Have organizational guidelines on how to research, develop, and provide health information
in clear language.

Individual-level actions:
» Media-related HL and the dissemination of trustworthy health information channels can be

supported through training and interventions by continuous education centers, healthy
communities, community and occupational health services, and many more local actors that
can be included in national strategies.
15.1.3.2 Improving HL in relation to mental health
The HLS19 questionnaires did not contain many questions related to mental health. The few items
on mental health that were used in the survey showed that finding information on mental health
was difficult for one out of every three respondents while understanding information on how to
maintain good mental health was difficult for one out of every four respondents.
Due to the Covid-19 pandemic, among other things, international experts and organizations are
warning that mental health issues will be on the rise in the near future. Supporting mental HL in
the fields of promotion and prevention may help reduce the burden of disease in this field. Mental
HL around treatment may encourage people to seek treatment in a timely way and add to the
effectiveness of services.
Recommendation 6: Focusing on mental HL and interventions to improve HL in relation to mental

health should be prioritized and supported by specific research.
15.2 Recommendations and proposed actions for specific

health literacies
Within the HLS19 Project, new tools were developed for collecting information on four specific
health literacies via optional packages that could be selected by participating countries.
» Navigational HL was measured in eight (ten) countries: AT, BE, CH, CZ, DE, FR, PT, SI, (BG*,
NO*),
» Communicative HL was measured in nine (ten) countries: AT, BE, BG, CZ, DE, DK, FR, HU, SI,
(NO*),
» Digital HL was measured in 13 countries: AT, BE, CH, CZ, DE, DK, FR, HU, IE, IL, NO, PT, SK,
» Vaccination HL was measured in 11 countries – of these, seven countries used the complete
optional package (AT, BE, CZ, HU, IE, PT, and SI) and a further four countries used the vac-
cination items from the HLS19-Q47 (BG, DE, IT, and NO).
* Not all of the items of the HL measure were implemented and therefore no comparable score was possible.
In the following paragraphs, recommendations and proposed actions pertaining to these specific
health literacies are provided in the order of number of countries that assessed the specific health
literacies.
Navigational HL
An instrument for measuring Navigational HL was developed and fully applied in eight European
countries for the first time. Overall, the Navigational HL tasks are perceived as challenging, alt-
hough with considerable variation among countries. The Average Percentage Response Patterns
(APRP) for “(very) difficult”, with an average of 45%, ranged from 33% (AT, SI) to 59% (DE). Like
General HL, Navigational HL follows a social gradient: a lower social status/level in society and
lower financial resources are linked to lower Navigational HL in most countries. Concerning spe-
cific tasks, navigating healthcare systems, and finding support for orientation in the system is
perceived as (very) difficult by many patients/users (total: 47.6%). Especially information tasks on
the systemic level (about the structure and functioning of the healthcare system) and concerning
health care organizations were perceived as (very) difficult, indicating that those structures form
a “black box” (Gui et al. 2018b) for many patients/users. Respondents viewed it especially as (very)
difficult to understand information about current health care reforms that may affect their health
care situation (total: 56.6%), to judge whether a particular health service would meet their expec-
tations (total: 52.0%), to find information about patients’/users’ rights in the healthcare system
(total: 51.6%), and to find information on the quality of health services (total: 48.8%). Furthermore,
having low Navigational HL does not remain without consequences: the HLS19 study found a rela-
tionship between Navigational HL and indicators for health and health care utilization in some
countries.
Recommendation 7: Health policy should develop strategies to improve people’s Navigational HL,
especially focusing on population groups which experience more problems for this type of com-
petence. Specifically, to improve Navigational HL, interventions on systemic and organizational
levels should be developed and implemented with the aim of making the health system more
health-literate, user-friendly, and easier to navigate.
Systems interventions:
» On a general level, more transparency, clarity, and user-friendliness for healthcare systems
as well as more comprehensibility and plausibility about health care reforms and their rele-
vance/effects on the residential population (or on specific subgroups) are needed.
» Overall, decision makers need to be made aware of the level of difficulties that people expe-
rience when attempting to navigate and use the healthcare system. Agenda-setting should
focus on effective strategies for improvements on the systemic level, with specific emphasis
on
» making healthcare systems, including insurance companies, more transparent and

user-friendly in the sense of health-literate healthcare systems and health care or-
ganizations (HLO) (if necessary, by legal regulations),
» making it easy to find the right contact in health care organizations/professions at
the right time by providing easily accessible and trustful guidance systems (using
easy-to-understand signage systems) and pathways (of care) through the healthcare
system in the form of specific programs),

» improving pathways to health information and services by involving users in rede-
signing these pathways to meet their needs,
» strengthening especially the availability and understandability of information about
what to expect from services, the quality of services (in those cases where people have
the choice between services), and how to ensure patients’ rights,
» training key staff in supporting patient navigation.
» Alliances should be sought to support Navigation HL also through cooperation with other
sectors (for example, by including education about the general functioning of national
healthcare system in school curricula; by providing essential health information via occupa-
tional health services).
Organizational-level interventions (compare e.g., Rudd, Rima E./Anderson, Jennie E. (2006)):

» Train staff to help clients/visitors find their way around in complex health care institutions.
» Use orientation systems (signage, guidance systems, etc.).
» Involve visitors/patients in improving the orientation system, for example by walking inter-
views for the accreditation of health literate organizations.
» Involve user/patient organizations to support people in navigation. User/patient organizations
could also be involved in the development of health literate organizations.
Individual-level interventions:
» Develop and offer training about the general functioning of the healthcare system, how to
navigate the system, and how to advocate one’s own patients’ rights.
» Support people with higher risks of low Navigational HL such as people with chronic illnesses
who may need specific guidance from patient navigators, care or case management along the
entire illness, and health care trajectories.
Communicative HL: Communicating with physicians
While the HLS-EU already contained some items on Communicative HL, the HLS19 included a spe-
cific optional package for Communicative HL with physicians, which came as either a long or a
short version of this specific scale (the HLS19-COM-Q11 and the HLS19-COM-Q6). It was intended
to measure Communicative HL in dialogue with health professionals in general. However, during
pretesting, the general term “health professionals” was not well accepted by the participants. Be-
cause the participants’ experiences differ depending on the kind of health professional, the term
was perceived to be too vague, which made it difficult to have an opinion and answer the ques-
tions. In addition, the status of different health professions varies widely across the participating
countries while that of physicians is quite similar and comparable. Therefore, the group decided
to focus on physician-patient communication.
For the various items in the HLS19-COM-Q11, the percentage of perceived difficulty for the total
sample ranged from 4.4% (HL-COM1: describe to your doctor your reasons for coming to the
consultation) to 25.3% (HL-COM4: get enough time in the consultation with your doctor). For the
HLS19-COM-Q6 items, they varied from 9.2% (HL-COM3: explain your health concerns to your
doctor) to 26.2% (HL-COM4: get enough time in the consultation with your doctor). General HL
was the strongest predictor of Communicative HL. Communicative HL was lower for lower socio-
economic status (social status/level in society and financial deprivation) and for poorer health,
confirming a “social gradient” in doctor-patient communication. Older people, people with a very
low educational level, those with a low level in society, the financially deprived, those with a poorer
health status, and those reporting frequent use of health care were found to be specifically prone
to low Communicative HL. Although a mixed pattern of the consequences of Communicative HL
was observed across the participating countries, low Communicative HL does appear to be linked
to poorer self-reported health and to more frequent utilization of healthcare services in most
countries. Associations were also found between Communicative HL and health behavior and at
least one health status indicator in most countries.
Concerning the specific Communicative HL tasks, most respondents found these comparably easy
in relation to the other specific health literacies. Most difficulties were encountered in getting
enough time in the consultation with doctors (difficult for 1 in 4 respondents) and in expressing
personal views and preferences to the doctor. Between 10% (AT and SI) and 27% (DE) of respond-
ents reported difficulties in communicating with their physicians. Communicative HL with physi-
cians is relevant for the outcomes of health care services.
Recommendation 8: Interventions to improve the communication of health professionals with pa-

tients should have high priority in health policy and practice. Specifically, interventions to support
health professionals, especially physicians, in dedicating more time to person-centered commu-
nication are needed.
Proposed actions for potential interventions:
» Healthcare systems should be improved to

» support health professionals in dedicating more time to person-centered communi-
cation and information to be able to better facilitate active patient participation in
the conversation and when negotiating their health care, by better financial remu-
neration for the time needed for communication,
» organize healthcare to better enable and facilitate communication and information,
for example by having specific health professionals (e.g., advanced nurses) support-
ing communication processes, in addition to good communication with physicians,
» make high-quality, evidence-based communication and information training an in-
tegrated part of professional training and the continuous education of healthcare
professionals,
» National action plans on HL that include a focus on Communicative HL can be an
important tool to facilitate implementation and dissemination (see, for example, the
Austrian policy for establishing a patient-centered culture of communication “Im-
proving the Quality of Healthcare Communication” (Nowak/Sator 2017)).
» Healthcare organizations should be encouraged to adopt organizational HL/responsiveness

including a focus on supporting high-quality patient-centered communication:

» Specifically, the management of health care services should support communication
training for all health professionals working for the organization,
» Communication training should also focus on ensuring good communication with
socio-economic disadvantaged people and those with chronic diseases,
» Supplementary education should be offered for physicians in communication and
counselling.
» Support to individuals can provide specific support for those who need it:
» Support Communicative HL through training and the empowerment of patients/us-
ers, especially those with chronic illness or multimorbidity and people with low so-
cio-economic resources (e.g., certain migrant communities, people with low formal
education) to support active participation (e.g., Ask me 3, question prompting),
» Include nonprofessional health workers/mediators (lay people with special training)
as specific support for vulnerable populations to improve communication in consul-
tations (e.g., (Katona et al. 2021) and to support the management of communica-
tion/information management over the course of an illness,
» Offer services in different languages and/or adequate translation services for people
not proficient in the local language (e.g., via video translation).
» Collaborate with other sectors, especially with education, to foster knowledge and skills on
how to express oneself about health and disease from an early age on.
Digital HL
Digital health is an important facilitator of self-care and the use of healthcare services, adopting
healthy lifestyles, and participating in the early detection of health problems. Accordingly, , by
encompassing a set of critical skills for empowerment in health and well-being, Digital HL is be-
coming increasingly important. It is specifically defined in the HLS19 as the ability to search for,
access, understand, appraise, validate, and apply online health information, the ability to formu-
late and express questions, opinions, thoughts, or feelings when using digital devices, taking into
account the frequency with which people use different digital sources and resources such as online
video consultations, digital personal health records, social media, and health related apps, etc. for
promoting health. The HLS19 provided the first international assessment of Digital HL for 13 coun-
tries in the WHO European Region.
In the HLS19 study, three aspects of Digital HL were measured: the frequency of use of digital tools
to access information on health; the perceived ease or difficulty to find, understand, appraise, and
apply health information from digital sources, and on-line interactivity regarding health issues.
The results of the HLS19 revealed that digital tools are widely used to access health-related infor-
mation but that in all but three participating countries it is less frequently used to interact with
the healthcare system itself. Overall, more than 25% of the total population reported difficulties
using digital devices to process health information, with huge variation among countries, ranging
from 11% (PT) to 48% (SK).
About a third of the respondents found it difficult to very difficult to find, understand, evaluate,
and apply digitally available health information (2%-18% of respondents found it “very difficult”,
and a further 20%-40% “difficult”). The distribution differed across participating countries, with
only one in five of the Norwegian respondents (22%) experiencing the tasks as being difficult or
very difficult, as opposed to three out of every five German respondents (58%).
Regarding specific tasks, respondents found it most difficult to judge whether information is of-
fered with commercial interests and to judge whether information is reliable.
General HL and Digital HL are strongly associated. Furthermore, Digital HL is positively associated
with high education, low financial deprivation, and younger age categories, although there were
exceptions in some countries (see Chapter 12 for details).
The degree to which Digital HL predicts selected outcome measures was highly variable across
participating countries. This probably indicates different levels of development in the process of
digitalizing health care: the more digitalized health care in a country is, the stronger the impact
of the Digital HL score is expected to be. Digital development is most likely to increase over the
next years.
Recommendation 9: Emphasis on easily accessible, high quality, trustworthy, understandable, as-

sessable, and applicable health information as well as communication via digital sources should
be increased.
Proposed actions for potential interventions:
Systems-level interventions:
» Introduce regulations and incentives for good-quality and appropriate digital health infor-
mation, resources, and communication systems.
» Set up public online health portals (for the general public, for healthcare professionals, for
journalists) to provide essential health information free from commercial interests (nation-
ally, or via national/regional healthcare providers), including information on seasonally rele-
vant health topics.
» Carry out research on the preferred information and communication channels for specific
vulnerable target groups (including social media) to create matching strategies to support
Digital HL.
» Include public participation when developing online interventions to determine policy on
digital health.
Organizational-level interventions:
» Support healthcare providers to improve the availability and user-friendliness of their digital
services including digital options for making appointments, innovative tools such as chat-
bots, and acknowledging the need for cultural appropriateness.

» Provide training for healthcare providers to empower groups in acclimatizing to a digital
system, without stigma regarding ability according to age, etc.
» Update health literate organization standards and attributes to include Digital HL.
» Provide training on critical Digital HL (how to recognize and scrutinize fake news and com-
mercial interests) for example, through school curricula, continuing education centers, and
skill building through social media avenues.
» Facilitate access to digital devices for financially deprived groups (for example, via public li-
braries, community centers, primary healthcare centers).
» Offer specific training in digital skills on the local level (for example, via senior clubs, cen-
ters for adult learners).
Vaccination HL
High confidence in vaccination programs is crucial for maintaining high coverage rates. However,
over the past few years, there has been a decline in the general public’s confidence in vaccines,
resulting in an increasing number of vaccine delays and refusals. This, in turn, contributes to
declining immunization rates and increases in disease outbreaks in several countries, as illustrated
by recent measles outbreaks. As such, vaccination hesitancy has become a major concern for
public health authorities. This concern became even more pressing with the Covid-19 pandemic.
Although there is a consensus among experts that vaccination of the population is by far the most
(and probably the only) effective strategy to curb the pandemic, doubts about the safety and ef-
fectiveness of the vaccines against the SARS-CoV-2 virus are rampant in large groups of the pop-
ulation. As such, this is an even more relevant topic than before and a much-debated issue.
Several studies have documented that health literacy, or its more specific form of Vaccination
Literacy, may play an important role in vaccination hesitancy. More specifically, Vaccination HL can
be regarded as a potential moderator of complacency and/or confidence issues and might, as
such, contribute to counteracting the negative effects of exposure to misleading information on
vaccinations. However, to date no national surveys have included measures of vaccination behavior
and its determinants (attitudes, trust, knowledge, groups, and social norms) along with refined
measures of functional, communicative, and critical HL to clarify the role of the latter in vaccination
behavior.
The HLS19 study is the first to provide an international assessment of Vaccination HL in the WHO
European Region. Eleven countries measured this type of HL. The results show that dealing with
information about vaccination is difficult for a major proportion of respondents in all countries,
although the size varies among countries (13% for PT to 45% for BG). Judging or assessing infor-
mation on vaccination was experienced as the most difficult aspect. Like General HL and other
specific forms of HL, Vaccination HL is lower for people with lower socio-economic status. In
contrast, age, gender and – surprisingly – being trained in a health profession are not consistently
correlated with Vaccination HL. The latter implies that not only lay people but also people working
in healthcare can be relevant target groups for interventions to improve Vaccination HL.
In all participating countries (except for PT), Vaccination HL is a relevant determinant of reported
vaccination behavior.
Recommendation 10: Improving Vaccination HL should have top priority, with a special focus on
the vulnerable groups identified. The aspect of judging information on vaccination (as opposed to
finding, understanding, or applying it) should be prioritized to improve the trustworthiness of
information and communication regarding vaccination.
Proposed actions for potential strategies that may improve Vaccination HL and particularly re-
sistance to false information:
Systems-level interventions
» Provide easily accessible, understandable, trustworthy, and quality-assured public infor-

mation on vaccination and vaccination services for the general public, especially via the in-
ternet and on social media. Consider the potential of innovative information tools such as
chatbots but also consider specific information needs and preferred, culturally appropriate
information channels for selected, hard-to-reach groups (jointly developed with the target
groups, in different languages).
» Communicate better about the role of relevant institutions and organizations, such as the
Food and Drug Administration (FDA) or the European Medicines Agency (EMA) and provide
transparent information about vaccine approval processes.
» Communicate clearly and openly about the potential benefits AND risks of vaccinations, for
example by using fact box formats. For the dissemination of information, it may make sense
to establish partnerships with relevant actors in health and other sectors, including commu-
nity care and occupational health services, education (school curricula), the media (journal-
ists, influencers), and NGOs representing diverse parts of civil society.
» Focus on strengthening the confidence of health professionals in vaccinations: They are the
ones conveying information to the public. Therefore, it would make sense to
» better understand what makes health professionals skeptical and what information
they need, for example, via focus groups,
» invest in vaccination-related training and further education for healthcare staff,
» provide guidelines and contact points for health professionals in charge of vaccina-
tion programs.
Organizational-level interventions (for centers and institutions providing vaccinations)
» Have a policy on staff training about vaccination to ensure coherent messages to vaccinees.
» Prepare staff to answer questions from vaccinees.
» Have high-quality information material available on vaccinations.

» Offer training and information on vaccination for interested individuals (for example, online
talks by medical experts, courses in community or senior centers, at places of worship).
» Involve representatives of target groups in the development of information on vaccination:
This may help to identify topics people might be specifically interested in, such as the ingre-
dients in vaccination products, the risk of vaccination injuries, etc. The relevant information
should be made available transparently and via trusted sources so as not to fuel conspiracy
theories and false information.
15.3 Recommendations and actions proposed for research
The HLS19 results have demonstrated the relevance of HL for population health and health policy
by identifying overarching general trends in the results while also revealing considerable variations
in the results across participating countries.
Recommendation 11: The HL of the adult resident population should be measured regularly in as
many countries in the WHO European Region as possible. The next wave of measuring should be
planned for data collection in 2024.
Proposed actions: Besides the measurement of HL in the adult population, the measurement of HL
in children and adolescents should be considered.
Proposed actions: Besides the measurement of general and specific personal HL, research on in-
tervention research/action research to improve HL should be supported.
Proposed actions: Besides the measurement of general and specific personal HL, the measurement
of organizational HL should be supported.
For numerous reasons, including the Covid-19 pandemic, the HLS19 was less standardized than
originally planned. Since different methods of data collection were applied, differences in results
between countries can only be interpreted with caution. There is, however, an ongoing transfor-
mation in the techniques of data collection, albeit at differing speeds across countries.
Recommendation 12: In preparation for the next wave of the survey, more specific research should
be funded to extend and apply the tool for measuring General HL and relevant correlates. This
should include more detailed, specific analyses and publications of the HLS19 data. On the basis
of the validation results, the wording of some items should probably be revised.
Proposed actions: The implementation, intervention, and action research for improving limited
General HL should be stipulated.
The HLS19-Q47 allows for a more comprehensive analysis of the task-related difficulties encoun-
tered by respondents than the HLS19-Q12 or the HLS19-Q16.
Recommendation 13: Especially countries measuring population HL for the first time should use
the HLS19-Q47, which will best support the selection of country-specific difficulties and interven-
tions responding to these difficulties.
Countries that have more experience with the survey can combine the shorter versions with the
scales for the specific health literacies that complement the assessment of General HL.
For all four specific HLs (Navigational HL, Communicative HL, Digital HL, Vaccination HL) included
in the HLS19, the assessment tools were developed and used for the first time in the HLS19.
Recommendation 14: For the four specific HLs, more detailed analyses and publications on the
HLS19 data as well as further research and development on improvements for later applications
are needed. More specific correlates, determinants, and consequences of specific HLs should be
considered, selected, and tested.
Proposed actions: The implementation, intervention, and action research for improving limited
specific HL should be stipulated.
Alongside those specific health literacies that were assessed in the HLS19, which partly followed
specific types of HL-related tasks, and which, in the case of vaccination, focused on one specific
aspect of prevention, other important fields of HL maybe of interest for health policy makers.
Recommendation 15: Additional specific health literacies or relevant topics of General HL should
be reviewed, selected, and researched to be included in the next wave of HLS.
Proposed action: Health research institutes should be encouraged to include HL research in their
research agendas.
Proposed action: It might be worthwhile to undertake research into HL related to specific chronic
diseases or conditions.
Recommendation 16: More detailed analyses are needed regarding the costs and economics of
General HL and of the four specific HLs studied in the HLS19. Further dissemination of the results
of the HLS19 through peer-reviewed scientific publications is required, as is further research and
development into how to improve the survey for future applications.

15.4 References
DeWalt, Darren A; Callahan, Leigh F.; Hawk, Victoria H.; Broucksou, Kimberley A.; Hink, Ashley
(2010): Health Literacy Universal Precautions Toolkit. Rockville, MD. Agency for Healthcare
Research and Quality
Gui, Xinning; Chen, Yunan; Pine, Kathleen H. (2018): Navigating the Healthcare Service “Black Box”:
Individual Competence and Fragmented System. In: Proceedings of the ACM on Human-Com-
puter Interaction 2/CSCW:Article 61
Katona, Cintia; Bíró, Éva; Kósa, Karolina (2021): Nonprofessional Health Workers on Primary Health
Care Teams in Vulnerable Communities. In: The Annals of Family Medicine 19/3:277-277-
277
Nowak, Peter; Sator, Marlene (2017): Verbesserung der Gesprächsqualität in der Krankenversor-
gung. Eine Strategie zur Etablierung einer patientenzentrierten Kommunikationskultur - die
mögliche Rolle der Balintarbeit. 13 Fachtagung der Österreichischen Balintgesellschaft. Salz-
burg, 1. April 2017
Rudd, Rima E; Anderson, Jennie E. (2010): The Health Literacy Environment Activity Packet. First
Impressions & A Walking Interview.
Rudd, Rima E.; Anderson, Jennie E. (2006): The health literacy environment of hospitals and health
centers. Health and Adult Literacy and Learning Initiative, Harvard School of Public Health,
Boston
Figures and Tables
Figures
Figure 1.1: The HLS-EU Conceptual Model of Health Literacy ...................................................... 5
Figure 5.1: Percentages of respondents who responded with “very difficult” or “difficult” to
the HLS19-Q12 items (ordered by the overall mean), for each country .................... 91
Figure 5.2: Average Percentage Response Patterns (APRP) for the HLS19-Q12 item set, for
each country and the mean for all countries (equally weighted) .............................. 92
Figure 5.3: Histograms of the HLS19-Q12 scores, for all countries ......................................... 102
Figure 5.4: Percentage of respondents by categorical level of General HL as measured by

the HLS19-Q12, for each country and the mean for all countries ........................... 107
Figure 5.5: Average self-perceived health by level of General HL, for each country and the
mean for all countries .......................................................................................... 109
Figure 6.1: Spearman correlations (ρ) between GEN-HL and selected determinants, for all
countries (equally weighted)* ............................................................................... 115
Figure 6.2: Percentage distribution of gender, for each country and for all countries (equally
weighted) ............................................................................................................ 117
Figure 6.3 Means of GEN-HL scores by gender, for each country and for all countries (equally
weighted) ............................................................................................................ 118
Figure 6.4: Percentage distribution of age groups in seven categories, for each country and
for all countries (equally weighted)* ..................................................................... 119
Figure 6.5 Means of GEN-HL scores by age in seven groups, for each country and for all
countries (equally weighted)................................................................................. 121
Figure 6.6: Percentage distribution of level in society, for each country and for all countries
(equally weighted)................................................................................................ 122
Figure 6.7: Means of GEN-HL scores by level in society, for each country and for all countries
(equally weighted)................................................................................................ 123
Figure 6.8: Percentage distribution of educational levels (four grouped categories), for each
country and for all countries (equally weighted).................................................... 125
Figure 6.9: Means of GEN-HL scores by education level (four grouped categories), for each
country and for all countries (equally weighted).................................................... 126
Figure 6.10.: Percentage distribution of financial deprivation levels, for each country and for
all countries (equally weighted) .......................................................................... 127
Figure 6.11: Means of GEN-HL scores by financial deprivation level, for each country and for
387
Figure 6.12: Percentage distribution of migration background, for each country and for all
countries (equally weighted)............................................................................... 130
Figure 6.13: Means of GEN-HL scores by migration background, for each country and for
Figure 6.14: Percentage distribution of the number of long-term illnesses/health problems,

for each country and for all countries (equally weighted) .................................... 132
Figure 6.15: Means of GEN-HL scores by the number of long-term illnesses/health problems,
for each country and for all countries (equally weighted) .................................... 133
Figure 6.16: Percentage distribution of training in a healthcare profession, for each country
and for all countries (equally weighted) .............................................................. 134
Figure 6.17: Mean of GEN-HL scores by training in a healthcare profession, for each country
and for all countries (equally weighted) .............................................................. 135
Figure 7.1: Spearman correlations (ρ) among indicators of health behaviors and lifestyles,
with GEN-HL, and selected socio-demographic and socio-economic
determinants, for all countries (equally weighted)* ............................................... 151
Figure 7.2: Percentage distribution of six categories of BMI by GEN-HL (10 groups from
lowest HL to highest HL), for all countries (equally weighted) ................................ 152
Figure 7.3: Percentage distribution of four categories of smoking behavior by GEN-HL

(10 groups from lowest HL to highest HL), for all countries (equally weighted) ...... 155
Figure 7.4: Percentage distribution of four categories of alcohol consumption by GEN-HL

Figure 7.5: Percentage distribution of four categories of physical activity by GEN-HL

Figure 7.6: Percentage distribution of four categories of fruit and vegetable consumption
by GEN-HL (10 groups from lowest HL to highest HL), for all countries (equally
weighted) ............................................................................................................ 161
Figure 8.1: Spearman correlations (ρ) among indicators of health status, with GEN-HL, and
selected socio-demographic and socio-economic determinants, for all countries
(equally weighted)* .............................................................................................. 168
Figure 8.2: Percentage distribution of five categories of self-perceived health by GEN-HL

(10 groups from lowest HL to highest HL), for all countries (equally weighted) ..... 169
Figure 8.3: Percentage distribution of three categories of long-term illnesses/health

problems by GEN-HL (10 groups from lowest HL to highest HL), for all
countries (equally weighted)* .............................................................................. 171
Figure 8.4: Percentage distribution of three categories of limitations due to health problems
weighted) ........................................................................................................... 174
Figure 9.1: Spearman correlations (ρ) among indicators of health care utilization, with
GEN-HL, and selected socio-demographic and socio-economic determinants, for all
countries (equally weighted)* .............................................................................. 182
Figure 9.2: Percentage distribution of four categories of utilization of emergency services

weighted) ........................................................................................................... 184
Figure 9.3: Percentage distribution of four categories of utilization of GPs/family doctors by

GEN-HL (10 groups from lowest HL to highest HL), for all countries (equally
weighted) ........................................................................................................... 186
Figure 9.4: Percentage distribution of four categories of utilization of medical or surgical

specialists by GEN-HL (10 groups from lowest HL to highest HL), for all
countries (equally weighted)................................................................................ 190
Figure 9.5: Percentage distribution of four categories of utilization of inpatient hospital

services by GEN-HL (10 groups from lowest HL to highest HL), for all countries
(equally weighted)............................................................................................... 192
Figure 9.6: Percentage distribution of four categories of utilization of day patient hospital
services by GEN-HL (10 groups from lowest HL to highest HL), for all countries
(equally weighted)............................................................................................... 195
Figure 10.1: Steps in the development of the HLS19-NAV (in accordance with
Griese et al., 2020, p. 3) .................................................................................... 205
Figure 10.2: Percentages of respondents who responded with “very difficult” or “difficult”
to the HLS19-NAV items (ordered by the overall mean), for each country ............. 208
Figure 10.3: Average Percentage Response Patterns (APRP) for the response categories
“very difficult”-“difficult”-“easy”-“very easy” of the 12 HLS19-NAV items,
for each country and the mean of all countries (equally weighted) ...................... 209
Figure 11.1: Steps in the development of the instrument to measure HL-COM ........................ 239
to the HLS19-COM-P-Q6 items (ordered by the overall mean),
for each country ................................................................................................ 243
Figure 11.4: Average Percentage Response Patterns (APRP) for the item set of the
HLS19-COM-P-Q11, for each country and the mean of all countries (equally
weighted) .......................................................................................................... 244
Figure 11.5: Average Percentage Response Patterns (APRP) for the item set of the
HLS19-COM-P-Q6, for each country and the mean for all countries (equally
weighted) .......................................................................................................... 245
Figure 11.6: Percentage distribution of five categories of self-perceived health by

HL-COM Q11 (7 groups from lowest HL to highest HL), for all countries
(equally weighted) .............................................................................................. 263
389
Figure 11.7: Percentage distribution of five categories of self-perceived health by
HL-COM Q6 (7 values from lowest HL to highest HL), for all countries
to the HL-DIGI items (ordered by the overall mean), for each country and
the mean for all countries (equally weighted)...................................................... 282
Figure 12.2: Average Percentage Response Patterns (APRP) for the HL-DIGI scale, for each
country and mean of all countries (equally weighted).......................................... 284
Figure 12.3: Average Percentage Response Patterns (APRP) for an index of the two items on
interaction with digital devices (HL-DIGI-INT), for each country and mean of all
Figure 12.4: Histograms of the distribution of the HL-DIGI scores, for each country ............... 292
Figure 12.5: Spearman correlations (ρ) among HL-DIGI, GEN-HL, and selected socio-
demographic and socio-economic determinants, for all countries
equally weighted) ............................................................................................... 296
Figure 12.6: Spearman correlations (ρ) among indicators of health status, with HL-DIGI,
GEN-HL, and selected socio-demographic and socio-economic determinants,
Figure 12.7: Spearman correlations (ρ) among indicators of health care utilization, with
HL-DIGI, GEN-HL, and selected socio-demographic and socio-economic
determinants, for all countries measuring digital HL (equally weighted) .............. 304
to the HLS19-VAC items (ordered by the overall mean), for each country ............ 323
Figure 13.2: Average Percentage Response Patterns (APRP) across all Vaccination HL items,
for each country and the mean for all countries (equally weighted) ..................... 324
Figure 14.1: The relationship between GEN-HL on the x-axis, and health-related quality of
life (EQ-5D-5L) on the y-axis for DK, IE, NO. The graphs are predictive margin
plots. They are based on the models presented in Table 14.2, and all the other
variables in the models are kept constant on their average. ................................ 353
Figure 14.2: The relationship between GEN-HL on the x-axis, and health-related quality
life (EQ-5-5LD) on the y-axis, shown for different levels of education in Denmark.
The graphs are predictive margin plots. They are based on the Danish model
presented in Table 14.2. The estimates are computed keeping all the other
variables in the model constant at their average. The vertical lines represent
the 95% confidence intervals for the point estimates .......................................... 354
Figure 14.3: The relationship between GEN-HL on the x-axis, and health-related quality of
life (EQ-5D) on the y-axis, shown for females and males in Ireland. The graphs are
predictive margin plots. They are based on the Irish model presented in
Table 14.2 and all the other variables in the model are kept constant at their
average. The vertical lines represent the 95% confidence intervals for
point estimates. ................................................................................................. 355
Figure 14.4: The relationship between GEN-HL on the x-axis, and health-related quality
of life (EQ-5D-5L) on the y-axis, shown for unemployed and employed
individuals in Ireland. The graphs are predictive margin plots. They are
based on the Irish model presented in Table 14.2, where all the other
in the model are kept constant at their average. The vertical lines represent
the 95% confidence intervals for the point ........................................................ 356
Figure 14.5: The relationship between GEN-HL on the x-axis, and days of absence per
year due to poor health on the y-axis. The graphs are predictive margin
plots. They are based on the country models presented in Table 14.3, and
all the other variables in the model are kept constant on their average ............. 358
Tables
Table 1.1: Matrix of subdimensions of HL based on the HLS-EU Conceptual Model

(Sørensen et al. 2012) used for developing the HLS19 instruments ............................. 6
Table 2.1: Overview of the different processes used by each country to translate the
instruments ............................................................................................................ 29
Table 2.2: Overview of the different processes used by each country to translate the
instruments (continued from Table 2.1) .................................................................. 30
Table 2.3: Overview of the different field-testing procedures used in countries where
CAPI/PAPI was the main method of data collection .................................................. 32
CATI was the main method of data collection .......................................................... 32
CAWI was the main method of data collection ......................................................... 33
mixed methods were the main method of data collection ....................................... 33
Table 2.7: Details of survey sampling and response rates in the countries using
CAPI/PAPI as the main method of data collection ..................................................... 35
CATI as the main method of data collection............................................................. 36
CAWI as the main method of data collection ............................................................ 37
Table 2.10: Details of survey sampling and response rates in the countries using mixed
methods for data collection.................................................................................. 39
Table 2.11: Details of study population and representativeness in countries using

CAPI/PAPI as the main method of data collection .................................................. 41
391
CATI as the main method of data collection .......................................................... 42

CAWI as the main method of data collection ......................................................... 43
Table 2.14: Details of study population and representativeness in countries using mixed
Table 2.15: Details of fieldwork in countries using CAPI/PAPI as the main method of data
collection ............................................................................................................. 46
Table 2.16: Details of fieldwork in countries using CATI as the main method of data
collection ............................................................................................................. 46
Table 2.17: Details of fieldwork in countries using CAWI as the main method of data
collection ............................................................................................................. 47
Table 2.18: Details of fieldwork in countries using mixed methods for data collection ............. 48
Table 2.19: Details of data quality checks and data weighting in countries using
CAPI or PAPI as the main method of data collection .............................................. 49
CATI as the main method of data collection .......................................................... 50
CAWI as the main method of data collection ......................................................... 50
Table 2.22: Details of data quality checks and data weighting in countries using mixed
Table 2.23: Details of ethical approval, compliance with data protection rules, and informed
consent ............................................................................................................... 52
Table 2.24: Details of ethical approval, compliance with data protection rules, and informed
consent (continued from Table 2.23) .................................................................... 52
Table 3.1: Overview of the HLS19 instruments including core and optional parts/items ............. 64
Table 3.2: The HLS19-Q12 instrument, item numbers in the Q12 and Q47,
and item wording................................................................................................... 65
Table 3.3: Categorization of the items in the HLS19-Q12 (using the item numbers for the
Q12 and Q47) in relation to the conceptual model of HL
according to SØrensen et al. (2012)........................................................................ 66
Table 3.4: Overview of the implementation of the HLS19 instruments, correlates,

and additional optional items by country ................................................................. 67
Table 4.1: Levels of person factors used for the analysis of differential item
functioning (DIF) ..................................................................................................... 84
Table 5.1: Percentages of respondents in each country who responded with “very difficult” or
“difficult” to the HLS19-Q12 items (ordered by the mean for the row), for each
country and the mean for all countries (equally weighted) ....................................... 90
Table 5.2: Cronbach’s alpha for the HLS19-Q12, for each country and the mean for all
countries (equally weighted).................................................................................... 93
Table 5.3: Fit indices for the CFA for the HLS19-Q12, for each country and the mean for
all countries (equally weighted) ............................................................................... 95
Table 5.4: Standardized parameter estimates for the HLS19-Q12 score in a single-factor CFA,
for each country and the mean for all countries (equally weighted) .......................... 96
Table 5.5: Overall analyses for the HLS19-Q12, for each country and data collection method
or “mode” ............................................................................................................... 98
Table 5.6: Pearson correlation of the HLS19-Q12 and HLS19-Q16 scores with the
HLS19-Q47 scores for BG, DE, IE, IT, NO, and SI and the mean for these
countries .............................................................................................................. 100
Table 5.7: Pearson correlation of the HLS19-Q16 scores with the HLS19-Q12 scores for
BE, BG, CZ, DE, DK, FR, HU, IE, IL, IT, NO, RU, SI, and SK, and the mean
correlation across these countries ......................................................................... 101
Table 5.8: Means, standard deviations, and percentiles for the HLS19-Q12 score,
by country ............................................................................................................ 103
Table 5.9: Differences in mean HLS19-Q12 scores between the country sample and selected
vulnerable subpopulations, for each country and the mean for all countries
(equally weighted)................................................................................................. 105
Table 5.10: Percentage of difficulty of each item for each level of General HL, for all
countries (equally weighted).................................................................................. 108
Table 6.1: Spearman correlations (ρ) between GEN-HL and selected determinants, for each
country and for all countries (equally weighted)..................................................... 116
Table 6.2: Means, standard deviations, and percentiles of age distributions, for each
Table 6.3: Means, standard deviations, and percentiles of distribution of levels in society,
for each country and for all countries (equally weighted) ....................................... 123
Table 6.4: Means, standard deviations, and percentiles of distribution of educational levels
(nine ISCED categories*), for each country and for all countries
Table 6.5: Model 1: linear regression models of GEN-HL by five core social determinants
(Multivariable standardized coefficients (β) and R2), for each country and
for all countries (equally weighted)........................................................................ 137
393
Table 6.6: Model 2: Multivariable linear regression models of GEN-HL by five core social
determinants and migration background (standardized coefficients (β) and R2),
determinants and long-term illness (standardized coefficients (β) and R2),
determinants and training in a healthcare profession (standardized coefficients
(β) and R2), for each country and for all countries (equally weighted) ..................... 143
Table 7.1: Multivariable linear regression models of BMI by GEN-HL and five core social
determinants (standardized coefficients (β) and R2), for each country and
Table 7.2: Multivariable linear regression models of smoking behavior by GEN-HL and
five core social determinants (standardized coefficients (β) and R2), for each country
and for all countries (equally weighted) ................................................................. 156
Table 7.3: Multivariable linear regression models of alcohol consumption by GEN-HL and five
core social determinants (standardized coefficients (β) and R2), for each country
and for all countries (equally weighted) ............................................................ 158
Table 7.4: Multivariable linear regression models of physical activity by GEN-HL and five core
social determinants (standardized coefficients (β) and R2), for each country
and for all countries (equally weighted) ............................................................ 160
Table 7.5: Multivariable linear regression models of fruit and vegetable consumption by
GEN-HL and five core social determinants (standardized coefficients
(β) and R2), for each country and for all countries (equally weighted) ..................... 162
Table 8.1: Multivariable linear regression models of self-perceived health by GEN-HL and
five core social determinants (standardized coefficients (β) and R2), for each
Table 8.2: Multivariable linear regression models of long-term illnesses/health problems

by GEN-HL and five core social determinants (standardized coefficients (β)
and R2), for each country and for all countries (equally weighted) .......................... 173
Table 8.3: Multivariable linear regression models of limitations due to health problems by
GEN-HL and five core social determinants (standardized coefficients (β) and R2),
Table 9.1: Multivariable linear regression models of utilization of emergency services by

each country and for all countries (equally weighted)............................................. 185
Table 9.2: Multivariable linear regression models of utilization of GPs/family doctors by

Table 9.3: Multivariable linear regression models of utilization of medical or surgical specialists
by GEN-HL and five core social determinants (standardized coefficients (β) and R2),
Table 9.4: Multivariable linear regression models of utilization of inpatient hospital services by
Table 9.5: Multivariable linear regression models of utilization of day patient hospital services
by GEN-HL and five core social determinants (standardized coefficients (β) and R2),
Table 10.1: Fit indices for the HLS19-NAV single-factor CFA, for each country and the mean
Table 10.2: Standardized Parameter Estimates, for each country and the mean for all countries
Table 10.3: Cronbach’s alpha and the Person Separation Index (PSI) for HLS19-NAV, for each
country and the mean of all countries (equally weighted) ....................................... 213
Table 10.4: Pearson correlation between HL-NAV and other HL scores used in the HLS19, for each
country and the mean for all countries (equally weighted) .................................. 214
Table 10.5: Means, standard deviations, quartiles, for HL-NAV, for each country and the mean
for all countries (equally weighted) ..................................................................... 214
Table 10.6: Deviation of Navigational HL mean scores for potentially vulnerable subpopulations
relative to the total mean score of the country, for each country and the mean for
Table 10.7: Spearman correlations between Navigational HL and selected determinants, for
each country and for all countries (equally weighted).......................................... 216
Table 10.8: Multivariable linear regression models of Navigational HL by five core social
determinants (standardized coefficients (β) and R2), for each country and for all
Table 10.9: Multivariable linear regression models of Navigational HL by five core social
determinants and GEN-HL (standardized coefficients (β) and R2), for each country
and for all countries (equally weighted) ............................................................... 218
Table 10.10: Spearman correlations (ρ) of Navigational HL with five indicators for health care
utilization, for each country and for all countries (equally weighted) ................... 219
Table 10.11: Multivariable linear regression models of utilization of GPs/family doctors by

Navigational HL and five social determinants (standardized coefficients
(β) and R2), for each country and for all countries (equally weighted) .................. 220
Table 10.12: Multivariable linear regression models of utilization of medical or surgical

specialists by Navigational HL and five social determinants (standardized
coefficients (β) and R2), for each country and for all countries
395
Table 10.13: Spearman correlations (ρ) of Navigational HL with three indicators for health
status, for each country and for all countries (equally weighted) ....................... 222
Table 10.14: Multivariable linear regression models of self-perceived health (standardized

coefficients (β) and R2) by Navigational HL and five core social determinants,
for each country and for all countries (equally weighted) .................................. 223
Table 10.15: Multivariable linear regression models of long-term illnesses/health problems by

Navigational HL and five core social determinants (standardized coefficients
(β) and R2), for each country and for all countries (equally weighted)................. 224
Table 10.16: Multivariable linear regression models of limited in activities due health problems
(by Navigational HL and five core social determinants standardized coefficients
(β) and R2), for each country and for all countries (equally weighted) ................ 225
Table 11.1: Overview of the main communicative practices of health professionals in the
Calgary-Cambridge Guide to the Medical Interview (C-CG) and the main
communicative tasks of patients, which together constitute the Conceptual
Framework for Communicative HL .................................................................... 237
Table 11.2: Final list of items for the HLS19-COM-P (the short form Q6 is on a grey
background) .................................................................................................... 239
Table 11.3: Countries applying the optional package on Communicative HL ........................... 241
Table 11.4: Fit indices for the one-factor model (CFA) of the HLS19-COM-P-Q11 (left) and
the HLS19-COM-P-Q6 (right), for each country ................................................. 247
Table 11.5: Standardized parameter estimates for the HLS19-COM-P-Q11 (left) and the
HLS19-COM-P-Q6 (right), for each country and the mean for all countries
(equally weighted) ............................................................................................ 248
Table 11.6: Cronbach’s alpha and the Person Separation Index for the HLS19-COM-P-Q11
(left) and the HLS19-COM-P-Q6 (right), for each country and the mean for
all countries (equally weighted). ....................................................................... 250
Table 11.7: Pearson correlations between the HLS19-COM-Q11 (left) and the HLS19-COM-P-Q6
(right) scores and general and other specific HL scores, for each country and
the mean for all countries (equally weighted).................................................... 252
Table 11.8: Means, standard deviations, median and the 25th percentile for the HLS19-COM-P-
Q11 (left) and the HLS19-COM-P-Q6 (right) scores, for each country and the
mean for all countries (equally weighted) ......................................................... 254
Table 11.9: Differences between the total mean score and the score for potentially vulnerable
subpopulations for the HLS19-COM-Q11 (left) and the HLS19-COM-Q6 (right),
for each country and the mean for all countries (equally weighted) ................... 256
Table 11.10: Spearman correlations (ρ) of the HLS19-COM-Q11 (left) and the HLS19-COM-Q6
(right) scores with General HL and HL determinants, for each country and the
mean for all countries (equally weighted) ......................................................... 258
Table 11.11 : Model 1: Multivariable linear regression models of HL-COM by five core social
determinants (standardized coefficients (β) and R2), for each country and for all
countries (equally weighted) (Results based on the HLS19-COM-Q11 score to the
left and the HLS19-COM-Q6 score to the right). ................................................ 260
Table 11.12: Model 2: Multivariable linear regression models of HL-COM by GEN-HL and five
core social determinants (standardized coefficients (β) and R2), for each country
and for all countries (equally weighted). (Results based on the HLS19-COM-Q11
score to the left and the HLS19-COM-Q6 score to the right). ............................. 261
Table 11.13: Spearman correlation coefficients and standardized coefficients (ß) and R2 for two
linear regression models, for each country and for all countries (equally weighted),
illustrating the hypothesized association between HL-COM and self-perceived
health (dependent variable) .............................................................................. 265
“difficult” to the HL-DIGI items (ordered by the mean for All), for each country and
the mean for all countries (equally weighted).................................................... 283
“difficult” to the two items on interaction with digital devices (HL-DIGI-INT)
(ordered by the mean for the All), for each country and the mean for all countries
Table 12.3: Cronbach’s alphas for the HL-DIGI scale, for each country and mean of all countries
Table 12.4: Fit indices for the one-factor confirmatory factor model with the eight HL-DIGI
items as indicators, for each country and mean of all countries
Table 12.5: Pearson correlation (r) between HL-DIGI score and the specific scales measuring HL
in HLS19, for each country and all countries (equally weighted) .......................... 290
Table 12.6: Means, standard deviations, medians, and quartiles of the HL-DIGI scores, for each
country and for all countries (weighted equally)................................................ 291
Table 12.7: Differences in mean HLS19-DIGI score between the country sample and selected
vulnerable subpopulations, for each country and the mean for all countries
Table 12.8: Percentage distributions of responses to use of digital resources, for all countries
Table 12.9: Means, standard deviations, medians, and quartiles of the average digital
use score, for each country and for all countries (equally weighted) .................. 295
Table 12.10: Model 1: Multivariable linear regression models of HL-DIGI by five social
determinants (standardized coefficients (β) and R2), for each country and for
all countries (equally weighted) ........................................................................ 299
397
Table 12.11: Model 2: Multivariable linear regression models of HL-DIGI by five social
determinants and use of digital resources (standardized coefficients (β) and R2),
for each country and for all countries (equally weighted) .................................. 300
Table 13.1: Countries applying the optional package on Vaccination HL or only the measure on
Vaccination HL ..................................................................................................... 320
Table 13.2: Fit indices for the one-factor confirmatory factor model with the four Vaccination HL
items as indicators, for each country ................................................................... 326
Table 13.3: Standardized factor loadings when the four dichotomized Vaccination HL items
load onto a single factor, for each country .......................................................... 327
Table 13.4: Cronbach’s alphas for the 4-item Vaccination HL scale, for each country and the
mean for all countries (equally weighted) ............................................................ 327
Table 13.5: Distribution of the Vaccination HL score, for each country and the mean for all
Table 13.6: Mean Vaccination HL standardized scores for potentially vulnerable/disadvantaged

subpopulations, for each country and all country (equally weighted) .................... 330
Table 13.7: Model 1: Multivariable linear regression models of Vaccination HL by five core
social determinants (standardized coefficients (β) and R2), for each country
and for all countries (equally weighted) ............................................................... 332
Table 13.8: Correlations between the Vaccination HL score and selected vaccination correlates
considered in the HLS19 optional package on vaccination, for each country and
for all countries (equally weighted) ...................................................................... 333
Table 13.9: Odds ratios of a multivariable logistic regression model on being vaccinated within
the last five years (vaccination behavior) with HL-VAC, gender, age, education, level
in society, financial deprivation, and trained in a health profession as determinants,
for each country and for all countries (equally weighted) ..................................... 335
Table 13.10: Odds ratios of a multivariable logistic regression model on being vaccinated within
the last five years (vaccination behavior) with Vaccination HL, gender, age,
education, level in society, financial deprivation, trained in a health profession, and
specific vaccination correlates as determinants, for each country and for all
Table 14.1: Independent variables used to explain variance in the dependent outcome
variables ............................................................................................................ 351
Table 14.2: Multiple linear regression with health-related quality of life (EQ-5D-5L) as the
dependent variable (unstandardized coefficients (b) and R2), for Denmark (DK),
Ireland (IE) and Norway (NO). Variance is estimated by using robust estimation.
Cells are empty because the variables were not part of the regression models. ... 352
Table 14.3: Multiple linear regression of absence from work due to health problems as the
dependent variable (unstandardized coefficients (b) and R2), for Denmark (DK),
Ireland (IE) and Norway (NO). Cells are empty because the variables were not
part of the regression models. ........................................................................... 357
Table 15.1: Average percentage shares for “very difficult” and “difficult” answers to the
different HL-related tasks*.................................................................................. 373
Table 15.2: Tasks in relation to the mass media that were experienced as “difficult” or
“very difficult” by more than 25% of respondents ................................................. 375
399

HLS19 International Report

Uploaded by

Copyright:

Available Formats

HLS19 International Report

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

HLS19 International Report

Uploaded by

Copyright:

Available Formats

International Report

on the Methodology, Results, and

The HLS19 Consortium of the WHO Action Network M-POHL

Authors: The HLS19 Consortium of the WHO Action Network M-POHL

Vienna, December 2021

ICC team members who contributed to the HLS19 Project

National HLS19 team members who contributed to the HLS19 Project

Further M-POHL members who contributed to the HLS19 Project

Working group leads of Optional Packages

Online publication: https://m-pohl.net/Results

Health Literacy cannot be separated from the development of health

© provided by Ilona Kickbusch

The publication of this Health Literacy Survey (HLS19) report is timely.

Low levels of health literacy among population groups are a chal-

Health literacy is an important element of the WHO Regional

Hans Henri P. Kluge, MD, WHO Regional Director for Europe

Health literacy is important for health and health equity. Therefore,

The recent European health literacy survey HLS19 with 17 participating

© BKA / Andy Wenzel 1

Short Summary VII

Health literacy survey, HLS19, recommendations, health literacy measurement

2.2 The HLS19 Instruments (Chapter 3)

2.3 Methods of Data Analyses (Chapter 4)

Scores were calculated for the following HL measures:

HLS19-Q12 General Health Literacy (12 items)

3. Key Results for General Health Literacy

3.1.1 Difficulty of Individual Items

Executive Summary XIII

3.1.3 Distributions of Scores and APRPs

3.1.4 Disadvantaged/Vulnerable Subpopulations

3.3 Social Gradient and Determinants (Chapter 6)

Models including additional predictors (respondents’ migration background, long-term illness,

3.3 Consequences for Health-Related Outcome Indicators

3.3.1 Health Behaviors and Lifestyles (Chapter 7)

3.3.2 Health Status (Chapter 8)

3.3.3 Extent of Healthcare Services Utilization (Chapter 9)

Executive Summary XVII

4.1 Navigational HL (Chapter 10)

4.1.2 Definition and Instrument

4.1.3 Data Collection

4.1.4 Difficulty of Individual Items

4.1.5 Construction of Scores, Validation, and Psychometric Properties

4.1.6 Distributions of Scores and APRPs

4.1.7 Disadvantaged/Vulnerable Subpopulations

4.1.8 Social Gradient and Determinants

4.1.9 Consequences for Health-Related Outcome Indicators

Executive Summary XIX

4.1.10 Discussion and Conclusions

4.2 Communicative HL with Physicians (Chapter 11)

4.2.2 Definition and Instrument

A comprehensive definition of Communicative HL is provided: “Communicative HL refers to pa-

4.2.3 Data Collection

4.2.5 Construction of Scores, Validation, and Psychometric Properties

4.2.6 Distributions of Scores and APRPs

4.2.7 Disadvantaged/Vulnerable Subpopulations

4.2.8 Social Gradient and Determinants

Executive Summary XXI

In most countries Communicative HL was a significant predictor of self-perceived health, while in

4.2.10 Discussion and Conclusions

4.3 Digital HL (Chapter 12)