JPM 13 01610
JPM 13 01610
JPM 13 01610
Personalized
Medicine
Systematic Review
Quality of Life and Audiological Benefits in Pediatric Cochlear
Implant Users in Romania: Systematic Review and Cohort Study
Gina Gundacker 1, *, Delia Emilia Trales 2 and Horatiu Eugen Stefanescu 2
1 Life Science Department, University of Applied Sciences Technikum Vienna, 1200 Vienna, Austria
2 Department of Otolaryngology, Victor Babes, Medical and Pharmaceutical University,
300041 Timis, oara, Romania; [email protected] (D.E.T.); [email protected] (H.E.S.)
* Correspondence: [email protected]
Abstract: Profound sensorineural hearing loss (SNHL) can be successfully treated with a cochlear
implant (CI), and treatment is usually accompanied by increased quality of life (QoL). Therefore, the
aim of this study was to investigate generic and health-related QoL, as well as the level of audiological
outcomes, of CI users, in addition to whether Qol can be restored to the extent of those with normal
hearing. Furthermore, different implantation timepoints were compared (early vs. late), and a
possible correlation between health and generic QoL questionnaires was investigated. The outcomes
from 93 pediatric CI users from Romania were analyzed in the study. Two QoL questionnaires (SSQ12,
AQoL-6D), as well as the HSM sentence test and Soundfield measurements, were assessed. The
outcomes revealed that the CI users were able to achieve the same QoL as their age- and-gender
matched peers with normal hearing, and hearing was restored with good speech comprehension. No
significant difference between early- and late-implanted children was detected, although a tendency
of a better Word Recognition Score (+10%) in the early-implanted group was discovered. A moderate
and significant correlation between the generic and health-related Qol questionnaire was observed.
Audiological examinations are still the standard practice by which to measure the benefit of any
hearing intervention; nonetheless, generic and health-related QoL should be assessed in order to
provide a full picture of a successful and patient-satisfactory cochlear implant procedure.
Keywords: quality of life; sensorineural hearing loss; cochlear implant; audiometry; rehabilitation
Citation: Gundacker, G.; Trales, D.E.;
Stefanescu, H.E. Quality of Life and
Audiological Benefits in Pediatric
Cochlear Implant Users in Romania:
Systematic Review and Cohort Study.
1. Introduction
J. Pers. Med. 2023, 13, 1610. https:// Hearing loss (HL) is the most common sensory impairment, and, as of today, the
doi.org/10.3390/jpm13111610 only one which can be replaced with an implant [1]. According to the world report on
hearing from 2021, conducted by the World Health Organization (WHO), more than 5%
Academic Editors: Georg Mathias
Sprinzl and Astrid Magele
of the world’s population, i.e., roughly 460 million individuals, have moderate to severe
hearing loss which would influence their everyday lives if left untreated [2]. Of these
Received: 13 October 2023 people, about 34 million children are affected with disabling hearing loss [3]. Hearing-
Revised: 7 November 2023 impaired people suffer from restrictions regarding their participation in daily life [3].
Accepted: 9 November 2023 Furthermore, individuals with communication difficulties have a risk of restricted cognitive
Published: 15 November 2023
skills, including reduced attention and concentration spans as well as lower than average
memory recall [2]. This can be avoided if they receive the right support and treatment as
soon as possible. Disabling hearing loss often also leads to low self-esteem, feelings of
Copyright: © 2023 by the authors.
seclusion, and social isolation, causing depression and anxiety [2,3]. Hence, people who
Licensee MDPI, Basel, Switzerland. are hearing-impaired often report a lower overall quality of life (QoL) [2].
This article is an open access article Depending on the type and severity of hearing impairment, several treatment options
distributed under the terms and are possible, from conventional hearing aids to medical interventions utilizing passive
conditions of the Creative Commons middle ear prostheses, active middle ear or bone conduction implants, and cochlear im-
Attribution (CC BY) license (https:// plants (CI). Cochlear implantation often requires personalized rehabilitation to not only
creativecommons.org/licenses/by/ hear sounds, but to enhance speech perception, speech production, and language devel-
4.0/). opment [3,4]. With the use of a CI, approximately one million partly or completely deaf
individuals worldwide have been able to regain functional hearing [5]. The literature on
children with CIs has demonstrated significant outcomes, especially in the development
and perception of speech [6]. Children who have received cochlear implants at a young
age have developed speech perception skills that allow them to develop fluent spoken
language at a rate resembling that of children with normal hearing [6,7].
The progress and benefit of a cochlear implant can be measured via objective measures
such as pure tone audiometry, which is measured at certain frequencies and sound levels [8].
Also, sentence tests are regularly applied in order to check whether speech comprehension
is developing. The Hochmair–Schulz–Moser (HSM) sentence test is one example of such
an assessment, and was developed especially for CI users, applying everyday words to test
their speech perception [9]. In general, beneficial audiometric measurements do not always
indicate a better QoL [10]. However, since the aim of such interventions is to improve
quality of life, attention should also be focused on how the person subjectively perceives
their new hearing. A vast amount of different QoL questionnaires, validated and translated
into numerous languages, are available and are used to validate the benefit to the patient.
These tests can be taken into account for future treatments and can be compared to other,
similar interventions [11].
Quality of life questionnaires validate individuals’ subjective perception of different
dimensions of life, such as mobility, mental health, sight, hearing, etc. One can distinguish
between generic and disease-specific QoL questionnaires and their respective impacts on
quality of life [10]. The evaluation of quality of life is still not a standard clinical routine
measure to evaluate the benefit of an intervention; thus, the available literature on the
quality of life of CI patients is very heterogenous.
Therefore, the aim of this study was to conduct a systematic literature review on the
generic QoL of CI users, as well as to provide additional audiological data and QoL scores
from a pediatric study cohort of CI users from Romania. Therefore, the following points
were the focus of this paper:
• A systematic review of the current literature regarding the QoL of CI users was
conducted to obtain an overview of the generic QoL questionnaires utilized in the
literature, and to see which questionnaires were employed for pediatric CI users.
• The generic QoL of the Romanian CI user study cohort was assessed, and they were
compared with age- and gender-matched peers with normal hearing.
• Data from the study cohort were collected to assess the level of speech comprehension
and hearing ability of pediatric CI users in Romania using the translated and recorded
HSM sentence test and pure tone audiometry.
• Possible influencing factors, such as early-implanted (≤3 years) CI users versus late-
implanted (>3 years) children, were investigated.
• Finally, possible correlations between the generic and health-related quality of life
questionnaires were assessed.
Table 1. Search terms and hits for the literature search in the PubMed database.
Inclusion Criteria
Subjects of any age, gender, or ethnicity with any hearing loss or single-sided deafness (SSD),
Population
as well as and their legal representatives
Intervention/treatment Cochlear implantation
Comparator Not applicable
Outcomes Generic quality of life scores
All studies with original data, including case series, case–control, and longitudinal studies.
Study design Systematic reviews without original data, as well as acknowledgements, recommendations, or general
topic descriptions were excluded.
Exclusion Criteria
Different device or treatment
Not a clinical study in humans
Other type of hearing loss (not SNHL, MHL, or SSD)
Topic not related to hearing loss or its treatment
Publication lacking sufficient information for evaluation
Overlap of data
The Oxford level of evidence guidelines were considered in order to evaluate the
quality of the included papers [13]. The five levels were defined based on the possible
outcomes/research standards, with level I representing the best and level V representing
the worst reporting standard. In our analysis, for example, we considered the study design,
the number of study participants, the follow-up examinations, etc.
Figure1.1.Flow
Figure Flowdiagram
diagramof of
thethe
study selection
study process
selection according
process to the PRISMA
according guidelines
to the PRISMA [12]
guidelines [12
(search conducted on 05.07.2022). QoL = quality of life.
(search conducted on 05.07.2022). QoL = quality of life.
J. Pers. Med. 2023, 13, 1610 6 of 21
An additional search, specifically for the AQoL with the search terms (cochlear im-
plant *) AND (aqol), was conducted on 28.7.2022/9:00. Two publications were identified,
but only one [15] met the inclusion criteria described in Table 2. Finally, 26 publications
were included for further data extraction.
Table 3. Extracted data regarding the systematic review for adults. All questionnaires refer to the generic quality of life of cochlear implant users before and after
the intervention. If more than one publication is indicated, the values correspond to the mean value of these studies and their standard deviation (SD), unless
otherwise stated. If medians and ranges are specified, they are listed separately. Darker blue fields represent total scores and lighter green or blue fields highlight the
subcategories of the SF-36.
Table 3. Cont.
Table 3. Cont.
Table 3. Cont.
Table 3. Cont.
Table 4. Extracted data regarding the systematic review for children. All questionnaires refer to the generic quality of life of cochlear implant users after the
intervention. If more than one publication is indicated, the values correspond to the mean value of these studies and their standard deviation (SD). If medians and
ranges are specified, they are listed separately. Darker blue fields represent total scores.
Table 4. Cont.
3.1.4.
3.1.4.Quality
QualityEvaluation
Evaluationof
ofExtracted
ExtractedPublications
Publications
To
Toevaluate
evaluatethethequality
qualityof ofthe
theincluded
includedpapers,
papers,thetheOxford
Oxfordlevel
levelofofevidence
evidencewas wasused,
used,
which
whichranges
rangesfrom
fromlevel
levelIItotoV,V, with
with the
thebest
bestrating
ratingcorresponding
correspondingto tolevel
levelI.I.Most
Mostpapers
papers
indicated only level
indicated level IV
IVororIV–V,
IV–V, because
becausethey were
they casecase
were series or case–control
series studies,
or case–control which
studies,
can becan
which seenbeinseen
Figure 2. The prospective
in Figure cross-sectional
2. The prospective study bystudy
cross-sectional IssingbyetIssing
al. [35]etachieved
al. [35]
level III because
achieved long-term
level III because follow up
long-term evaluations
follow with more
up evaluations withthan
more30than
participants, as well
30 participants,
as comparison group data, were provided, and the outcomes were
as well as comparison group data, were provided, and the outcomes were presented presented clearly and
understandably to the reader. In addition, we investigated whether the
clearly and understandably to the reader. In addition, we investigated whether the studies studies had been
financially
had supported,
been financially which waswhich
supported, the case
wasforthe
13case
publications out of 26. Itout
for 13 publications canof also
26.beIt seen
can
in the
also bebar
seenchart in Figure
in the bar chart2 that 18 papers
in Figure did18not
2 that report
papers conflicts
did of interest,
not report only
conflicts four did,
of interest,
and four
only four did,
did not
andprovide
four didthis
notinformation.
provide this information.
Figure
Figure2.2.Quality
Qualityevaluation
evaluationofofthe
theincluded
includedpublications
publicationsofofthe
thesystematic
systematicreview
reviewaccording
accordingto
tothe
the
Oxford
Oxfordlevel
levelof
ofevidence
evidence[13].
[13].Additional
Additionalrepresentation
representationof
offinancial
financialsupport
supportand
andpossible
possibleconflicts
conflicts
of
ofinterest.
interest.
3.2.Data
3.2. DataCollection
Collectionfrom
fromStudy
StudyCohort
CohortininRomania
Romania
3.2.1.Assessment
3.2.1. Assessmentof ofQuality
Qualityof
ofLife
Life(AQoL-6D)
(AQoL-6D)
Figure33represents
Figure representsthe theresults
resultsof
ofthe
thetotal
totalutility
utilityscore
scoreand
and utility
utilitysub-scores
sub-scoresof ofthe
the
AQol-6D questionnaire from the pediatric CI users of this study in
AQol-6D questionnaire from the pediatric CI users of this study in blue. It is important to blue. It is important
to note
note thatthat
thethe
blueblue total
total utility
utility score
score in in
thethe left
left chart
chart ofofFigure
Figure3a3aisiscomposed
composedofofthe thesix
six
dimensions of the right bar chart (b), which provides the utility sub-scores. In general, aa
dimensions of the right bar chart (b), which provides the utility sub-scores. In general,
utilityscore
utility scoreofof00indicates
indicatesno noQoL,
QoL,while
whileaamaximum
maximumscore scoreofof11represents
representsaaperfect
perfectQoL.
QoL.
The results of the individual dimensions of the 74 participants in this
The results of the individual dimensions of the 74 participants in this study ranged in study ranged in utility
score from
utility score 0.71
fromto0.71
0.87.
to In Figure
0.87. 3, the3,highest
In Figure scores
the highest werewere
scores achieved
achievedin the
in relationships
the relation-
(0.87 (0.17)) and pain (0.87 (0.15)) dimensions. However, the participants had the lowest
ships (0.87 (0.17)) and pain (0.87 (0.15)) dimensions. However, the participants had the
results for the sub-scores of mental health (0.71 (0.20)) and coping (0.74 (0.26)).
lowest results for the sub-scores of mental health (0.71 (0.20)) and coping (0.74 (0.26)).
The total utility score of the AQoL-6D of this study was 0.79 (0.17) for the children
The total utility score of the AQoL-6D of this study was 0.79 (0.17) for the children
and adolescents who had been treated with a cochlear implant, which can be seen in the
and adolescents who had been treated with a cochlear implant, which can be seen in the
blue bar in the left bar chart of Figure 3a. Furthermore, in the left bar chart in Figure 3a,
blue bar in the left bar chart of Figure 3a. Furthermore, in the left bar chart in Figure 3a,
norm values for the total utility scores of age- and gender-matched peers with normal
norm values for the total utility scores of age- and gender-matched peers with normal
hearing are presented in green. These norm values were extracted from the publication
hearing are presented in green. These norm values were extracted from the publication by
by Maxwell et al. [43] (represented as a dark green bar), as well as from the study by
Maxwell et al. [43] (represented as a dark green bar), as well as from the study by Haw-
Hawthorne et al. [44] (represented as a light green bar). Both values from the indicated
thorne et al. [44] (represented as a light green bar). Both values from the indicated publi-
publications [43,44] were compared to the total utility score of the AQoL-6D of this study.
cations [43,44] were compared to the total utility score of the AQoL-6D of this study. Nei-
Neither the norm values of Hawthorne et al. [44] (p = > 0.999) nor those from Maxwell
ther
et al.the norm
[43] (p = values of Hawthorne
0.559) were et al. [44] (pa =significant
able to demonstrate > 0.999) nor those from
difference (p <Maxwell et al.
0.05) between
[43] (p = 0.559) were able to demonstrate a significant difference
the total utility score of CI users and children with normal hearing. (p < 0.05) between the total
utility score of CI users and children with normal hearing.
J.J.Pers.
Pers.Med.
Med.2023,
2023,13,
13,xxFOR
FORPEER
PEERREVIEW
REVIEW 12 of
12 of 19
19
J. Pers. Med. 2023, 13, 1610 14 of 21
p = 0.559
p = 0.559
p > 0.999
p > 0.999
1.0 1.0
1.0 1.0
Subscores
Score
UtilitySubscores
0.873 0.870
UtilityScore
Utility
0.4
Total
0.2 0.2
0.2 n = 314 n = 246 n = 74 0.2
n = 314 n = 246 n = 74
0.0 0.0
0.0 0.0 Independent Relation- Mental Coping Pain Senses
normal hearing normal hearing CI-users Independent Relation- Mental Coping Pain Senses
normal hearing normal hearing CI-users Living ships Health
norm values norm values from this study Living ships Health
norm values norm values from this study
from Maxwell et al. from Hawthrone et al.
from Maxwell et al. from Hawthrone et al.
(a)
(a) (b)
(b)
Figure 3. Graphical
Figure Graphical representation of of the total
total utility score
score ((a), blue
blue bar)and and the utility
utility sub-scores
Figure 3.3. Graphicalrepresentation
representation ofthe the totalutility
utility score((a),
((a), bluebar)
bar) and thethe utility sub-scores
sub-scores
(b) on the assessment of quality of life questionnaire, which included
(b) on the assessment of quality of life questionnaire, which included six dimensions (AQoL-6D), six dimensions (AQoL-6D),
(b) on the
for cochlear assessment
cochlear implant of
implant (CI)treatedquality of
(CI)treated childrenlife questionnaire,
children (n (n == 74)
74) of
of thiswhich
this study included
studyAn six
An additional dimensions
additional comparison (AQoL-6D),
comparison with with the
the
for
for cochlear
total utility implant
scores on (CI)treated
the AQoL-6D children
using (nnorm
= 74)values
of this ofstudy
peers Anofadditional
the same comparison
age and genderwithwith
the
total utility scores on the AQoL-6D using norm values of peers of the same age and gender with
total
normal
normal utility scores
hearing
hearing on the
isisalso
also shown,AQoL-6D
shown, obtained
obtained usingfrom
from norm values
thestudies
the studiesof of peers
ofMaxwell
Maxwellof the same
etal.
et al. [43]age
[43] ((a),and
((a), dark
dark gender
green)with
green) and
and
Hawthorne
normal hearing
Hawthorne et al.
et al. is[44]
also
[44] ((a), light
((a),shown, green).
obtained
light green). Significance level:
from thelevel:
Significance studies p < 0.05.
p < of The
Maxwell
0.05. mean
The mean et al.values and
[43] and
values correspond-
((a),correspond-
dark green)
ing standard
and
ing standard
Hawthorne deviations
et al. [44]
deviations inin parentheses
parentheses are displayed
((a), light green).
are displayed in the
Significance
in thelevel:
bars. The
bars. The scoresThe
p < scores
0.05. range
rangemean from
from the worst
values
the worst
and
score of 0,
corresponding
score indicating
standard
of 0, indicating no quality
deviations
no quality of life (QoL),
of lifein(QoL), to
parenthesesthe maximum
are displayed
to the maximum score
score of
inof
the1, indicating
1, bars. a
The scores
indicating perfect
range
a perfect generic
from
generic
QoL.worst score of 0, indicating no quality of life (QoL), to the maximum score of 1, indicating a
QoL.
the
perfect generic QoL.
3.2.2. Speech
3.2.2. Speech Spatial
Spatial Qualities
Qualities of of Hearing
Hearing ScaleScale (SSQ12)
(SSQ12)
3.2.2.The
Speech Spatial
results of of theQualities
the SSQ12 of Hearing
SSQ12 questionnaire
questionnaire forScale (SSQ12)
for 79
79 pediatric
pediatric CI CI users
users are
are shown
shown in in Figure
Figure
The results
4. TheThe individual
results ofscores
the on
SSQ12 the speech
questionnaire spatial
for qualities
79 of
pediatric hearing
CI
4. The individual scores on the speech spatial qualities of hearing test are displayed. The userstest
are are displayed.
shown in Figure The 4.
The
totalindividual
total score is
score scores onin
is presented
presented the
in speech
blue.
blue. Thespatial
The resultsqualities
results ranged from
ranged of
fromhearing
00 to testwith
to 10,
10, are displayed.
with 00 correspondingThe total
corresponding to
to
score
the is
worst presented
result andin blue.
10 to The
the results
best. A ranged
maximum from 0
score toof
the worst result and 10 to the best. A maximum score of 10 points would indicate that10,10with 0
points corresponding
would indicate to the
that
worst
thereis
there isresult
no and 10 toto
nolimitation
limitation tothe best. of
quality
quality A maximum
oflife
lifedue
dueto score
tothe of 10 points
thehearing
hearing wouldIn
impairment.
impairment. indicate
In general,
general, thatthe
the there
mean
mean is
no limitation
values in to
Figure quality
4 are of life
representeddue toasthe hearing
crosses, and impairment.
the mean
values in Figure 4 are represented as crosses, and the mean value of the overall score of In general,
value of the the mean
overall values
score of
in
theFigure
SSQ124for are CI
for represented
users is as crosses,
is 6.05
6.05 and highest
(1.63). The
The the mean value
score ofof the(1.63)
6.43 overallwasscore of the SSQ12
achieved in the
the
the SSQ12 CI users (1.63). highest score of 6.43 (1.63) was achieved in
for CI users
qualities of is 6.05
of hearing (1.63).
hearing section. The
section. Hence,highest
Hence, the score
the lowest of 6.43
lowest score, (1.63)
score, 5.79 was
5.79 (2.19), achieved
(2.19), was
was given in the
given for qualities
for the
the speech
speech of
qualities
hearing
category. section. Hence, the lowest score, 5.79 (2.19), was given for the speech category.
category.
5.793
5.793 5.946
5.946 6.434
6.434 6.049
6.049
(2.186)
(2.186) (2.074)
(2.074) (1.631)
(1.631) (1.634)
(1.634)
10
10
88
points
Meanpoints
66
Mean
44
22
00
Speech
Speech Spatial
Spatial Qualities
Qualities Overall
Overall
Figure
Figure 4.Boxplot
Figure 4.
4. Boxplotpresentation
Boxplot presentation
of the
presentation of
Speech
of the
the Spatial
Speech
Speech
Qualities
Spatial
Spatial
of Hearing
Qualities
Qualities
Scale questionnaire
of Hearing
of Hearing (SSQ12)
Scale questionnaire
Scale questionnaire
outcomes from 79 cochlear
(SSQ12) outcomes
outcomes from 79 implant implant
79cochlear
cochlear users. Median and quartiles
users.Median
Median for the sub-scores
and quartiles
quartiles for the and the and
the sub-scores
sub-scores overall
the
(SSQ12) from implant users. and for and the
overall score (blue) are presented. An additional representation of the mean values are
overall score (blue) are presented. An additional representation of the mean values are shown as shown as
J. Pers. Med. 2023, 13, 1610 15 of 21
score (blue) are presented. An additional representation of the mean values are shown as crosses
inside the boxplots and as numerical values above the boxplots, with their standard deviations in
parentheses. The score, in general, ranges from a worst possible score of 0 to an optimal score of 10.
A maximum score of 10 represents no limitations on quality of life due to hearing impairment.
Table 5. Mean values of the hearing thresholds and their standard deviations (SDs) at the specific test
frequencies
deviations of the pure tone
in parentheses. audiometry
The score, for the
in general, cochlear
ranges from aimplant users ofscore
worst possible this study. The
of 0 to an degrees
optimal
score of 10. Aloss
of hearing maximum score
(HL) were of 10 represents
determined by theno limitations
guidelines of on quality Speech-Language-Hearing-
American of life due to hearing im-
pairment.
Association (ASHA) [46].
Frequency
3.2.3. Pure Hearing Threshold,
Tone Audiometry n Degree of Hearing Loss
SD
(kHz) Mean (dB) (Ears) according to ASHA
Pure tone audiometry refers to the total number of examined ears which were treated
0.25
with a cochlear 24.4
implant (Table 2.62 hearing level
5). The best aided 55 was 24.4 dB, Slight HL
measured at a
frequency0.5 26.3 in (Figure 5,3.47
of 0.25 kHz, as shown Table 5). The56hearing threshold
Mild HL from
ranged
24.4 dB (2.62
1 dB), at a test frequency
24.6 of 0.25 4.06
kHz, to 27.3 dB
55 (1.52 dB) at aSlight
frequency
HL of 6
kHz. 2 24.6 2.34 55 Slight HL
The blue area at the bottom of the audiogram represents the cochlear implant indica-
4 26.8 2.21 51 Mild HL
tion range of the manufacturer (MED-EL [45]) (Figure 5). The hearing thresholds of most
6
CI users improved by at least27.3 1.52
40 dB after implantation. 45 Mild HL
Figure 5. Audiogram with averaged hearing thresholds and standard deviations of the children
Figure 5. Audiogram
and adolescents with
with averaged
cochlear hearing
implants thresholds
(CI). For each and standard
frequency, deviations
data of the
from at least 40children
CI usersand
were
adolescents with cochlear implants (CI). For each frequency, data from at least 40 CI users were
considered, but the exact numbers of participants are noted in Table 5. The blue area shows the CI
considered, but the exact numbers of participants are noted in Table 5. The blue area shows the CI
indication range based on the manufacturer’s recommendation [45].
indication range based on the manufacturer’s recommendation [45].
The blue area at the bottom of the audiogram represents the cochlear implant indica-
Table 5. Mean values of the hearing thresholds and their standard deviations (SDs) at the specific
tion range of the manufacturer (MED-EL [45]) (Figure 5). The hearing thresholds of most
test frequencies of the pure tone audiometry for the cochlear implant users of this study. The degrees
CI users improved by at least 40 dB after implantation.
of hearing loss (HL) were determined by the guidelines of American Speech-Language-Hearing-
Association (ASHA) [46].
Table 6. Results of the Hochmaier–Schulz–Moser (HSM) sentence test for 41 cochlear implant users.
The scores generally ranged from a minimum score of 0% to a maximum score of 100%, indicating
perfect speech comprehension.
Properties Scores
Participants (n) 41
Mean (%) 71.6
Standard deviation (%) 23.8
Minimum (%) 15.1
Maximum (%) 100
Table 7. Comparative values between early (≤3 years) and late (>3 years) implantation timepoints
for cochlear implant users.
Table 8. Correlation characteristics of the total scores of the generic (Assessment of Quality of Life in
six dimensions (AQoL-6D)) and health-related (Speech Spatial Qualities of Hearing Scale (SSQ12))
quality of life. The correlation is significant when the p-value is <0.05.
4. Discussion
From the generic QoL results, it can be concluded that CI users can achieve the
same level of quality of life as their healthy, normal-hearing peers. After comparing
the normative values from two different publication populations [43,44], no significant
difference regarding the QoL was detected between normal-hearing and CI-treated children
and adolescents (p < 0.05). Although the absolute numerical norm values achieved by
Hawthorne et al. [44] (0.870 (0.170)) and Maxwell et al. [43] (0.873 (0.112)) were somewhat
different, a multiple comparison method revealed no significant difference in utility scores
compared to the outcomes of this study (0.794 (0.169)), supporting the statement that
CIs increase the QoL of hearing-impaired people after treatment [4] and confirming that
normal hearing standards can be achieved. A possible influence of the fact that some
questionnaires were answered by or together with the parents due to the age of the child
and more heterogenous outcomes, as reflected by a rather high standard deviation (SD)
such as that reported by the study of Khadka et al., could not be seen in our results, which
achieved a rather low SD of 0.169 [48]. The validated norm values were retrieved from age
groups of 15–19 years in the publication by Hawthorne et al. [44] and 16–24 years in the
study of Maxwell et al. [43].
However, the overall positive outcomes are probably the result of an increase in self-
confidence bolstered by the ability to communicate [6]. As no statistical gender differences
could be found, the outcomes were not split (p > 0.05).
A total of 79 participants achieved a mean overall score of 6.05(1.63) for the health/hearing-
related QoL, assessed by the Speech Spatial Qualities of Hearing Scale (SSQ12), which appears
to be rather low in comparison to other publications [49–52]. Emphasis needs to be placed on
the rather young study cohort, in which speech might not have been as developed due to age,
not due to hearing status. It is also possible that the comparative studies simply presented
the results of small, top-performing groups. For example, the studies by Rauch et al. [49] and
Falcon et al. [50] involved single-sided deaf patients with normal hearing on one side [53].
A moderate and significant correlation between the outcomes of the generic and
health/hearing-specific QoL was detected (Table 8). Unsurprising, the AQoL-6D subdi-
mension of senses, which deals with seeing, hearing, and communication, showed the
best correlation with the SSQ12, again reflecting the importance of hearing on overall
quality of life. Parameters like anxiety, feelings of seclusion, or depression, which very
often accompany untreated hearing loss [2], are often evaluated, and may be eliminated
via treatment.
Even though the literature on AQoL assessments in the population with hearing loss is
sparse, the questionnaire was used as the assessment of choice for utilization in the clinical
process. The reasons were as follows: first, access was free of charge; second, the statistical
software SPSS and STATA provided easy and, therefore, bias-free evaluation tools; and
third, the availability of validated population norms enabled statistical comparisons. The
provided data calculation syntax for STATA and SPSS ensures correct data analysis and,
therefore, better comparability. Thus, no random deviations or obvious calculation errors
occurred, as was found in the published outcomes of several HUI 3 questionnaire studies.
The HUI was often evaluated incorrectly, which unfortunately made the results unusable
for our systematic review and comparisons, especially since a license fee was required to
for this questionnaire.
During the review process of the different QoL studies, it became evident that a
consensus on the application of QoL instruments would be advisable.
Unfortunately, not enough comparable data were available to perform a meta-analysis
(Tables 3 and 4) [1,15,16,22–42,54,55]. Although no age group was excluded from the
literature review, it is apparent that there is limited data available regarding QoL surveys
on children [1,38–42]. This was one of the reasons why this study focused on collecting
data from children and adolescents.
In conclusion, the systematic literature review clearly showed improved QoL after
implantation, comparable to that of healthy, normal-hearing children, but also showed some
J. Pers. Med. 2023, 13, 1610 18 of 21
clear limitations: first, a clear classification of health-related and generic QoL questionnaires
is necessary. Second, uniform and correct assessments are needed in order for comparisons
to be made. Third, more day-to-day evidence is required, which points towards the
necessity to implement QoL questionnaires into clinical routines. More data on the QoL of
the pediatric population are especially required.
The 41 tested CI users achieved impressive speech comprehension, with an overall
score of 71.6% (23.8%) on the HSM sentence test (Table 6). The rather high standard
deviation might be because of overly complex sentences or words, even though the HSM
test includes everyday phrases in order to eliminate this possible bias. Another possibility
might be the concentration level or the unfamiliar environment, or that the children were
simply too shy to repeat the words [56].
The tested CI users showed an improvement of around 40 dB with the use of a cochlear
implant, as the mean aided hearing threshold was between 24.4 dB (2.62 dB) and 27.3 dB
(1.52 dB).
The literature repeatedly points out the importance of early implantation to ensure the
optimal developmental effects [57–59]. Our study group outcomes were split into early- and
late-implanted children (Table 7). Children under 3 years of age at the time of implantation
were considered to fall into the early implantation group, and the rest comprised the
late implantation group. Despite the outcomes not being significantly different, the early
implantation group performed about 10% better than the comparison group on the HSM
sentence test.
In conclusion, both age groups achieved generally good results for all measurements
which were undertaken. These great results are probably related to the support of the
families and their social environments. This assumption is based on the fact that all
examinations, as well as rehabilitation and training (in the camp as well as in the hospital
in Timisoara), were voluntary, and the parents went above and beyond to provide their
children with all the opportunities necessary to further improve their hearing. Obviously,
they were doing everything right.
5. Conclusions
Studies investigating the effect of a CI on different aspects of QOL and audiologic
performance are important in order to provide realistic expectations to our patients, espe-
cially when children are involved. This study has proven that cochlear implant users can
achieve the same generic quality of life as their normal-hearing peers. The audiological
examinations showed a clear benefit after cochlear implantation. Since speech comprehen-
sion is challenging, the good outcomes achieved on the HSM sentence test by the CI users
in Romania were impressive. Even though no significant difference was found between
late and early implantation, for the QoL and audiological parameters examined, the early
implantation group exhibited roughly 10% greater speech understanding compared to the
later implantation cohort. No significant correlation between the generic and health-related
quality of life questionnaires was found. Thus, both should be administered to obtain a
better understanding of children and adolescents treated with cochlear implants and their
hearing/health, as well as their general satisfaction in life.
Author Contributions: Conceptualization, H.E.S.; methodology, H.E.S. and G.G.; validation, G.G.,
D.E.T. and H.E.S.; formal analysis, G.G.; investigation, G.G. and D.E.T.; resources, G.G. and D.E.T.;
data curation, G.G. and D.E.T.; writing—original draft preparation, G.G.; writing—review and editing,
all authors; visualization, G.G.; supervision, H.E.S. All authors have read and agreed to the published
version of the manuscript.
Funding: This research received no external funding.
Informed Consent Statement: Informed consent was obtained from all parents and legal guardians
of the subjects involved in the study.
Conflicts of Interest: The authors declare no conflict of interest.
J. Pers. Med. 2023, 13, 1610 19 of 21
References
1. Alnuhayer, O.; Alshawi, Y.; Julaidan, B.; Alromaih, N.; Alakeel, N.; Alballaa, A. Quality of Life and Cochlear Implant: Results in
Saudi Children. Cureus 2020, 12, e11968. [CrossRef] [PubMed]
2. World Health Organization. World Report on Hearing; World Health Organization: Geneva, Switzerland, 2021; ISBN 978-92-4-
002048-1.
3. World Health Organization. Deafness and Hearing Loss. Available online: https://www.who.int/news-room/fact-sheets/
detail/deafness-and-hearing-loss (accessed on 29 August 2022).
4. Lee, Y.; Sim, H. Bilateral Cochlear Implantation versus Unilateral Cochlear Implantation in Deaf Children: Effects of Sentence
Context and Listening Conditions on Recognition of Spoken Words in Sentences. Int. J. Pediatr. Otorhinolaryngol. 2020, 137, 110237.
[CrossRef] [PubMed]
5. Zeng, F.-G. Celebrating the One Millionth Cochlear Implant. JASA Express Lett. 2022, 2, 077201. [CrossRef] [PubMed]
6. Hirschfelder, A.; Gräbel, S.; Olze, H. The impact of cochlear implantation on quality of life. The role of audiologic performance
and variables. Otolaryngol. Head Neck Surg. 2008, 138, 357–362. [CrossRef] [PubMed]
7. Gatehouse, S.; Noble, W. The Speech, Spatial and Qualities of Hearing Scale (SSQ). Int. J. Audiol. 2004, 43, 85–99. [CrossRef]
8. Bess, F.H.; Humes, L. Audiology: The Fundamentals, 4th ed.; Lippincott Williams & Wilkins: Philadelphia, PA, USA, 2008;
ISBN 978-0-7817-6643-2.
9. Hochmair-Desoyer, I.; Schulz, E.; Moser, L.; Schmidt, M. The HSM Sentence Test as a Tool for Evaluating the Speech Understanding
in Noise of Cochlear Implant Users. Am. J. Otol. 1997, 18, S83.
10. Haraldstad, K.; Wahl, A.; Andenæs, R.; Andersen, J.R.; Andersen, M.H.; Beisland, E.; Borge, C.R.; Engebretsen, E.; Eisemann, M.;
Halvorsrud, L.; et al. A Systematic Review of Quality of Life Research in Medicine and Health Sciences. Qual. Life Res. 2019, 28,
2641–2650. [CrossRef]
11. Real Life Gains: The Importance of Quality of Life|ExpertsONLINE. 2022. Available online: https://www.youtube.com/watch?
v=hQMgpqxWfww&t=2s (accessed on 2 July 2023).
12. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.;
Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ 2021, 372, n71.
[CrossRef]
13. Ball, C.; Phillips, R.; Guyatt, G. OCEBM Levels of Evidence—Centre for Evidence-Based Medicine (CEBM). Available online:
https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (accessed on 24 August 2022).
14. Iezzi, A. Homepage of the Assessment of Quality of Life. Available online: https://www.aqol.com.au/ (accessed on 17 August
2022).
15. Rostkowska, J.; Skarzynski, P.H.; Kobosko, J.; Gos, E.; Skarzynski, H. Health-Related Quality of Life in Adults with Profound
Postlingual Hearing Loss before and after Cochlear Implantation. Eur. Arch. Otorhinolaryngol. 2021, 278, 3393–3399. [CrossRef]
16. Sarant, J.; Harris, D.; Busby, P.; Maruff, P.; Schembri, A.; Dowell, R.; Briggs, R. The Effect of Cochlear Implants on Cognitive
Function in Older Adults: Initial Baseline and 18-Month Follow Up Results for a Prospective International Longitudinal Study.
Front. Neurosci. 2019, 13, 789. [CrossRef]
17. Noble, W.; Jensen, N.S.; Naylor, G.; Bhullar, N.; Akeroyd, M.A. A Short Form of the Speech, Spatial and Qualities of Hearing Scale
Suitable for Clinical Use: The SSQ12. Int. J. Audiol. 2013, 52, 409–412. [CrossRef] [PubMed]
18. ASHA. Sound Field Measurement Tutorial. Available online: https://www.asha.org/policy/rp1991-00025/ (accessed on 1 March
2023).
19. Gates, G.; Hoffman, H. What the Numbers Mean: An Epidemiological Perspective on Hearing. Available online: https://www.nidcd.
nih.gov/health/statistics/what-numbers-mean-epidemiological-perspective-hearing (accessed on 27 February 2023).
20. Herzog, M.; Schön, F.; Müller, J.; Knaus, C.; Scholtz, L.; Helms, J. Langzeitergebnisse nach Cochlear-Implant-Versorgung älterer
Patienten. Laryngorhinootologie 2003, 82, 490–493. [CrossRef] [PubMed]
21. Louw, C.; Swanepoel, D.W.; Eikelboom, R.H. Self-Reported Hearing Loss and Pure Tone Audiometry for Screening in Primary
Health Care Clinics. J. Prim. Care Commun. Health 2018, 9, 2150132718803156. [CrossRef]
22. Anzivino, R.; Conti, G.; Di Nardo, W.; Fetoni, A.R.; Picciotti, P.M.; Marra, C.; Guglielmi, V.; Fortunato, S.; Forli, F.; Paludetti, G.;
et al. Prospective Evaluation of Cognitive Functions After Rehabilitation with Cochlear Implant or Hearing Aids: Preliminary
Results of a Multicentric Study on Elderly Patients. Am. J. Audiol. 2019, 28, 762–774. [CrossRef] [PubMed]
23. Calvino, M.; Sánchez-Cuadrado, I.; Gavilán, J.; Gutiérrez-Revilla, M.A.; Polo, R.; Lassaletta, L. Effect of Cochlear Implantation
on Cognitive Decline and Quality of Life in Younger and Older Adults with Severe-to-Profound Hearing Loss. Eur. Arch.
Otorhinolaryngol. 2022, 279, 4745–4759. [CrossRef]
24. Hey, M.; Wesarg, T.; Mewes, A.; Helbig, S.; Hornung, J.; Lenarz, T.; Briggs, R.; Marx, M.; Ramos, A.; Stöver, T.; et al. Objective,
Audiological and Quality of Life Measures with the CI532 Slim Modiolar Electrode. Cochlear Implant. Int. 2019, 20, 80–90.
[CrossRef]
25. Sivonen, V.; Sinkkonen, S.T.; Willberg, T.; Lamminmäki, S.; Jääskelä-Saari, H.; Aarnisalo, A.A.; Dietz, A. Improvements in
Hearing and in Quality of Life after Sequential Bilateral Cochlear Implantation in a Consecutive Sample of Adult Patients with
Severe-to-Profound Hearing Loss. J. Clin. Med. 2021, 10, 2394. [CrossRef]
26. Sorrentino, T.; Donati, G.; Nassif, N.; Pasini, S.; Redaelli de Zinis, L.O. Cognitive Function and Quality of Life in Older Adult
Patients with Cochlear Implants. Int. J. Audiol. 2020, 59, 316–322. [CrossRef]
J. Pers. Med. 2023, 13, 1610 20 of 21
27. Tang, L.; Thompson, C.B.; Clark, J.H.; Ceh, K.M.; Yeagle, J.D.; Francis, H.W. Rehabilitation and Psychosocial Determinants of
Cochlear Implant Outcomes in Older Adults. Ear Hear. 2017, 38, 663–671. [CrossRef]
28. Forli, F.; Lazzerini, F.; Fortunato, S.; Bruschini, L.; Berrettini, S. Cochlear Implant in the Elderly: Results in Terms of Speech
Perception and Quality of Life. Audiol. Neurootol. 2019, 24, 77–83. [CrossRef]
29. Peters, J.P.M.; van Heteren, J.A.A.; Wendrich, A.W.; van Zanten, G.A.; Grolman, W.; Stokroos, R.J.; Smit, A.L. Short-Term
Outcomes of Cochlear Implantation for Single-Sided Deafness Compared to Bone Conduction Devices and Contralateral Routing
of Sound Hearing Aids-Results of a Randomised Controlled Trial (CINGLE-Trial). PLoS ONE 2021, 16, e0257447. [CrossRef]
30. Chen, S.L.; Chan, K.C.; Hu, C.Y.; Fang, H.Y.; Wu, C.M. Experience of Adult Cochlear Implantation at a Tertiary Hospital. J. Chin.
Med. Assoc. 2022, 85, 469–477. [CrossRef] [PubMed]
31. Peter, N.; Kleinjung, T.; Probst, R.; Hemsley, C.; Veraguth, D.; Huber, A.; Caversaccio, M.; Kompis, M.; Mantokoudis, G.; Senn, P.;
et al. Cochlear Implants in Single-Sided Deafness—Clinical Results of a Swiss Multicentre Study. Swiss Med. Wkly. 2019, 149,
w20171. [CrossRef] [PubMed]
32. Saraç, E.T.; Batuk, M.O.; Sennaroglu, G. Evaluation of the Quality of Life in Adults with Cochlear Implants: As Good as the
Healthy Adults? Am. J. Otolaryngol. 2019, 40, 720–723. [CrossRef] [PubMed]
33. Sousa, A.F.; Couto, M.I.V.; Martinho-Carvalho, A.C. Quality of Life and Cochlear Implant: Results in Adults with Postlingual
Hearing Loss. Braz. J. Otorhinolaryngol. 2018, 84, 494–499. [CrossRef]
34. Issing, C.; Baumann, U.; Pantel, J.; Stöver, T. Cochlear Implant Therapy Improves the Quality of Life in Older Patients—A
Prospective Evaluation Study. Otol. Neurotol. 2020, 41, 1214–1221. [CrossRef]
35. Issing, C.; Holtz, S.; Loth, A.G.; Baumann, U.; Pantel, J.; Stöver, T. Long-Term Effects on the Quality of Life Following Cochlear
Implant Treatment in Older Patients. Eur. Arch. Otorhinolaryngol. 2022, 279, 5135–5144. [CrossRef]
36. Völter, C.; Götze, L.; Dazert, S.; Falkenstein, M.; Thomas, J.P. Can Cochlear Implantation Improve Neurocognition in the Aging
Population? Clin. Interv. Aging 2018, 13, 701–712. [CrossRef]
37. Forli, F.; Turchetti, G.; Giuntini, G.; Bellelli, S.; Fortunato, S.; Bruschini, L.; Barillari, M.R.; Berrettini, S. Cochlear Implant in
Prelingually Deafened Oralist Adults: Speech Perception Outcomes, Subjective Benefits and Quality of Life Improvement. Acta
Otorhinolaryngol. Ital. 2017, 37, 416–422. [CrossRef]
38. Piromchai, P.; Tanamai, N.; Kiatthanabumrung, S.; Kaewsiri, S.; Thongyai, K.; Atchariyasathian, V.; Thanawirattananit, P.;
Wacharasindhu, C.; Mukkun, T.; Isipradit, P.; et al. Multicentre Cohort Study of Cochlear Implantation Outcomes in Thailand.
BMJ Open 2021, 11, e054041. [CrossRef]
39. Hendriksma, M.; Bruijnzeel, H.; Bezdjian, A.; Kay-Rivest, E.; Daniel, S.J.; Topsakal, V. Quality of Life (QoL) Evaluation of Children
Using Cochlear Implants: Agreement between Pediatric and Parent Proxy-QoL Reports. Cochlear Implant. Int. 2020, 21, 338–343.
[CrossRef] [PubMed]
40. Aldriweesh, B.; Alharbi, M.; Alzahrani, M. Correlation of Quality of Life with Speech and Hearing Performance after Pediatric
Cochlear Implantation. Int. J. Pediatr. Otorhinolaryngol. 2021, 150, 110889. [CrossRef] [PubMed]
41. Dev, A.N.; Nahas, G.; Pappas, A.; Ambrose, T.; Craun, P.; Fustos, E.; Reilly, B.K.; Preciado, D. Underinsurance in Children
Is Associated with Worsened Quality of Life after Cochlear Implantation. Int. J. Pediatr. Otorhinolaryngol. 2022, 157, 111119.
[CrossRef]
42. Vermi Sli Peker, S.; Demi, R.K.F.; Cukurova, I. Quality of Life and Parental Care Burden in Cochlear Implanted Children: A
Case-Control Study. Int. J. Pediatr. Otorhinolaryngol. 2020, 136, 110164. [CrossRef] [PubMed]
43. Maxwell, A.; Özmen, M.; Iezzi, A.; Richardson, J. Deriving Population Norms for the AQoL-6D and AQoL-8D Multi-Attribute
Utility Instruments from Web-Based Data. Qual. Life Res. 2016, 25, 3209–3219. [CrossRef] [PubMed]
44. Hawthorne, G.; Osborne, R. Population Norms and Meaningful Differences for the Assessment of Quality of Life (AQoL) Measure.
Aust. N. Z. J. Public. Health 2005, 29, 136–142. [CrossRef] [PubMed]
45. MED-EL. Medical Electronics Indications—MED-EL Pro. Available online: https://www.medel.pro/indications (accessed on 16
April 2023).
46. Clark, J.G. Degree of Hearing Loss. Available online: https://www.asha.org/public/hearing/degree-of-hearing-loss/ (accessed
on 9 August 2022).
47. Schober, P.; Boer, C.; Schwarte, L.A. Correlation Coefficients: Appropriate Use and Interpretation. Anesth. Analg. 2018, 126, 1763.
[CrossRef] [PubMed]
48. Khadka, J.; Kwon, J.; Petrou, S.; Lancsar, E.; Ratcliffe, J. Mind the (Inter-Rater) Gap. An Investigation of Self-Reported versus
Proxy-Reported Assessments in the Derivation of Childhood Utility Values for Economic Evaluation: A Systematic Review. Soc.
Sci. Med. 2019, 240, 112543. [CrossRef]
49. Rauch, A.-K.; Arndt, S.; Aschendorff, A.; Beck, R.; Speck, I.; Ketterer, M.C.; Jakob, T.F.; Hassepass, F. Long-Term Results of
Cochlear Implantation in Children with Congenital Single-Sided Deafness. Eur. Arch. Otorhinolaryngol. 2021, 278, 3245–3255.
[CrossRef]
50. Falcón Benítez, N.; Falcón González, J.C.; Ramos Macías, Á.; Borkoski Barreiro, S.; Ramos de Miguel, Á. Cochlear Implants in
Single-Sided Deafness. Comparison Between Children and Adult Populations with Post-Lingually Acquired Severe to Profound
Hearing Loss. Front. Neurol. 2021, 12, 760831. [CrossRef]
51. Beijen, J.-W.; Snik, A.F.M.; Mylanus, E.A.M. Sound Localization Ability of Young Children with Bilateral Cochlear Implants. Otol.
Neurotol. 2007, 28, 479. [CrossRef] [PubMed]
J. Pers. Med. 2023, 13, 1610 21 of 21
52. Galvin, K.L.; Hughes, K.C.; Mok, M. Can Adolescents and Young Adults with Prelingual Hearing Loss Benefit from a Second,
Sequential Cochlear Implant? Int. J. Audiol. 2010, 49, 368–377. [CrossRef] [PubMed]
53. Cleveland Clinic. Medical Professional Unilateral Hearing Loss (Single-Sided Deafness): Symptoms & Management. Available
online: https://my.clevelandclinic.org/health/diseases/21625-unilateral-hearing-loss-single-sided-deafness (accessed on 12
April 2023).
54. Wick, C.C.; Kallogjeri, D.; McJunkin, J.L.; Durakovic, N.; Holden, L.K.; Herzog, J.A.; Firszt, J.B.; Buchman, C.A. Hearing and
Quality-of-Life Outcomes After Cochlear Implantation in Adult Hearing Aid Users 65 Years or Older: A Secondary Analysis of a
Nonrandomized Clinical Trial. JAMA Otolaryngol. Head Neck Surg. 2020, 146, 925–932. [CrossRef] [PubMed]
55. Marx, M.; Mosnier, I.; Venail, F.; Mondain, M.; Uziel, A.; Bakhos, D.; Lescanne, E.; N’Guyen, Y.; Bernardeschi, D.; Sterkers, O.;
et al. Cochlear Implantation and Other Treatments in Single-Sided Deafness and Asymmetric Hearing Loss: Results of a National
Multicenter Study Including a Randomized Controlled Trial. Audiol. Neurootol. 2021, 26, 414–424. [CrossRef] [PubMed]
56. Moschner, D.B. Forschen mit Jungen Kindern—Eine Herausforderung; Universität Oldenburg: Oldenburg, Germany, 2012.
57. Tajudeen, B.A.; Waltzman, S.B.; Jethanamest, D.; Svirsky, M.A. Speech Perception in Congenitally Deaf Children Receiving
Cochlear Implants in the First Year of Life. Otol. Neurotol. 2010, 31, 1254–1260. [CrossRef] [PubMed]
58. Colletti, L.; Mandalà, M.; Zoccante, L.; Shannon, R.V.; Colletti, V. Infants versus Older Children Fitted with Cochlear Implants:
Performance over 10 Years. Int. J. Pediatr. Otorhinolaryngol. 2011, 75, 504–509. [CrossRef] [PubMed]
59. Naik, A.N.; Varadarajan, V.V.; Malhotra, P.S. Early Pediatric Cochlear Implantation: An Update. Laryngoscope Investig. Otolaryngol.
2021, 6, 512–521. [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.