Human Systems Engineering and Design (IHSED2023), Vol.

112, 2023, 368–376

https://doi.org/10.54941/ahfe1004149

Usability Testing of Instructions for Use

for Cleaning, Disinfection and
Sterilization of Ultrasound Probes
M. Oliverio, D. Fiesoli, S. Bellavia, and R. De Luca
Esaote S.p.A, via di Caciolle 15, 50127, Firenze, Italy

ABSTRACT
Infection Prevention and Control (IPC) represents an area of big interest in healthcare.
In this context, ‘reprocessing’ is a process including cleaning, disinfection and/or ster-
ilisation carried out on a used device in order to allow its safe reuse. A typical IPC
responsibility assignments matrix includes the processor (i.e. the healthcare orga-
nization and professional), manufacturers of reprocessing agents and equipment,
and manufacturers of medical devices (EN ISO 17664:2021). This project aims to
analyze and validate the Instructions for Use (IFUs) for reprocessing of ultrasound
(US) probes, from the point of view of the usability, in accordance with standards IEC
62366-1:2015+AMD1:2020 and IEC TR 62366-2:2016. US probes enable a wide range
of applications, with the level of infection risk based on the Spaulding Classification,
going from non-critical and semi-critical probes (in contact with intact skin and mucous
membrane respectively), to critical probes (intraoperative applications). In addition to
the diverse intended uses and clinical procedures, the risk of contamination depends
also on the variability in the healthcare environments and user knowledge and exper-
tise. Safe reprocessing of an US probe requires that its IFUs, considered as part of the
user interface, are clear, legible, and complete. For this reason, to ensure their correct
execution and workflow, a usability testing for the reprocessing IFUs is performed.
According to the IEC 62366 standards, such test consists of two steps: formative and
summative evaluations. The overall aim of the first step is to explore if the instructions
are recognizable, understandable and operable by the user and to identify worst-case
scenarios and critical tasks which will be further investigated during the second step
by a user group, including lay and professional users with different profiles and needs.
For the purpose of this project, the usability testing has confirmed a powerful tool to
verify that the investigated reprocessing IFUs for a US probe can be easily and effec-
tively used. The preliminary analysis and the following interviews of diverse users
operating in different environments, have provided evidence that such IFUs enable
the user to perform an appropriate and reliable reprocessing for a safe reuse of the
probe.
Keywords: Usability, Reprocessing, IFU, Ultrasound probes

INTRODUCTION
Diagnostic ultrasound (US) imaging is a non-invasive technique widely used
worldwide, being non-hazardous, compact, portable, real-time and versa-
tile. Dedicated and specialized US probes allow many clinical applications,

© 2023. Published by AHFE Open Access. All rights reserved. 368

Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 369

including abdomen, cardiovascular, musculoskeletal, obstetrics and gynae-

cology, interventional and intraoperative procedures. Different users (e.g.
sonographers, physicians, radiologists, surgeons, anaesthesiologists, mid-
wives, paramedics) may perform an US scan/exam in diverse environments
such as emergency room, intensive care unit, operating theatre (De Luca et al.
2021; Andreoni et al. 2015). In this complex context characterized by diverse
intended uses and environments, a correct reprocessing of probes is a cru-
cial process, to guarantee a reliable and safe use of the probe. Spaulding
divided medical instruments and equipment into three categories (critical,
semi-critical, and non-critical) on the basis of the risk of infection from
contamination on the item:
• Non-critical: medical device in contact only with the intact skin of the
patient;
• Semi-critical: medical device in contact with intact mucous membranes or
nonintact skin of the patient;
• Critical: medical device introduced directly into the human body, either in
contact with the bloodstream or into other normally sterile areas of the
body (Spaulding, 1972).
Manufacturers of reusable medical devices have the responsibility to
support product label claims of reusability by providing complete and com-
prehensive written instructions for the cleaning, disinfection, and sterilization
of their products. Manufacturers also have the responsibility to conduct and
document any testing necessary to validate the suitability of these instructions
(AAMI TIR12:2010). Then, it is important to provide users with suitabil-
ity information aimed at the proper reprocessing of probes according to
their use, to avoid cross-contamination and consequently possible infec-
tion. Usability is an important topic in the risk analysis to minimize use
errors that can lead to hazardous scenarios. As defined in the IEC 62366-
1:2015 + AMD1:2020, usability is the characteristic of the user interface that
facilitates use and thereby establishes effectiveness, efficiency, and user satis-
faction in the intended use environment (IEC 62366-1:2015 + AMD1:2020).
The use of medical devices could be a normal or abnormal use. An abnor-
mal use is an intentional improper use of the medical device, which is not
managed by the standards. In particular, the standards help to manage the
normal use of the medical device, that can be divided in correct use and use
error. The use error is a user action or lack of user action while using the
medical device that leads to a different result than that intended by the man-
ufacturer or expected by the user. The relationship between different types of
use of medical device is shown in Figure 1. In this scenario, the main role is
played by the user interface, which is the means by which the user and the
medical device interact (IEC 62366-1:2015 + AMD1:2020). The analysis of
user errors and of critical errors aims to either approve the user interface or to
identify non-acceptable use risks that would lead to device interface re-design
(Ravizza et al. 2019; Zhang et al. 2003).
Accompanying documentation is considered part of the medical device and
its user interface. When information for safety is used as a risk control mea-
sure, the medical device manufacturer shall subject this information to the
370 Oliverio et al.

usability engineering process with the following aim: to determine that the
information is perceivable by, is understandable to, and supports correct use
of the medical device by users of the intended user profiles in the context of
the intended use environment (IEC 62366-1:2015 + AMD1:2020). The pur-
pose of this paper is to evaluate the usability of the reprocessing IFUs defined
according to the probe category.

Figure 1: Relationship of the types of use (IEC 62366-1:2015 + AMD1:2020).

USABILITY ENGINEERING PROCESS

As reported in the usability standards, the usability engineering process con-
sists of two main steps: formative and summative sessions. The formative
session is made of iterative tests performed by different types of user and by
the design team. It aims to improve the user interface before the summative
session, performed by a significant sample of final users.
In the specific study, the usability test has been performed following the
steps described by the IEC 62366 standards (Stuppia et al. 2022).

Formative and Summative Assessments

To perform the formative test, the following techniques, recommended in the
IEC TR 62366-2:2016 standard, have been implemented: focus group, cog-
nitive walkthrough, brainstorming, task analysis and standard review. The
focus group is defined as an effective technique for understanding the per-
ceptions, opinions, beliefs and attitude of medical device users. The focus
group of the first formative session was composed by the design team and
two experts of the product, which collected feedback from the field. Then,
the cognitive walkthrough was conducted, to predict what we expect from
the end user. Thereafter the task analysis techniques were carried out to study
the interaction between the user and the manual and to understand what
actions hinder or facilitate the user performance (IEC TR 62366-2). In the
second formative session, other three experts were involved to brainstorming
to identify other possible hazard related use scenarios. In this study, the risk
assessment started with a high-level risk analysis, depending on the improper
reprocessing, and continued with a deeper and detailed level of risk analy-
sis including an erroneous understanding of the instructions to reprocess the
Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 371

probe following the provided documentation (IFUs). According to the stan-

dards, the information provided to the final user, shall be easy to read, easy
to find and easy to understand. The risk analysis of the correct probes repro-
cessing instructions was based on these three fundamentals (IEC TR 62366-2;
ISO 14971:2019 + A11 2021; ISO 15233-1:2021).
The study through formative sessions, aimed to identify if some instruc-
tions contained in the manual could lead the user to conduct an improper
reprocessing of the probe and therefore to define the related tasks. The task
analysis was carried out to produce detailed descriptions of the activities of
personnel performing the reprocessing (IEC TR 62366-2:2016).
The output of the formative sessions was the identification of the primary
operating functions (POFs) that are functions that involve user interaction
that is related to the safety of the medical device, and their related tasks,
reported in Table 1 (IEC TR 62366-1:2015 + AMD:2020).

Table 1. POFs and critical tasks.

POFs Tasks
POF1 Understanding of probe Choose of clinical procedure with
classification and clinical procedure probe classification
POF2 Identification of reprocessing Choose of reprocessing methods to
methods to be used with respective be used following the instructions
instructions
POF3 Association of the products to be Choose of appropriate products for
used to perform the appropriate each probe and reprocessing
probe reprocessing method
POF4 Follow the instructions for Preliminary steps: probe
performing reprocessing for disconnection, cover and
non-critical, semi-critical, critical accessories removal
probes, biopsy application Performing reprocessing within the
specified time frame
Performing reprocessing according
to the immersion level
Performing cleaning before
disinfection/sterilization

Finally, a standards review was conducted with an internal usability expert,

to evaluate the user interface according to established usability engineering
practices. Though the feedback from the field regarding the manuals, and
the two formal formative sessions with a group of experts, primary operat-
ing functions and tasks were listed. For this study, 15 specialists with different
types of expertise, were chosen to perform the tasks and fill out an associ-
ated questionnaire. Such questionnaire required to associate an agreement or
disagreement score to each question related to the task list. Hereafter the list
of questions is reported in the Table 2.
The different colours have the scope to link each POF to one or more tasks,
listed in the Table 1, that is linked to one or more questions (Table 2).
372 Oliverio et al.

Table 2. Questionnaire.

Questions
POF1 Is it easy to identify the classification of the probe?
Is it easy to identify the type of procedure?
Is the search for such information within the manual sufficiently intuitive?
POF2 Is it easy to identify the reprocessing methods associated with probe
classification?
Is it easy to identify the reprocessing methods associated with the biopsy
procedure?
Depending on the classification and access and the tissues it comes into
contact with, is it clear which reprocessing method to use?
Is the difference between LLD, MLD and HLD clear?
Is the difference between manual liquid and wiping methods clear?
POF3 Is it easy to identify the reprocessing agent to be used for each method?
Is the table containing product information is easy to consult?
POF4 Before starting reprocessing, is clear that the probe must be disconnected
from the system?
Before reprocessing can begin, is clear that the cover must be removed?
Is it clear that any accessories must be removed before reprocessing can
begin?
Are the times given in the reprocessing instructions understandable?
Is it clear that the process has a certain duration, which is mandatory?
Is it clear where to find, within the manual, the inherent indication of
immersion level?
Is it sufficiently clear when to perform the cleaning step within a
disinfection and/or sterilization procedure?

For each question an agreement or disagreement score was associated as

reported in the Table 3.

Table 3. Agreement/disagreement scores.

Agreement/Disagreement Related score

I like it, no problem 0
I might need some time to get comfortable on how to use it 1
It is hard to use. It makes me nervous 2
The use and/or the information for use are too complicated; I can’t 3
understand it
Don’t use it! Possible risk 4

Severity rating scale was used to identify the critical points within the man-
ual referred to the reprocessing (J. Zhang et al. 2003). There is a wide variety
of end users involved in the use of an ultrasound probe. As reported in the
annex K of the standard 62366-2, a representative sample needs to be cho-
sen in the summative session. Fifteen participants are a practical minimum
number of participants for human factors validation testing (Health et al.
2019). User group involved in the summative session, was composed by dif-
ferent specialists with different age comprised between 28 and 65 years old
Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 373

and different levels of expertise and experience, by entry level to senior. The
areas of expertise and applications included urology, breast care, obstetrics,
gynaecology, interventional, cardiology, service, and usability.
The standard shows the decrease in cumulative for a number of users
greater than 10. However, the factors that cause cumulative decreases
are number of users, probability of a usability error occurring, formative
iterations.
Added to the formative iterations, there is the important statistic feature
of the presence of the manual in the field and the feedback reported by the
internal experts.

Summative Evaluation Results

To analyze the IFUs understanding, identification and execution, the over-
all score associated to each task was calculated as the average of the scores
assigned to the relative questions. No critical error (corresponding to totally
unsatisfied in the bar chart) was observed during the simulated use indicat-
ing the absence of patient-related risk linked to intended user/user-interface
interaction. Therefore, the risk of cross-contamination and, consequently,
infection is effectively mitigated.

Figure 2: Task analysis: average of the scores assigned to the questions related to each
task.

The task including preliminary steps (probe disconnection, cover and

accessory removal), achieved the 90% of totally satisfied score, with a not
significant percentage of partial unsatisfied and neutral. A generic suggestion
collected during the summative test regarded the possibility to have a leaflet
with the IFUs described by pictograms to have an easier access and under-
standing to the information for an appropriate reprocessing. Such suggestion
374 Oliverio et al.

does not affect the current IFUs, but it represents a potential additional
information to be provided to the user and therefore an improvement of
the manual. The question regarding the probe immersion level limit (“Is it
clear where to find, within the manual, the inherent indication of immersion
level?”, POF4, Table 2), registered almost the 30% of neutrals, however this
result does not correspond to a potential user error. It is considered a useful
input to improve the manual section showing the probe immersion limit. A
remarkable result is on performing cleaning before sterilization and disinfec-
tion. It means that the information is clearly expressed and that there is the
confidence that this step is performed as indicated. It is an important result
because it is a crucial step to begin the correct reprocessing.
Heuristic summary is hereby proposed in Figure 3 through a radar-graph
with all the participants’ responses. The underlying area of the graph is
approximately proportional to the perceived risk (larger area, major per-
ceived risk) and therefore inversely proportional to the degree of under-
standing and implementation of the provided reprocessing IFUs (small area,
overwhelming positive response). Since all the responses are centred in the
middle, it is confirmed that the reprocessing IFUs were well-received by the
participants.

Figure 3: Heuristic analysis: participants scores assigned to each question.

The following questions:

• Is the search for such information within the manual sufficiently intuitive?
(POF 1, Table 2),
• Before starting reprocessing, is clear that the probe must be disconnected
from the system? (POF 4, Table 2)
Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 375

registered the highest scores (higher score, lower favourableness) that came
from entry-level users, who are less familiar with manuals and workflow.
Note that this feedback regards the capability to easily find the correct infor-
mation in the manual, that is operator-dependent, and it does not affect the
correct understanding of a proper probe reprocessing execution. A training
dedicated to the information flow showed in the manual represents a suitable
mean to aid the manual readability.
The polarized graph in Figure 4 shows the average of the scores (refer to
Table 3) assigned by the fifteen participants to each question. This graph
allows to identify the most critical aspects under investigations, correspond-
ing to the highest score. Note that all the obtained results are lower than
1; this means that the overall summative evaluation result proved that the
provided IFUs minimize the risk of improper reprocessing. Therefore, the
summative evaluation has confirmed that essential information for the effec-
tive use of the IFUs represents a powerful mean directed to the user to
perform an appropriate probe reprocessing to optimize infection prevention
and control in healthcare environments.

Figure 4: Average of the scores assigned by the participants to each question.

CONCLUSIONS
Safe reprocessing of an US probe requires that its IFUs, considered as part
of the user interface, are clear, legible, and complete. According to the
IEC 62366 standards, formative and summative evaluations have been per-
formed, to evaluate the reprocessing IFUs from the usability point of view. For
the purpose of this project, the usability testing has confirmed a powerful tool
to verify that our reprocessing IFUs for a US probe may be easily and effec-
tively used. The preliminary analysis and the following interviews of diverse
users operating in different environments, have provided evidence that such
IFUs enable the user to perform a proper and reliable reprocessing for a safe
reuse of the probe.
376 Oliverio et al.

REFERENCES
AAMI TIR12:2010, Designing, testing, and labeling reusable medical devices for
reprocessing in health care.
Andreoni G., Mazzola M., Matteoli S., D’Onofrio S., Forzoni L. (2015), Ultrasound
system typologies, user interfaces and probes design: a review, Procedia Manuf
Elsevier, 3: 112–119, doi: 10.1016/ j.promfg.2015.07.115.
De Luca R., Dattoma T., Forzoni L., Bamber J., Palchetti P., Gubbini A. (2018),
Diagnostic ultrasound probes: a typology and overview of technologies, Current
Directions in Biomedical Engineering, 4(1): 49–53, doi: 10.1515/cdbme-2018-
0013.
De Luca R., Forzoni L., Gelli F., Bamber J., An Educational Overview of Ultrasound
Probe Types and Their Fields of Application, Archives of Acoustics – Volume 46,
Number 1, 2021.
EN ISO 14971:2019 + A11 2021 Medical devices. Application of risk management
to medical devices.
EN ISO 17664:2021 Processing of health care products—Information to be provided
by the medical device manufacturer for the processing of medical devices — Part
1: Critical and semi-critical medical devices.
Health, C. for D. and R. (2019, September 2). Applying Human Factors and
Usability Engineering to Medical Devices. U.S. Food and Drug Administration;
FDA. https://www.fda.gov/regulatory-information/searchfda-guidance-document
s/applying-human-factors-andusability-engineering-medical-devices
IEC 62366-1:2015 + AMD1:2020, Medical devices - Part 1: Application of usability
engineering to medical devices.
IEC TR 62366-2:2016, Medical devices - Part 2: Guidance on the application of
usability engineering to medical devices.
ISO-TR 15223-1:2021 Medical devices – Symbols to be used with information to be
supplied by the manufacturer – Part 1: General requirements.
ISO-TR-24971:2020 Medical devices—Guidance on the application of ISO 14971.
Ravizza, A., Lantada, A. D., Sánchez, L. I. B., Sternini, F., & Bignardi, C. (2019).
Techniques for Usability Risk Assessment during Medical Device Design. Proceed-
ings of the 12th International Joint Conference on Biomedical Engineering Systems
and Technologies, 207–214. https://doi.org/10.5220/0007483102070214
Spaulding EH. Chemical disinfection and antisepsis in the hospital. J Hosp Res,
9:5-31, 1972.
Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L., & Kubose, T. (2003). Using usabil-
ity heuristics to evaluate patient safety of medical devices. Journal of Biomedical
Informatics, 36(1), 23–30. https://doi.org/10.1016/S1532-0464(03)00060-1