Circular 356

Guidelines on Digital Learning for

Cabin Crew Training

Approved by and published under the authority of the Secretary General

INTERNATIONAL CIVIL AVIATION ORGANIZATION

Circular 356
Guidelines on Digital Learning for
Cabin Crew Training

Approved by and published under the authority of the Secretary General

INTERNATIONAL CIVIL AVIATION ORGANIZATION

Published in separate English, Arabic, Chinese, French, Russian
and Spanish editions by the
INTERNATIONAL CIVIL AVIATION ORGANIZATION
999 Robert-Bourassa Boulevard, Montréal, Quebec, Canada H3C 5H7

For ordering information and for a complete listing of sales agents

and booksellers, please go to the ICAO website at www.icao.int

Cir 356, Guidelines on Digital Learning for Cabin Crew Training

Order Number: CIR356
ISBN 978-92-9258-886-1

© ICAO 2020

All rights reserved. No part of this publication may be reproduced, stored in a

retrieval system or transmitted in any form or by any means, without prior
permission in writing from the International Civil Aviation Organization.
 TABLE OF CONTENTS

Page

Glossary .................................................................................................................................................................. (v)

Chapter 1. Introduction to digital learning ........................................................................................................ 1

1.1 General..................................................................................................................................................... 1
1.2 Baseline considerations............................................................................................................................ 1
1.3 E-learning versus digital learning ............................................................................................................. 2
1.4 Benefits of implementing digital learning .................................................................................................. 2
1.5 Blended learning approach....................................................................................................................... 4
1.6 Levels of user interactivity ........................................................................................................................ 6
1.7 Limitations of digital learning .................................................................................................................... 6
1.8 Adapting cabin crew training via digital learning ....................................................................................... 7

Chapter 2. Applying digital learning .................................................................................................................. 9

2.1 How to build a digital learning programme ............................................................................................... 9

2.2 Instructional design process ..................................................................................................................... 9
2.3 Instructional design model ........................................................................................................................ 10
2.4 Training delivery methods ........................................................................................................................ 12
2.5 Instructor availability ................................................................................................................................. 14
2.6 Evaluation methods .................................................................................................................................. 14
2.7 User training ............................................................................................................................................. 15

Chapter 3. Quality assurance in digital learning............................................................................................... 16

3.1 Content management system................................................................................................................... 16

3.2 Learning management system ................................................................................................................. 16
3.3 Quality management process ................................................................................................................... 17
3.4 Contingency planning ............................................................................................................................... 18

Chapter 4. Regulatory considerations ............................................................................................................... 19

4.1 Regulatory approval process .................................................................................................................... 19

4.2 Aspects for consideration ......................................................................................................................... 19
4.3 Surveillance .............................................................................................................................................. 20

______________________

(iii)
 GLOSSARY

DEFINITIONS

Approved training — cabin crew. Training conducted under special curricula and supervision approved by a Member
State that, where applicable, is conducted within an approved training organization.

Cabin crew member. A crew member who performs, in the interest of safety of passengers, duties assigned by the
operator or the pilot-in-command of the aircraft, but who shall not act as a flight crew member.

Classroom training. In-person, instructor-led training which may include group exercises and interactive instructional
sessions.

Competency. A dimension of human performance that is used to reliably predict successful performance on the job. A
competency is manifested and observed through behaviours that mobilize the relevant knowledge, skills and
attitudes to carry out activities or tasks under specified conditions.

Computer-based training. Training involving instructional aids, such as computers and tablets. Computer-based
training may encompass the use of data storage medium (such as CD-ROM or flash drive), as well as web-based
training (commonly referred to as e-learning), distance learning and digital learning (such as virtual learning and
gamification).

Hands-on exercise. Exercise on the use of equipment or aircraft systems that is conducted without a specific context.
Equipment that is removed from operation, or other representative training equipment considered acceptable by the
State, can be used for the purposes of this training.

Operator. The person, organization or enterprise engaged in or offering to engage in an aircraft operation.

Safety performance indicator. A data-based parameter used for monitoring and assessing safety performance.

Simulated exercise. Exercise representing a full context scenario (e.g. aircraft evacuation) where cabin crew apply the
operator’s procedures and associated crew responsibilities for dealing with the specific situation. This is typically
conducted in a representative training device capable of reproducing the appropriate environment or equipment
characteristics (e.g. cabin, flight deck, accessible cargo compartment, crew rest area, etc.), or on an actual aircraft.

State of the Operator. The State in which the operator’s principal place of business is located or, if there is no such
place of business, the operator’s permanent residence.

______________________

(v)
(vi) ICAO Circular 356

ABBREVIATIONS AND ACRONYMS

App Application
CMS Content management system
ILT Instructor-led training
LMS Learning management system
VR Virtual reality

______________________
 Chapter 1

INTRODUCTION TO DIGITAL LEARNING

1.1 GENERAL

1.1.1 The purpose of this circular is to provide detailed guidance on designing, developing and using digital
learning for cabin crew safety training. It also provides basic concepts and information on the processes and resources
involved in digital learning development. These guidelines focus on formal learning, specifically on structured courses
designed to meet job-related learning objectives. They do not address assessment needs or evaluation stages of a
training programme but rather the design, development and delivery methods that are specific to digital learning.

1.1.2 Digital learning should support classroom training and hands-on exercises with new technologies. These
guidelines provide information about digital learning and the strengths and weaknesses of approaches to implementing
technology-based learning in general. In addition, to keep trainees involved and motivated, and to improve the training
and learning experience, the aspects covered in these guidelines should be taken into consideration when designing
technology-based training.

1.1.3 Digital learning offers opportunities to improve competencies, including knowledge and skills. It can offer
enhanced safety via well-trained and competent aviation professionals.

1.1.4 Digital learning is not meant to replace classroom training, hands-on exercises or traditional simulated
exercises in their entirety. Based on the technological methods used, digital learning can recreate a realistic environment
that best simulates line operations. This level of realism in training reinforces the development and application of
competencies alternatively trained in a classroom setting or through hands-on exercises. Digital learning should be
selected as a training delivery method when it is deemed suitable to attain the learning outcomes and its use results in
an equivalent or increased level of competence for the trainee.

1.2 BASELINE CONSIDERATIONS

1.2.1 The way organizations view the learning process and how people learn has evolved over the years.
Additionally, the aviation industry is constantly evolving and deals with new challenges, diverse cultures, different
generations of trainees and new regulations. These issues raise the question of how to make training more effective and
sustainable. Organizations should focus on what can be done and what kind of technology can be used to enhance
training. Digital learning is an opportunity to improve and change the way training content is retained.

1.2.2 Points to consider when choosing digital learning include, but are not limited, to:

a) Is learning best delivered in one unit or spread out over time?

b) Does training address an immediate learning objective?

c) Do trainees have access to hardware and software?

d) Are trainees sufficiently self-motivated for digital learning or self-directed modes of learning?

1
2 ICAO Circular 356

e) Does the content of the training require regulatory approval?

f) What is the operator’s current training environment (e.g. hardware, software, training facilities)?

g) What are the learning objectives (e.g. what is taught — knowledge-based items, skills, etc.)?

h) What are the benefits of either a learning environment with in-person, instructor-led training or a
blended learning approach (see section 1.5)?

i) Will greater competence be achieved through digital learning?

j) Are the available technological methods suitable to achieve the specific learning objectives (e.g.
recreating operational constraints such as the forces required to open a jammed door)?

k) What other training opportunities are available to support the learning (e.g. blended learning, hands-on
exercises, classroom training, etc.)?

1.2.3 Digital learning or the use of new technology in training should fit the organization’s purpose. The content
and the learning outcomes should be continually evaluated and modified, as necessary.

1.2.4 Digital learning can improve the effectiveness and retention of knowledge and skills; it is not meant to
reduce costs.

1.3 E-LEARNING VERSUS DIGITAL LEARNING

1.3.1 E-learning involves the use of hardware and software technologies to deliver a broad array of training
delivery methods to enable learning and improve performance.

1.3.2 Digital learning is any training delivery method that effectively uses technology, such as virtual and
augmented reality (AR), or gamification, to strengthen the trainee’s knowledge and skills. It emphasizes instruction and
provides access to content and feedback through formative assessments. Digital learning can be used as a primary
training (main source of training) for certain topics (e.g. knowledge-based items) or as a secondary training solution (any
supplementary learning to compliment the primary training) for certain topics (e.g. reinforcing certain steps of a
procedure to support a hands-on exercise). An effectively implemented digital learning programme complements the
dynamic aviation training environment by providing positive learning and training enhancements.

1.4 BENEFITS OF IMPLEMENTING DIGITAL LEARNING

1.4.1 Existing and emerging technologies offer opportunities to create a learning experience that may not be
possible in traditional learning environments. Digital learning presents specific benefits to different stakeholders, as
described below.

1.4.2 Benefits for the trainees

Benefits of using digital learning technologies for trainees may include, but are not limited, to:

a) grasping concepts more quickly and thoroughly;

 ICAO Circular 356 3

b) connecting theory and application more adeptly;

c) engaging in learning more readily;

d) facilitating the widespread sharing of knowledge by providing continuous access to training materials;

e) access to accurate updated information at any time;

f) personalized training — the learning environment reflects individual preferences (knowledge, learning
styles, self-paced);

g) engagement of trainees — based on the interactivity within the training modules, digital learning can
be more engaging for trainees;

h) opportunities to practice competencies and tasks through the use of technologies with the ability to
replicate real life events within a safe training environment; and

i) theoretical elements covered using digital learning may allow additional time for trainees to practice
hands-on exercises.

1.4.3 Benefits for the operator

Operators can benefit from digital learning technology as it enables improved methods of increasing competencies for a
large remote workforce, such as cabin crew members. Digital learning provides access to real time tracking of trainee
performance and feedback, and can be used to assess the level of training required. It enables expanded learning
opportunities through highly engaging learning. Digital learning offers effective training delivery methods, such as
practising with associated feedback, combining collaborative activities with self-directed learning, personalizing learning
paths based on the trainee’s needs, standardizing content delivery, and using simulation and games.

1.4.4 Additional benefits

Additional benefits of using digital learning technology may include, but are not limited, to:

a) training on exceptional situations (e.g. noisy environments or obfuscation by smoke) can be safely
trained through virtual training;

b) exercises can be repeated as often as required, with or without variation of parameters and
complexity;

c) cooperative scenarios (e.g. firefighting) may be trained within virtual multi-user training scenarios;

d) in-situation hints (text, audio, photo, video) can be shown on demand and integrated into the learning
process;

e) virtual facilities (e.g. virtual cabin doors) can be “damaged” without financial implication (e.g. during
simulated exercises) and they do not sustain damage that naturally and inevitably occurs as a result of
normal use or aging;

f) virtual reality (VR) training allows individual trainee performance to be live-monitored and reviewed by
the instructor at any time;

g) flexibility of learning – trainees can access the content anywhere and anytime. Depending on the
system and technology being used by the operator, it can be done online and/or offline. The trainee
decides when, how and where to do the training. It is self-directed learning;
4 ICAO Circular 356

h) updates, highlights and revisions:

1) allow the operator to distribute critical and non-critical updates in a minimal amount of time and to
monitor receipt of messages;

2) learning analytics — more data is made instantly available, with greater context around the data.
The operator can use this data to adapt and update the training content and methods to benefit
the trainee and the instructor; and

3) enhanced face-to-face engagement — training is more focused on face-to-face and hands-on

exercises, as the theoretical elements can be learned prior to instructor-led classroom training.

1.5 BLENDED LEARNING APPROACH

1.5.1 A blended learning approach is a combination of training delivery methods adapted to the content. When
developing or revising a training programme, the operator should use a variety of training delivery methods (classroom,
computer-based training, hands-on exercises, simulated exercises in representative training devices, etc.) aligned to the
subject matter. The operator should ensure a balance between independent learning (e.g. distance learning) and
instructor-led training to achieve the learning outcomes. A blended learning approach combines traditional classroom
training with digital learning technology to address a specific topic during training (e.g. first aid). It has multiple
possibilities adaptable to operator and trainee needs. An example of this approach could be a combination of the
following methods, as presented in Figure 1-1.

1.5.2 Technological methods used as part of a blended learning approach include:

a) virtual learning — this allows trainees to connect, interact and learn individually or in a group outside
the classroom using realistic virtual environments;

b) mobile applications (Apps) — these are learning applications that are accessible through portable
electronic devices such as smart phones, tablets, and e-readers;

c) e-learning — this is the delivery of a learning, training or educational programme by electronic means;
and

d) gamification — this is an educational approach to motivate trainees to learn by using video game style
design and game elements in a learning environment.

1.5.3 Organizational methods used as part of a blended learning approach include:

a) instructor-led training (ILT) — classroom training facilitated by an instructor;

b) self-directed learning — a learning method without direct supervision or attendance in a classroom;

c) pre- and post-learning — takes place before and after the ILT classroom training. Trainees can learn
certain topics, definitions and theory before the training so that they can focus more on the practical
aspects of the training;

d) coaching/mentoring — assists with the facilitation of the trainee’s learning experience by providing
support, encouragement, information and guidance. Coaching can be done within the training
department, for example with e-tutors; and
 ICAO Circular 356 5

e) pop-up training refreshers — mini courses of short duration that can be distributed to trainees via a
digital form (e.g. reinforcement of a new procedure) and should capture the attention of the intended
audience (they should be entertaining as well as educational).

Virtual
learning

Pop-up
training Mobile Apps
refreshers

Coaching/
E-learning
mentoring
Blended learning
approach

Pre- and
post- Gamification
learning

Self-directed Instructor-led
learning training

Figure 1-1. Methods used for a blended learning approach

6 ICAO Circular 356

1.6 LEVELS OF USER INTERACTIVITY

1.6.1 There are different levels of user interactivity when it comes to digital learning. The level of interactivity
depends on the content, trainees, objectives, time and technology.

Table 1-1. Levels of interactivity

Digital learning
Level interactivity User interactivity Learning methods/tools

1 — Passive No interaction Trainees are not required to Simple images and graphics, simple video
interact with digital learning and audio, test questions, etc.
resources and the digital
learning process is strictly
linear.

2 — Limited Limited Trainees may have some Animations, clickable menus, drag and
participation control over their digital learning drop interactions, and multimedia.
experience as they are required
to make simple interactions with
the digital learning material.

3 — Moderate Moderate Trainees gain more control over Animated video, customized audio,
interaction the digital learning experience, complex drag and drop interactions,
which is more customized and simulations, stories and branching
complex. scenarios, and multimedia.

4 — Full Full immersion Trainees have great control over Interactive games, simulated job
their digital learning experience performance exercises, customized audio
as they are required to fully or videos, avatars, stories and scenarios,
interact with the digital learning as well as multimedia.
content and give feedback.

5 — Full VR Full immersion Trainees have enhanced control This encompasses Level 4 in combination
into a over their digital learning with a head-mounted display and
stereoscopic VR experience as they are required enhanced interactivity (gesture and
environment to fully interact with the digital speech recognition).
learning content and give
feedback.

1.6.2 All levels (including level 5) could be conducted as a mobile or stationary solution. Mobile solutions support
“anytime, anywhere”, while stationary ones may support blended learning in training centres, for example.

1.7 LIMITATIONS OF DIGITAL LEARNING

1.7.1 The operator should consider the technological infrastructure and support required for digital learning, prior
to its implementation. It should also take into consideration online cyber security standards, data protection and data
restrictions.
 ICAO Circular 356 7

1.7.2 The operator should also consider the impact of digital learning on its compliance with national regulations
(e.g. flight time limitations and rest time requirements) and the willingness of the State which oversees the approved
training programme to accept digital learning as a training delivery method (refer to Chapter 4).

1.7.3 The operator should conduct a risk assessment when implementing new hardware and software to support
digital learning. It should also identify the knowledge and skills that need to be addressed in training and determine if
digital learning is the most appropriate means to deliver a specific portion of the training programme. The operator
should not use digital learning as a means to conduct formal assessments on all topics. It can be used to practice or
reinforce certain procedures and support hands-on and simulated exercises.

1.7.4 The State should verify that the digital learning is appropriate to the learning objectives and trainee
characteristics.

1.7.5 The State should also verify that there is a limited use of digital learning in the approved training
programme. It can be used as a complementary part of training but actual practical training should be maintained (e.g.
simulated exercises such as an aircraft evacuation). For example, for the operation of aircraft doors, digital learning may
not replicate the weight of a door, the force required to open a jammed door, associated sounds, the help of another
person or persons to open the door, unexpected conditions outside and inside, or coordination with other crew members.
Therefore, digital learning can be acceptable as a means to complement the hands-on exercise of door operation but
should not fully replace it. Likewise, group discussions, problem-solving during scenario-based training, sharing
experiences, as well as providing and receiving inputs as part of scenarios to practice crew resource management,
should not be fully substituted by digital learning.

1.8 ADAPTING CABIN CREW TRAINING VIA DIGITAL LEARNING

1.8.1 Each operator is required to establish and maintain a training programme that is approved by the State of
the Operator, and which is to be completed by each person before their assignment as a cabin crew member as per the
requirements in Annex 6 — Operation of Aircraft, Part I — International Commercial Air Transport — Aeroplanes. Annex
6 requirements also state that cabin crew members complete a recurrent training programme annually.

1.8.2 The Cabin Crew Safety Training Manual (Doc 10002) provides guidance on initial training for cabin crew,
which should include:

a) aviation indoctrination;

b) cabin crew tasks, including duties and responsibilities;

c) normal, abnormal and emergency procedures;

d) aircraft type training;

e) dangerous goods;

f) human performance;

g) cabin health and first aid;

h) aviation security; and

i) identifying and responding to trafficking in persons.

8 ICAO Circular 356

1.8.3 To train cabin crew members on these topics, Doc 10002 recommends that the operator use a variety of
training delivery methods (classroom, computer-based training, hands-on exercises, simulated exercises in
representative training devices, etc.) as appropriate to the subject matter. Digital learning can be used to support hands-
on training by practicing procedural steps such as door operation, evacuations or firefighting. Based on the scenario,
hands-on exercises and simulated exercises should be conducted utilizing representative training devices, or on an
actual aircraft. Digital learning may be used to complement or replace some training devices or the use of an actual
aircraft, or to overcome the limitations of some training devices in reproducing the appropriate environment and
equipment characteristics (e.g. realistic simulation of a ditching environment).

______________________
 Chapter 2

APPLYING DIGITAL LEARNING

2.1 HOW TO BUILD A DIGITAL LEARNING PROGRAMME

2.1.1 Prior to building a digital learning programme, the operator should define the learning outcome of the
training and select the preferred learning methodology and/or technology to achieve that goal. The application of a digital
learning programme begins with identifying applicable training content that is suitable for digital learning, taking into
consideration possible limitations (refer to Chapter 1, 1.7). The operator should determine methods that will be used to
deliver the training content. Considerations should be given to the level of competence required when selecting the
delivery method. Digital learning may be a component of a blended learning approach (refer to Chapter 1, 1.5).
Therefore, digital learning may be an effective tool for prequalification of trainees prior to attending training so that they
begin the training programme having already acquired a basic level of knowledge.

2.1.2 The operator should consider the potential gap between different generations of cabin crew members that
it employs. Some cabin crew members may not be at the same user level, or as technology savvy, as others and may
experience difficulties during the training due to limited technological proficiency.

2.1.3 The operator should also consider access to technology by its cabin crew members and whether it will be
required to provide devices to them, such as smart phones or tablets.

2.1.4 The operator should take into account the time allocated for cabin crew members to complete digital
learning courses, so as to not affect minimum crew rest periods or encroach on days off duty.

2.1.5 Overall, structure design and planning, while crucial for every type of training programme, are even more
important for digital learning programmes. From the start, consideration should be given to the framework. For example:
rostering of crew members (if required); mandatory versus self-development training; timeline and life cycle of the
content; and trainee access to the required technology.

2.2 INSTRUCTIONAL DESIGN PROCESS

2.2.1 The process of instructional design should focus on factors that influence learning outcomes, including, but
not limited, to:

a) the level of knowledge regarding content and the technology utilized that individual trainees need for
accomplishing the objectives;

b) the appropriateness of instructional strategies in terms of objectives and trainee characteristics;

c) definition and implementation of the support needed for successful learning; and

d) periodic revisions or any change to operational documents that are necessary to ensure the
programme meets expected outcomes.

9
10 ICAO Circular 356

2.2.2 In determining the instructional design, the following characteristics should be considered:

a) hardware and software (i.e. the training platform);

b) audience (e.g. skills, knowledge level, etc.);

c) trainee and instructor objectives;

d) training content and structure;

e) assessment criteria; and

f) evaluation process (of content and results).

2.3 INSTRUCTIONAL DESIGN MODEL

2.3.1 For the operator, there are many different instructional models that can assist in developing digital learning.
(e.g. situated cognition theory; sociocultural learning theory; the analysis, design, development, implementation and
evaluation (ADDIE) Model; Merrill’s principles of instruction; individualized instruction; and Bloom’s taxonomy of learning
objectives). The ADDIE Model components may be considered when implementing digital learning. Chapter 2 of the
ICAO Procedures for Air Navigation Services — Training (PANS-TRG, Doc 9868) contains a detailed description of an
ADDIE methodology.

Analyse

2.3.2 As part of the ADDIE Model, the operator should identify all the variables, as listed below, that need to be
considered when designing the course.

2.3.3 Trainee characteristics and knowledge — One of the most important components in the transformation to
digital learning is ensuring that trainees have adequate knowledge and training to effectively use digital tools and
applications.

2.3.4 Further trainee characteristics — These are critical to achieve the specific learning objectives. They should
be identified and considered. Consideration should be given, but not be limited, to:

a) validation with the State, if required;

b) generation of trainees (technological proficiency);

c) language;

d) framework (how and when do the trainees complete the training, will they be given an adequate time
in the roster — depending on the training being mandatory or for self-development);

e) accessibility to technology; and

f) learning styles (preferred ways of learning);

2.3.5 Resources required — Content, technology, facilities, subject matter experts for content and design,
hosting platform, e-tutor, etc., and potential training delivery methods.
 ICAO Circular 356 11

2.3.6 Learning objectives — The operator should identify the requirements and meet the learning outcomes.

Design

2.3.7 This phase focuses on identifying the learning objectives for the training course and how material will be
created and designed (for instance, it may include describing what content areas are to be covered, and a storyboard
outlining what will be covered in text, audio and video and in what order), and deciding on the selection and use of
technology, such as a hosting platform, video or social media.

2.3.8 Learning objectives — Specific measurable actions should be defined enabling trainees to fulfil learning
objectives.

2.3.9 Instructional strategies — Clear links should be established between training content and learning
objectives. Content and learning activities should be introduced in a logical sequence that supports the trainee’s
construction of knowledge and skills. Consideration should be given, but not be limited, to:

a) repetition, review and mnemonics;

b) questions, examples and paraphrasing;

c) analysing, categorizing, creation of diagrams and interrelation of new ideas to existing ones;

d) realistic interactivity (the virtual system behaves like the real one);

e) self-paced exploration (“what happens if I do this…”);

f) assistance on demand (text, audio, photo, video); and

g) gamification (e.g. scoring for increased motivation);

2.3.10 Testing strategies — Feedback should be provided on the trainee’s progress in meeting the defined
learning objectives.

2.3.11 Task Analysis — A task analysis is a critical part of instructional design. It should include a detailed
description of theoretical components and practical activities. Consideration should be given, but not be limited, to:

a) task durations;

b) frequency;

c) complexity;

d) whether the task is mandatory or for self-development;

e) single-person task or multiple users;

f) skills needed to complete the task;

g) environmental conditions (real and/or virtual); and

h) necessary equipment and people.

12 ICAO Circular 356

Develop

2.3.12 During this phase, the training should be built and tested. Feedback and observations collected at this
stage offer insight into final adjustments that should be made before implementing the learning solution. Several means
may be used to gather feedback and observations, such as small group trials.

Implement

2.3.13 This phase begins with a notification (of new training being available) and enrolment, followed by pre-
training communication and interaction with the newly developed learning resources. Especially when transitioning from
traditional classroom training, the operator should invest enough time for the introduction and explanation of the new
training method. An appropriate timeframe should be selected for the roll-out (e.g. consideration should be given to
operational factors). The operator should also ensure the availability of support functions (e.g. information technology,
course content experts).

Evaluate

2.3.14 Evaluation methods are varying methods that can determine the effectiveness of the operator’s digital
learning programme. They should be appropriate to the required application of knowledge and skills. Consideration
should be given, but not be limited, to the:

a) level of acquired competence of the individual (Did the trainee obtain the knowledge and skills set out
in the learning objectives and meet the learning outcomes?);

b) level of application of new competence by measuring how the new skills and knowledge are applied
on the job., This may include observation and analysis of the trainee’s behaviour;

c) level of achievement of the learning outcomes of the training content by assessing the learning
outcomes and the degree to which learning objectives have been achieved; and

d) level of trainee’s satisfaction should be measured (Did the trainee like the training? Did the trainee find
it meaningful and relevant? Did the trainee experience difficulties with usability of the digital learning
method?).

2.3.15 The operator should have a process in place to maintain, modify und update the digital learning
programme. Digital learning should be adapted and updated when needed or when the evaluation shows that it does not
meet the trainings objectives.

2.4 TRAINING DELIVERY METHODS

2.4.1 The use of existing and emerging technologies is only limited by creativity and availability of current
technology. The operator can consider the use of the following delivery methods as training delivery methods.

Audio/video

2.4.2 Converting theoretical content into audio-visual content is an effective way to engage trainees and draw
their attention with practical examples. It is an easy way to standardize, for example, a certain procedure. A well-
constructed narrative (audio) is fundamental to reinforce the didactic character of the video.
 ICAO Circular 356 13

Application (App)

2.4.3 An App is a software programme that is often installed on handheld, portable electronic devices. There are
web or online Apps that require data connectivity and other Apps that operate independently on the device. The degree
of complexity varies from a simple source of information to extensive user interaction incorporating artificial intelligence.

Virtual classroom

2.4.4 A virtual classroom is an online learning environment where the participants can interact through
discussion and access multiple sources of media. The operator should conduct a safety risk assessment on the
constraints of replacing live classroom instruction with a virtual classroom for safety-related topics addressed as part of
cabin crew training.

Gamification

2.4.5 Gamification is the process of applying gaming designs and concepts to learning or training scenarios to
make them more engaging and entertaining for the trainee. This game-based approach to instruction can provide an
effective approach to enhance learning as it allows the operator to present educational content to trainees in a way that
is visually compelling and personalized.

2.4.6 Gamification is often delivered to the trainee’s computer, tablet or smart phone and the trainee can
engage with the content whenever and wherever they happen to be. In game-based learning events, trainees compete
directly against one or more individuals or participate individually in an interactive experience that rewards learning
performance in some way.

2.4.7 When trainees receive content on a daily or weekly basis and are quizzed on that content with some
additional game elements, it affects long-term retention and knowledge application. When considering whether to
integrate gamification into learning, the operator should ensure there will be a resulting structure and framework, and not
just something that is new and fun to play. Trainees should understand learning outcomes of the training session.

Virtual reality (VR)

2.4.8 VR is an interactive computer-generated experience which takes place within a simulated environment. It
incorporates mainly auditory and visual feedback, but may also allow other types of sensory feedback. This immersive
environment can be similar to the real world or it can be fictional. Current VR technology most commonly uses VR
headsets or multi-projected environments, to generate realistic images, sounds and other sensations that simulate a
user’s physical presence in a virtual or imaginary environment. A person using VR equipment is able to “look around” the
artificial world, move around in it, and interact with virtual features or items in complete safety.

Augmented reality (AR)

2.4.9 AR is an enhanced version of the real world, where live direct or indirect views of physical real world
environments are augmented with multiple sensory cues, including visual, auditory, haptic, somatosensory, and olfactory,
over a user’s view of the real world, therefore enhancing the trainee’s current perception of reality. One example of AR
could be mixed interactive reality such as a hologram, a three-dimensional image, created with photographic projection.

Computer-based training

2.4.10 Computer-based training is an individual interactive process. The trainee interacts with various types of
learning materials via a computer. Computer-based training courses can be multimedia-enhanced textbooks, tutorials,
practice drills or even virtual simulations. The learning material comes in computer-based training software packages.
14 ICAO Circular 356

Web-based training

2.4.11 Web-based training is an internet- or intranet-based training that can be accessed remotely online. Two
primary models of web-based instruction are instructor-facilitated or self-directed.

2.5 INSTRUCTOR AVAILABILITY

Instructor availability during digital learning

2.5.1 Depending on the type of training delivery method used, the trainee should have access to an instructor to
resolve questions throughout the training programme. Therefore, the operator should consider the availability of
instructors when evaluating the competence acquired by the trainee through digital learning. This provides trainees with
the opportunity to resolve questions that may arise as a result of the digital learning material.

Instructor-led training following digital learning

2.5.2 The operator should consider instructor follow-up with trainees after the completion of the digital learning
portion of the training. This may be accomplished when the trainee arrives at a centralized training location and prior to
hands-on and simulated exercises (e.g. a scenario-based training such as an aircraft evacuation).

2.6 EVALUATION METHODS

2.6.1 Evaluating learning activities is crucial for both self-directed and facilitated digital learning methods. The
operator can assess trainee progress, the quality and effectiveness of the training, the methods used, and improve
future learning activities and outcomes. The operator should consider the use of the following evaluation methods:

a) tracking the trainee’s progress;

b) surveys (pre- and post-training);

c) trainee achievement of the learning outcomes (e.g. the ability to retain and apply the competencies
developed during the digital learning programme);

d) safety performance indicators (e.g. number of safety reports submitted by cabin crew members,
increase or decrease in the number of cabin-related occurrences);

e) trainee and instructor feedback;

f) assessment results; and

g) system reliability and user challenges (information technology support requests).

2.6.2 Data collated in the evaluation process should be analysed and corrective actions addressing any
identified deficiencies should be implemented in a timely manner.
 ICAO Circular 356 15

2.7 USER TRAINING

2.7.1 The operator should have a procedure to ensure all users are provided with the appropriate level of system
training. This should include specific training for various user groups such as:

a) managers;

b) instructors;

c) trainees; and

d) administrative personnel.

Media characteristic definitions

2.7.2 To simplify training, make the training efficient and to facilitate the trainee’s use of the digital learning
programme, the operator should, to evaluate suitability when selecting training media, consider the following media
characteristics:

a) navigational interaction — the amount of control the trainee has to move about within a lesson and
determine their own learning path;

b) instructional interaction — the amount of interaction the trainee has within the training content, also
referred to as interactivity;

c) interface — representation of an actual device, component, or environment;

d) graphics — graphical items within the lesson content, such as control panels and instructions for the
use of equipment, which are directly related to the training content and are not part of the training
programme interface;

e) tracking and security — the level of tracking available from within the content and reporting to some
type of logging system (“flat file”, learning management system, database, etc.);

f) familiarization — the way in which the user (instructor and trainee) is introduced to the system and
whether there is a support system (help desk) in case of difficulties;

g) integrated user assistance — integrated help or assistance for both navigational interaction (e.g.
tutorial) as well as instructional interaction (on-demand hints during a training session);

h) completeness — how much of the overall curriculum is covered by digital media and whether all
relevant aspects of featured training procedures are sufficiently covered; and

i) detail/abstraction — the level of detail chosen to convey information (graphics, sound, text, “speech
out”, charts, etc.).

______________________
 Chapter 3

QUALITY ASSURANCE IN DIGITAL LEARNING

3.1 CONTENT MANAGEMENT SYSTEM

A content management system (CMS) manages the creation and modification of digital content. It typically supports
multiple users in a collaborative environment. Most CMS include web-based publishing, format management, history
editing, and version control. CMS supports the separation of content and presentation and can provide the following
features:

a) search engine optimization;

b) integrated and online documentation;

c) modularity and extensibility;

d) user and group functionality;

e) template support for changing designs;

f) installation and upgrade wizards;

g) integrated audit logs;

h) compliance with various accessibility frameworks and standards;

i) tracking;

j) unified user experience;

k) version control; and

l) edit permission.

3.2 LEARNING MANAGEMENT SYSTEM

3.2.1 A learning management system (LMS) is a type of software for the administration, documentation, tracking,
and reporting of digital learning. The benefit of the LMS is its ability to provide, manage and store data for each
individual trainee and content. The information stored in an LMS should include, but is not limited, to:

a) trainee’s information;

b) course duration;

16
 ICAO Circular 356 17

c) time invested in lessons;

d) start date of the course activation by the trainee and date of completion of the course by the trainee
(hour, minute, second that the trainee passes the exam or completes of the course);

e) number of attempts made in the assessment;

f) the percentage of progress in the course by the trainee;

g) actual assessment of each attempt (display questions, responses marked and which should be
answered by the result);

h) proof of completion;

i) trainee course feedback;

j) result of the course (pass or fail if there is a mandatory assessment);

k) system reliability/functionality; and

l) validity and expiry of regulatory content (revision service).

3.2.2 Depending on the type of digital learning, an LMS or a CMS may not be required and the operator can
decide the best option to store relevant training programme material and records.

3.3 QUALITY MANAGEMENT PROCESS

3.3.1 To ensure content quality is consistently delivered and maintained, the operator should have an auditing
and quality management process. The content of digital learning should remain in line with the objectives set out by the
operator and should have the capability to undergo potential changes without deviating or affecting the original
objectives.

3.3.2 To maintain the highest level of quality, the following aspects should be taken into account:

a) process monitoring to ensure compliance;

b) cyber security measures to avoid unauthorized access (e.g. user identification, password, multi-device
use);

c) notifications received in relation to technical problems;

d) system ability to update and replace the contents of the course to ensure regulatory compliance,
trainee needs and the operator’s requirements; and

e) the system flexibility to upload and update the content of the course.

3.3.3 Digital learning platforms can provide data and feedback to measure instructor effectiveness, trainee
competence and programme quality. The platform should include, but is not be limited, to:

a) examination results, competency-based assessments, etc.;

18 ICAO Circular 356

b) feedback loop to address personalization of learning path, remediation, acceleration, improvement

progress and response to intervention;

c) tracking of trainee qualification in relation to trainee competence;

d) quality of the content (e.g. usability, engagement); and

e) instructor performance.

3.4 CONTINGENCY PLANNING

The operator should have a contingency plan to monitor the digital learning platform to ensure that the trainee can
complete the specified course within the allotted timeframe in the event of a system failure.

______________________
 Chapter 4

REGULATORY CONSIDERATIONS

4.1 REGULATORY APPROVAL PROCESS

Digital content that is used as a training delivery method for elements of the approved cabin crew training programme is
subject to applicable national regulations and may require approval by the State’s competent authority prior to its
implementation. The State approval process may include, but is not limited, to:

a) review of a risk assessment conducted by the operator;

b) a mitigation plan for system failures and back-ups;

c) change management, including procedural updates;

d) programme overview;

e) compliance with regulatory requirements;

f) surveillance/monitoring activities;

g) LMS and records management; and

h) quality management process.

4.2 ASPECTS FOR CONSIDERATION

4.2.1 The State should consider the following aspects as part of the regulatory approval process:

a) training delivery methods (refer to Chapter 2, 2.4);

b) media characteristic definitions (refer to Chapter 2, 2.7.2);

c) operator experience with digital learning (refer to 4.2.2);

d) limitations and suitability of digital learning (refer to Chapter 1, 1.2 and 1.7);

e) trainee characteristics (refer to Chapter 2, 2.3.3 and 2.3.4);

f) system security (refer to Chapter 3, 3.3.2);

g) instructor availability during digital learning (refer to Chapter 2, 2.5.1);

h) instructor-led training following digital learning (refer to Chapter 2, 2.5.2); and

i) testing evaluation (refer to 4.2.3).

19
20 ICAO Circular 356

Operator experience with digital learning

4.2.2 An operator that has never delivered digital learning should require additional oversight during the launch
phase, including a need for small group trials to test the training delivery method and the level of competence acquired
by trainees.

Testing evaluation

4.2.3 The State should consider the following points when evaluating digital learning:

a) Is there a testing timeline?

b) Can there be multiple attempts?

c) What is the pass/fail criteria?

d) Is there an opportunity to retake tests?

e) Is there remedial training if the trainee cannot pass the retake?

f) Does the system provide immediate feedback on all tests, whether the answer is right or wrong?

4.2.4 Remediation also may be provided in some tests at the discretion of the course developer.

4.3 SURVEILLANCE

The State should look at the aspects listed in this chapter during scheduled surveillance.

— END —