Orofc20 25

European Aviation Safety Agency NPA 2018-07(A)
3. Impact assessment (IA)
The final list of retained policy options is presented in the table below.
Table 3: Final list of policy options
Option Short title Description
0 No policy change Continuation of legacy training or ATQP for the conduct of recurrent training
and checking to flight crew. In addition, ED Decision 2015/027/R developed
interim guidance material (GM) in order to promote a standardised and
consistent means for the implementation of EBT within the existing rule
structure. It allows therefore mixed EBT implementation where EBT is
implemented only for a certain portion of the recurrent assessment and
training.
The risks and the problems stay unresolved, as mentioned in the issue analysis.
1 Voluntary EBT It provides an opportunity for the AOC holders to take a decision to implement
the EBT system for the recurrent training and checking of their pilots and to shift
It envisages a separation of the administrative work for revalidation of the pilot

licence from the technical work for assessment of the pilot.
The administrative work will be maintained and performed by the TREs, as the
current system suggests. TREs will continue to revalidate the licence; however,
not based on the simulator results, but based on evidence provided by EBT. This
option guarantees that the pilots will be assessed and their licences will be
revalidated based on additional evidence from EBT evaluation modules. The
assessment will be disconnected from the revalidation of the licence and this
would ensure a more objective revalidation process.
The revalidation of the licence will be signed by the TREs and the validity of the
licence will be maintained for 1 year (as currently).
The technical part will be performed by the AOC TRIs based on the concept of
the continuous monitoring of the pilots performance.
2 Mandatory EBT This option envisages mandatory EBT for all operators and discontinuation of
the legacy training. Implementation of mandatory use of competency-based
training in all flight crew training performed by an AOC holder. Furthermore, for
those types of aircraft subject to EBT (see Appendix 2 to Doc 9995), its
implementation would be mandatory by removing the current prescriptive
rules, thus making EBT the only alternative to ATQP.
TE.RPRO.00034-006 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified.
Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 20 of 56
An agency of the European Union
3.5. Methodology and data
3.5.1. Data collection

This RIA is performed based on several sources:
Questionnaires to the operators representatives of which participate in the RMG
Six AOC holders provided data on their experience in commencing and/or running mixed EBT
recurrent training and checking, starting from a different basis. As mentioned in the issue
analysis, within the current regulatory framework, it is only possible to achieve mixed EBT
implementation28. Due to the novelty of the EBT approach in the EU context, there is not
extensive expertise in the EASA Member States operators/ATOs for its implementation.
Information was, therefore, sought also outside the EU. In the analysis of the responses to the
questionnaires, an important distinction was made with regard to the type of recurrent training
and checking the operator had before they implemented mixed EBT. Some operators before the
implementation of mixed EBT, performed legacy training, others provide ATQP to their pilots. In
addition, the size of the operator has also been considered in estimating the impacts. The
analysis, therefore, recognises different impacts for small/medium operators in starting and
running EBT.
Questionnaire to other operators who implement EBT for recurrent training and checking
part from data from the RMG, complementary sources (e.g. data from other operators) were
sought to ensure that different business models and operators who have initiated a shift
towards EBT are represented. Hence, the analysis of the expected costs and benefits is based on
real cases.
Questionnaire to the competent authorities
Data regarding the impact of EBT on the workload, fees, charges and internal work was analysed
from EU and non-EU competent authorities. They were contacted as part of or through the RMG
members, some of whom were representatives of competent authorities.
3.5.2. Methodology applied

The RIA is developed by combining different impact assessment tools:
Cost Benefit Analysis (CBA): it is applied for the economic impacts, because the data on benefits and
costs was sufficiently quantified and monetarised. The method entails identification and evaluation of
the expected economic benefits and compliance costs for the industry to implement EBT. The outcome
is expressed in terms of net present value (NPV), cost-benefit ratio and year of return of investments in
EBT. Apart from the advantage of measuring and quantifying the net benefits, the CBA is also chosen
due to its quality to quantify the costs and benefits over time. This is considered highly relevant,
because the benefits of EBT are unevenly distributed in the years of the EBT implementation and the
CBA captures that feature. However, the CBA is performed mainly for the economic impacts. There is
no quantification of the safety impacts, because of the limitations of data currently available.
Consequently, safety impacts will be qualitatively assessed. The reference period in the CBA is 10 years
28
Mixed EBT implementation means that only for a portion of the recurrent assessment and training there is application of EBT.
(2018-2027) and the analysis is made following the European Commission guidelines for the CBA29. The
CBA is performed in several steps:
STEP 1 Estimation of the costs of recurrent legacy/prescriptive training and checking
implementation (baseline scenario)
STEP 2 Estimation of the costs (one-off and recurrent) for preparation and implementation of EBT
STEP 3 Calculation of the difference between costs for EBT recurrent training and checking and
legacy recurrent training and checking (delta of EBT costs)
STEP 4 Distribution of the one-off and recurrent costs for EBT, as well as recurrent costs for legacy
STEP 5 Quantification and monetisation of the economic benefits of EBT, e.g. alleviations envisaged
in the regulatory proposal for operators who might be granted with these privileges upon the
discretion of the com
time
STEP 6 Comparison of the economic benefits and costs for EBT. Calculation of cost-benefit ratio and
year of return of investments in EBT
Multi-criteria analysis: multi-criteria analysis allows comparison of all options by scoring them against
trade-off between different impact assessment criteria, e.g. low scores on one criterion may be
compensated by high scores on another. MCA covers a wide range of techniques that aim at combining
a range of positive and negative impacts into a single framework to allow easier comparison of
scenarios. The scoring of the impacts uses a scale of 5 to + 5 to indicate the negative and positive
Table 4: Scale with scoring of the impacts

Positive Score Negative Score
impact impact
+5 Very high positive impact 5 Very high negative impact
+4 High positive impact 4 High negative impact
+3 Medium positive impact 3 Medium negative impact
+2 Low positive impact 2 Low negative impact
+1 Very low positive impact 1 Very low negative impact
0 Neutral
29
http://ec.europa.eu/smart-regulation/better_regulation/documents/com_2015_215_en.pdf
For the economic impacts, the above scale is further detailed with the scores using the budget of the
CAs and the turnover of the airlines (for the EASA MS geographic area). The CBA provides the outcome
of the economic impact of the regulatory options. These impacts can be measured in relative share
against the budget or turnover of a stakeholder group.
Table 5: Definition for the economic scale

COMPETENT AIRLINES
AUTHORITIES
2 800 220 000
2016
QUALITATIVE DESCRIPTION Score Turnover impact
VERY HIGH IMPACT +/- 5 > +1.5% >42.0 >3316.5

]1 to 1.5 %[ 42.0 3316.5
HIGH IMPACT +/- 4 ]0.8 to 1 %[ 28.0 2211.0
]0.6 to 0.8 %[ 22.4 1768.8
MEDIUM IMPACT +/- 3 ]0.4 to 0.6 %[ 16.8 1326.6
]0.2 to 0.4 %[ 11.2 884.4
LOW IMPACT +/- 2 ]0.1 to 0.2 %[ 5.6 442.2
]0.05 to 0.1 %[ 2.8 221.1
VERY LOW IMPACT +/- 1 ]0.02 to 0.05 %[ 1.4 110.6
]0 to 0.02 %[ 0.6 44.2
NEUTRAL 0 0.3 22.1
Case studies: Due to the complexity of the issue and the existence of different business models for
operators30, it is difficult to represent all different models and analyse their impact. Therefore, it is
suggested to illustrate the impact through examples of the impacts for the options mentioned above.
Hence, there are two examples of operators who undertake EBT. One is a small size operator (with a
small fleet and 100 pilots) and the other one is a medium/large operator (with a medium fleet and
1 000 pilots). For larger operators, the medium fleet/1 000 pilots can be easily extrapolated with
simple maths (e.g. 1 000 pilots multiplied by 10 10 000 pilots). All assumptions regarding the type of
the operators are presented below.
Impacts analysed: The present RIA analysed the impacts considering several criteria: safety, social,
economic. These criteria follow the main objectives of the Basic Regulation. The analysis of the
General Aviation is not affected as their
aircraft types are not currently included in EBT. Therefore, General Aviation is not in the scope of this
proposal31 and proportionality issues are analysed in the economic impacts. The proposed approach
ensures that the impact is assessed only once, avoiding any risks of double counting.
In addition, it is important to note that each option was analysed separately, considering the
baseline. The assessment of the impacts took into consideration potential costs and benefits
having in mind the baseline scenario. However, as policy options 1 and 2 are very similar in
terms of content (both bear the same elements and regulatory proposals), the differentiation
30
Point to point operator (P2P), Hub and Spoke operator (H&S), Operators running mainly long-haul and/or and short-haul flights or
combination of them, etc.
31
RMT.0599 may study the extension of EBT to General Aviation in 2019.
between them is mainly in the way EBT will be implemented by the stakeholders: in the first case
on a voluntary basis, in the second case as a compulsory requirement.
3.6. What are the impacts
3.6.1. Safety impact

Option 0 No policy change
Safety risks continue to exist as mentioned in the issue analysis. The exposure to the safety risks will
remain and the current safety level will be maintained. Therefore, the safety level score is assessed as
neutral (score 0).
Option 1 Voluntary EBT
Due to the nature of EBT, it is difficult to assess its safety impact. However, there is evidence showing
that the proper implementation of EBT will significantly improve aviation safety by strengthening the
core competencies of flight crew and helping them to handle abnormal and unexpected situations
safely.
There have been several research studies as regards the development of the EBT concept and its
implementation assessment. In the context of this RIA, it is worth mentioning the Man4Gen Study32,
which showed the transferability of the core competencies between scenarios. This conclusion would
reinforce the idea that training the flight crew in core competencies, rather than in executing specific
manoeuvres, enables them to handle a wider range of scenarios with higher levels of resilience.
In addition, the experience of operators that have implemented EBT or training methodologies with
similar goals (i.e. ATQP, the Federal Aviation Administration Advanced Qualification Program AQP)
shows that flight crew are better prepared to take over highly-automated operations and to apply a
more consistent and quick decision-making in those moments with high workload. These operators
have also noticed the positive feedback from the flight crew being trained with those training
methodologies, as they feel better prepared, especially with regard to the performance of the line-
orientated evaluation (LOE). The data33 shows that the remedial training for flight crew who fail in the
LPCs and OPCs is reduced by half (50 %) after the implementation of mixed EBT. Apart from a
significant benefit from a safety perspective, this also has a positive economic implication.
It is expected that the safety benefit of EBT would be demonstrated over time by continually improving
a system targeted at focused learning34. The implementation of the programme should ensure a level
of safety at least equivalent to that provided by compliance with ORO.FC.230 of Appendix 9 to Annex I
(Part-FCL) to the Aircrew Regulation, by continuing to focus on legacy items of check, albeit within a
different structure. Safety benefits should be expected through a qualitative approach, using core
competencies to develop resilience by exposure to challenging situations.
This option would have the potential to deliver a significant improvement in safety. An EBT programme
is intended to identify operational risks by using multiple sources of operational data to determine the
32
www.man4gen.eu
33
Based on the feedback by operators who implemented full EBT worldwide, 2008-2015, EASA questionnaire 2016
34
IATA, Data Report for Evidence-based Training, 2013.
prioritisation of training topics linked to a competency-based training framework. As examples35, the

analysis of worldwide data revealed consistent and significant risks in the following areas:
Flight path Manual aircraft control: Manual aircraft control skills of pilots can be expected to
deteriorate over time as aircraft design improves and the use of automation increases unless
supported by training to maintain and further develop these skills.
The unstable approach paradox: Despite the reduction in unstable approaches, the go-around
remains a high-risk flight phase, and increased training focus on go-arounds mitigates this risk.
Crew resource management: According to the Data Report for EBT, flights where outstanding
communication and leadership have been observed involve significantly less errors and
undesired aircraft states than flights where poor leadership and communication have been
observed. This reinforces the need for a continued focus on these skills.
Surprise: The effect of the current high levels of operational safety is that the effects of surprise
may compound any event. Training to react and recover from surprise events forms a key part of
the EBT programme.
Prioritisation of training topics: The prioritisation of training topics is the most important result
from the data analysis of the EBT report. This process involves the assessment of inputs from
multiple data sources and ranking of threats, errors and competencies, as well as causal factors
from incidents and accidents. The process is a key part in translating data into useful training
events and scenarios that can be used to assess and improve pilot performance in recurrent
training programmes. The process used for the Data Report for EBT is transparent and
repeatable and results in a list of prioritised training topics. Three levels of priority (A, B and C,
with A having the highest priority) are used to determine the frequency of pilot exposure to the
training topics within a 3-year rolling recurrent training programme.
In addition to the improvement from an increased take-up of EBT, aircraft operators would be able to
develop less complex training programmes, tailored to the identified risks, and to implement the
principles of competency-based training in all training programmes. The overall result would be better
training of the pilots involved and a lower flight-crew-related accident rate in the future. Although, the
implementation of EBT within this option is voluntary, it is expected that a significant number of
operators will apply EBT due to the benefits it renders. Hence, the safety impact of Option 1 is scored
as +2 (low positive impact).
35
IATA, Data Report for Evidence-based Training, 2013

Orofc20 25

Uploaded by

Copyright:

Available Formats

Orofc20 25

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Orofc20 25

Uploaded by

Copyright:

Available Formats

European Aviation Safety Agency NPA 2018-07(A)

3. Impact assessment (IA)

Option Short title Description

It envisages a separation of the administrative work for revalidation of the pilot

3.5. Methodology and data

3.5.1. Data collection

3.5.2. Methodology applied

Table 4: Scale with scoring of the impacts

Table 5: Definition for the economic scale

VERY HIGH IMPACT +/- 5 > +1.5% >42.0 >3316.5

3.6. What are the impacts

3.6.1. Safety impact

prioritisation of training topics linked to a competency-based training framework. As examples35, the

You might also like