06/02/2019

SMS Auditing
Focus on the main points of a SMS
How to implement?
How to audit?
Toulouse March 2019

Pierre ROSELLINI
[email protected]

Training

Monday : Theoretical training.

Friday : Theoretical training.

Wednesday : Practical training => audit preparation.

Thursday : Practical training => audit realization, writing audit report.

Friday : Practical training => writing audit report, audit restitution.

1
 06/02/2019

SMS auditing
Focus on the main points of a SMS : policy, responsibilities,
documentation, records, process of safety occurrences, safety
studies, indicators, internal and external audits, corrective actions,
continuous improvement, safety reviews ...
For each point : how to implement? How to audit?
Context of the audit : definitions, goal of an audit, different types of
audit, role and responsibility of the auditors.

Audit organization : preparation, realisation, report.

Behaviour : ethics, technique of audit, traps to avoid.
Writing of audit guides.
Audit exercises :
- Preparation
- Realisation
- Report.

SMS auditing
Focus on the main points of a SMS : policy, responsibilities,
documentation, records, process of safety occurrences, safety
studies, indicators, internal and external audits, corrective actions,
continuous improvement, safety reviews ...
For each point : how to implement? How to audit?
Context of the audit : definitions, goal of an audit, different types of
audit, role and responsibility of the auditors.

Audit organization : preparation, realisation, report.

Behaviour : ethics, technique of audit, traps to avoid.
Writing of audit guides.
Audit exercises :
- Preparation
- Realisation
- Report.

2
 06/02/2019

Convention of Chicago

54 of the 55 States invited attended the

Chicago Conference, and by its conclusion on 7
The 1944 Chicago
December, 1944, 52 of them had signed the
Convention new Convention on International Civil
Aviation.

ICAO Framework

The International Civil Aviation Organization (ICAO) is an UN

specialized agency, established by States in 1944 to manage the
administration and governance of the Convention on International Civil
Aviation (Chicago Convention).

ICAO works with the Convention’s 192 Member States and industry
groups to reach consensus on international civil aviation Standards
and Recommended Practices (SARPs) and policies in support of a
safe, efficient, secure, economically sustainable and environmentally
responsible civil aviation sector.

3
 06/02/2019

ICAO Framework

A catalogue with all the ICAO publications

is available on ICAO internet site. You can
find there all the documentation you need.

ICAO Framework

DOC 9859 = Safety

Management Manual
(Guidance on Annex 19
implementation of SMS, requirements for
SSP …). SMS, SSP …

4
 06/02/2019

ICAO Framework

Annexes must be
translated into State laws
and regulations to become
mandatory. Annex 19 has
been translated into Civil
Aviation Safety Regulation
CSAR Part 19.

CSAR 19 has been built on

annex 19 and DOC9859.

ICAO Framework

The first edition of Annex 19 (July 2013)

was adopted by the council on 25 February
2013 and becomes applicable on 14
November 2013.
The second version of Annex 19 (July
2016) will be applicable on 7th november
2019. This module has been built on
Annex 19 V2.

Annex 19 is dedicated to :
- States authorities for implementation of
State Safety Program (SSP) and others
items,
- Service providers for implementation of
Safety Management Systems (SMS).

5
 06/02/2019

ICAO Annex 19

Standards and recommended

practices for states authorities :
SSP and safety oversight.

ICAO Annex 19

Standards and recommended

practices for service providers.

Service providers :
- Approved training organizations
- Air operator
- Approved maintenance
organizations
- Manufacturers of aircrafts
- ANSP
- Operators of certified aerodromes

6
 06/02/2019

ICAO Annex 19

Both states and providers

are concerned.

ICAO Annex 19

Annex 19 Appendix 2 :
framework for a safety
management system. 4 SMS
main items that are the 4
pillars of the SMS.
1. Safety Policy and Objectives

2. Safety Risk Management

1. Safety Policy and Objectives

4. Safety Promotion
3. Safety Assurance

2. Safety Risk Management

3. Safety Assurance

4. Safety Promotion

7
 06/02/2019

Quality approach QMS

ISO 9001 will be useful to implement a SMS.

Quality can be consider as a toolbox to build the SMS : procedures

(management of documentation, audit, records, safety committee,
corrective actions …), use of mapping, process approach …

Quality concept can be used without going to an ISO9001 certification.

SMS vs QMS

DOC9859 V3
5.4.2.8

Relationship between
SMS and QMS.

8
 06/02/2019

Integrated management System (IMS)

If you have both a SMS and a QMS, you have an Integrated management
System (IMS).
In an IMS you can have :
Safety Management System (SMS)

Quality Management System (QMS)

Security Management System (SeMS)

Environmental Management System (EMS)

Occupational Health and Safety Management System (OHSMS)

Financial Management System (FMS)

Documentation Management System (DMS)

Fatigue Risk Management System (FRMS)

Integrated management System (IMS)

If you have an IMS you must define priorities.

Example of priorities if you have SMS, SeMS, EMS and QMS.

Safety Management System (SMS) Security Management System (SeMS)

Environmental Management System

(EMS)

Quality Management System (QMS)

9
 06/02/2019

QMS ISO 9001 V2015

4 CONTEXT 5 Leadership 6 Planning 7 Support 8 Operations 9 Evaluation 10 Improvement
4.1 Understand 5.1 Provide 6.1 Define actions to 7.1 Support your QMS 8.1 Develop, implement, 9.1 Monitor, 10.1 Determine
your organization leadership by manage risks and by providing the and control your measure, improvement
focusing on quality address necessary resources operational processes analyze, and opportunities and
and customers opportunities evaluate QMS make improvements
performance
4.2 Clarify the 5.2 Provide 6.2 Set quality 7.2 Support your QMS 8.2 Determine and 9.2 Use internal 10.2 Control
needs and leadership by objectives and by ensuring that document product and audits to nonconformities and
expectations of establishing a develop plans to people are competent service requirements examine take appropriate
interested suitable quality achieve them conformance and corrective action
parties policy performance

4.3 Define the 5.3 Provide 6.3 Plan changes to 7.3 Support your QMS 8.3 Establish a process to 9.3 Carry out 10.3 Enhance the
scope of your leadership by your quality by explaining how design and develop management suitability, adequacy,
quality defining roles and management system people can help products and services reviews and and effectiveness of
management responsibilities document your your QMS
system results
4.4 Develop a 7.4 Support your QMS 8.4 Monitor and control
QMS and by managing your external processes,
establish communications products, and services
documented
information
7.5 Support your QMS 8.5 Manage and control
by controlling production and service
documented provision activities
Exercise : try to find what information

requirements are the same 8.6 Implement

arrangements to control
than for SMS. product and service
release

8.7 Control
nonconforming outputs
and document actions
taken

ICAO Annex 19

Annex 19 Appendix 2 :
framework for a safety
management system. 4 SMS
main items that are the 4
pillars of the SMS.
1. Safety Policy and Objectives

2. Safety Risk Management

1. Safety Policy and Objectives

4. Safety Promotion
3. Safety Assurance

2. Safety Risk Management

3. Safety Assurance

4. Safety Promotion

10
 06/02/2019

ICAO Annex 19

1st pillar of a SMS :

SMS

1 Safety policy and objectives

1.1 Management commitment.
1.2 Safety accountabilities and responsibilities.
1.3 Appointment of key safety personnel.
1.4 Coordination of emergency response planning.
1.5 SMS documentation.

1 Safety Policy and Objectives

1.1 Management commitment

Main points :
Commitment at the
highest level
Safety policy
Just culture

11
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment

At the highest level, the commitment must be very important.

The highest level must allocate resources in order to implement the SMS.

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy
DOC9859 V3
5.3.12 Example of safety policy

Safety first

Everybody is
involved

Resources

Responsibilities
Safety risk
management

12
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Just culture

Compliance

Training

Safety indicators
and objectives
Survey : audits,
safety management
review.
External providers

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Examples of safety policy.

Exercises : analysis of these policies.

13
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Extract of policy of the French ANSP.

The policy or strategy of DSNA is defined by the Director of the DSNA under the
authority of the Director General of Civil Aviation (DGCA) and related services, to
achieve the objectives of the LOLF.
It is materialized by a set of strategic directions defined in the 5 years plan and
translated in each annual action plan …
At this end we give priority to the following strategic axes:
1) Ensure a high level of safety and security of air navigation
2) Controlling the environmental impact of air traffic
3) Improve delays.
4) Improve the economic efficiency of air navigation services
5) Participate in European construction
6) Consider the needs of general aviation in the establishment procedures and
airspace structures.
I rely on the involvement of all staff to address these objectives at all levels, in order
to strengthen the unit and the image of DSNA.
I will ensure its implementation and its effectiveness.

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

No compromise
EasyJet does not compromise on safety. We have established a leading-edge
Safety Management System and Fatigue Risk Management System which are well
established and incorporate rigorous reporting processes. Through these
systems EasyJet works to drive safety performance improvements and reduce
risks to its people, passengers and suppliers.
EasyJet has long pioneered innovative solutions to improve safety and continues
to seeks better ways to support our people to improve safety performance.
Our strong focus on safety has helped easyJet to meet the new regulatory
compliance requirements prescribed by the European Aviation Safety Agency
(EASA).
EasyJet continues to work closely with EASA on the development of future
safety regulations. This has included EasyJet’s Director of Safety and Security
being a member of EASA’s taskforce following the Germanwings incident.

14
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

15
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

16
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

The policy must explain how the service provider is trying to reach a
good safety level. Safety must be the highest priority (vs other items :
commercial, social, environmental and operational).

The policy must include several axes of improvement.

The axes of improvement must be translated in an annual action plan.

For each axe you must find several actions.

The policy must be periodically renewed.

The policy must be communicated, known and applied.

17
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Safety policy diffusion

Policy

Statement pinned up in
organization’s facilities

SMS manual

During briefings or
safety trainings
Letter sent to
every employee

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Links between policy, business plan, action plan and organization.

Business plan (5 years) Annual action Plan

Policy
Actions
Axes of
improvement

The action plan is a tool that must be used daily by the main executives
of the organization.

18
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy
Action Plan
Summary

Every year the main objectives of the policy

are translated in an action plan.

For the French ANSP the policy defines 4 main

axes :
- Safety and security
- Environment
- Management and user services
- European construction

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Example of action (Paris ACC)

19
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Monitoring of the Action Plan

Implementation of the action plan is

monitored through monthly reviews.

1 Safety Policy and Objectives

1.1 Management commitment
Safety Policy

Action completed and closed

Dates of monthly Actions done

monitoring

20
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture
JUST CULTURE means a culture in which front-
line operators or other persons are not punished
for actions, omissions or decisions taken by
them that are commensurate with their
experience and training, but in which gross
negligence, wilful violations and destructive acts
are not tolerate.

Unintentional, unwilling error Non punitive culture

Unacceptable behavior Punitive culture
Voluntary violation of the rule, the law

1 Safety Policy and Objectives

1.1 Management commitment
Safety Culture

21
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

22
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

This video has been done in Maastricht

UAC (Operated by Eurocontrol with 4
countries : Belgium, Germany,
Luxembourg, Netherlands).
Try to note : who, most important
words …

Debriefing after video.

How evaluate the level of safety culture and of just culture in a

company?

23
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture
Example of questionnaire for evaluation of just culture.

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

24
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

25
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

1 Safety Policy and Objectives

1.1 Management commitment
Just Culture

Results for French ANSP (DSNA)

Target associated Training on just culture has not been

No target associated done by French ANSP

26
 06/02/2019

How to audit a requirement?

How audit a requirement?

Providers : what must I implement?

Civil Aviation authority (CAA) : what are we going to audit and how?

Writing an audit guide for a requirement or a family of requirements.

Who and how? Providers with CAA.

Audit guide

Content of an audit guide.

We are going to build

an audit guide for each
requirement or family of
requirements..

27
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment

1 Safety Policy and Objectives

1.1 Management commitment

28
 06/02/2019

1 Safety Policy and Objectives

1.1 Management commitment

1 Safety Policy and Objectives

1.1 Management commitment

29
 06/02/2019

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities

Main points :
Accountable executive
Safety responsibilities : document and communicate

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
Accountable executive

The accountable executive is the chief of the organization (chief of ACC,

chief of airport, DG airline …).

30
 06/02/2019

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
Document and communicate responsibilities

Everyone involved in safety must have responsibilities and duties

clearly defined (decision, manual, function sheet …)..

Everyone must know and apply his responsibilities.

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
Document and communicate responsibilities

Example of responsibilities : safety manager

ACTIVITIES
He is directly linked to the chief of the center and is independent from the operational
hierarchy. It participates in weekly management meetings to have a cross-sectional view of the
center safety and security issues.
Under the authority of the chief of the center:
• He coordinates the implementation and operation of the SMS.
• He ensures that the SMS provisions are applied.
• He collects and proposes, at the appropriate decision-making level, modifications to improve
the functioning of the SMS.
• He organizes the management safety reviews of the center and provides secretarial support.
• He ensures that the security monitoring function is provided in the center.
• He coordinates the security coordination of the center.
• He organizes internal audits.
• He is the point of entry for all external audits taking place in the center
• He manages the ACAP table (corrective actions / preventive actions).
• He ensures the implementation of risk assessment and mitigation actions.
• He notifies safety, security, … events of which it is aware

31
 06/02/2019

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

Main points :
Appointment of safety manager
Position of safety manager

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

DOC 9859 V3 Functions of a safety manager

32
 06/02/2019

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

You must identify a safety management function with responsibility for

development and maintenance of the safety management system.

This function must be accountable directly to the highest organizational

level.

This function must be independent of the operational management.

If you have also a Quality Management System (QMS) quality manager

and safety manager can be the same.

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

French ANSP Headquarters

French ANSP : a dedicated organization (sub-directorate) has been

identified and created at the national level.

33
 06/02/2019

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

RSMS = Responsible of the

ACC or Airport Safety Management System.
Organization RSMS = safety manager

Deputy ACC Chief RSMS

Operations Technical Administrative

Division Division Division

Airports and ACC : a dedicated function (RSMS = Responsible of the

SMS = safety manager) has been identified and created in each ACC and
airport.

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

Airline organization
Accountable
Manager
Safety Quality
Manager Manager

Flight OPS Maintenance Ground OPS Training

Manager Manager Manager Manager

Safety Safety Safety

officer officer officer

34
 06/02/2019

1 Safety Policy and Objectives

1.3 Appointment of key safety personnel

Airline organization
Accountable
Manager
Safety & Quality
Manager

Flight OPS Maintenance Ground OPS Training

Manager Manager Manager Manager

Safety Safety Safety

officer officer officer

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
1.3 Appointment of key safety personnel

35
 06/02/2019

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
1.3 Appointment of key safety personnel

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
1.3 Appointment of key safety personnel

36
 06/02/2019

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
1.3 Appointment of key safety personnel

1 Safety Policy and Objectives

1.2 Safety accountabilities end responsibilities
1.3 Appointment of key safety personnel

37
 06/02/2019

1 Safety Policy and Objectives

1.4 Coordination of emergency response
planning

Main point :
Contingency plan

1 Safety Policy and Objectives

1.4 Coordination of emergency response
planning

A contingency plan contains arrangements to ensure the continued

safety of air navigation in the event of partially or total disruption of air
traffic services (ATS).

The contingency plan should be designed to provide alternative routes,

using existing airways in most cases.

38
 06/02/2019

1 Safety Policy and Objectives

1.4 Coordination of emergency response
planning

Example of events : Volcanic eruption in Iceland (2010)

1 Safety Policy and Objectives

1.4 Coordination of emergency response
planning

Example of events : Volcanic eruption in Iceland (2010)

Airspace has been closed over

Europe during 4/5 days.

At the end of April 2010, the

European Commission estimates
losses for the aviation sector and
tour operators between 1.5 and
2.5 billion euros.

39
 06/02/2019

1 Safety Policy and Objectives

1.4 Coordination of emergency response
planning

Other cases (weather, runway excursion …)

Storms Snow Runway excursion

Computer failure Strike (ATCO, pilots …)

1 Safety Policy and Objectives

1.4 Coordination of emergency response
planning

To be defined

40
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation

The documentation of
the SMS can be SMS The SMS manual is at the top of the pyramid. It can be also an IMS
represented by Manual manual (if SMS + QMS + …).
a five floors pyramid.
Mapping, Mapping, processes are quality items (ISO9001).
Process, Activities

Procedures : Audits,
Some procedures are used to be
Documentation, Records, compliant with safety and quality
Corrective Actions, requirements.
Management review …

Operational Documentation, ATCO Manuals

Technical documentation, ATSEP Manuals, Pilots
documentation …

Records: Minutes management review, Corrective Actions, A record is :

- A proof of an activity (minutes),
Dashboards, Safety occurrences, Reports of audits, Strips, Data - A result of an activity (indicator).
radar , Flight plan data …

1 Safety Policy and Objectives

1.5 SMS Documentation
SMS manual

DOC 9859 V3 Contents of a SMS manual

41
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation
SMS manual
Example of SMS
manual summary 1 General informations
(Paris ACC). 1.1 Missions
1.2 Organization
The SMS manual
explains how the SMS 2 Policy and objectives
requirements are 2.1 Policy
2.2 Safety accountability and responsibilities
implemented by the 2.3 Safety manager
provider. 2.4 Coordination of emergency response planning
2.5 SMS documentation

3 Safety risk management

3.1 Safety occurrences
3.2 Changes and risk assessment and mitigation

4 Safety assurance
4.1 Safety survey
4.2 Continuous improvement of the SMS

5 Safety promotion
5.1 Training
5.2 Safety communication

1 Safety Policy and Objectives

1.5 SMS Documentation
SMS manual

Anything you write in the SMS manual must be done ​and demonstrated.
It is a new layer of requirements.

So avoid writing too much.

42
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

What is the SMS documentation ?

SMS documentation = all the safety documentation of the provider

(operational documentation is included in SMS documentation).

How to manage documentation ?

We can use ISO 9001 (quality) tools in order to explain in a procedure

how we manage the documentation.

We can identify different families of documents. For each family we

define management rules.

Who write, who verify, who approve ? What is the current version ?
Paper or electronic document ? Where is the document of reference ?

SMS documentation must be kept on date.

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

Management of documentation : example SMS manual

Responsibilities
Writing Checking Approving
Name : Name : Name :
Function : Safety Manager Function : Safety Manager Function : Chief ACC
Date : 15/01/2019 Date : 15/01/2019 Date : 20/01/2019
Signature : Signature : Signature :

Evolutions
Date/Version Content Who
28/10/2018 V1.0 Initial version Safety manager
20/01/2019 V1.1 Integration risk assessment and Safety manager
mitigation new procedure

43
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

The ANSP shall provide and keep up-to-date operations manual.

What is operations manual ? For French ANSP operations manual is :

- Manual for ATCOs
- Manual for Chief of ATCOs room.
- Manuals for sectors.
- ATFM manual.
- Instructions.
…

We have to define rules in a dedicated procedure in order to manage all

these documents.

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

In this procedure we must put rules of management : who write, who

verify, who approve ? What is the current version ? Paper or electronic
document ? Where is the document of reference ?

In this procedure we can put practical informations : name of a file, path

of a file, how to modify the file, when we must modify the file …

With such a procedure it’s very easy to build an electronic

documentation.

44
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

Example : Paris ACC procedure for ATCO documentation.

Summary
1 Goal
2 Principle of application
2.1 General organization
2.2 Path and access rights on the network
2.3 Instructions
2.4 ATCO manual
2.5 Chief of ATCO room manual
2.6 Sectors manuals
2.7 ATFM manual
…

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

Example : Paris ACC procedure for ATCO documentation.

2.6 Sectors manuals

2.6.1 Goal
2.6.2 Responsibilities
2.6.3 Identification
2.6.4 Diffusion
2.6.5 Classifying, filing
…

2.6.2 Responsibilities
The sector manuals are drafted by the control subdivision, verified and approved
by the head of the control subdivision.
The other subdivisions may be requested by the control subdivision for re-
reading operations.

45
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation
Management of the documentation

Operational documentation site

1 Safety Policy and Objectives

1.5 SMS Documentation
Records

A demonstrative element is :
- A proof of an activity (minutes),
- A result of an activity (indicator).

An ISO tool exists : a documented procedure for the demonstrative

elements. This documented procedure defines several items :
identification, storage, protection, retrieval, and retention of records

46
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation
Records

Record Acces Who Where Support Duration Archivage

Minutes of Safety
management Free Manager
G:\Dir_CRNA\SMQS\revues Electronic 3 years 5 years
review
Audit reports Safety
Free Manager
G:\Dir_CRNA\SMQS\audits Electronic 3 years 5 years

Chief
Free Subd G:\EXPLOITA\Doc_Ops\LOA Electronic 3 years 5 years
LOAs
OPS
Safety Chief
occurrences Confide Subd Office Chief Subd Safety Paper 3 years 5 years
ntial
analysis Safety

1 Safety Policy and Objectives

1.5 SMS Documentation

47
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation

1 Safety Policy and Objectives

1.5 SMS Documentation

48
 06/02/2019

1 Safety Policy and Objectives

1.5 SMS Documentation

1 Safety Policy and Objectives

1.5 SMS Documentation

49
 06/02/2019

ICAO Annex 19

SMS

2. Safety Risk Management

2.1 Hazard identification
2.2 Safety risk assessment and mitigation

2.1 Hazard identification = a safety occurrence has occurred.

2.2 Risk assessment and mitigation : before a planned change.

ICAO Annex 19

SMS

2. Safety Risk Management

2.1 Hazard identification
2.2 Safety risk assessment and mitigation

2.1 Hazard identification = a safety occurrence has occurred.

2.2 Risk assessment and mitigation : before a planned change.

50
 06/02/2019

2 Safety Risk Management

2.1 Hazard identification

Main points :
Safety occurrences : report, notification, analyze, corrective action,
feed-back ….
Pro-active actions

Safety Occurrences
Methodology

Regulations :
ICAO Annex 19, DOC 9859,

This methodology explains

how to do the assessment
of the safety occurrences :
French ANSP Methodology report, notification,
analysis …

Safety events : How to do

analysis ? How to classify
Assessment of severity and frequency ?
safety occurrences How to search causes ?
The regulations are not
enough explicit.

51
 06/02/2019

Safety Occurrences
Methodology

French ANSP Methodology

Procedure : Manual : Tools

Findings/ Assessment of Data base ECCAIRS
Corrective Actions Safety Occurrences Risk Analysis Tool RAT

Safety Occurrences
Methodology: procedure

Procedure :
Findings/
Corrective Actions

This procedure take into account different findings :

- Main causes of safety occurrences,
- Audit reports (non compliance, weakness points …),
- Recommendations …
This procedure defines :
- The links between findings and corrective actions,
- The management of the corrective actions.

52
 06/02/2019

Safety Occurrences
Methodology: manual

This manual is used by Safety Subdivisions (ATM and CNS).

Safety Occurrences
Organization (French ANSP)

Paris ACC ATM Division :

The Safety Team
ATM Division

OPS
Studies Training Safety
Safety
ATCO Team
Team
Chief
Deputy
Chief and deputy of safety 4 ATCOs
team are very often ATCOs. 4 Assistants

53
 06/02/2019

Safety Occurrences

Runway incursions : Danger

Bad radio
Vehi … grrr Causes communications,
unway Human error …

Incursion of a
What did he Safety Event vehicle on the
say ? runway
We could go
… I believe

Be careful. Be sure that the radio Interrupted landing,

communications are understood Consequences collision …
by your correspondent

Safety Occurrences
Methodology: procedure

To find the causes, we have to

Causes
investigate the safety event.

Safety Event A safety event must be reported.

Unfortunately the only thing we

Consequences can do is to try to minimize the
consequences.

54
 06/02/2019

Safety Occurrences
Methodology: procedure

X
Causes If we delete the causes …

To avoid the occurrence of a safety

Safety Event event we have to delete the causes.

X
Consequences … we also delete the consequences.

Columbia disaster (2003)

The Space Shuttle Columbia crash was a space accident that occurred
on February 1, 2003, during Mission STS-107.
During the atmospheric reentry phase, the Columbia shuttle was
destroyed over Texas and Louisiana and the seven crew members were
killed.

55
 06/02/2019

Columbia disaster (2003)

This normalization of deviance has led to no longer consider, little by

little, that a risk can actually cause a serious accident, even a disaster.

This risk was no longer considered a risk to be solved since it had

never caused an accident before ... while failing to assess the
probability that this risk could lead to accidents in the future.

All the safety occurrences known must be

reported … even if they have no immediate
consequences on safety…

Safety Occurrences process

Beginning Report

Notification Immediate action

(if necessary) (if necessary)

Record in database

Analyse (severity,
frequency, causes)

Corrective actions
(decide, implement,
monitor)

56
 06/02/2019

Safety Occurrences report

Iceberg theory The more safety

occurrences we know, the
Safety more corrections we do.
occurrences
reported Of course a SMS will help
us, year after year, to
increase the number of
occurrences reported.

Safety
occurrences
not
reported.

Safety Occurrences report

JUST CULTURE means a culture in which front-line operators or other

persons are not punished for actions, omissions or decisions taken by
them that are commensurate with their experience and training, but in
which gross negligence, wilful violations and destructive acts are not
tolerate.

Unintentional, unwilling error Non punitive culture

Unacceptable behavior
Punitive culture
Voluntary violation of the rule, the law

With JUST CULTURE a provider increase the number of safety events reported.

57
 06/02/2019

Safety Occurrences report

Report

Safety
Occurrence

ATCOs, ATSEPs,
Pilots, Cabin crews …
List of events
to be notified Immediate Actions
Real Time
to managers if necessary

Record event
in database
ECCAIRS
Database
ECCAIRS : European Coordination
Centre for Accident and Incident
Toward next step Reporting Systems.
(analysis)

Safety Occurrences
How to report

How to Report safety occurrences ?

The reports of the safety occurrences are done by using a form or

electronic report submission.

Different forms are available :

- ANSP : form for ATCO (AIRPROX, Losses of separation …).
- ANSP : form for ATSEP (technical events, failure of systems …).
- Airlines : Air Safety Report (AIRPROX ATC, NAVAID, TCAS …).
- Airlines : Cabin form (event in cabin involving passage or crew, injury,
evacuation …).

58
 06/02/2019

ATCO Form 1/2

X RATCAS

X
TP FL280 28/10/2014 12:15 X

AFR F-ABCD A320 LFPG LFML 1457 X IFR

BAW G-KLMN B737 EGLL LIRF 2812 X IFR

123,45 YES NO

A AWY Radar control

2,4
X X X
600 X

ATCO Form 2/2

Explanation of the incident by ATCO involved.

ATCO Paris/ACC 01:05 04:05 X

None
X

59
 06/02/2019

Proactive reports
(without safety occurrences)

Personnel is encouraged to report any incident or problem.

The form used to report safety occurrences can be used to

mention any problem encountered.

For each problem mentioned a reply is done.

The safety ATCO (2 in each team) must allow a good

communication between ATCO teams and safety subdivision.

Notification Process
French ANSP (DSNA) notification Process (mail, phone …).
Crisis unit END of notification process
ACC or Airport
N Chief
Operational Chief ATM
ATCO or
Duty
Analyze Event Y Chief CNS
ATSEP with experts must be notify ?
Engineer Safety
manager

Safety BEA
Chief
manager CAA Office
ANSP investigation
headquarters
analysis
Outside Crisis units (ANSP, NSA, ministry …)

60
 06/02/2019

Safety occurrence: Analysis

Analysis

Debriefing of ATCO
or ATSEP involved by Safety Subdivision
or safety commission
safety subdivision

Safety Subdivision
First
Severity and Frequency To determine if causes
Classification must be searched a risk matrix
can be used by the safety
The database is updated during Subdivision
analysis

N Search
Causes ?

END Y
ECCAIRS : European Coordination
Centre for Accident and Incident
Toward next step
Reporting Systems.
(search causes)

Safety occurrences
Risk Analysis Tool (RAT)

Risk in the RAT methodology is calculated taking into account ‘Severity’ and
‘Repeatability’ of the occurrence.

61
 06/02/2019

Severity classification

5 levels of severity have been defined for ATM operational and 6 for
ATM specific (technical). Explanations on different levels are given in
the next slides.

ATM operational
Severity
A B C D E
levels
Incident Serious Major Significant Not No effect
determined

ATM specific
Severity
AA A B C D E
levels
Incident Total Serious Partial Ability to Not Not effect on
inability to inability to inability to provide safe determined ATM service
provide safe provide safe provide safe but
ATM service ATM service ATM service degraded
ATM service

Frequency classification

5 levels of frequency have been defined for ATM operational and for
ATM specific (technical). Explanations on different levels are given in
the next slides.

Frequency
5 4 3 2 1
levels
Frequency Very Frequent Frequent Occasional Rare Extremely Rare

62
 06/02/2019

Safety occurrences
Risk Matrix ATM operational

The risk of a safety occurrence is characterized by two items :

- Severity
- Frequency

Risk severity
Risk Serious Major Significant Not No effect
frequency A B C determined E
D
Very frequent 5 A5 B5 C5 D5 E5
Frequent 4 A4 B4 C4 D4 E4
Occasional 3 A3 B3 C3 D3 E3
Rare 2 A2 B2 C2 D2 E2
Extremely rare 1 A1 B1 C1 D1 E1

Safety occurrences
Risk Matrix ATM specific (technical)

Risk severity
Risk Total Serious Partial Ability to Not No effect
frequency inability to inability to inability to provide determined on ATM
provide provide provide safe but D service
safe ATM safe ATM safe ATM degraded E
service service service ATM service
AA A B C
Very frequent 5 AA5 A5 B5 C5 D5 E5
Frequent 4 AA4 A4 B4 C4 D4 E4
Occasional 3 AA3 A3 B3 C3 D3 E3
Rare 2 AA2 A2 B2 C2 D2 E2
Extremely rare 1 AA1 A1 B1 C1 D1 E1

63
 06/02/2019

Safety occurrence: analysis

Analysis

Debriefing of ATCO Safety Subdivision

or ATSEP involved by or safety commission
safety subdivision

Safety Subdivision
First
Severity and Frequency
Classification
The databases
are updated during analysis
ECCAIRS : European Coordination
Centre for Accident and Incident Toward next step
Reporting Systems.
(search causes)

Safety occurrence: analysis

Analysis
A safety commission must
be held if the risk (severity +
frequency) reaches a high
Y Safety N level (defined by provider
Commission ? and CAA).

Causes evaluated Causes evaluated

by safety by safety
commission subdivision
The databases
are updated during
analysis
ECCAIRS : European Coordination
Centre for Accident and Incident
Reporting Systems.

Toward next step

(corrective actions)

64
 06/02/2019

Safety occurrence: analysis

ATM Division : Safety Commission

Commission: Head of ATM division. Head of Subdivisions OPS,
Studies, Training, Safety.
ATCO involved, safety ATCO from teams, experts ...
Brainstorming to identify severity, frequency and causes.

Replaying radio
Replaying radar data communications

Safety occurrence: analysis

CNS Division : Safety Commission

Commission: Chief of CNS Division, Chief of Subdivisions
Flight Plan, Radar, Telecom Energy, Training, Safety and Chief of
Studies subdivision (ATM division).
ATSEP involved, others ATSEP, experts ...
Brainstorming to identify causes.

65
 06/02/2019

Safety occurrence: analysis causes

We have to identify the different categories of causes, specially for

the human factors.

The numbers are the references in the database.

A safety occurrence can have several causes (455, 443 …).

It is very important to define corrective actions on the main causes.

4 Safety Promotion
4.2 Safety communication

Lesson dissemination, feed-back

66
 06/02/2019

Corrective Actions

Decide
Monitoring commission
Corrective Actions
Chief of ATM or CNS.
and Feedback
Implement Responsible of the
Corrective Actions Corrective Action.
and Feedback Responsible of the
Feedback.
The databases Monitoring
are updated during analysis Corrective Actions and Monitoring commission
ECCAIRS : European Coordination Feedback.
Centre for Accident and Incident
Reporting Systems.

Corrective N
Action Monitoring commission
effective?

END

4 Safety Promotion
4.2 Safety communication

Besides Corrective Actions, we must inform ATCOs, ATSEPs, pilots …

(feedback).

Information content :
- Safety occurrences
- Causes
- Corrective Actions
- How to avoid same causes

The feedback can be done by email, during briefings, with bulletins ....

The feedback can be sent to others entities.

67
 06/02/2019

4 Safety Promotion
4.2 Safety communication

Example : Feedback in French ANSP Paris ACC.

Feedback is sent by
Email to all people involved
(ATCO and/or ATSEP).
Feedback is presented during
briefings.

4 Safety Promotion
4.2 Safety communication
Example : Briefings in French ANSP Paris ACC.

The briefing is provided

during 12 days because
we have 12 teams on East
area and 12 teams on
West area.
The ATCOs must sign the
attendance sheet. Briefings are
included in continuous training.
The different items are linked to
the main causes of safety
occurrences causes.

68
 06/02/2019

4 Safety Promotion
4.2 Safety communication

In each team 2 ATCO are safety experts. These ATCO participate to the safety
commissions. They can disseminate safety information to and from teams.

Commission: Head of ATM division. Head of Subdivisions

OPS, Studies, Training, Safety.
ATCO involved, safety ATCO from teams, experts ...
Brainstorming to identify severity, frequency and causes.

Replaying radio
Replaying radar data communications

Safety data collection and processing systems

Safety Occurrences must be recorded in a

database.
Example ECCAIRS database.
ECCAIRS : European Coordination Centre for Accident and
Incident Reporting Systems.

A link must be done between Safety occurrences and Corrective

Actions.
So you can easily explain what has been done.

AGATA
Corrective Actions must be stored and
recorded in a database or in a file.

Corrective Actions database or Table

69
 06/02/2019

2 Safety Risk Management

2.1 Hazard identification

2 Safety Risk Management

2.1 Hazard identification

70
 06/02/2019

2 Safety Risk Management

2.1 Hazard identification

2 Safety Risk Management

2.1 Hazard identification

71
 06/02/2019

ICAO Annex 19

SMS

2. Safety Risk Management

2.1 Hazard identification
2.2 Safety risk assessment and mitigation

2.1 Hazard identification = a safety occurrence has occurred.

2.2 Risk assessment and mitigation : before a planned change.

ICAO Annex 19

SMS

2. Safety Risk Management

2.1 Hazard identification
2.2 Safety risk assessment and mitigation

2.1 Hazard identification = a safety occurrence has occurred.

2.2 Risk assessment and mitigation : before a planned change.

72
 06/02/2019

2 Safety Risk Management

2.2 Safety risk assessment and mitigation
3 Safety Assurance
3.2 The management of change

Main points :
Safety studies
Management of changes

Challenger disaster (1986)

The US Space Shuttle Challenger accident is the January 28, 1986

astronaut accident that resulted in the disintegration of the NASA
Challenger Space Shuttle 73 seconds after takeoff, causing the death of
the seven astronauts of the crew of the mission STS-51-L

73
 06/02/2019

Challenger disaster (1986)

Weather forecasts for January 28 predicted an

unusually cold morning, with temperatures close
to −1°C (30° F), the minimum temperature
permitted for launch. The Shuttle was never
certified to operate in temperatures that low. The
O-rings, as well as many other critical
components, had no test data in such conditions.

The Rogers Commission investigated the

accident and found that the corporate culture of
the National Aeronautics and Space
Administration (NASA) and the decision-making
process had been one of the main factors that led
to the accident.
NASA had no risk assessment and mitigation
process.

What is a Change ?
A change is :
- A new equipment (hardware and/or software),
- A modification of an equipment (hardware and/or software),,
- A modification of airspace (airways, design …)
- A significant change in working methods.

74
 06/02/2019

Airspace change and Technical change

SU
AP
FL345
AP
FL285 FL285

AO AO

Initially we have 2 layers : sector AO and sector AP. To improve safety and
capacity in this area we have created a third layer sector : SU

This change has an impact on airspace. But it has also a technical impact : new
radio frequency, new ATCO position ….

General principles

t
Definition of the system perimeter and environment

Functional Hazard Assessment (FHA) :

System Analysis
Identification of the hazards
Definition of the safety objectives

Preliminary System Safety Assessment (PSSA) :

Identification of the Mitigation Means (MM).

System Safety Assessment (SSA) :

Demonstrate the implementation of Mitigation Means
Analysis of transition phases
Definition of safety assurance means

75
 06/02/2019

General principles

Definition of the system perimeter and environment

FHA System analysis

Identification of Hazards (severity, frequency …)

Definition of the Safety Objectives

NO

Identification
Conversion of objectives
of safety the Mitigation Means
in requirements
PSSA Objectives YES
Achieved ?
SSA Demonstrate
Quantitativethe
andimplementation of mitigation
qualitative assessment means
of safety Stored

Analysis of transition phases

Definition of safety assurance means

Synthesis

Identification of Hazards

Hazard = danger affecting the provision of services and that can lead to an
accident or incident.

Causes
Human Factor Technical Procedural
of hazard
Error Dysfunction Error

Brainstorming
Combination with experts
to identify
causes

Brainstorming with
Hazard
experts to identify hazards.
Brainstorming
with experts to find
severity and frequency
Consequences
Severity
Frequency

76
 06/02/2019

Example of Hazards hazard 1

This change is done to

SU increase the safety and the
capacity on this area.
FL345
The aircrafts that are in AO,
AP
AP and SU are all visualized
FL285 on each position AO, AP and
AO SU.
Radar The first hazard identified is :
screen of Too many aircrafts on the
AO, AP, SU screen at the same time,
positions screen overloaded.
Hazard 1 : «Radar screen overloaded».

Example of Hazards hazard 2

SU

FL345
AP
X FL285

ATCO

Hazard 2 : «Transfer to wrong sector».

The upstream sector gives the aircraft to the wrong sector (AP
instead SU, SU instead AP).

77
 06/02/2019

Consequences of hazards

For each hazard we have to identify the potential consequences.

We must determine the severity and the frequency by imagining the

worst possible case.

Hazard

Consequences

Severity
Frequency

Severity classification

DOC9859 V3
Chap 2

5 levels of
severity.

78
 06/02/2019

Example of severity hazard 1

Hazard 1
Radar screen
overloaded

Brainstorming
with experts
Radar screen of AO, AP, SU
positions overloaded

Severity = C
Major incident

Example of severity hazard 2

The ATCO makes

Hazard 2 an error, he gives
Transfer SU to the aircraft the
to bad sector frequency of AP
FL345 instead of SU.
X AP
FL285

Brainstorming
ATCO
with experts

Severity = C
major
incident

79
 06/02/2019

Frequency classification

DOC9859 V3
Chap 2

Example of frequency hazard 1

Hazard 1

Brainstorming Radar screen of AO, US, SU

with experts positions overloaded

Frequency = 4
Occasional

80
 06/02/2019

Example of frequency hazard 2

The ATCO
makes an error,
he gives to the
Hazard 2 SU aircraft the
frequency of AP
instead of SU.
FL345
X AP
FL285
ATCO
Brainstorming
with experts

Frequency = 3
Remote

General principles

Definition of the system perimeter and environment

FHA System analysis

Identification of Hazards (severity, frequency …)

Definition of the Safety Objectives

NO

Identification
Conversion of objectives
of safety the Mitigation Means
in requirements
PSSA Objectives YES
Achieved ?
SSA Demonstrate
Quantitativethe
andimplementation of mitigation
qualitative assessment means
of safety Stored

Analysis of transition phases

Definition of safety assurance means

Synthesis

81
 06/02/2019

Safety Objectives

For each level of severity we must define the maximum acceptable

frequency.

A B C D E
Severity
Catastrophic Hazardous Major Minor Negligible

Safety
Extremely Improbable Occasional Frequent -
Objective
improbable

Safety risk assessment Matrix

The risk of a safety occurrence is characterized by two items severity and

frequency.
Intolerable region. Tolerable region Acceptable region.
Acceptable with risk
mitigation.

Risk severity
Risk Catastrophic Hazardous Major Minor Negligible
frequency A B C D E

Frequent 5 A5 B5 C5 D5 E5
Occasional 4 A4 B4 C4 D4 E4
Remote 3 A3 B3 C3 D3 E3
Improbable 2 A2 B2 C2 D2 E2
Extremely A1 B1 C1 D1 E1
improbable 1

82
 06/02/2019

Safety risk assessment Matrix

For each hazard we must check if the couple severity/frequency is in the

acceptable area of the matrix.

Risk severity
Risk Catastrophic Hazardous Major Minor Negligible
frequency A B C D E

Frequent 5
Occasional 4 H1 = C4
Remote 3 H2 = C3
Improbable 2
Extremely
improbable 1

General principles

Definition of the system perimeter and environment

FHA System analysis

Identification of Hazards (severity, frequency …)

Definition of the Safety Objectives

NO

Identification
Conversion ofobjectives
of safety the Mitigation Means
in requirements
PSSA Objectives YES
Achieved ?
SSA Demonstrate
Quantitativethe
andimplementation of mitigation
qualitative assessment means
of safety Stored

Analysis of transition phases

Definition of safety assurance means

Synthesis

83
 06/02/2019

Mitigation Means
Causes
of hazard

Human Factor
Error

Technical
Combination Hazard Consequences
Dysfunction

Procedural Frequency
Frequency Severity
Error

Has
effect on
Decrease Decrease

Has
effect on
Preventive Protective
MMS MMS

Example of Protective Mitigation Means hazard 1

SU

FL345
AP
FL285

AO
Radar screen SU position with Radar screen of
all aircrafts (AO, AP, SU) SU position

Hazard 1 : «too many aircrafts on a position, radar screen overloaded».

Mitigation mean : the aircrafts that belongs to SU are in red color, the
others (AO and AP) are in black (software update).

84
 06/02/2019

Example of Protective Mitigation Means hazard 2

SU

FL345
X AP
FL285

ATCO

Hazard 2 : «Transfer to wrong sector».

The upstream sector gives the aircraft to the wrong sector (AP instead
SU, SU instead AP).

Mitigation means :
- ATCO training.
- ATCO documentation.
- Warning on positions.

New brainstorming
after Mitigation Means (MM) Hazard 1

Hazard 1
Radar screen
overloaded

Brainstorming Radar screen of AO, AP, SU positions

with experts overloaded
after MM

Severity = D
(C before MM)
Frequency = 4
(same before MM)

85
 06/02/2019

New brainstorming
after Mitigation Means (MM) Hazard 2

The ATCO
Hazard 2 SU makes an error,
he gives to the
FL345 aircraft the
frequency of AP
X AP
instead of SU.
FL285
Brainstorming
with experts ATCO
after MM

Severity = C
(same before MM)
Frequency = 2
(3 before MM)

Safety risk assessment Matrix

after mitigation means

For each hazard we must check if the couple severity/frequency is in the

acceptable area of the matrix.

Risk severity
Risk Catastrophic Hazardous Major Minor Negligible
frequency A B C D E

Frequent 5
Occasional 4 H1 = D4
Remote 3
Improbable 2 H2 = C2
Extremely
improbable 1

86
 06/02/2019

Safety risk assessment Matrix

before & after mitigation means

Before mitigation
means

After mitigation
means

General principles

Definition of the system perimeter and environment

FHA System analysis

Identification of Hazards (severity, frequency …)

Definition of the Safety Objectives

NO

Identification
Conversion ofobjectives
of safety the Mitigation Means
in requirements
PSSA Objectives YES
Achieved ?
SSA Demonstrate
Quantitativethe
andimplementation of mitigation
qualitative assessment means
of safety Stored

Analysis of transition phases

Definition of safety assurance means

Synthesis

87
 06/02/2019

Demonstrate the implementation

of the Mitigation Means (MM)

This phase has to demonstrate that :

- The safety objectives allocated to hazards are achieved.
- The Mitigation Means are effectively enforced.

It consists in the provision of :

- Reports of technical tests.
- Updated documentation (Letters of agreement, Protocols, Manuals,
Maintenance procedures…).
- Traceability of training (ATCO, ATSEP …).

Demonstrate the implementation

of the Mitigation Means (MM)

Hazard 1 : The update of software to

make appear aircrafts with different
colors must be validated before the
implementation of the change.

Radar screen
of SU position

Hazard 2 :
The traceability of ATCO training must be done and recorded.
The ATCO manuals have been updated. A new manual has been
created for the new sector.

88
 06/02/2019

General principles

Definition of the system perimeter and environment

FHA System analysis

Identification of Hazards (severity, frequency …)

Definition of the Safety Objectives

NO

Identification
Conversion of objectives
of safety the Mitigation Means
in requirements
PSSA Objectives YES
Achieved ?
SSA Demonstrate
Quantitativethe
andimplementation of mitigation
qualitative assessment means
of safety Stored

Analysis of transition phases

Definition of safety assurance means

Synthesis

Transition phase

Transition phase
work in progress
Initial situation New situation
before change after change

Risk assessment and Risk assessment and

mitigation on the risks mitigation on risks
during the works. after the
implementation of the
change.
Remember Überlingen
The transition phase (work during the night) have been forgotten : no
risk and assessment and mitigation process, no internal and external
coordination, no internal information.

The risks during the transition phase must be take into account in a
specific safety study.

89
 06/02/2019

General principles

Definition of the system perimeter and environment

FHA System analysis

Identification of Hazards (severity, frequency …)

Definition of the Safety Objectives

NO

Identification
Conversion ofobjectives
of safety the Mitigation Means
in requirements
PSSA Objectives YES
Achieved ?
SSA Demonstrate
Quantitativethe
andimplementation of mitigation
qualitative assessment means
of safety Stored

Analysis of transition phases

Definition of safety assurance means

Synthesis

Safety Assurance

Safety assurance defines the means to monitor the new system :

- Verify that an acceptable safety level is maintained on the long run.
- Verify that the initial hypothesis are always valid.
- Verify if new hazards occur.

They can take different forms :

- Definition of safety indicators.
- Procedure to monitor the indicators.

If necessary the safety study is updated.

90
 06/02/2019

Example of Safety Assurance

Safety is the most important item. Safety

SU assurance : monitoring of monthly
indicators to check the safety on AO, AP
and SU positions.
FL345
1 indicator to check the incidents due to
AP
«screen overload».
FL285
1 indicator to check incidents due to
AO «wrong sector».

Is the change efficient ?

Project assurance : indicator to check the delays on the positions
AO, AP, SU.

French ANSP Methodology

Regulations :
ICAO Annex 19, DOC 9859

This methodology explains

how to do a safety analysis :
French ANSP Methodology procedure, guides, templates.

How to do a safety analysis ?

The regulations and the
Safety Analysis general principles are not
enough explicit.

91
 06/02/2019

French ANSP Methodology

Procedure

Guides explain how

Guides Templates
to use templates.

Procedure, guides and templates are in compliance with regulations

and with general principles.

Procedure, guides and templates are approved by CAA.

Documentation

Procedure

Technical Airspace
Preliminary Preliminary Transition Structure for
Study Study Study Safety Analysis
(FHA, PSSA, SSA) (FHA, PSSA, SSA) template Report
template template

List of Hazards (built with experience)

EPIS-TIL EPIS-CA MISO Safety analysis

guide guide guide guide

EPIS = French acronym for preliminary study (FHA, PSSA, SSA).

MISO = French acronym for safety study on risks during transition phase.

92
 06/02/2019

Chronology

Activities linked to Project Management

Safety Assurance
Activities
Preliminary study
Technical change Is the safety
(FHA,PSSA,SSSA) Severity study pertinent ?
Yes Safety study
and/or Complexity
high
Report
Preliminary study MISO Monitoring indicators
Airspace change No Documentation reviews
(FHA,PSSA,SSA)
Audits

Beginning of Implementation
the change decided by Provider
t
after approval by
CAA if necessary

Brainstorming

Brainstorming

Leader : Safety Coordinator of the change

Experts : ATCOs, ATSEPs, engineers,
pilots …
(experts are very often front line personnel)

The safety coordinators follow a training (initial and continuous) on

safety studies (method, cause tree, requirements …).

The safety manager follow same training than safety coordinators.

He can follow the brainstorming.

The brainstorming is done to define hazards, severity, frequency and

mitigation means.

93
 06/02/2019

Oversight of safety studies by CAA

Notification of planned changes regarding safety Provider

t
CAA
Decision
Followed up Not followed up
of follow up
NSA ?
by CAA
Nomination of a correspondent
New hazard during
Y
Analysis with Severity
Validation of Safety Plan (if exists) A1 or B
2 ??

N
Writing the Coordination Plan
Safety analysis, not
followed up by CAA,
NSA,
Review of Safety Analysis archived by ANSP
Safety
Analysis

CAA
NSA correspondent sends his report Y

Request for
N Complementary
Change approved ? information ?
Y N
Data archived in CAA
NSA Refusal of change
Implementation
of change
Verification : follow up of safety assurance Documentary reviews

2 Safety Risk Management

2.2 Safety risk assessment and mitigation
3 Safety Assurance
3.2 The management of change

94
 06/02/2019

2 Safety Risk Management

2.2 Safety risk assessment and mitigation
3 Safety Assurance
3.2 The management of change

2 Safety Risk Management

2.2 Safety risk assessment and mitigation
3 Safety Assurance
3.2 The management of change

95
 06/02/2019

2 Safety Risk Management

2.2 Safety risk assessment and mitigation
3 Safety Assurance
3.2 The management of change

2 Safety Risk Management

2.2 Safety risk assessment and mitigation
3 Safety Assurance
3.2 The management of change

96
 06/02/2019

ICAO Annex 19

3rd pillar of a SMS :

SMS

3. Safety assurance
3.1 Safety performance monitoring and measurement.
3.2 The management of change.
3.3 Continuous improvement of SMS.

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

Main points :
Indicators
Monitoring of Indicators
Audits

97
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

The survey (safety performance monitoring and measurement) is

done by using :

- Safety Performance Indicators (SPI),

- Audits (external and internal).

The monitoring is done during dedicated meetings at different levels

of the organization.

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Indicators
Indicators:
They must be clear and easy (if possible) to understand.
For each indicator we have to define :
- the source,
- the method of calculation,
- the responsible of the indicator,
- the date the indicator is published,
- the periodicity,
- the alert,
- the target …

98
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Indicators

The provider define safety performance indicators (SPIs) and their

associated target and alert levels.
The target can be quantitative or qualitative and must be reviewed
each year.
The alert level is a threshold that means results are very bad and it is
mandatory to take immediate corrective actions.
Target and alert are chosen by observation of results of last years.
The SPIs with target and alert level must be agreed by the CAA.
The SPIs must be periodically monitored by the provider (daily,
weekly, monthly).
Some other indicators are also monitored (delays …).

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Indicators

Example of SPIs for ANSP

Source SPI
Report Number of AIRPROX
Report Number of losses of separation
Report Number of losses of separation 70%
Report Number of RA TCAS
Safety subdivision Number of losses of separation with severity A
Safety subdivision Number of losses of separation with severity B
Report Number of runway incursions

99
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Indicators
Indicators to evaluate the impact of a change.

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

Different types of audits : internal, external (safety by CAA, quality by

ISO).
Crossed audits
with others
providers
CAA ISO
CAA certificate
PROVIDER certificate ISO9001
Safety Quality
Internal
External External
Audits
Audits Audits
Monitoring

Who : CAA auditors Who : provider auditors Who : ISO auditors

Periodicity : To be defined Periodicity : yearly Periodicity : Yearly

100
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

Audit : Systematic and independent review to determine whether

activities and results related to the quality and safety management
system meet the pre-established requirements and whether these
provisions are implemented effectively and are able to achieve the
objectives.

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

DSNA (French ANSP) : a national provider with more than 50 sites.

DSNA provides ATS, CNS and AIS.
DSAC (French CAA) performs regular coordination with DSNA (twice a year).

2006 DSNA obtained the certificate of Air Navigation Services Provider

issued by the DSAC. The certificate ended in 2010.
2010 Renewal the certificate, valid until 2016.
2016 Renewal the certificate, valid UFN.

DSNA has an ISO9001 global certification since 2010.

101
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits
Periodicity of Lille
Audits CAA.

N CDG &
Brest Orly Strasbourg
E
Each year W Headquarters

Main
Airports2
Lyon
years

AIS CESNAC
ACC
3 years
Bordeaux SW Indian
SE Ocean
Nice
Toulouse West Indies
Marseille Guiana

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits
Periodicity of Lille
Audits ISO
(quality)
N CDG &
Brest Orly Strasbourg
E
Each year
W Headquarters

Year 1

Year 2 Lyon

Year 3 AIS CESNAC

Bordeaux SW Indian
SE Ocean
Nice
Toulouse West Indies
Marseille Guiana

102
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

A procedure “Internal Audits” must describe how are realized the

internal audits.

Internal auditors are trained by an approved organization.

Internal auditors must done at least 1 audit per year (example, to be

defined by each provider).

Each year a program of internal audits is planned … and must be

realized.

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

Internal audit on a specific target : Protection against cyber attacks.

After the audit a dedicated

action plan has been done. The
implementation of this plan is
monitor each month.

103
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

Internal audit on a specific target : Operational documentation for ATCO.

Each year an internal audit is done to verify if the procedure is applied

and if the documentation is updated.

Audit findings are immediately take into account by OPS subdivision

(ATM).

3 Safety Assurance
3.1 Safety performance monitoring
and measurement : Audits

Internal audit on a specific target : Documentation for ATCO training.

The chief of training subdivision for ATCO ask an audit to improve the
management of documentation.

Consequences : documentation only on electronic support and creation

of an intranet site.

104
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

105
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

106
 06/02/2019

3 Safety Assurance
3.1 Safety performance monitoring
and measurement

3 Safety Assurance
3.3 Continuous improvement

Main points :
Continuous improvement
Corrective actions
Safety management reviews

107
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement

Continuous improvement is a ISO 9001 quality concept but it applies

also to safety.

Deming wheel : Plan, Do, Check, Act … PDCA cycle.

3 Safety Assurance
3.3 Continuous improvement

Provide services :
- Safety
Policy - Safety occurrences
Action plan
Objectives
Resources
Safety occurrences
- Analyze
- Severity, frequency
- Causes
Audits Findings
Indicators
Corrective
actions to
improve the SMS

108
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement

Continuous
improvement is
never completed.

You must climb

slowly, step by
step.

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

The corrective actions are used to deal with :

- Causes of safety occurrences,
- Finding of audits (non conformities, weakness points ...)
- Other events …

A corrective action must be stored in a file or a data base.

A corrective action is a record of the SMS.

109
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

FINDINGS CORRECTIVE
ACTIONS
Event
Proposal Decide and
Non-Compliance Implement END :
Weakness Corrective The finding can
Recommendation Action be closed
Indicators (drifts)

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

110
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement
Corrective actions
Paris ACC : Corrective Actions Table
Last Update : 15 November 2014

Ref. Origin State of Observation Responsible Target Actions planned Effectiveness

& Date progress Date and carry out Criteria

MM/YY
In progress, Name
Completed, or better List of actions
Closed. Function planned
and carried out

Safety review
Internal audits,
External audits, Findings of audits, If the target date
Process reviews Causes of safety has passed, a new Criteria to check
Others occurrences target date should before closing the
Proposal, be set observation
Others …

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

Description of jobs not Ask a national coordination All job description

Note DO completed march Wait RIMS meeting in Brest done and signed by
075075 13 ATSEP description in progress ATCO and ATSEP
In P
27/2011 du 03 progress march ATCO description to plan
Rosellini
décemb 14
re 2011 march
15

111
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

2 safety management reviews per year to take high level decisions

and strategic actions.

Safety committee :
- Chief of DG is the chairman of the review,
- Main managers,
- Safety manager is the secretary of the review,
- Safety experts …

The safety review must be prepared by the safety manager.

112
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement
Corrective actions

Safety indicators

Audit reports (internal,

external)

List of planned changes

Preparation
Analysis Safety
Corrective actions table Synthesis
done by
Management
safety review
manager
Action plan

Program of audits (project)

Actions List of
Minutes
decided changes

Corrective Action
Program
actions table plan
of faudits
updated updated

3 Safety Assurance
3.3 Continuous improvement
Safety management review

Agenda :

1. Safety indicators
2. Risk assessment and mitigation
3. Implementation of SMS
4. Audit reports
5. Actions plan
6. Corrective and Preventive Actions

113
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement

3 Safety Assurance
3.3 Continuous improvement

114
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement

3 Safety Assurance
3.3 Continuous improvement

115
 06/02/2019

3 Safety Assurance
3.3 Continuous improvement

ICAO Annex 19

4th pillar of a SMS :

SMS

4.1 Training and education.

4.2 Safety communication.

116
 06/02/2019

4 Safety Promotion
4.1 Training and education

Main points :
Training and competencies of personnel involved in safety (pilots,
ATCO, ATSEP …).
Training and competencies of personnel involved in the SMS
(managers, safety manager, personnel in charge of safety
occurrences, safety studies, audits …).

4 Safety Promotion
4.1 Training and education
Personnel involved in safety

The overall safety objective is to ensure the competence of personnel

responsible for safety related tasks within the provision of ATM
services.

Pilots, Air Traffic Controller (ATCO) and Air Traffic Safety Engineers
Personnel (ATSEP) are mainly concerned by this requirement.

117
 06/02/2019

4 Safety Promotion
4.1 Training and education
Personnel involved in safety

ATCO

To be allowed to work on a position, ATCOs must have a valid license

issued by the CAA.

To obtain the license (initial training) :

- Training in ENAC.
- Training in ACC or Airport.
- On job training.
- Medical certificate.

4 Safety Promotion
4.1 Training and education
Personnel involved in safety

To maintain the license (continuing training) :

- Evaluation and individual training plan.
- Proficiency in English.
- Medical certificate.
- Annual number of hours worked.

A database is used to monitor the competency of ATCOs.

For each ATCO all the trainings done are recorded in this database.

To manage, monitor and organize all these trainings we must have a

dedicated team.

118
 06/02/2019

4 Safety Promotion
4.1 Training and education
Personnel involved in safety
Paris ACC ATM Division :
The Training Team
ATM Division

OPS
Studies
Training
Training
Safety
ATCO Team
Team
Chief
Deputy
6 ATCOs
4 Engineers

Paris ACC :
- 550 ATCOs
- Among them 100 ATCOs are in training.

4 Safety Promotion
4.1 Training and education
Personnel involved in safety
Planning Paris ACC

Units
Continuing
Training 1

Continuing
Training 2
Chief
Controller

Individual
Training

English

Instructors

119
 06/02/2019

4 Safety Promotion
4.1 Training and education
Personnel involved in safety

SUMMARY

Initial training
Continuous training
Phraseology
QCM
Instructions
Manual ATCO
Manual Chief ATCO Training ATCO Paris ACC
Manual ATFM
Manual sectors Last update 20.01.17

4 Safety Promotion
4.1 Training and education
Personnel involved in SMS

Personnel involved in the SMS

The service provider shall develop and maintain a safety training

program that ensures that personnel are trained and competent to
perform their SMS duties.

Who is involved ?
- Main managers
- Safety manager
- Personnel in charge of safety events
- Personnel in charge of safety studies
- Internal auditors

120
 06/02/2019

4 Safety Promotion
4.1 Training and education
Personnel involved in SMS
SMS training (The safety manager defines and monitors these trainings).
WHO ? WHAT ? BY WHO ?
Main managers General concepts of a management External company.
system.
Safety manager General concepts of a management External company.
system, quality training, audit training,
safety case training.

Personnel involved in the Training on methodology, procedure, ENAC

events process (safety, manual. External company.
security, quality …)
Personnel involved in Training on methodology, procedure, ENAC
safety case. cause tree … External company.
Auditors Training on audit. External company.
Everybody Sensibilization to SMS. Safety manager.

4 Safety Promotion
4.1 Training and education

121
 06/02/2019

4 Safety Promotion
4.1 Training and education

4 Safety Promotion
4.1 Training and education

122
 06/02/2019

4 Safety Promotion
4.1 Training and education

4 Safety Promotion
4.1 Training and education
Safety communication

Lesson dissemination, feed-back

See Part 3 SRM Safety occurrences

123
 06/02/2019

EU Requirement : external services

or external providers.

We have to define what are «external services and external providers.

You must check, survey the external services that have a link with
safety.

EU Requirement : external services

or external providers.

Remember when we talked about Überlingen accident :

«This accident happened over the territory of the Federal Republic of
Germany, July 1, 2002 at 09h35 pm UTC. Control of the airspace in
this area is delegated to the Swiss Air Navigation Services.»

According to the Letter of Agreement (LOA) we can say that for the
German ANSP, the Swiss ANSP is an external service provider.

124
 06/02/2019

EU Requirement : external services

or external providers.

French ANSP has identified a list of suppliers and external services

that can have an impact on safety.

Key elements of the list of Paris ACC:

- Energy.
- Air conditioning.
- Fire Detection.
- Telecommunications.
- Cleaning the operational room.
- Cleaning technical rooms.
- Training in English.

For these companies we have added safety requirements and

objectives in the contracts.

EU Requirement : external services

or external providers.

Date & Responsible

External services Supplier Contract
Duration for monitoring
Maintenance of S5204 01.01.12 Chief of energy
SDMO
generator sets (fuel) 14504 5 years subdivision

Safety objectives
Impact on Safety for supplier (contract) Others Safety barriers

Major. - 2 preventive - 2 generator sets.

Energy : interventions by year. - SDMO supplier can
1 External (EDF) - Call : intervention be helped by local
2 Generator sets (fuel) within 4 hours. technicians.
3 Batteries (4h max)

125
 06/02/2019

EU Requirement : external services

or external providers.

Date & Responsible

External services Supplier Contract
Duration for monitoring
10.12.07 Chief of
Fire detection SICLI 11/07
5 years administrative
division

Safety objectives
Impact on Safety Others
for supplier (contract)
Safety barriers
Major. - 2 preventive interventions - SICLI supplier can
- Risk of late detection by year. be helped by local
of a fire - Call : intervention within 4 technicians.
- False alarms. hours.
- All the be fire detectors
must changed within 4 years.

EU Requirement : external services

or external providers.

Date & Responsible

External services Supplier Contract
Duration for monitoring
Chief of ATCO
English training for ELS -
1 year training
ATCO
subdivision

Safety objectives
Impact on Safety Others
for supplier (contract)
Safety barriers
- Major. - 90% good quality evaluation
- Risk of bad practice - Paris ACC English teachers
of english. participate to some trainings.
- Paris ACC English teachers
participate to all debriefings.

126
 06/02/2019

EU Requirement : external services

or external providers.

To be defined

127