EPRI-TVA Alarm Improvement Project
EPRI-TVA Alarm Improvement Project
EPRI-TVA Alarm Improvement Project
10707048
10707048
Alarm Management Implementation Using the EPRI
Alarm Management Guidelines
Tennessee Valley Authority Alarm Improvement Project
1026497
Technical Update, July 2013
This is an EPRI Technical Update report. A Technical Update report is intended as an informal report of
continuing research, a meeting, or a topical study. It is not a final EPRI technical report.
NOTE
For further information about EPRI, call the EPRI Customer Assistance Center at 800.313.3774 or
e-mail [email protected].
10707048
ACKNOWLEDGMENTS
The following organization, under contract to the Electric Power Research Institute (EPRI),
prepared this report:
Plant Automation Services, Inc. (PAS)
16055 Space Center Blvd., Suite 600
Houston, TX 77062
Principal Investigators (in alphabetical order)
G. Nagarajan
L. Martinez
K. Phelps
R. Carlton
T. Younts
This report describes research sponsored by EPRI.
This publication is a corporate document that should be cited in the literature in the following
manner:
Alarm Management Implementation Using the EPRI Alarm Management Guidelines: Tennessee
Valley Authority Alarm Improvement Project. EPRI, Palo Alto, CA: 2013. 1026497.
10707048 iii
10707048
PRODUCT DESCRIPTION
The concept of alarm management is being increasingly identified as a “best practice” for
increasing the safety, availability, and reliability of plant operations. To address the growing
need for information and improvement processes in this area, the Electric Power Research
Institute (EPRI) formed a strategic alliance with Plant Automation Services (PAS), a global
leader in alarm management solutions, to create the EPRI Alarm Management and Annunciator
Applications Guidelines (EPRI report 1014316). Tennessee Valley Authority (TVA), contracted
with EPRI to guide TVA through steps 2, 3, and 4 of the Alarm Management Improvement
Process.
This document covers basic overviews of the concepts and work processes that were involved in
the steps of the improvement program.
Background
Unplanned unit outages, derates, and process upsets represent a significant cost in the power
industry. Many times, these costs can be avoided with operator intervention and response to a
burgeoning abnormal situation. The concept and methodology of alarm management represent
proven solutions that seek to ensure that the operators are provided with information that guides
them to the proper response to prevent an evolving abnormal situation from having safety,
environmental, financial, or equipment availability impacts.
Objectives
Effective alarm management techniques often represent a significant departure from traditional
standards of control system alarms since the introduction of the distributed control system
(DCS). This report documents real-world information regarding the techniques and benefits of an
alarm management improvement program and the results of such a program.
Approach
The goal of this report is to document the application of the EPRI Alarm Management and
Annunciator Applications Guidelines at TVA facilities and present the results of the alarm
management improvement program. The program was implemented first by developing the
alarm philosophy documentation.
Results
The alarm systems at most of the TVA plants were classified as “Overloaded” based on the
alarm data provided. The alarm systems were generating high rates of alarms, and it was
probable that operators might overlook critical alarms. The failure of the alarm systems to
provide useful, relevant, and timely information hindered the ability of the operators to detect an
emerging situation and respond properly to minimize the disturbance. Steps from the EPRI
Alarm Management and Annunciator Applications Guidelines resulted in a reduction of alarm
settings for all the TVA plants.
Applications, Value, and Use
Several published guidelines exist in the area of alarm management, which is generally regarded
as a best practice across all industries. The EPRI Alarm Management and Annunciator
Applications Guidelines provides the recommended guidelines that are specific for power
generation facilities. This focus provides value by eliminating some of the learning curve in
taking generalized specifications and determining how to apply them properly.
10707048 v
Keywords
Alarm analysis software
Alarm management
Alarm management benefits
Alarm rationalization
Annunciator guidelines
Distributed control system (DCS) alarms
10707048 vi
CONTENTS
1 THE ALARM MANAGEMENT PROBLEM AT TVA ..............................................................1-1
Nature of the Alarm Management.......................................................................................1-1
Alarming in the Pre-Digital Age ..........................................................................................1-1
Alarming on the Distributed Control System .......................................................................1-1
The Alarm Management Improvement Process..................................................................1-2
2 ALARM MANAGEMENT SOLUTION FOR TVA ..................................................................2-1
3 TVA ALARM PHILOSOPHY DEVELOPMENT .....................................................................3-1
TVA Alarm Philosophy .......................................................................................................3-1
TVA Alarm Philosophy Development and Summary ...........................................................3-1
4 APPLICATIONS AND SOFTWARE FOR ALARM MANAGEMENT .....................................4-1
5 ALARM MANAGEMENT AT BULL RUN FOSSIL POWER PLANT .....................................5-1
Alarm Analysis and Baseline for Bull Run...........................................................................5-1
Documentation and Rationalization of the Bull Run Alarm System .....................................5-6
6 ALARM MANAGEMENT AT WIDOWS CREEK FOSSIL POWER PLANT ..........................6-1
Alarms Analysis and Baseline for Widows Creek Unit 7 .....................................................6-1
Alarms Analysis and Baseline for Widows Creek Unit 8 .....................................................6-6
Documentation and Rationalization of the Widows Creek Alarm System..........................6-11
7 ALARM MANAGEMENT AT LAGOON CREEK COMBUSTION TURBINE PLANT ............7-1
Alarm Analysis and Baseline for Lagoon Creek ..................................................................7-1
Documentation and Rationalization of the Lagoon Creek Alarm System ............................7-6
8 ALARM MANAGEMENT AT GALLATIN FOSSIL PLANT ...................................................8-1
Alarm Analysis and Baseline for Gallatin Units 1 & 2 ..........................................................8-1
Alarm Analysis and Baseline for Gallatin Units 3 & 4 ..........................................................8-6
Documentation and Rationalization for the Gallatin Alarm System ...................................8-10
9 ALARM MANAGEMENT FOR SOUTHAVEN.......................................................................9-1
Alarm Analysis and Baseline for Southaven .......................................................................9-1
Documentation and Rationalization of the Southaven Alarm System .................................9-6
10 ALARM MANAGEMENT FOR MAGNOLIA......................................................................10-1
Alarm Analysis and Baseline for Magnolia........................................................................10-1
Documentation and Rationalization of the Magnolia Alarm System ..................................10-7
11 ALARM MANAGEMENT FOR ALLEN .............................................................................11-1
Alarm Analysis and Baseline for Allen ..............................................................................11-1
Documentation and Rationalization of the Allen Alarm System ........................................11-6
12 PATH FORWARD FOR FURTHER IMPROVEMENTS .....................................................12-1
13 SUMMARY .......................................................................................................................13-1
10707048 vii
A ALARM ANALYSIS DESCRIPTIONS ................................................................................. A-1
B ALARM SYSTEM CLASSIFICATION LEVELS .................................................................. B-1
C TVA ALARM PHILOSOPHY ............................................................................................... C-1
D BAD ACTOR RESOLUTIONS ............................................................................................ D-1
E REFERENCES .................................................................................................................... E-1
Books: ............................................................................................................................... E-1
Reports: ............................................................................................................................ E-1
F PRINCIPAL INVESTIGATORS............................................................................................ F-1
Ganapathy Nagarajan ....................................................................................................... F-1
Kenneth Phelps ................................................................................................................. F-1
Laura J. Martinez .............................................................................................................. F-1
Ronald W. Carlton ............................................................................................................. F-1
Tim Younts ........................................................................................................................ F-2
10707048 viii
LIST OF FIGURES
Figure 1-1 Number of Alarms Configured per Operator ...........................................................1-2
Figure 2-1 - Five Levels of Alarm Systems ...............................................................................2-1
Figure 5-1 Bull Run Recorded Alarms per Day........................................................................5-3
Figure 5-2 Bull Run Annunciated Alarms per Day with and without the 10 Most Frequent
Alarms .....................................................................................................................................5-3
Figure 5-3 Bull Run Alarm Flood Counts .................................................................................5-4
Figure 5-4 Bull Run Alarm Flood Duration ...............................................................................5-5
Figure 5-5 Bull Run Average Alarm Rates...............................................................................5-6
Figure 6-1 Widows Creek Unit 7 Annunciated Alarms per Day................................................6-3
Figure 6-2 Widows Creek Unit 7 Annunciated Alarms per Day with and without the 10
Most Frequent Alarms ..............................................................................................................6-3
Figure 6-3 Widows Creek Unit 7 Alarm Flood Count ...............................................................6-4
Figure 6-4 Widows Creek Unit 7 Alarm Flood Duration ...........................................................6-5
Figure 6-5 Widows Creek Unit 7 Average Alarm Rates ...........................................................6-6
Figure 6-6 Widows Creek Unit 8 Recorded Alarms per Day ....................................................6-8
Figure 6-7 Widows Creek Unit 8 Annunciated Alarms per Day with and without the 10
Most Frequent Alarms ..............................................................................................................6-8
Figure 6-8 Widows Creek Unit 8 Alarm Flood Count ...............................................................6-9
Figure 6-9 Widows Creek Unit 8 Flood Duration ....................................................................6-10
Figure 6-10 Widows Creek Unit 8 Alarm Rates .....................................................................6-11
Figure 7-1 Lagoon Creek Recorded Alarms per Day ................................................................7-3
Figure 7-2 Lagoon Creek Annunciated Alarms per Day with and without the 10 Most
Frequent Alarms ......................................................................................................................7-3
Figure 7-3 Lagoon Creek Alarm Flood Count ..........................................................................7-4
Figure 7-4 Lagoon Creek Alarm Flood Duration ......................................................................7-5
Figure 7-5 Lagoon Creek (Average Alarm Rates) ...................................................................7-6
Figure 8-1 Gallatin Units 1 & 2 Recorded Alarms per Day .......................................................8-2
Figure 8-2 Gallatin Units 1 & 2 Annunciated Alarms per Day with and without the 10 Most
Frequent Alarms ......................................................................................................................8-3
Figure 8-3 Gallatin Units 1 & 2 Alarm Flood Count ..................................................................8-4
Figure 8-4 Gallatin Units 1 & 2 Alarm Flood Duration ..............................................................8-5
Figure 8-5 Gallatin Units 1 & 2 Average Alarm Rates ..............................................................8-5
Figure 8-6 Gallatin Units 3 & 4 Recorded Alarms ....................................................................8-7
Figure 8-7 Gallatin Units 3 & 4 Annunciated Alarms with and without the 10 Most
Frequent Alarms ......................................................................................................................8-7
Figure 8-8 Gallatin Units 3 & 4 Alarm Flood Count ..................................................................8-8
Figure 8-9 Gallatin Units 3 & 4 Alarm Flood Duration ..............................................................8-9
Figure 8-10 Gallatin Units 3 & 4 Average Alarm Rates ............................................................8-9
Figure 9-1 Southaven Recorded Alarms per Day ....................................................................9-3
Figure 9-2 Southaven Annunciated Alarms per Day with and without the 10 Most
Frequent Alarms ......................................................................................................................9-3
Figure 9-3 Southaven Alarm Flood Count ...............................................................................9-4
Figure 9-4 Southaven Alarm Flood Duration ...........................................................................9-5
Figure 9-5 Southaven – (Average Alarm Rates) ......................................................................9-6
Figure 10-1 Magnolia Recorded Alarms per Day...................................................................10-3
Figure 10-2 Magnolia Annunciated Alarms per Day with and without the 10 Most Frequent
Alarms ...................................................................................................................................10-3
Figure 10-3 Magnolia Alarm Flood Count ..............................................................................10-5
10707048 ix
Figure 10-4 Magnolia Alarm Flood Duration ..........................................................................10-6
Figure 10-5 Magnolia Average Alarm Rates ..........................................................................10-6
Figure 11-1 Allen Annunciated Alarms per Day with and without the 10 Most Frequent
Alarms ...................................................................................................................................11-3
Figure 11-2 Allen Alarm Flood Count ....................................................................................11-4
Figure 11-3 Allen Alarm Flood Duration ................................................................................11-5
Figure 11-4 Allen – Average Alarm Rates .............................................................................11-5
10707048 x
LIST OF TABLES
Table 3-1 Severity of Consequences ..................................................................................... 3-2
Table 3-2 Determination of Priority from the Severity of Consequence and the Maximum
Time to Respond..................................................................................................................... 3-3
Table 5-1 Bull Run Top 20 Nuisance Alarms .......................................................................... 5-2
Table 5-2 Bull Run Alarms per Day ........................................................................................ 5-4
Table 5-3 Bull Run Alarm Flood Count Analysis ..................................................................... 5-5
Table 5-4 Bull Run Average Alarm Rates ............................................................................... 5-6
Table 6-1 Widows Creek Unit 7 Top 20 Nuisance Alarms ...................................................... 6-2
Table 6-2 Widows Creek Unit 7 Alarms per Day .................................................................... 6-4
Table 6-3 Widows Creek Unit 7 Alarm Flood Count Analysis ................................................. 6-5
Table 6-4 Widows Creek Unit 7 Average Alarm Rates ........................................................... 6-6
Table 6-5 Widows Creek Unit 8 Top 20 Nuisance Alarms ...................................................... 6-7
Table 6-6 Widows Creek Unit 8 Alarms per Day ..................................................................... 6-9
Table 6-7 Widows Creek Unit 8 Alarm Flood Count Analysis ............................................... 6-10
Table 6-8 Widows Creek Unit 8 Average Alarm Rates ......................................................... 6-11
Table 7-1 Lagoon Creek Top 20 Nuisance Alarms ................................................................. 7-2
Table 7-2 Lagoon Creek Alarms per Day ............................................................................... 7-4
Table 7-3 Lagoon Creek Alarm Flood Count Analysis ............................................................ 7-5
Table 7-4 Lagoon Creek Average Alarm Rates ...................................................................... 7-6
Table 0-1 Gallatin Units 1 & 2 Top 20 Nuisance Alarms ......................................................... 8-2
Table 8-2 Gallatin Units 1 & 2 Alarms per Day ....................................................................... 8-3
Table 8-3 Gallatin Units 1 & 2 Alarm Flood Count Analysis ..................................................... 8-4
Table 8-4 Gallatin Units 1 & 2 Average Alarm Rates .............................................................. 8-6
Table 8-5 Gallatin Units 3 & 4 Top 20 Nuisance Alarms ......................................................... 8-6
Table 9-1 Southaven Top 20 Nuisance Alarms ...................................................................... 9-2
Table 9-2 Southaven Alarms per Day .................................................................................... 9-4
Table 9-3 Southaven Alarm Flood Count Analysis ................................................................. 9-5
Table 9-4 Southaven Average Alarm Rates ........................................................................... 9-6
Table 9-5 Southaven Delta V Alarm Change Summary.......................................................... 9-8
Table 9-6 Southaven GE Mark V CTG Annunciated Alarm Change Summary ....................... 9-8
Table 9-7 Southaven STG GE Mark Alarm Change ............................................................... 9-9
Table 10-1 Magnolia Top 20 Nuisance Alarms ..................................................................... 10-2
Table 10-2 Magnolia Alarms per Day ................................................................................... 10-4
Table 10-3 Magnolia Alarm Flood Count Analysis ................................................................ 10-5
Table 10-4 Magnolia Average Alarm Rates .......................................................................... 10-7
Table 10-5 Magnolia Alarm Change Summary ..................................................................... 10-8
Table 11-1 Allen Top 20 Nuisance Alarms ........................................................................... 11-2
Table 11-2 Allen Alarms per Day.......................................................................................... 11-3
Table 11-3 Allen Alarm Flood Count Analysis ...................................................................... 11-4
Table 11-4 Allen Average Alarm Rates ................................................................................ 11-6
10707048 xi
10707048
1
THE ALARM MANAGEMENT PROBLEM AT TVA
The Tennessee Valley Authority (TVA) has numerous power plants and a variety of control
systems in each plant. The power plants are of several different types, for example, fossil based
(coal fired), combustion turbine, or combined cycle. Each plant is controlled by operators
through a control system; these systems are provided by a number of vendors. Most of these
systems had alarm management issues. The following briefly explains the nature and origin of
these typical issues.
10707048 1-1
A major problem associated with the change from a benchboard to a DCS was the much more
limited view of the process by the operator. Early displays could display only a few values at a
time. The operator’s “at-a-glance” overview perspective was thus replaced by a “keyhole” view
of the process, and a multiplicity of new alarms was often used to notify the operator to shift that
view from one part of the process to another. Figure 1-1 plots the increase in configured alarms
per operator over the transition to the DCS era.
Figure 1-1
Number of Alarms Configured per Operator
Many systems generate, even in steady-state process operation, far more alarm events than can
be possibly individually understood and acted upon by the operator. During an upset, there is an
order of magnitude increase in the number and speed of alarm occurrences, not only rendering
the alarm system useless, but also creating an active hindrance to the operator’s ability to deal
with the situation.
It is well known that ordinary process upsets can be either made worse or made to last longer if
they are associated with an ineffective alarm system. The situation is made worse by the ease of
modifying alarms in a DCS. Not only can engineers change the alarm configuration, but so can
operators, maintenance technicians, managers, and college interns. Security and change tracking
is generally inadequate for the task. For many years, the configuration, alteration, and bypassing
of alarms in a DCS has often been ineffectively covered by management of change (MOC)
policies and practices.
The result is alarm systems that are overloaded and ineffective. Such systems are common
throughout industry and also typical of the systems in TVA plants.
10707048 1-2
Table 1-1
Seven-Step Methodology for Alarm System Improvement
“Always-needed” steps:
Step 1: Develop, Adopt, and Maintain an Alarm Philosophy.
Step 2: Collect Data and Benchmark the Alarm Systems.
Step 3: Perform “Bad Actor” Alarm Resolution.
Steps to implement based on alarm system performance after the first three steps:
Step 4: Perform Alarm Documentation and Rationalization (D&R).
Step 5: Implement Alarm Audit and Enforcement Technology.
Step 6: Implement Real-Time Alarm Management.
Step 7: Control and Maintain the Improved System.
10707048 1-3
Step 6: Implement Real-Time Alarm Management.
Certain advanced alarm capabilities may be needed on some systems to address specific issues:
• Alarm shelving: A method to safely and securely suppress nuisance or other alarms until
their underlying problems can be corrected. Uncontrolled alarm suppression is a major
problem with many DCSs.
• State-based alarming: In state-based alarming, the process state is determined, and alarm
settings are altered in predetermined ways to match the alarming requirements of that process
state. The state determination and alarm configuration alteration can be done in a variety of
manual, semi-automated, and fully automated ways.
• Alarm flood suppression: Alarm floods are usually associated with events such as an
inadvertent equipment trip. Scores of distracting and non-relevant alarms can result,
impeding the operator’s ability to handle the situation correctly. Flood suppression
temporarily eliminates the expected and distracting alarms, leaving the relevant alarms that
assist the operator and ensuring that the alarm system is effective in abnormal situations.
Step 7: Control and Maintain the Improved System.
An ongoing program of system analysis, key performance indicator (KPI) monitoring, and the
correction of problems as they occur is needed for an effective alarm system.
10707048 1-4
2
ALARM MANAGEMENT SOLUTION FOR TVA
The TVA corporate office identified alarm management as a way to increase the safety,
availability, and cost effectiveness of their operations. TVA had previously developed an alarm
philosophy as described in step 1 of the Electric Power Research Institute (EPRI) guideline.
TVA for this project contracted with EPRI to facilitate and complete steps 2, 3, and 4 of the
Alarm Management Improvement Process described earlier in this document. EPRI further sub-
contracted the work to Plant Automation Services (PAS) due to their expertise in the subject
matter. PAS and EPRI had worked together previously to create the EPRI Alarm Management
and Annunciator Applications Guidelines (EPRI report 1014316), which is specific to the power
industry. The steps from these guidelines were applied to the TVA alarm management problems.
Step 1 involved the development of an alarm philosophy. Refer to Chapter 3 for details. Step 2
involved analyzing the alarms data for each plant or unit for each operator position and
establishing the benchmark for the system. Based on a variety of performance measurements, an
alarm system was benchmarked at one of five levels, which are shown in Figure 2-1.
OVERLOADED REACTIVE STABLE ROBUST PREDICTIVE
Improvement Plans : Specific Steps to move from each classification to the next.
Table 2-1
Five Levels of Alarm Systems
Overloaded: A continuously high rate of alarms, with rapid performance deterioration during
process upset.
Reactive: Some improvement compared to Overloaded, but the peak rate during upset is still
unmanageable. The alarm system is still an unhelpful distraction to the operator for much of the
time.
Stable: A system well defined for normal operation, but less useful during plant upset.
Compared to Reactive, there are improvements in both the average alarm and peak alarm rates.
“Bad actors” are resolved and under systematic control. Problems remain with the burst alarm
rate.
Robust: The average and peak alarm rates are under control for foreseeable plant operating
scenarios. Dynamic and state-based techniques are used to improve the real-time performance.
Predictive: Implementation of fully adaptive alarming and state prediction, breakthrough
performance on both the average and the peak alarm rate.
Refer to Appendix B, Alarm System Classification Levels, for further details on each of the
categories.
Along with baseline information, a “bad actor” report was also provided. These are the top
frequent alarms provided in several categories, such as chattering, consequential, and duplicate
alarms. The likely causes of these alarms and their fixes were also suggested.
10707048 2-1
Step 3 involved discussing these alarms with the plant personnel and identifying the resolution
taken to address the nuisance alarms. This was done just before Step 4, so that information from
the resolution could be used in Documentation and Rationalization.
Step 4 used a sound, consistent, and logical method known as Documentation & Rationalization
(D&R) for determining, prioritizing, and documenting alarms. D&R was performed as a team-
based effort that involved a thorough re-examination of every existing and possible alarm
configured on a system in order to ensure that they support the TVA philosophy.
A team of knowledgeable people reviewed each point on the system and did the following:
• Discussed each configured and possible alarm on that point.
• Verified that operator action is required to respond to the alarm.
• Verified that the alarm is created from a truly abnormal situation.
• Verified that any configured alarm should exist at all.
• Verified that an alarm does not duplicate another similar alarm that occurs under the same
conditions. If it did, the one that was retained best indicated the root cause of the abnormal
condition.
• Determined the proper priority of each alarm according to the method later described in this
section.
• Determined the proper alarm limits for an alarm based on an examination of:
− Process history
− Relevant operating procedures
− Equipment and safety system specifications
• Documented as much of the following as was practical:
− Possible causes of the alarm
− Method of alarm verification
− Proper operator response to the alarm
− Other points likely to be involved with the alarm
− Relevant operating procedure, alarm response procedure, process hazard analysis, or other references for
dealing with the alarm
• Noted any needed modifications to an existing alarm, such as introduction of logic, change of
the alarm type, rewording of the alarm message, changing DCS graphics, and so forth.
During D&R, all DCS points that could be alarmed were examined, along with any other
systems that provided alarm or abnormal situation notification to the board operator. PAS’s Plant
State Suite (PSS) software built an alarm database from the DCS point configuration, which was
then used to facilitate the D&R sessions.
A much more detailed description of the D&R work process that was followed can be found in
EPRI Alarm Management and Annunciator Applications Guidelines (EPRI report 1014316).
The above steps were performed at TVA sites listed below. The details on each of the steps are
provided in the individual sections.
Bull Run –Fossil Power Plant
Widows Creek Unit 7 – Fossil Power Plant
10707048 2-2
Widows Creek Unit 8 – Fossil Power Plant
Lagoon Creek – Combustion-Turbine Power Plant.
Gallatin Units 1 & 2 – Fossil Power Plant
Gallatin Units 3 & 4 – Fossil Power Plant
Southaven - Combined-Cycle Combustion-Turbine Power Plant
10707048 2-3
10707048
3
TVA ALARM PHILOSOPHY DEVELOPMENT
TVA Alarm Philosophy
The alarm philosophy document is the guiding design document of any alarm management
process. It contains the Alarm Management Improvement Process, measureable goals, and
targets for the alarm system. The TVA alarm philosophy is applicable to all TVA sites.
10707048 3-1
Table 3-1
Severity of Consequences
Public or No effect Opacity, NOx, SOx, or other Opacity, NOx, Sox, or Opacity, NOx, Sox, or other
Environment environmental problem not other environmental environmental problem
requiring reporting or problem involving involving reporting with the
resulting in fines. reporting but not fines. likelihood of fines.
Release to on-site Minor environmental Significant adverse impact,
environment, contained impact, but possible significant long-term
immediately. Amount below permit violation with liability, enforcement
reportable quantities. Local minor administrative action. Limited or extensive
environmental effect only. penalties. toxic release. Crosses fence
Does not cross fence line Contamination causes line. Impact involving the
and is not detected off-site. some non-permanent community. Operating
Little, if any, cleanup. damage. Possible Permit violation. Clear
Negligible financial detection off-site or a public concern. Repeated
consequences. possible matter of exceedance. Uncontained
minor public concern. release of hazardous
Single complaint materials with major
likely. Single environmental impact and
exceedance of statutory third-party impact.
or prescribed limit. Extensive cleanup measures
Reportable quantity. and financial consequences.
North American Submittal of a NERC report Submittal of a NERC Continued NERC violation
Electric Reliability with no fines. report that may involve or impact to grid.
Corporation fines.
(NERC) Reporting
Generation No loss Unit derate of < 10% MW Unit derate more than A unit trip.
Capacity capacity for < 24 hr. “Minor” but less than
full load MW capacity.
Generation Impact No effect Generation loss or dip that is Generation loss Generation loss that is likely
on Electrical Grid automatically compensated involving significant to produce brownout or
for by computerized grid adjustment of grid cascade to blackout
load adjustment. resources, requiring conditions.
human response in
planning and
intervention.
Costs/Production No loss Event costing < $50,000. Event costing $50,000 Event costing > $250,000.
Reporting required at the to $250,000. Reporting Reporting required above
Unit Mgr level. required at the site the site level.
level.
10707048 3-2
The maximum time to respond (MTR) is the maximum allowable time after the alarm has
occurred that an operator has to take action to avoid the consequences. A grid combined the
severity of consequence and the maximum time to respond to determine the priority, as
shown in Table 3-2.
Table 3-2
Determination of Priority from the Severity of Consequence and the Maximum Time to
Respond
10 to 30 minutes
3 3 2
Delayed No Alarm
Abnormal Abnormal Urgent
Response
3 to 10 minutes 3 2 1
No Alarm
Quick Response Abnormal Urgent Critical
< 3 minutes
2 1 1
Immediate No Alarm
Urgent Critical Critical
Response
As the philosophy was applied to each site, the knowledge and experience gained from the
sites in terms of philosophy was incorporated into the document.
10707048 3-3
10707048
4
APPLICATIONS AND SOFTWARE FOR ALARM
MANAGEMENT
Plant Automation Services (PAS) has developed the PSS software suite that has capabilities
for alarm management and loop optimization. Two modules that were actively used for TVA
were Alarm Analysis and Alarm Advanced Elements. The Alarm Analysis module is used
for analysis of alarms and event data, and the Advanced Alarm Elements module is aimed to
optimize the D&R effort. The software imports point configurations for multiple systems and
creates the tag and alarm database. The Alarm Advanced Elements module provides a
window with all tags. It allows selection of single or multiple tags and displays all configured
and non-configured alarms pertaining to those tags in a separate window. This window
provides drop-down lists and manual entry for trip setpoints, causes, actions, time to respond,
severity of consequences, and all other alarm information. This module calculates the
proposed priority based on the priority determination matrix (Refer to Section 3). It also
provides an override option to manually override the recommended priority. It supports
import and export options for Microsoft Excel.
This module is designed to improve the time required for the D&R effort by reducing manual
entry errors through drop-down lists and permitting copying and pasting of the D&R
information without overwriting the alarm information itself.
This application software has been used for the D&R activity for all the plants and has been
extremely effective in reducing the time for the amount of work required. The database is
then exported for the implementation of the D&R effort and also for reference through
TVA’s portals.
10707048 4-1
10707048
5
ALARM MANAGEMENT AT BULL RUN FOSSIL
POWER PLANT
The alarm improvement effort at Bull Run involved the analysis of alarms data to create a
baseline and the D&R of all alarms.
10707048 5-1
Table 5-1
Bull Run Top 20 Nuisance Alarms
Figure 5-1 shows the recorded and annunciated alarm daily rates throughout the analysis
period. Both recorded and annunciated alarms continuously exceeded best practice
guidelines. The average recorded alarm rate for this period was 7,200 alarms per day with a
peak of 22,685 alarms. The average annunciated alarm rate for this period was 6,883 alarms
per day with a peak of 21,495.
10707048 5-2
25000 Recorded Alarms Per Day
Recorded Alarms
20000
Annunciated
Alarms
15000 'Manageable'
(300/day)
10000
5000
0
1/07/2011 - 1/27/2011
Figure 5-1
Bull Run Recorded Alarms per Day
Figure 5-2 shows annunciated alarms and the load that would be seen if the top 10 most
frequently alarmed tags were eliminated. The annunciated alarm rate could be reduced by
approximately 60% by removing the 10 most frequent alarms.
Figure 5-2
Bull Run Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
10707048 5-3
Table 5-2 summarizes the daily alarm rates.
Table 5-2
Bull Run Alarms per Day
% Of % Of
Days Days
Median
Average Maximum More More
Total Alarms
Alarms Alarms Than Than
Alarms per
per Day per Day 300 150
Day
Alarms Alarms
per Day per Day
Recorded
151,194 7,200 4,150 22,685 100% 100%
Alarms
Non-
6,648 317 80 2,087 33% 38%
Annunciated
Annunciated - 4 83,586 3,980 1,457 14,905 100% 100%
Annunciated - 3 26,175 1,246 831 3,764 100% 100%
Annunciated - 2 14,895 709 12 8,412 38% 38%
Annunciated - 1 19,890 947 634 3,012 62% 71%
All Annunciated 144,546 6,883 4,045 21,495 100% 100%
Annunciated
Alarms Without 49,664 2,365 1527 6,070 100% 100%
the 10 Most
Frequent
Figure 5-3 shows the alarm counts in the alarm floods during the analysis period. Table 5-3
provides an analysis of the alarm flood counts.
Figure 5-3
Bull Run Alarm Flood Counts
10707048 5-4
Table 5-3
Bull Run Alarm Flood Count Analysis
Alarm floods were a problem for this system. In fact, more than 99% of annunciated alarms
produced by the system were during flood periods. Flood magnitude was high as there were
about 8.4 floods per day on average.
This system spent 92.1% of the time in a flood condition as shown in Figure 5-4. Flood
duration periods (in one case lasting more than four days) presented alarms at a rate higher
than an operator could handle.
Duration in
01/07/2011 - 01/27/2011
Figure 5-4
Bull Run Alarm Flood Duration
Figure 5-5 shows the average alarm rates through the analysis period, and Table 5-4 provides
a 10-minute time-slice view of the entire period.
10707048 5-5
Annunciated Alarms per 10
Minutes
Highest
10-minute
Rate = 437
Alarm
01/07/2011 - 01/27/2011
Figure 5-5
Bull Run Average Alarm Rates
Table 5-4
Bull Run Average Alarm Rates
In the above 10-minute time-slice view, the system spent about 82.9% of the time producing
alarms at an unacceptable rate.
10707048 5-6
Table 5-5
Bull Run D&R Participants
In all, the alarms on more than 28,000 tags were evaluated. The evaluation yielded a
reduction of 95.44% of configured annunciated priorities for the power house and 89.46% for
the scrubber. The result is a fully rationalized and documented alarm database that will
enhance an operator’s ability to respond to abnormal situations without hindering their
efforts.
Table 5-6
Bull Run Power House Annunciated Alarm Changes
Configured
711 699 1,501 2,911
Priorities
After D&R
% of Configured
24.43% 24.01% 51.56% ---
Priorities
Configured
-58,171 -446 -2,273 -60,890
Priorities
Resulting
Changes
% of Change -98.79% -38.95% -60.23% -95.44%
10707048 5-7
Table 5-7
Bull Run Scrubber Alarms Changes
Configured
207 305 995 1507
Priorities
After D&R
% of
Configured 13.73% 20.24% 66.03% ---
Priorities
Configured
-11,992 +261 -1,049 -12,780
Priorities
Resulting
Changes
% of Change -97.7% +593.18% -51.32% -89.46%
During D&R, all the recommendations for the bad actors were discussed, and a resolution
action was noted for several bad actors as needed. Refer to Attachment A, TVA Bull Run
Bad Actor resolutions.xls, for the list of resolutions.
10707048 5-8
6
ALARM MANAGEMENT AT WIDOWS CREEK FOSSIL
POWER PLANT
The alarm improvement effort at Widows Creek involved the analysis of alarms data to create a
baseline and the D&R of all alarms. Since EPRI provides guidelines on an operator basis and
because Units 7 and 8 are operated from different consoles, a separate analysis was done for each
of the units. Units 7 and 8 are almost identical. Unit 7 is slightly larger in terms of tags and
alarms since it includes the common scrubber.
10707048 6-1
Table 6-1
Widows Creek Unit 7 Top 20 Nuisance Alarms
Figure 6-1 shows the daily alarm rates. Both recorded and annunciated alarms are the same and
continuously far exceed best practice guidelines. The average recorded or annunciated alarm rate
for this period was 282,520 alarms per day, with a peak of 379,966 alarms. With this number of
alarms, the operator could not possibly evaluate and respond to each one.
10707048 6-2
Annunciated Alarms Per Day
600000
500000
400000
300000
'Manageable' (300/day)
100000 'Acceptable' (150/day)
0
8/02/2011 - 8/23/2011
Figure 6-1
Widows Creek Unit 7 Annunciated Alarms per Day
Figure 6-2 shows that the annunciated alarm rate could be reduced by approximately 60% by
removing the 10 most frequent alarms.
'Acceptable' (150/day)
400000
300000
200000
100000
0
8/02/2011 - 8/23/2011
Note: Manageable and Acceptable overlay each other due to the magnitude of the scale.
Figure 6-2
Widows Creek Unit 7 Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
Table 6-2 summarizes the daily rates and the percentage of days the alarms were above the limit.
For 100% of days, the alarm rate exceeded the both the EPRI Acceptable range of 150 alarms per
day and the Manageable range of 300 alarms per day.
10707048 6-3
Table 6-2
Widows Creek Unit 7 Alarms per Day
% Of
% Of Days
Days
Average Median Maximum More Than
More
Total Alarms Alarms Alarms Alarms 300
Than 150
per Day per Day per Day Alarms
Alarms
per Day
per Day
Recorded Alarms 6,215,444 282,520 267,400 379,966 100% 100%
Non-Annunciated 0 0 0 0 0% 0%
Annunciated - >= 4 5,766,658 262,121 245,658 354,386 100% 100%
Annunciated - 3 45 2 0 45 0% 0%
Annunciated - 2 77,037 3,502 669 17,431 68% 77%
Annunciated - 1 371,704 16,896 16,412 28,120 100% 100%
All Annunciated 6,215,444 282,520 267,400 379,966 100% 100%
Figure 6-3 shows the alarm count in the alarm floods during the analysis period. Table 6-3
provides an analysis of the alarm flood counts.
10000000
Peaks Above 3,500,000
9000000
8000000
3000000
2000000
1000000
08/02/2011 - 08/23/2011
Figure 6-3
Widows Creek Unit 7 Alarm Flood Count
10707048 6-4
Table 6-3
Widows Creek Unit 7 Alarm Flood Count Analysis
Alarm floods were a problem for this system. In fact, more than 99% of annunciated alarms
produced by the system were during flood periods. Flood magnitude was high. There were only
two flood periods identified during the whole period, but they covered almost the entire period.
This system spent 99.9% of the time in a flood condition as shown in Figure 6-4. Flood duration
periods (in one case lasting more than 15 days) presented alarms at a rate higher than an operator
can handle.
600
Duration in
Hours
2 Separate Floods
400
300
200
100
0
08/02/2011 - 08/23/2011
Figure 6-4
Widows Creek Unit 7 Alarm Flood Duration
Figure 6-5 shows the average alarm rates throughout the alarm period, and Table 6-4 provides a
time-slice view of the rates. The system spent about 99.8% of the time producing alarms at an
unacceptable rate. There were approximately 3,161 instances when more than 10 alarms
occurred in 10 minutes.
10707048 6-5
Annunciated Alarms per 10 Minutes
3100
2600
2100
1600
1100
600
100
-400
08/02/2011 - 08/23/2011
Figure 6-5
Widows Creek Unit 7 Average Alarm Rates
Table 6-4
Widows Creek Unit 7 Average Alarm Rates
Annunciated
Alarms per 10
Minutes =0 >0 >=10 >20 >30 >50 >100
Cumulative 0.2% 99.8% 99.8% 99.8% 99.8% 99.8% 99.7%
Flood Flood Flood Flood Flood
10707048 6-6
• The Widows Creek Unit 8 area spent the entire 27 days of the analysis period in a flood
condition.
• The top 10 most frequent alarms accounted for 55% of the total annunciated alarm events.
Many of the duplicate high and abnormal (HI2, HI3, LO2, and LO3) alarms were eliminated
during the D&R process.
• The system produced alarms with a priority distribution outside the best practice guidelines.
• Nuisance alarm generation exceeded best practices. During the evaluation period, 325 unique
alarms were noted as chattering. These chattering events produced over 2 million
annunciated alarms. Table 6-5 lists the top 20 nuisance alarms.
• A total of 1,021 alarms were found that went stale at least once during the analysis period.
There were 3,742 instances of stale alarms.
Table 6-5
Widows Creek Unit 8 Top 20 Nuisance Alarms
10707048 6-7
Figure 6-6 shows the annunciated alarm daily rates throughout the analysis period. The
annunciated alarm rate significantly exceeded best practice guidelines for the 27-day analysis
period. The average annunciated alarm rate for this analysis period was over 100,000 alarms per
day, with a peak of 145,651 alarms on August 6, 2011.
0
7/28/2011 - 8/23/2011
Figure 6-6
Widows Creek Unit 8 Recorded Alarms per Day
As seen in Figure 6-7, the annunciated alarm rate could be reduced by approximately 55% by
removing the 10 most frequent alarms.
10707048 6-8
Table 6-6 summarizes the daily alarm rates.
Table 6-6
Widows Creek Unit 8 Alarms per Day
% Of Days % Of Days
Average Median Maximum More Than More Than
Total
Alarms per Alarms Alarms 300 150
Alarms
Day per Day per Day Alarms Alarms per
per Day Day
Annunciated - ≥4 1,374,852 50,920 49,853 75,856 100% 100%
Annunciated - 3 1,318 49 16 806 4% 4%
Annunciated - 2 0 0 0 0 0% 0%
Annunciated - 1 1,373,006 50,852 49,692 86,852 100% 100%
All Annunciated 2,749,176 101,821 99,407 145,651 100% 100%
Annunciated
Alarms Without 1,214,468 44,980 46,308 60,395 100% 100%
the 10 Most
Frequent
Figure 6-8 shows the alarm count in alarm floods during the analysis period.
1,500,000
1,000,000
500,000
0
07/28/2011 - 08/23/2011
Figure 6-8
Widows Creek Unit 8 Alarm Flood Count
Table 6-7 provides an analysis for the alarm count in alarm floods. Alarm floods were a problem
for this system. The system was under a flood period for the entire analysis period.
10707048 6-9
Table 6-7
Widows Creek Unit 8 Alarm Flood Count Analysis
Figure 6-9 shows the duration in hours that the system spent in floods. In one case, the system
spent almost seven days in flood.
Alarm Floods - Duration
Duration in Hours
180
160
100
80
60
40
20
0
07/28/2011 - 08/23/2011
Figure 6-9
Widows Creek Unit 8 Flood Duration
The Widows Creek Unit 8 alarm rates in Figure 6-10 show that the system was in flood 99.8% of
the time. Figure 6-10 shows the average alarm rates throughout the alarm period.
10707048 6-10
Annunciated Alarms per 10 Minutes
Highest 10-minute Rate =
1,548
1600
Alarm Flood = 10+ in 10
minutes
1400
1200
1000
800
600
400
200
0
07/28/2011 - 08/23/2011
Figure 6-10
Widows Creek Unit 8 Average Alarm Rates
Table 6-8 separates the alarms rates into 10-minute time-slice views.
Table 6-8
Widows Creek Unit 8 Average Alarm Rates
Annunciated
Alarms per 10 Min. =0 >0 >=10 >20 >30 >50 >100
Cumulative 0.1% 99.9% 99.9% 99.9% 99.9% 99.9% 99.9%
Flood Flood Flood Flood Flood
No. of Alarms 0 1–9 10–20 21–30 31–50 51–100 >100
Instances 2 0 0 0 0 0 3,886
% of time 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 99.9%
In this 10-minute time-slice view, the system spends 99.9% of the time producing alarms at an
unacceptable rate. There were only two instances when the alarm rate was less than 10 alarms in
10 minutes
10707048 6-11
Table 6-9
Widows Creek D&R Participants
All alarmable tags configured in the site’s ABB Infi 90 DCS were evaluated. The results of this
D&R were provided for review and implementation by site personnel at the site. In all, the
alarms on almost 16,000 tags were evaluated. The evaluation yielded a reduction of 91.35% of
configured annunciated priorities for Unit 7 and 89.20% for Unit 8. Following implementation of
D&R, priorities 1 through 3 would be annunciated, and priority 16 would be journaled. Tables 6-
10 and 6-11 show the changes in alarm numbers and priorities.
Table 6-10
Widows Creek Unit 7 Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3~16
Description Configured
(Critical) (Urgent) (Abnormal)
Priorities
Configured
38,68 695 20,783 25,346
Priorities
Before D&R % of
Configured 15.26% 2.74% 82.00%
Priorities
Configured
367 929 896 2,192
Priorities
After D&R % of
Configured 16.74% 42.38% 40.88%
Priorities
Configured
-3,501 234 -19,887 -23,154
Resulting Priorities
Changes
% of Change -90.51% 33.67% -95.69% -91.35%
10707048 6-12
Table 6-11
Widows Creek Unit 8 Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Urgent) (Abnormal)
Priorities
Configured
6,643 108 16,275 23,026
Priorities
Before D&R % of
Configured 28.85% 0.47% 70.68%
Priorities
Configured
373 841 1,273 2,487
Priorities
After D&R % of
Configured 15.00% 33.82% 51.19%
Priorities
Configured
-6,270 733 -15,002 -20,539
Priorities
Resulting Changes
% of Change -94.39% 678.70% -92.18% -89.20%
During D&R, all the recommendations for the bad actors were discussed, and a resolution action
was noted for several bad actors as needed. Refer to Attachment B, TVA Widows Creek U7 Bad
Actor Resolutions.xls, and Attachment C, TVA Widows Creek U8 Bad Actor Resolutions.xls,
for the list of resolutions for Unit 7 and 8, respectively.
10707048 6-13
10707048
7
ALARM MANAGEMENT AT LAGOON CREEK
COMBUSTION TURBINE PLANT
Lagoon Creek is a combustion-turbine power generation facility operated from a single operator
position. The alarm management included baseline and bad actor reports and D of the alarm
system.
10707048 7-1
Table 7-1
Lagoon Creek Top 20 Nuisance Alarms
10707048 7-2
Recorded Alarms Per Day
2500
Recorded Alarms
2000
Annunciated Alarms
1500 'Manageable'
(300/day)
1000
500
0
17 DAYS
7/25/2011 - 8/10/2011
Figure 7-1
Lagoon Creek Recorded Alarms per Day
Figure 7-2 shows the alarm rates that could be reduced by fixing the 10 most frequent alarms.
Annunciated Alarms
2000
Annunciated Alarms w/o
10 Most Frequent
1500 'Manageable' (300/day)
1000
500
0 17 DAYS
7/25/2011 - 8/10/2011
Figure 7-2
Lagoon Creek Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
10707048 7-3
Figure 7-3
Lagoon Creek Alarms per Day
% Of % Of
Days Days
Median
Average Maximum More More
Total Alarms
Alarms Alarms Than Than
Alarms per
per Day per Day 300 150
Day
Alarms Alarms
per Day per Day
Recorded
13,613 801 561 2,160 88% 94%
Alarms
JOURNAL 2,594 153 122 585 6% 35%
Annunciated - 4 491 29 21 125 0% 0%
Annunciated - 3 7,206 424 274 1,418 41% 65%
Annunciated - 2 1,745 103 46 581 12% 18%
Annunciated - 1 1,577 93 90 211 0% 18%
All Annunciated 11,019 648 423 2,038 65% 88%
Annunciated
Alarms Without 9,312 548 332 1,765 59% 88%
the 10 Most
Frequent
For 88% of the days, the alarm rate exceeded the EPRI Acceptable range, and for 65% of the
days, it exceeded the Manageable range. See Figure 7-3. The alarm system’s effectiveness was
significantly compromised during such periods.
300
200
100
0
7/25/2011 - 8/10/2011
Figure 7-4
Lagoon Creek Alarm Flood Count
10707048 7-4
Table 7-3 provides an analysis of the counts in alarm floods. Alarm floods were a problem for
this system. Almost 77.5% of annunciated alarms produced by the system were during flood
periods. Flood magnitude was high. There were 8.1 floods per day on average.
Table 7-2
Lagoon Creek Alarm Flood Count Analysis
2.5
1.5
0.5
0
7/25/2011 - 8/10/2011
Figure 7-5
Lagoon Creek Alarm Flood Duration
Figure 7-5 shows the average alarm rates over a 10-minute period throughout the analysis period,
and Table 7-4 shows the alarm rates in 10-minute slices.
10707048 7-5
Annunciated Alarms per 10 Minutes
Highest 10-Minute Rate = 686
100 Peaks Exceed 100
90
80
70
60
50
40
30
20
10
0
7/25/2011 - 8/10/2011
17 Days
Figure 7-6
Lagoon Creek Average Alarm Rates
Table 7-3
Lagoon Creek Average Alarm Rates
10707048 7-6
In all, the alarms on 14,476 tags were evaluated (53,336 potentially alarmable points). As shown
in Table 7-6, the D&R evaluation effort yielded a 26% reduction of configured annunciated
alarms within LCC’s Ovation control system.
Table 7-5
Lagoon Creek Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Urgent) (Abnormal)
Priorities
Configured Priorities 1,170 685 3,570 5,425
Before D&R % of Configured
21% 13% 66% ---
Priorities
Configured Priorities 976 868 2,165 4,009
After D&R
% of Configured
24% 22% 54% ---
Priorities
Configured Priorities -194 +183 -1405 -1416
Resulting
Changes
% of Change -16% +27% -39% -26%
All bad actor recommendations provided along with the baseline report were discussed, and the
resolutions taken on all the bad actors were noted as part of the D&R database.
10707048 7-7
10707048
8
ALARM MANAGEMENT AT GALLATIN FOSSIL PLANT
The TVA Gallatin fossil-power generating facility has four generation units, which are operated
from two separate console positions. EPRI provides guidelines on an operator basis, so a separate
analysis was performed for the two positions—Units 1 & 2 and Units 3 &4.
10707048 8-1
Table 8-1
Gallatin Units 1 & 2 Top 20 Nuisance Alarms
Figure 8-1 shows the plot of the number of recorded and annunciated alarms per day. Both
recorded and annunciated alarms continuously far exceeded best practice guidelines. The average
recorded alarm rate for this period was 2,174 alarms per day, with a peak of 6,805 alarms. The
average annunciated alarm rate for this period was 1,291 alarms per day, with a peak of 5,679.
Figure 8-1
Gallatin Units 1 & 2 Recorded Alarms per Day
10707048 8-2
Figure 8-2 shows that the annunciated alarm rate could be reduced by as much as 77% by
removing the 10 most frequent alarms. Table 8-3 provides a statistical breakdown of the total
daily alarms being received.
5000
Annunciated Alarms
4000 Annunciated Alarms
w/o 10 Most Frequent
3000 'Manageable'
(300/day)
2000
1000
0
10/1/2011 - 10/26/2011
Figure 8-2
Gallatin Units 1 & 2 Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
Figure 8-3
Gallatin Units 1 & 2 Alarms per Day
% Of % Of
Days Days
Average Median Maximum
Total More More
Alarms Alarms Alarms per
Alarms Than 300 Than 150
per Day per Day Day
Alarms Alarms
per Day per Day
Recorded Alarms 56,515 2,174 1,517 6,805 100% 100%
Non-Annunciated 22,955 883 70 6,132 27% 27%
Annunciated - 4 1,743 67 59 214 0% 4%
Annunciated - 3 3,751 144 120 632 8% 27%
Annunciated - 2 10,056 387 358 753 73% 100%
Annunciated - 1 18,010 693 199 5,138 42% 62%
All Annunciated 33,560 1,291 791 5,679 100% 100%
Annunciated
Alarms Without
7,438 286 219 854 46% 73%
the 10 Most
Frequent
Figure 8-3 is a graph of the flooding impact at the station based on both frequency and number of
alarms in each flood event. Table 8-3 has the breakdown of the flood statistics to highlight their
impact on operator response to alarms in total.
10707048 8-3
Alarm Floods - Alarm Count
1000
Peaks Above 1000
900
800
700 397 Separate Floods
Figure 8-4
Gallatin Units 1 & 2 Alarm Flood Count
Table 8-2
Gallatin Units 1 & 2 Alarm Flood Count Analysis
Figure 8-4 shows the time duration spent in floods. Alarm floods were a problem for this system.
This system spent 27.7% of the time in a flood condition.
10707048 8-4
Alarm Floods - Duration
20
18
16
14
8
Duration in
6
0
10/01/2011 - 10/26/2011
Figure 8-5
Gallatin Units 1 & 2 Alarm Flood Duration
Figure 8-5 is a plot of the average alarm rates though the analysis period. Table 8-5 provides 10-
minute slices of the alarm rates.
160
140
120
100
80
60
40
20
0
10/1/2011 - 10/26/2011
Figure 8-6
Gallatin Units 1 & 2 Average Alarm Rates
10707048 8-5
Table 8-3
Gallatin Units 1 & 2 Average Alarm Rates
Table 8-4
Gallatin Units 3 & 4 Top 20 Nuisance Alarms
Figure 8-6 shows the recorded and annunciated daily alarm rates. Both recorded and annunciated
alarms continuously far exceeded best practice guidelines. The average recorded alarm rate for
this period was 5,359 alarms per day, with a peak of 11,778 alarms. The average annunciated
alarm rate for this period was 2,659 alarms per day, with a peak of 5,861.
10707048 8-6
12000 Recorded Alarms Per Day
Recorded
10000 Alarms
Annunciated
8000 Alarms
6000
4000
2000
0 10/01/2011 - 10/26/2011
Figure 8-7
Gallatin Units 3 & 4 Recorded Alarms
Figure 8-7 shows the annunciated alarms and the reduction that could be achieved by fixing the
top 10 most frequent alarms. The annunciated alarm rate could be reduced by as much as 54% by
removing the 10 most frequent alarms. Table 8-6 shows the analysis on the daily alarm rates.
6000
Annunciated Alarms
5000
Annunciated Alarms
4000 w/o 10 Most Frequent
'Manageable'
(300/day)
3000
2000
1000
0
10/1/2011 - 10/26/2011
Figure 8-8
Gallatin Units 3 & 4 Annunciated Alarms with and without the 10 Most Frequent Alarms
10707048 8-7
Table 8-5
Gallatin (Units 3 & 4) Alarms per Day
% Of % Of
Days Days
Median
Average Maximum More More
Total Alarms
Alarms Alarms Than Than
Alarms per
per Day per Day 300 150
Day
Alarms Alarms
per Day per Day
Recorded
139,337 5,359 3,826 11,778 100% 100%
Alarms
Non-
70,209 2,700 1,931 5,917 100% 100%
Annunciated
Annunciated - 4 13,867 533 182 3,864 31% 58%
Annunciated - 3 27,923 1,074 405 3,660 77% 96%
Annunciated - 2 26,396 1,015 1,090 1,580 100% 100%
Annunciated - 1 942 36 28 107 0% 0%
All Annunciated 69,128 2,659 1,896 5,861 100% 100%
Annunciated
Alarms Without 31,931 1,228 989 2,695 100% 100%
10 Most
Frequent
Figure 8-8 shows that more than 95.4% of annunciated alarms produced by the system were
during flood periods. Flood magnitude was high. There were 22.6 floods per day on average.
900 588Separate
Floods
800
Highest Count in
700 an Alarm Flood =
10,327
600
500
400
300
200
100
0
10/1/2011 - 10/26/2011
Figure 8-9
Gallatin Units 3 & 4 Alarm Flood Count
10707048 8-8
Figure 8-9 is a plot of the time the system spent in flood. This system spent about 64.0% of the
time in a flood condition.
40
35
30
588 Separate Floods
25
Longest Duration of Flood = 42.6
20
Duration
15
10
0
10/01/2011 - 10/26/2011
Figure 8-10
Gallatin Units 3 & 4 Alarm Flood Duration
Figure 8-10 is a plot of the average alarm rates. Table 8-7 shows a 10-minute slice view of the
alarm rates. The system spent over 57.7% above the recommended alarm rates.
160
140
120
100
80
60
40
20
0
10/1/2011 - 10/26/2011
Figure 8-11
Gallatin Units 3 & 4 Average Alarm Rates
10707048 8-9
Table 8-6
Gallatin Units 3 & 4 Alarm Rate Distribution
The purpose of this D&R was to evaluate all alarmed and potentially alarmable tags configured
in the GAF’s Emerson Ovation DCS.
According to TVA’s Corporate Alarm Management Philosophy document, the term alarm
identifies audibly annunciated alarms, which will be allocated to Emerson Ovation Priorities 1, 2,
or 3. Status messages will be allocated to Priorities 4 through 8. D&R evaluation effort yielded a
54% reduction of configured annunciated alarms (Priorities 1, 2, and 3) within GAF’s Units 1 &
2 Ovation control system. Table 8-9 and 8-10 show the alarm reduction achieved through D&R
for Units 1 & 2 and 3 & 4, respectively.
10707048 8-10
Table 8-8
Gallatin Units 1 & 2 Alarm Changes
Total
Priority 2 Priority 3
Description Priority 1 (Critical) Configured
(Urgent) (Abnormal)
Priorities
Configured
1,604 593 564 2,761
Priorities
Before D&R % of
Configured 58% 21% 21% ---
Priorities
Configured
234 178 857 1,269
Priorities
After D&R % of
Configured 18% 14% 68% ---
Priorities
Configured
-1,370 -415 +293 -1,492
Priorities
Resulting
Changes
% of Change -85% -70% +52% -54%
Table 8-9
Gallatin Units 3 & 4 Alarm Changes
Total
Priority 2 Priority 3
Description Priority 1 (Critical) Configured
(Urgent) (Abnormal)
Priorities
Configured
1,537 624 512 2,673
Priorities
Before D&R % of
Configured 58% 23% 19% ---
Priorities
Configured
204 176 850 1,230
Priorities
After D&R
% of
Configured 17% 14% 69% ---
Priorities
Configured
-1,333 -448 +338 -1,443
Priorities
Resulting
Changes
% of Change -87% -72% +66% -54%
All bad actor recommendations provided along with the baseline report were discussed, and the
resolution was noted as part of the D&R database.
10707048 8-11
10707048
9
ALARM MANAGEMENT FOR SOUTHAVEN
A study was done on the alarm data from February 8 to March 13, 2012, (33 days) from the
Emerson Delta-V System for the TVA Southaven combined-cycle combustion-turbine plant.
10707048 9-1
Table 9-1
Southaven Top 20 Nuisance Alarms
Figure 9-1 shows the recorded alarm daily rates. The average recorded alarm rate for this period
was 197 alarms per day, but there was an instance of more than 600 alarms per day and an
occurrence of a peak of over 1,600 alarms per day. The number of recorded and annunciated
alarms is almost identical, as shown in the graph with recorded and annunciated results
overlaying each other. There were only 19 alarms journaled during the analysis period. With this
number of alarms, the operator was able to respond properly except during the periods of high
alarm rates when the operator could not possibly be evaluating and responding to each one.
10707048 9-2
Figure 9-1
Southaven Recorded Alarms per Day
Figure 9-2 shows the annunciated daily rates and the rate that could be achieved by fixing the top
10 most frequent alarms. The annunciated alarm rate would be reduced by approximately 36%
by removing the 10 most frequent alarms. For 100% of days, the alarm rate exceeded the EPRI
Acceptable range of 150 alarms per day and 77% of days, the alarm rate exceeded the
Manageable range of 300 alarms per day. Table 9-2 provides an analysis of the daily rates.
Figure 9-2
Southaven Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
10707048 9-3
Table 9-2
Southaven Alarms per Day
% Of % Of
Days Days
Median
Average Maximum More More
Total Alarms
Alarms Alarms Than Than
Alarms per
per Day per Day 300 150
Day
Alarms Alarms
per Day per Day
Recorded
6,517 197 112 1,623 9% 42%
Alarms
JOURNAL 19 1 0 7 0% 0%
Annunciated - A 76 2 2 9 0% 0%
Annunciated - W 3,777 114 83 854 6% 18%
Annunciated - C 2,645 80 42 765 3% 9%
All Annunciated 6,498 197 112 1,623 9% 42%
Annunciated
Alarms without 4,137 125 95 637 6% 21%
10 Most
Frequent
800
179 Separate
Floods
600
Highest Count in
an Alarm Flood
400
200
0
01/01/2011 - 01/31/2011
Figure 9-3
Southaven Alarm Flood Count
10707048 9-4
Table 9-3
Southaven Alarm Flood Count Analysis
More than 54% of annunciated alarms produced by the system are during flood periods. Flood
magnitude is high. There were about 5.8 floods per day on average.
10
179 Separate Floods
0
01/01/2011 - 01/31/201
Table 9-4
Southaven Alarm Flood Duration
This system spends 93.05% of the time in a flood condition. Flood duration periods, in one case
lasting almost 12 hours, present alarms at a rate higher than the operator can handle.
10707048 9-5
Annunciated Alarms per 10 Minutes
120
100
Highest 10-
minute Rate =
80 124
60
40
20
0
01/01/2011 - 01/31/2011
Figure 9-4
Southaven Average Alarm Rates
Table 9-5
Southaven Average Alarm Rates
10707048 9-6
Table 9-6
Southaven D&R Participants
All alarmed and potentially alarmable tags configured in the SCC’s Emerson DeltaV and GE
Mark V DCS systems, representing all DCS alarms coming into the SCC’s single control room,
were evaluated.
Per TVA’s Corporate Alarm Management Philosophy document, the term alarm identifies
audibly annunciated alarms which are subdivided into Priority 1 (Critical), 2 (Urgent), or 3
(Abnormal) alarms, corresponding to Emerson DeltaV’s default Critical, Warning and Advisory
alarm priorities. Status messages were allocated to Priority 7.
In all, the alarms contained within 2,071 Emerson DeltaV tags and 1,064 GE Mark V tags were
evaluated, for SCC Units 1, 2, and 3, three one-on-one combined-cycle combustion turbines.
At SCC, each of the combustion-turbine generator and steam-turbine generator pairs are
controlled by its own GE Mark V control system, which has no alarm priority capability.
For control of the heat recovery steam generators and the Balance of Plant, the Emerson DeltaV
control system is used.
Since all alarms found in the three GE Mark V control systems (one for each of the three SCC
units) are identical, it was decided that only the Unit 1 Mark V alarms would be D&R’d, as the
same data are applicable to all three units.
Also, due to GE Mark V/Cimplicity control system limitations (that is, no capability for alarm
priorities), the Master Alarm database would be created and kept for a time in the future when it
could be fully implemented (that is, a future DCS upgrade/migration).
Table 9-6 shows that the D&R evaluation effort yielded a 15% reduction of configured
annunciated alarms (Priorities 1, 2, and 3) within SCC’s DeltaV control system
10707048 9-7
Table 9-7
Southaven Delta V Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
1,224 1,337 97 2,658
Priorities
Before D&R % of
Configured 46% 50% 4% ---
Priorities
Configured
355 737 1,160 2,252
Priorities
After D&R
% of
Configured 16% 33% 51% ---
Priorities
Configured
-869 -600 +1,063 -406
Priorities
Resulting
Changes
% of Change -71% -45% +1,096% -15%
D&R evaluation effort yielded a 22% reduction of configured annunciated alarms (Priorities 1, 2,
and 3) within SCC’s GE Mark V CTG control system as shown in Table 9-7.
Table 9-8
Southaven GE Mark V CTG Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
323 0 0 323
Priorities
Before D&R
% of
Configured 100% 0% 0% ---
Priorities
Configured
90 84 79 253
Priorities
After D&R % of
Configured 36% 33% 31% ---
Priorities
Configured
-233 +84 +79 -70
Priorities
Resulting
Changes
% of Change -72% N/A N/A -22%
10707048 9-8
D&R evaluation effort yielded a 12% reduction of configured annunciated alarms (Priorities 1, 2,
and 3) within SCC’s GE Mark V STG control system as shown in Table 9-8.
Table 9-9
Southaven STG GE Mark Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
254 0 0 254
Priorities
Before D&R % of
Configured 100% 0% 0% ---
Priorities
Configured
55 90 79 224
Priorities
After D&R % of
Configured 25% 40% 35% ---
Priorities
Configured
-199 +90 +79 -30
Priorities
Resulting
Changes
% of Change -78% N/A N/A -12%
All bad actor recommendations provided along with the baseline report were discussed and the
resolutions taken on all the bad actors were noted as part of the D&R alarm database.
10707048 9-9
10707048
10
ALARM MANAGEMENT FOR MAGNOLIA
A study was done on the alarm data from April 30 through May 28, 2012, (29 days) from the
alarm journals from the Emerson Ovation system at TVA Magnolia combined-cycle power plant.
10707048 10-1
Table 10-1
Magnolia Top 20 Nuisance Alarms
Figure 10-1 shows the recorded alarm daily rates. The average recorded alarm rate for this period
was 3,757 alarms per day, with a peak of 9,618 alarms. The average annunciated alarm rate for
this period was 3,281 alarms per day, with a peak of 9,438. With this number of alarms, the
operator could not have possibly evaluated and responded to each one.
10707048 10-2
Recorded Alarms Per Day
12000 Recorded
Alarms
10000 Annunciated
Alarms
'Manageable'
8000 (300/day)
6000
4000
2000
0
April 30 - May 282012 -
Figure 10-1
Magnolia Recorded Alarms per Day
Figure 10-2 shows the annunciated daily rates and the rate that could be achieved by fixing the
top 10 most frequent alarms. The annunciated alarm rate would be reduced by approximately
36% by removing the 10 most frequent alarms. Table 10-2 provides an analysis of the daily rates.
Figure 10-2
Magnolia Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
10707048 10-3
Table 10-2
Magnolia Alarms per Day
% Of % Of
Days Days
Median
Average Maximum More More
Total Alarms
Alarms Alarms Than Than
Alarms per
per Day per Day 300 150
Day
Alarms Alarms
per Day per Day
Recorded
108,946 3,757 2,778 9,618 100% 100%
Alarms
JOURNAL 13,803 476 389 1,713 59% 86%
Annunciated - A 33,462 1,154 976 3,928 79% 97%
Annunciated - W 54,271 1,871 396 7,922 59% 66%
Annunciated - C 7,410 256 233 9,618 31% 72%
All Annunciated 95,143 3,281 2,270 9,438 97% 100%
Annunciated
Alarms Without 25,676 885 855 2,303 93% 100%
the 10 Most
Frequent
For 100% of days, the alarm rate exceeded both the EPRI Acceptable range of 150 alarms per
day and the Manageable range of 300 alarms per day.
All bad actor recommendations provided along with the baseline report were discussed, and the
resolutions taken on all the bad actors were noted as part of the D&R alarm database.
More than 95% of annunciated alarms produced by the system are during flood periods. Flood
magnitude is high. See Figure 10-3. As shown in Table 10-3, there were about 11.3 floods per
day on average.
10707048 10-4
Alarm Floods - Alarm Count
25000
327 Separate
Floods
20000
Highest Count in an
Alarm Flood =
15000
10000
5000
0
- April 30 to May 28, 2012 -
Figure 10-3
Magnolia Alarm Flood Count
Table 10-3
Magnolia Alarm Flood Count Analysis
This system spent 93.05% of the time in a flood condition. Flood duration periods, in one case
lasting almost 12 hours, present alarms at a rate higher than the operator can handle. See Figure
10-4.
10707048 10-5
Alarm Floods - Duration
80
327 Separate
Duration
Floods
70
Longest
60 Duration of
50
40
30
20
10
0
- April 30 to May 28, 2012 -
Figure 10-4
Magnolia Alarm Flood Duration
Figure 10-5 shows the average alarm rates over a 10-minute period throughout the analysis
period, and Table 10-4 shows the alarm rates in 10-minute slices.
250
200
150
100
50
0
- April 30 - May 28, 2012-
Figure 10-5
Magnolia Average Alarm Rates
10707048 10-6
Table 10-4
Magnolia Average Alarm Rates
MCC, a three-unit, one-on-one combined-cycle gas-turbine plant, has the three combustion-
turbine generators (CTGs) controlled directly by one GE Mark VI control system, while the three
steam turbine generators (STGs) are controlled directly by one Alstom control system (Units 1,
2, and 3 identify each CTG/STG pair).
Additionally, there is a single Emerson Ovation control system that oversees the GE Mark VI
CTG and Alstom STG control systems, and controls the balance of plant.
Some GE Mark VI CTG and Alstom STG alarms are transmitted to the Emerson Ovation control
system for monitoring and annunciation. The remaining GE Mark VI and Alstom alarms are not
transmitted to the Emerson Ovation control system, and annunciate as “stand-alone” alarms.
At MCC, each of the three units is controlled from a single control room, with a single operator
position.
In all, the alarms contained within 14,266 tags were evaluated for MCC Units 1, 2, and 3.
According to TVA’s Corporate Alarm Management Philosophy document, the term alarm
identifies audibly annunciated alarms, which are subdivided into Priority 1 (Critical), 2 (Urgent),
10707048 10-7
or 3 (Abnormal) alarms, corresponding to Emerson Ovation’s default Priority 1, 2, and 3 alarm
priorities. Status/journal messages were allocated to Priority 8, and diagnostic messages were
assigned to Priority 7.
As Table 10-6 shows, the D&R evaluation effort yielded a 75% reduction of configured
annunciated alarms (Priorities 1, 2, and 3) across MCC’s four Emerson Ovation control systems.
Table 10-6
Magnolia Alarm Changes
Configured
Priorities
1,117 954 1,300 3,371 (P1-P3)
After D&R
% of
Configured
Priorities 33% 28% 39% ---
Configured
Priorities
Resulting -1,417* -2,320* -4,265* -10,153
Changes
% of Change
-56%* -71%* -77%* -75%
* The priority distribution was determined by calculating the ratio of Priority 1, 2, and 3 alarms to the total
number of all currently annunciated alarms (Priorities 1 through 8 since TVA Magnolia has annunciated alarms
across all eight priority levels). Therefore, 84% of all currently annunciated alarms (Priorities 1 through 8) are
Priority 1, 2, and 3, and the remaining 16% are Priorities 4 through 8. After D&R, 100% of their annunciated alarms
will be only Priority 1, 2, or 3.
All bad actor recommendations provided along with the baseline report were discussed, and the
resolutions taken on all the bad actors were noted as part of the D&R alarm database.
10707048 10-8
11
ALARM MANAGEMENT FOR ALLEN
A study was done on the alarm data from July 23 through August 22, 2012, (31 days) from the
ABB Infi System for - Allen Fossil Plant Unit 3.
10707048 11-1
Table 11-1
Allen Top 20 Nuisance Alarms
Figure 11-1 shows the annunciated daily rates and the rate that could be achieved by fixing the
top 10 most frequent alarms. The annunciated alarm rate would be reduced by approximately by
77% removing the 10 most frequent alarms. Table 11-2 provides an analysis of the daily rates.
For 100% of days, the alarm rate exceeded both the EPRI Acceptable range of 150 alarms per
day and the Manageable range of 300 alarms per day.
10707048 11-2
9000
Annunciated Alarms Per Day
8000
Annunciated Alarms
Annunciated Alarms w/o 10 Most Frequent
7000 'Manageable' (300/day)
'Acceptable' (150/day)
6000
5000
4000
3000
2000
1000
0
- July 23, 2012 to August 22, 2012 -
Figure 11-1
Allen Annunciated Alarms per Day with and without the 10 Most Frequent Alarms
Table 11-2
Allen Alarms per Day
% Of % Of
Days Days
Median
Average Maximum More More
Total Alarms
Alarms Alarms Than Than
Alarms per
per Day per Day 300 150
Day
Alarms Alarms
per Day per Day
All Annunciated 91,166 2,941 2,629 8,270 100% 100%
Annunciated
Alarms without 21,127 682 589 1,850 100% 100%
the 10 Most
Frequent
More than 95% of annunciated alarms produced by the system are during flood periods. Flood
magnitude is high. There were about 15 floods per day on average. See Figure 11-2 and Table
11-3.
10707048 11-3
Alarm Floods - Alarm Count
Alarm Floods - Alarm Count
473 Separate
Floods
7000
1000
Peaks Above Highest
Count in an
Alarm Flood
6000 = 7,039
800 Longest
Duration of
5000 Flood = 16.58
179 Separate Hours
Floods
600
4000
Highest Count in
an Alarm Flood
3000
400
2000
200
1000
00
01/01/2011
- July 23 - 01/31/2011
to August 22, 2012-
Figure 11-2
Allen Alarm Flood Count
Table 11-3
Allen Alarm Flood Count Analysis
This system spent 64.1% of the time in a flood condition. Flood duration periods, in one case
lasted almost 17 hours, present alarms at a rate higher than the operator can handle. See Figure
11-3.
10707048 11-4
Alarm
AlarmFloods
Floods- Duration
- Duration
Duration in
Hours 473 Separate
Floods
14
18
Longest
Duration Duration of
16 Flood = 16.58
12 Hours
14
10
12 179 Separate Floods
8
10 Longest Duration of Flood =
68
6
4
4
22
00
01/01/2011
- July - 01/31/201
23 to August 22, 2012-
Figure 11-3
Allen Alarm Flood Duration
Figure 11-4 shows the average alarm rates over a 10-minute period throughout the analysis
period, and Table 11-4 shows the alarm rates in 10-minute slices.
350
Highest 10-
minute Rate
300 = 310
Alarm Flood
250 = 10+ in 10
minutes
200
150
100
50
0
- July 23, 2012 to August 22, 2012 -
Figure 11-4
Allen Average Alarm Rates
10707048 11-5
Table 11-4
Allen Average Alarm Rates
All alarmed and potentially alarmable tags configured in ALF’s four ABB DCS systems,
representing all DCS alarms coming into ALF’s four control rooms, were evaluated.
At ALF, each of the three coal-fired units (Units 1, 2 and 3), as well as the coal yard, is
controlled by its own ABB control system, with a single operator per system.
In all, the alarms contained within 12,386 ABB tags were evaluated for ALF Units 1, 2, 3 and the
coal yard.
According to TVA’s Corporate Alarm Management Philosophy document, the term alarm
identifies audibly annunciated alarms, which are subdivided into Priority 1 (Critical), 2 (Urgent),
or 3 (Abnormal) alarms, corresponding to ABB’s default Priority 1, 2, and 3 alarm priorities.
Status/journal messages were allocated to Priority 8, and non-annunciated human-machine
interface (HMI) graphics drivers and operator graphics inputs were assigned to Priority 16.
By combining Tables 11-6 through 11-9, the D&R evaluation effort is shown to have yielded a
67% reduction of configured annunciated alarms (Priorities 1, 2, and 3) across ALF’s four ABB
control systems.
10707048 11-6
Table 11-6
Allen Unit 1 ABB Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
1,249 1,085 959 3,293
Priorities
Before D&R % of
Configured 38% 33% 29% ---
Priorities
Configured
165 226 817 1208
Priorities
After D&R
% of
Configured 14% 19% 68% ---
Priorities
Configured
-1,084 -859 -142 -2,085
Resulting Priorities
Changes
% of Change -87% -79% -15% -63%
Table 11-7
Allen Unit 2 ABB Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
2,120 1,127 1,023 4,270
Priorities
Before D&R % of
Configured 50% 26% 24% ---
Priorities
Configured
172 224 805 1,201
Priorities
After D&R
% of
Configured 14% 19% 67% ---
Priorities
Configured
-1,948 -903 -218 -3,069
Priorities
Resulting
Changes
% of Change -92% -80% -21% -72%
10707048 11-7
Table 11-8
Allen Unit 3 ABB Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
2,484 1,134 968 4,586
Priorities
Before D&R % of
Configured 54% 25% 21% ---
Priorities
Configured
181 230 864 1,275
Priorities
After D&R
% of
Configured 14% 18% 68% ---
Priorities
Configured
-2,303 -904 -104 -3,311
Priorities
Resulting
Changes
% of Change -93% -80% -11% -72%
Table 11-9
Allen Coal Yard ABB Annunciated Alarm Changes
Total
Priority 1 Priority 2 Priority 3
Description Configured
(Critical) (Warning) (Advisory)
Priorities
Configured
204 277 46 527
Priorities
Before D&R % of
Configured 39% 52% 9% ---
Priorities
Configured
59 100 354 513
Priorities
After D&R
% of
Configured 12% 19% 69% ---
Priorities
Configured
-145 -177 +308 -14
Priorities
Resulting
Changes
% of Change -71% -64% +670% -3%
All bad actor recommendations provided along with the baseline report were discussed, and the
resolutions taken on all the bad actors were noted as part of the D&R alarm database.
10707048 11-8
12
PATH FORWARD FOR FURTHER IMPROVEMENTS
Alarm management is a continuous improvement effort. Changing conditions within the plant
can lead to the appearance of nuisance alarms. Frequent reporting is often needed to identify and
resolve these nuisance alarms. Alarm analysis software can be an invaluable tool to generate
automatic reports delivered to the persons responsible for the alarm system performance. A
review of alarm system performance should be included in any plant performance review
meeting.
For TVA, Steps 1 through 4 have been performed. Continuous monitoring of the annunciated
priority distribution should be done to ensure that the operator is presented with a distribution
similar to best practices.
Audit and Enforce is Step 5 of the Alarm Management Improvement Process found in the EPRI
Alarm Management and Annunciator Applications Guidelines (EPRI report 1014316) and briefly
discussed in the introduction of this document.
In order to achieve another step change in alarm system performance, TVA plants will have to
progress through Step 6, Real Time Alarm Management, of the Alarm Management
Improvement Process. Specifically, the use of the state-based alarming and alarm flood
suppression methodologies will address the issue of nuisance alarms on startup and shutdown as
well as alarm flooding in the TVA alarm system.
Another important part of Step 6 is alarm shelving. As equipment ages and sensors go in and out
of calibration, it becomes necessary to “turn off” alarming on tags that need to undergo
maintenance. Turning off an alarm requires great scrutiny and care. Each alarm has an identified
response, and turning off the alarm effectively delays or prevents that response from occurring.
Alarm shelving is a robust methodology to turn off an alarm with the appropriate controls to
ensure that it is not forgotten. Use of the methodologies in Step 6 will ensure that all alarms
presented to the operator are relevant and timely.
10707048 12-1
10707048
13
SUMMARY
An alarm philosophy workshop was conducted in order to obtain information for the
development of a philosophy. The Alarm Philosophy document was developed to provide the
guiding design of any alarm management process and to document the Alarm Management
Improvement Process, measureable goals, and targets for the alarm system.
Based on the alarm and event history provided, PAS analyzed and benchmarked the alarm
system for Tennessee Valley Authority plants in accordance with the EPRI Guidelines. The point
configuration for all of the systems was imported into PAS’s PSS software, which created a
database of all alarms and their settings. During D&R, every alarm setting was reviewed with
experienced plant personnel, using the philosophy as a guideline.
The summary by plant is shown in Table 15-1.
Table 13-1
Alarm Summary by Plant
10707048 13-1
10707048
A
ALARM ANALYSIS DESCRIPTIONS
Alarm System Performance Analysis
The result of the setting up of alarms (configuration) is the creation of alarm events. These
events are just referred to as alarms in System Performance Analysis. The actual alarms
delivered by the control system to the operator over a certain period of time are analyzed.
Important knowledge around the system and the operator interaction is obtained. Nuisance
alarms are identified.
Alarm Floods
Alarm floods are defined as periods of alarm activity with presentation rates higher than the
operator can respond. Alarm floods can make a difficult process situation much worse. In a
severe flood, the alarm system becomes a nuisance, a hindrance, or a distraction, rather than a
useful tool.
For calculations, an alarm flood event is defined as beginning when the alarm rate exceeds 20 or
more alarms occurring in 10 minutes, and ending when the rate drops below 5 alarms in 10
minutes. Only Annunciated alarms are considered. The analysis examines Flood Events for
number, magnitude (severity), and duration. The overall percentage of time the system spends in
flood is calculated.
10707048 A-1
The most frequently occurring alarms include the Bad Actors and Nuisance Alarm.
Bad actors are alarms whose states are suspect and cannot be relied upon for delivering accurate
information to the operator. Nuisance Alarms may or may not be delivering accurate information
but for other reasons (generally alarm behavior) constitute a nuisance and distraction for the
operator. Potentially hazardous consequences may result when operators distrust the validity of
the alarms and do not respond to them in a timely fashion.
Substantial performance improvement can be made by addressing these alarms.
A common response to a nuisance alarm is to disable the tag, rather than to specifically analyze
the behavior and perform a solution. Such disabling is often done without proper Management-
of-Change (MOC) and without proper notification to affected personnel. This often results in
thousands of non-annunciated alarms being generated and recorded. Such disabled alarms are
often forgotten and never addressed.
Chattering Alarms
Chattering alarms are nuisance alarms that transition into and out of alarm in a short amount of
time. This results in a significant distraction for the operators, drawing their attention away from
the normal process control tasks. Chattering alarms rapidly fill the Alarm Historian and make
data analysis difficult. Chattering alarms may be the result of instrument problems in the field,
poor control, or improperly specified deadband or delay times.
For this analysis, a criterion of three alarms per minute is used as the definition of a chattering
alarm. Recorded and Annunciated chattering alarms are analyzed.
Stale Alarms
Stale alarms are in the alarm state continuously for more than 24 hours. Following their initial
appearance, stale alarms provide no valuable information to the operators. They clutter the alarm
displays and interfere with the operator’s ability to detect and respond to new and meaningful
alarms. Stale alarms are candidates for state-based alarming solutions.
Note: Some DCSs may not capture all RETURN-TO-NORMAL events and the Stale Alarm
analysis is therefore not 100% accurate. This analysis does well identifying long duration alarms
even if the exact count or duration is slightly off.
10707048 A-2
Alarms by Type
Alarm events are separated into various types and ranked for frequency, such as Instrument
Diagnostic alarms, High Process Value alarms, Digital alarms, and so forth. Insight into the
health of the control system is provided, as well as abnormalities compared to typical industry
values.
Duplicate Alarms
Duplicate alarms are alarms that persistently occur within a short time period of other alarms. In
this report, alarms are considered duplicate or redundant when they consistently occur within one
second of each other. A high quantity of potential duplicates shows the need for rationalization to
eliminate them. While these numbers must be reduced to allow the operator to identify the root
cause of abnormal situations, the first step should be alarm improvement of the other identified
bad actor categories. Improving these tags may not eliminate all the duplicates, but should reduce
this category to a more manageable level.
Consequential Alarms
Consequential alarms are a subset of most frequently occurring alarms. They are source alarms
around which other alarms are occurring within a specific time. A very simple example would be
an alarm on a pump operating state being “OFF”, along with a low flow alarm on the discharge.
(If the response is fast enough – one second - this might also show up as a duplicate alarm.)
Consequential alarms are often multiple alarms from the same event, essentially telling the
operator the same thing in different ways. The analysis of consequential alarms is a fundamental
step in developing dynamic alarm management systems.
The result of this analysis depends on the specifications given. In this analysis, source alarms
with less than 2 occurrences during the analysis period are excluded. With a 75% occurring
probability around the source alarm, only alarms occurring 15 minutes before or after the source
alarm are considered.
The rationalization of consequential alarms can result in substantial performance improvement.
PlantStateSuite identifies the source alarm and before/after alarms for analysis.
Alarm Settings
Alarm Settings constitute the configuration of a tag and its alarms. The alarm algorithm, alarm
trip points, priority, and deadband are examples of alarm settings.
10707048 A-3
Prioritization of alarms provides a mechanism for placing a qualitative value on the importance
of the alarm. The priority of an alarm determines its significance and how quickly the operator
should respond to an alarm. The EPRI Alarm Management and Annunciator Application
Guidelines (document# 1014316) provides guidelines for priority distribution, namely, 80%
Priority 3 - Low, 15% Priority 2 - Medium, and 5% Priority 1 - High. EPRI addresses only
priorities that are seen by the operator, not those used to only record information without
annunciation to the operator.
10707048 A-4
B
ALARM SYSTEM CLASSIFICATION LEVELS
Overloaded
A continuously high rate of alarms, with rapid performance deterioration during process upsets.
Typically characterized by the following:
• Alarm system is difficult to use during normal operation and in practice ignored during plant
upset as it becomes unusable
• Low operator confidence in the alarm system, which is often ignored for long periods
• Important alarms are difficult or impossible to discriminate from less important ones, and the
alarm system gives little or no advance warning of plant upsets
• Many alarms are meaningless or of little value
Alarms are often disabled by the operator because they represent a nuisance, and are frequently
then forgotten (i.e. never re-enabled).
Reactive
Some improvement compared to Overloaded, but the peak rate during upset is still
unmanageable. The alarm system is still an unhelpful distraction to the operator for much of the
time. Typically characterized by the following:
• Alarm system is more stable and useful during normal operation, but is often unusable in
practice during plant upsets
• The operator reacts more to the rate of alarm generation rather than to the detail of the alarms
themselves
• Alarm prioritization known to be unreliable, but of some use
• The alarm system gives some early warning of plant upsets
• Some alarms are still meaningless or of little value contributing to overall noise level
Alarms are often disabled by the operator because they represent a nuisance, and are sometimes
then forgotten about.
Stable
A system well defined for normal operation, but less useful during plant upsets. Compared to
Reactive, there were improvements in both the average alarm and peak alarm rates. ‘Bad Actors’
are resolved and under systematic control. Problems remain with the burst alarm rate. The
system functions well for normal operations but are less useful during plant upsets. Typically
characterized by the following:
• Alarm system is reliable during normal operation, providing early warning of impending
plant upset, but is less useful during plant upset
10707048 B-1
• Operators are confident in the appropriateness of the alarm prioritization, and react
consistently and quickly based on priority
All alarms are meaningful and have a defined response.
Robust
Average and peak alarm rates are under control for foreseeable plant operating scenarios.
Dynamic and state based techniques are used to improve the real time performance. Typically
characterized by the following:
• Alarm system is reliable during all plant modes, including normal operation and plant upsets
• Operators have a high degree of confidence in the alarm system, and have time to read and
understand all alarms.
Predictive
Breakthrough performance on both the average and the peak alarm rate and fully encapsulates all
of the aspirations of the guidelines contained in EPRI Alarm Management and Annunciator
Applications Guidelines. Typically characterized by the following:
• The alarm system is stable at all times and provides the operator with the right information at
the right time – in order to avoid process upset or minimize the impact of any upset that does
occur
• The operator actively ‘patrols’ the process schematics and corrects deviations before they are
significant enough to cause an alarm.
10707048 B-2
C
TVA ALARM PHILOSOPHY
TVA Corporate Alarm Management Philosophy was used as the guideline for Alarm
Management for TVA. The philosophy was updated based on the lessons learned and
improvements during the Documentation and Rationalization for each plant. The Corporate
Alarm Management, Revision 5, is below.
10707048 C-1
FPG-SPP-10.xxx
Alarm Management Standard Rev. 0000
Page 1 of 32
FPG Standard
Programs and
Processes
Reviewed by:
Peer Team Chair Date
Approved by:
Peer Team Sponsor Date
Approved by:
Corporate Functional Area Manager Date
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 2 of 32
Revision Log
Revision or Affected
Change Effective Page
Number Date Numbers Description of Revision/Change
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 3 of 32
Table of Contents
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 5 of 32
1.0 PURPOSE
2.0 SCOPE
This procedure is to be used at any Fossil and Combustion turbine plant to provide a
consistent standard approach to alarm management.
3.0 PROCESS
The Alarm System Champion role will be fulfilled by the site system engineer or the site's
engineering manager's designee.
Responsible for maintaining the integrity of the alarm system, analyzing and reporting alarm
system performance and ensuring that corrective action is taken in accordance with this
procedure. Other responsibilities include oversee software and work processes, manage
alarm system improvement efforts (issue improvement work orders) and participate in the
Documentation and Rationalization process.
The CCSO should be responsible for the content of the alarm system and its beneficial use.
The CCSO:
The FED:
• Reviews and approves changes to the design basis of critical and urgent priority
alarms
• Ensures proper documentation and archiving through Configuration Control
Management procedures
• Develop and maintain alarm management design guides and standards for fleet-
wide use
• Participate in all acceptance testing and review and approve vendor system
deliverables pertaining to alarm management prior to start-up to ensure compliance
with alarm system configuration standards and proper functionality
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 7 of 32
The Operator:
A. Alarms must be properly chosen and implemented. Items that must be considered to
build an effective alarm system are:
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 8 of 32
1. Priority 1 alarm: Any annunciated alarm that results in a potential adverse effect
on the environment, causes >2% derate, initiates runback/rundown, unit trip,
immediately damages the asset, or creates unsafe conditions requiring a manual
unit or equipment trip initiated by the operator. A priority 1 alarm shall prompt the
UO/SOS to take immediate action to stabilize conditions. UO will notify SOS
immediately. A Service Request (SR) may be initiated.
B. The alarm system must be reserved for events that require operator action. Only such
events shall be configured as alarms. An alarm is not an information only device and
so should not reflect normal cycling of valves or equipment, out of service status not
requiring action, or normal variations in pressure or temperature or other process
parameters.
C. Alarms shall only be produced upon abnormal situations. Normal operation such as
startup and shutdown of equipment and systems is normal operation and should not
produce alarms.
D. Alarms should be placed, configured, and handled so that a single process event does
not produce multiple alarms.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 9 of 32
Overloaded
A continuously high rate of alarms, with rapid performance deterioration
during process upsets.
Reactive
Some improvement compared to Overloaded, but the peak rate during
upset is still unmanageable. The alarm system is still an unhelpful
distraction to the operator for much of the time.
Stable
A system well defined for normal operation, but less useful during plant
upsets. Compared to Reactive, there are improvements in both average
alarm and peak alarm rates. “Bad Actors” are resolved and under
systematic control.
Robust
Average and peak alarm rates are under control for the foreseeable
plant operating scenarios. Dynamic and state-based techniques are
used to improve the real time performance.
Predictive
Breakthrough performance on both the average and the peak alarm rate
and fully encapsulates all of the aspirations of the guidelines contained
in Engineering Equipment and Materials Users Association (EEMUA)
PUBLICATION 191.
Measurement is fundamental to control and improvement. The following are the goals and
KPI’s for FPG assets Alarm Systems performance. Values shown are for the span of control
and alarm authority of a single operating position.
Annunciated Alarms Per Day per Operator Position ~150 alarms per day ~300 alarms per day
Annunciated Alarms Per Hour per Operator Position ~6 (average) ~12 (average)
~1 (average) ~2 (average)
Annunciated Alarms Per 10 Minutes per Operator
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 10 of 32
Position
Percentage contribution of the top 10 most frequent ~<1% to 5% maximum, with action plans to
alarms to the overall alarm load address deficiencies.
Quantity of chattering and fleeting alarms Zero, action plans to correct any that occur.
~80% LOW
Configured Priority Distribution ~15% HIGH
~5% EMERGENCY
A. Performance against these listed KPIs, with their respective interim and long term goals
clearly defined on each graph, shall be reported every month by the site’s Alarm System
Champion. The report will also include:
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 11 of 32
Average Alarms/Day √ √ √ √
Frequent Alarms √ √ √ √
Chattering Alarms √ √ √ √
Frequent BADPV √ √ √ √
Stale Alarms √ √ √ √
A. The control system operator interface system shall be designed to minimize the
number of keystrokes required to identify, verify, and assess an alarm. All alarms
should be acknowledged only once.
B. Every configured process alarm should have an associated graphic display on the
control system. This associated display should aid the operator in the proper diagnosis
and mitigation of the event that caused the alarm and should show the alarm in context
of other appropriate system values.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 12 of 32
A. For all new designs monitored in the control system, no external light box or Pan alarm
hardware should be required for the annunciation of alarms and requires design
engineering approval.
B. For any existing installations that use external annunciators, the alarms on the external
device should meet the same configuration criteria for control system alarms priority.
The external alarms should be placed in a consistent, logical spatial pattern, be well
labeled, and integrated with the control system.
A. FPG assets will utilize a maximum of three levels of control system annunciated alarm
priorities and a minimum of one status priority for the alarm system. These levels shall
be consistent throughout the sites to ensure maximum ergonomic effectiveness and
operator understanding.
B. Industrial studies and best practices recommend the following breakdown of priorities:
1 (Critical) 3 – 7%
2 (Urgent) 12 – 18%
3 (Abnormal) 75 – 85%
C. These numbers are guidelines only, and it is understood that the results for individual
units may vary from these.
D. The color convention for the three annunciated alarm priorities shall be as follows:
1 Red
2 Yellow
3 Cyan
E. Color-coding standards are mandatory across the fleet for critical and urgent priority
alarms. For abnormal priority alarms, existing control systems implementations that do
not adhere this convention are not required to modify the configuration until an HMI
upgrade or control system modification is performed as long as the existing convention
does not conflict with the conventions for the critical and urgent alarms.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 13 of 32
G. A special alarm priority is the status priority. The status priority makes it possible to
assign alarms to a separate priority in which the actuation of the alarm does not
annunciate to the operator. Instead, the event only produces a time-stamped indication
that is recorded in the alarm event journal for archiving purposes. Journal alarms are
not truly alarms and they should not appear in the Alarm Summary.
H. Non-annunciated alarms are useful for determining the sequence of events in a post-event
analysis. They can also be used in verifying the proper activation of certain elements of trips,
runbacks, rundowns, and external system activations and alarms.
I. As alarm system and DCS graphic design are closely related, the existing DCS graphic
design shall be reviewed and updated and an alarm system design procedure shall be
developed to determine future alarm system standards. These two procedures shall
determine consistent alarm priority naming and color usage conventions to be used in
both DCS graphic design and alarm system design in future implementations and
upgrades of these systems.
A. Most DCS systems utilize more than three alarm priorities for example Emersion
Ovation DCS system utilize 8 alarm priorities. In addition to the annunciated priorities
as noted above the table below lists other possible uses for the non-annunciated
priorities.
1. Separate and distinct visual and audible indications should be provided for each
alarm priority.
2. The alarm indication color and priority standards shall be consistent on each
control system.
3. A process schematic must visually and consistently identify process alarms,
alarm acknowledge status and the priority of the alarm.
4. Color and shape must be used in a consistent way to identify the priority of
alarms on a schematic.
K. Existing graphics that do not follow the above best practices need not be altered
except through a comprehensive HMI update. Due to the significant change effort that
would be required to modify existing graphics to comply with new standards, the
present graphics on existing TVA assets are not required to undergo any major
changes at the present time. Any new graphics needed in existing facilities will be
examined and designed on a case-by-case basis. However, all graphics related to
new facilities will be designed according to this standard.
A. Nuisance alarms must be identified and properly addressed to ensure optimal system
performance, while meeting all change management and communication requirements
with operators. Chattering, frequent, fleeting, stale, out-of-service and other nuisance
alarms must be analyzed and an engineering solution or repair applied. They must not
be ignored or indefinitely suppressed.
B. Alarms that occur repeatedly over a short period of time (i.e. three times per minute)
are considered chattering alarms.
B. Alarm suppression with proper control is called Alarm Shelving. Regardless of whether
manual inhibited alarm-tracking procedures or automated solutions are implemented,
alarm suppression must be performed in a way that meets the following rigorous
requirements.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 15 of 32
C. Alarm inhibiting and shelving cannot be indefinite. The duration an alarm has been
shelved must be shown or discernable from any manual process. At periodic intervals,
the shelving system must present the alarms for reactivation upon confirmation by the
operator and manually inhibited alarms must be reviewed. It is essential that operators
must know, each shift, which alarms have been removed from service and for how
long. Shelving solutions must supply this information and operators must be required
to query the system upon shift change and prior to startup of equipment. It is not
acceptable that a shelving solution produce an alarm flood by automatically
reactivating alarms without the operator’s knowledge that it is about to occur.
D. Shelving solutions must be controllable by priority, with the ability to set time limits or
require certain approvals. Tennessee Valley Authority assets have these specific
requirements:
A. Most alarms in a process unit pertain to the normal operating state of a piece of
equipment. But, equipment often has several normal, but differing, operating states.
Control system alarm capabilities are normally only for single-state, single-value set
points and priorities. State examples include Startup, Shutdown, Half Rate Operation,
Maintenance State, etc.
B. State detection for state-based alarming uses available process information (which can
include operator input if desired) to correctly identify the current operating state of the
equipment. When the state changes, the system changes the alarm settings to
predetermined values appropriate for the new start. These tasks may be automated.
C. If multiple process states producing differing alarms are identified, these must be
documented during the alarm rationalization. State transitions requiring alarm system
modifications should be handled by one of the following methods:
A. Periods of alarm activity with annunciation rates greater than the operator can
effectively handle are defined as alarm floods. Flood Suppression is the dynamic
management of pre-defined groups of alarms based on detection of equipment state
and triggering events.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 16 of 32
A. Operator change of the alarm limits is a practice to be avoided and the system should
be configured to disallow this practice.
B. Therefore, the operator may need a configurable set of tools with which to meet such
normal operating needs. This tool set is called an Operator Alert System, which is
separate from the alarm system. Alerts have the following characteristics:
• To reduce, on an existing system, the number of configured alarms and thus the alarm
load created from them
• To correct a miss-configured system for performance improvement
• To ensure consistency in alarm settings
• To eliminate duplicate alarms
• To ensure proper and meaningful alarm limits and priority settings
• To configure alarms on points added or modified by projects in conjunction with PHA or
SIL revalidation if alarms are specified
• To verify proper configuration of nuisance alarms as they are identified
• To create the Master Alarm Database, used as a reference for State-Based alarm
management, Flood suppression, and Audit/Enforce mechanisms
During an alarm rationalization exercise, all alarm able control system points shall be
rationalized, along with any other systems that provide alarm or abnormal situation
notification to the board operator. The impact, severity, and response time matrices defined
in this section should be used to rationalize each alarm and will be documented in the
results. Background information on the matrix components (impact assumptions, severity,
etc.) should also be provided in the documentation for future reference. Any deviation from
the alarm priority as defined in the rationalization matrices must be identified during the
course of the rationalization and documented.
• Instrument Mechanic
Other individuals with knowledge of the process unit, its operation and specific equipment,
its advanced control schemes, unit hazards, and the alarm philosophy, will be needed
periodically. The entire team must understand the alarm philosophy before starting the
rationalization.
• P&IDs
• Operating procedures
• Control system configuration data
• Results from risk assessment component assessment reviews
• Control logic diagrams
• Control system graphics
A. For each alarm to be rationalized, the potential consequences, without any operator
intervention, must be identified Tennessee Valley Authority assets will use two criteria
as input to the final table that determines alarm priority:
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 18 of 32
D. Use of this matrix requires that for each impact category, a consequence category (if
any) of an operator failure to take action will be selected. Tennessee Valley Authority
assets will use the worst case severity method to determine the overall event impact.
This impact is then classified as MINOR, MAJOR or SEVERE.
A. Maximum time to respond is the time within which the operators can take action(s) to
prevent or mitigate the undesired consequence(s) caused by an abnormal condition.
This response time must include the action of outside personnel following direction
from the console operator.
B. To clarify, this is not how long it actually takes the operator to take the action. It is how
much time is available to take effective action from when the alarm sounds to when the
consequence is unavoidable.
C. For each alarm being rationalized, the maximum time allowable to respond will be
identified.
< 3 minutes
(Immediate Response)
3 to 10 minutes
(Quick Response)
10 to 30 minutes
(Delayed Response)
> 30 Minutes
(Determined Response)
NOTE
Note that a maximum time allowable to respond of greater than 30 minutes does not meet the
criteria for an alarm. This is not an absolute principle; there will be exceptions. The need for the
alarm system to retain a sense of urgency allows for such exceptions.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 20 of 32
Determining the most appropriate priority for an alarm requires consideration of both severity
of consequences and the time within which the operator can effectively correct the alarm. By
combining the severity factor and the response time, the systematic approach for setting
alarm priorities is defined. The following matrix provides the guideline for determining the
priority of an alarm.
> 30 Minutes
No Alarm No Alarm No Alarm No Alarm
Determined Response
10 to 30 minutes Delayed P3 P3 P2
No Alarm
Response Abnormal Abnormal Urgent
3 to 10 minutes P3 P2 P1
No Alarm
Quick Response Abnormal Urgent Critical
< 3 minutes P2 P1 P1
No Alarm
Immediate Response Urgent Critical Critical
A. All rationalized process alarms within an operating unit should be documented. The
documentation should include all information required to define the alarm, its purpose,
and the data required for rationalization. For new projects and incremental changes to
the unit, full alarm justification and documentation should be provided as part of the
project scope, accompanying any other required project documentation.
B. For ease of access and maintainability, the alarm system documentation should be
maintained through a uniform electronic database system across all TVA Fossil sites.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 21 of 32
D. Operations should have on-demand access to the above documentation of the alarm
system, preferably electronically, in the form of a Master Alarm Database. The Master
Alarm Database has several other important uses, particularly for alarm auditing and
settings enforcement.
A. Alarm limits allow alarms to actuate when there is a need to alert the operator to
conditions or events. When the need for an alarm has been identified, the selection of
set points and values must then be provided. Alarm limits should be selected to provide
adequate response time to plant operations. Care must also be taken to ensure alarms
on analog signals (analog alarms) do not duplicate independent digital alarms provided
by control system-based trips, runback or rundown logic or other interfaces, or vice
versa.
C. For processes with different operating states, several alarm values may be required.
All alarm limits and their corresponding process state must be documented.
D. To minimize chattering alarms, which activate repeatedly over a short period of time,
appropriate dead bands must be selected for all alarms. This may involve the
programming of a dead band for analog setpoint values, and a delay time for digital
points. Determination by historical performance is recommended. Best practice
starting points for design are:
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 22 of 32
** ON-Delays should be used with much care on Urgent and Critical priority alarms. OFF-
Delays of more than two minutes should be evaluated for all alarms for suitability.
Up-front decisions around alarm configuration will reduce time spent in rationalization.
Decisions can usually be made in advance around several topics:
P1 - Critical Priority
A. Automated shutdown systems are used to return a process to a safe state if the control
system becomes ineffective. For modern plants with such systems that are properly
designed, these alarms are monitored by the DCS.
B. For Tennessee Valley Authority, these cases are as follows, and these alarms shall be
assigned P1- Critical Priority:
The Flammable and Toxic Gas detectors sound an alarm and this alarm is monitored
by the DCS. The operator must ensure that all personnel leave the affected area until
the situation is verified and controlled. The alarms shall be displayed on a graphic that
indicates their geographic location. Wind direction and velocity is a recommended
element on this graphic.
When a safety shower / eyebath actuates, it may be due to a person that has become
incapacitated. The operator response is to send immediate assistance. The alarms
shall also be displayed on a graphic that indicates their geographic location.
B. Following instrument repairs, operators must follow-through and re-activate tags if they
were inactivated, or unshelve them if they were shelved.
1. By default, all Analog-In and Analog-out points will have Bad quality alarms
2. Bad quality alarms will take on the highest priority of other alarms on that tag
3. If no other priority exists on the tag, Bad quality alarm will have P3 - Abnormal
Priority
4. Where appropriate, group related Bad quality alarms into a common alarm, then
provide a display that shows the particular sensor involved
5. If an indicator point is an input to a controller point, alarm the Bad quality on the
controller point, not the indicator point (since that is where action will be taken)
6. Bad quality alarms often propagate through several points. These must be revised
so that a Bad quality event produces only one, not multiple, Bad quality alarms.
There are several techniques to accomplish this based on the point type.
3.7.4 35B Alarms for Redundant Sensors and Voting Systems (continued)
3. Alarms indicating the disagreement of multiple sensors should utilize logic and
settings such that only significant and sustained disagreement generates the
alarm.
B. The case-by-case review of these redundant installations may require further study
outside of the normal alarm system documentation and rationalization (D & R) process.
Safety considerations inherent in these redundant installations may necessitate
dynamic alarm changes in the logic equipment, instead of in the control system.
A. External systems such as analyzers, equipment cabinets, and PLCs are often
connected to the control system directly or via serial, Modbus, or similar methods. It is
common for these systems to have multiple health status indicators. Often these are
all individually alarmed, which is not a best practice.
C. The individual status points feeding the common point shall be configured with Journal
priority (if it is desired to record their individual time of activation).
A. Duplicate alarms, where several alarms on different process parameters indicate the
same abnormal situation, should be removed. In most cases, the Documentation and
Rationalization team shall select the best indicator of the root cause and place the
alarm on that device.
A. Often a consequential alarm can be handled by the same methods as duplicate alarms
and voting alarms or incorporated into a state-based alarming strategy.
A. Combination Alarms are where PV HIGH or PV LOW alarms are configured with, and
often followed immediately by, the next alarm (PV High-High or PV Low-Low)
1. The operator actions for the pre-alarm vs. the next alarm must be significantly
different in kind or in degree (In other words, do not alarm twice for the operator to
do the same thing)
2. There must be enough time after the first alarm to perform effective corrective
action before the process activates the next alarm
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 25 of 32
C. Deviation alarms are used to notify the operator that the process variable has moved
away from the controller setpoint beyond a certain amount, usually as a percentage of
total range. It is much more common to set alarms at a particular activation point
expressed in engineering units, such as “230 degrees C”, since the bad consequence
to be averted almost always occurs at a particular value.
A. These alarms occur when the process value changes faster than a configured
maximum rate. Use this type of alarm sparingly since it can easily generate unwanted
alarms during transitions.
A. A digital point has two configured states. One of those states is designated as Normal,
the other as Off-Normal. It is never appropriate to generate an alarm just because, for
example, a piece of equipment is turned Off.
B. While this technique is normally inconsistent with the alarm management concept that
operators will respond to every alarm, certain situations justify the practice of re-
annunciation. Some examples of this are turbine bearing temperature, pressure
vessels, turbine steam-driven speed pumps, turbine water induction and large-
component vibration measurement alarms. Each alarm will be reviewed for necessity
during D&R sessions.
A. In general, a program implemented in a control system does a task that the operator
relies on. Failure of that program means the operator must act in a different way – very
similar to the treatment of an external device health alarm. Programs must be
accompanied by documentation for the operator regarding the action to take based on
the specific alarm.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 26 of 32
A. Alarms specific to the internal workings of a control system should be absent under
normal operating conditions, and they should not be tolerated when they occur.
B. All alarms related to Drop alarms and Ethernet switches will be assigned a priority 2 at
a minimum.
C. All alarms related to Module and Node signal types will be assigned the diagnostic
priority 4.
B. Any elimination or alteration of input alarms therefore would alter the performance of
the logic action – or even possibly eliminate it, without anyone knowing about it.
C. The preferred practice is to configure such logic blocks with the Process Value (PV) as
an input, and compare it to a numeric contained in the logic block. This is better
because even though the numeric could be changed, logic blocks are more obscure
and people are not nearly as likely to alter them.
A. Some control systems incorporate an Operator Message system that operates similarly
to the Alarm system, where messages produce a sound and must be acknowledged.
Since such systems provide an equal demand on an operator’s attention as an alarm,
their configuration and actuation must follow the same principles as the alarm system.
B. Use of the messaging system other than status annunciation or confirmation prompting
is beyond the original intent and should be avoided. There are other ways to announce
status to the operator (e.g., graphic elements that display sequence status without
generating messages). Only status changes that require operator confirmation before
proceeding should use messages (e.g., actions, such as entering needed data).
C. Some control system vendors allow the use of Journal messages whereby the
messages are recorded in the message log but are not shown in the message
summary or annunciated to the operator. Journal messages should be used for the
same purposes in which Journal priority alarms are used.
To maintain the integrity of the alarm system, Configuration Control Management through
existing procedures must be in effect that address changes to alarm systems. Such
changes must be properly evaluated, authorized and communicated to all affected
personnel and shifts.
Configuration Control Management procedures and TVA standards define the minimum
level to invoke appropriate approvals and documentation.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 27 of 32
The proper settings reside in a Master Alarm Database. The Configuration Control
Management procedures must ensure timely update of that database so that proper
changes do not get undone by enforcement.
Audit and enforcement software may be used to periodically check for changes from the
proper settings, to report such changes, and to restore the system to the proper settings.
Audit and enforcement software must understand any state-based, flood suppression,
shelving, or other alarm handling strategies being employed and work correctly in
conjunction with them. The audit report shall be delivered weekly to the control system
owner who will act on variances.
To emphasize, best practices support that the integrity of the overall alarm system is of such
importance as to require Configuration Control Management procedures around all alarm
priorities, including abnormal. This is the reason a separate operator alert system is a best
practice.
3.9 Training
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 28 of 32
Each asset should provide Essentials of Alarm Management Training to control room
operators and control and instrumentation engineers. The training should cover several
areas and include specific points:
The ability to reach maximum potential for handling alarms can only be achieved by
continually monitoring, tuning and enhancing the existing alarm strategies. A workflow
process must be in place to allow the alarm system champion to continuously improve the
alarm system. The figure illustrates the interdependence of the various components of the
alarm management maintenance workflow process:
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 29 of 32
DEFINE
Define the Problem, Objective, and Goals of the Alarm Management effort. Decide on the
desired performance level (Overloaded, Reactive, Stable, Robust, Predictive) Develop the
Project Plan and Assign Resources
MEASURE
Develop the Data Collection and Sampling Plan, in general the proper use of Alarm Analysis
software and the proper determination of Key Performance Indicators.
ANALYZE
Analyze Key Performance Indicators. A statistical analysis of alarm events must be used to
identify trends, patterns, and bad actor alarms on the system. Initial Baseline analyses of a
system are highly valuable. Publish periodic reports. Upon identifying the specific
problems, perform root cause analyses, develop alternative solutions, and decide upon the
proper solution.
IMPROVE
The objective of alarm analysis is to identify opportunities to improve the alarm management
system. In many environments a significant number of problems can be eliminated through
minor changes to the system. Based on the problem areas identified, determine and
implement solutions. Any changes to the alarm management system must be carefully
evaluated and planned before being implemented. Essential to successful implementation
of changes is to identify and communicate the changes to the operators and other affected
individuals, usually in the form of content-specific training and the standard Configuration
Control Management process.
The subsequent cycles of analysis indicate whether the changes made were effective.
CONTROL
Ensure that gains are not lost over time. Properly transition the needed responsibilities from
an Alarm Management Improvement “Project” organization, to the ongoing organization that
will continue to use the system long term. At a minimum, this involves continuing analysis of
the alarm system performance KPIs and action plans for new deficiencies that occur.
4.0 RECORDS
4.1 QA Records
None
None
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 30 of 32
5.0 DEFINITIONS
Alarm - An audible and/or visible means of indicating to the operator an abnormal condition
requiring an operator response.
Alarm Log - The historical record of alarm, acknowledgement, and return-to-normal events.
Alarm Summary - A display that lists alarm events with selected information (e.g. date,
time, priority, acknowledgement status, alarm type, grouping, etc.) Such displays generally
have a variety of sorting, filtering, and other features.
Alarm System - The collection of hardware and software that detects an alarm state,
transmits the indication of that state to the operator, and records changes in the alarm state.
Calculated Alarm - An alarm generated from a calculated value instead of a direct process
measurement.
Chattering Alarm - An alarm that rapidly transitions between the alarm state and the normal
state. Generally three alarms in one minute is a starting definition for a chattering alarm.
Configure - To arrange the settings in a control system such that a particular alarm will be
produced. The opposite (deconfigure) is different than suppressing a configured alarm.
Suppression is an override.
Console - The interface for an operator to monitor the process, which may include multiple
displays and workstations, and defines the boundaries of the operator’s span of control.
10707048
FPG Standard Alarm Management Standard FPG-SPP-10.xxx
Programs and Rev. 0000
Processes Page 31 of 32
Control System - A system that responds to input signals from the equipment under control
and/or from an operator and generates output signals that cause the equipment under
control to operate in the desired manner.
(Alarm) Dead band - The range through which an input must be varied from the alarm
setpoint necessary to clear the alarm.
Deviation Alarm - An alarm generated based upon the difference between two analog
values, typically a controller setpoint and a process value. Deviation alarms are also used to
indicate a difference between the measurements from redundant instruments.
Digital Alarm - An alarm based upon a digital (binary) signal, such as Off-Normal or
Change-of-State.
(Alarm) Flood - A period of time when the alarm system generates a greater number of
alarms than the operator can effectively manage. Generally defined as beginning when the
alarm rate exceeds 10 alarms in 10 minutes and ending when the rate drops below 5 alarms
in 10 minutes.
Initiating Event - A malfunction, failure or other condition that can cause an alarm
indication.
Master Alarm Database - The document that contains the authorized list of rationalized
alarms and associated settings and parameters.
Operator - The person directly responsible for ensuring the process variables are
maintained within limits.
Operator Response Time - The time between the annunciation of the alarm and when the
operator completes the corrective action in response to the alarm.
(Alarm) Philosophy - A document that establishes the basic definitions, principles, and
processes to design, implement, and maintain an alarm system.
Plant State - A defined state of operation of a process plant (such as shutdown, start-up, full
rates, varying feedstocks, differing modes, etc.).
Rate-of-Change Alarm - An alarm generated when a limit value for the rate of change of a
process variable is exceeded.
Return to Normal (also Clear) - The alarm system indication that an alarm condition has
transitioned to the normal state. Generally a time stamped event record is produced when
this occurs.
(Alarm) Setpoint (Alarm Limit, Alarm Trip Point) - The threshold value or discrete state of
a process variable that triggers the alarm indication.
Stale Alarm - An alarm that remains in the alarm state for an extended period of time.
Standing Alarms - A measure of the number of alarms in the alarm state at a specified
time.
State-Based Alarming - A process in which multiple settings for alarms are determined to
match varying plant states, and the settings dynamically swapped as plant state change is
detected. This ensures alarms are always relevant and appropriate.
Suppress (Suppression) - Any mechanism to prevent the indication of the alarm to the
operator when the base alarm condition is present, initiated automatically, by logic, or
manually. Suppression is generally an override to a configured alarm. Uncontrolled alarm
suppression can be hazardous
System Alarm - An alarm generated from faults within the control system hardware,
software or components.
Unacknowledged - A state in which an alarm has not been acknowledged by the operator.
10707048
10707048
D
BAD ACTOR RESOLUTIONS
The bad actors for Bull Run, Widows Creek Units 7 and 8 were discussed. For below plants, the
resolutions have been provided as attachments.
1. Bull Run (See Attachment A)
2. Widows Creek Unit 7 (See Attachment B)
3. Widows Creek Unit 8 (See Attachment C)
For Lagoon Creek, Gallatin, Southaven, Magnolia, and Allen plants, the resolutions were
provided as part of the alarm database along with the D&R results.
10707048 D-1
10707048
E
REFERENCES
Books:
Bill R. Hollifield, and E. Habibi, The Alarm Management Handbook, ISBN 0-9778969-0-0
Reports:
EPRI Alarm Management and Annunciator Applications Guidelines. EPRI, Palo Alto, CA: 2008.
1014316.
10707048 E-1
10707048
F
PRINCIPAL INVESTIGATORS
(In alphabetical order)
Ganapathy Nagarajan
Ganapathy Nagarajan is a senior controls engineer for Plant Automation Services (PAS), Inc.
Ganapathy has international multi-company, multi-industry experience in the area of systems
integration, DCS, and safety systems engineering, and alarm management including
rationalization and alarm analysis. His expertise also includes high-performance human machine
interface (HP HMI) philosophy development, style guide development, and HP HMI graphic
design.
Gana holds a Bachelor of Science Degree in Electrical & Electronics Engineering, India.
Kenneth Phelps
Kenneth Phelps is a senior applications consultant for Plant Automation Services (PAS), Inc.
Kenneth has international multi-company, multi-industry experience in the area of alarm
management including alarm philosophy development, rationalization, and alarm analysis. In
addition, Kenneth has a number of years of chemical industry experience with a focus on safe
work practices, product/process improvement, statistical process control, process automation,
and production.
Kenneth holds degrees in A.A.S. Instrumentation from Texas State Technical Institute, Waco,
Texas, and in A.A.S. HVAC from North Harris County College.
Laura J. Martinez
Laura Martinez is a control systems engineer for Plant Automation Services (PAS), Inc. Laura
has international multi-company, multi-industry experience in the area of alarm management
including alarm philosophy development, rationalization, alarm analysis, and dynamic alarming.
Laura holds a Bachelor of Science degree in Chemical Engineering from Texas A&M
University, Kingsville, Texas.
Ronald W. Carlton
Ronald Carlton is a senior alarm manager for Plant Automation Services (PAS), Inc. Ronald has
international multi-company, multi-industry experience in the area of alarm management
including alarm philosophy development, rationalization, and alarm analysis. His expertise also
includes high performance human machine interface (HP HMI) philosophy development, style
guide development, and HP HMI graphic design. Ronald has over 30 years of chemical industry
experience in both operations and process control.
Ronald holds a Bachelor’s Degree in Industrial Management from the University of Houston at
Clear Lake, Houston, Texas.
10707048 F-1
Tim Younts
Tim Younts is an alarm management consultant with Plant Automation Services (PAS), Inc. Tim
has international, multi-industry experience in the areas of alarm management, systems safety,
operational risk management, control engineering, and real-time, mission-critical console
operations.
Tim holds a Bachelor of Science Degree in Physics from the University of North Texas, Denton,
Texas, and has prior aerospace industry experience, which is now being applied to the industrial
plant automation industry.
10707048 F-2
10707048
Export Control Restrictions The Electric Power Research Institute, Inc.
Access to and use of EPRI Intellectual Property is granted (EPRI, www.epri.com) conducts research and
with the specific understanding and requirement that development relating to the generation, delivery
responsibility for ensuring full compliance with all applicable and use of electricity for the benefit of the public. An
U.S. and foreign export laws and regulations is being independent, nonprofit organization, EPRI brings
undertaken by you and your company. This includes an
together its scientists and engineers as well as
obligation to ensure that any individual receiving access
hereunder who is not a U.S. citizen or permanent U.S. experts from academia and industry to help
resident is permitted access under applicable U.S. and address challenges in electricity, including
foreign export laws and regulations. In the event you are reliability, efficiency, affordability, health, safety and
uncertain whether you or your company may lawfully obtain the environment. EPRI also provides technology,
access to this EPRI Intellectual Property, you acknowledge policy and economic analyses to drive long-range
that it is your obligation to consult with your company’s legal
research and development planning, and supports
counsel to determine whether this access is lawful.
Although EPRI may make available on a case-by-case research in emerging technologies. EPRI’s
basis an informal assessment of the applicable U.S. export members represent approximately 90 percent of the
classification for specific EPRI Intellectual Property, you and electricity generated and delivered in the United
your company acknowledge that this assessment is solely States, and international participation extends to
for informational purposes and not for reliance purposes. more than 30 countries. EPRI’s principal offices and
You and your company acknowledge that it is still the
laboratories are located in Palo Alto, Calif.;
obligation of you and your company to make your own
assessment of the applicable U.S. export classification and Charlotte, N.C.; Knoxville, Tenn.; and Lenox, Mass.
ensure compliance accordingly. You and your company
Together…Shaping the Future of Electricity
understand and acknowledge your obligations to make a
prompt report to EPRI and the appropriate authorities
regarding any access to or use of EPRI Intellectual Property
hereunder that may be in violation of applicable U.S. or
foreign export laws or regulations.
© 2013 Electric Power Research Institute (EPRI), Inc. All rights reserved.
Electric Power Research Institute, EPRI, and TOGETHER…SHAPING THE
FUTURE OF ELECTRICITY are registered service marks of the Electric
Power Research Institute, Inc.
1026497