Quantifying Operational Risk PDF

B.A.J.
10, V, 919-1026 (2004)
QUANTIFYING OPERATIONAL RISK IN

GENERAL INSURANCE COMPANIES
Developed by a Giro Working Party

By M. H. Tripp, H. L. Bradley, R. Devitt, G. C. Orros,
G. L. Overton, L. M. Pryor and R. A. Shaw
[Presented to the Institute of Actuaries, 22 March 2004]
abstract
The paper overviews the application of existing actuarial techniques to operational risk. It
considers how, working in conjunction with other experts, actuaries can develop a new
framework to monitor/review, establish context, identify, understand and decide what to do in
terms of the management and mitigation of operational risk. It suggests categorisations of risk to
help analyses and proposes how new risk indicators may be needed, in conjunction with more
normal quantification approaches.
Using a case study, it explores the application of stress and scenario testing, statistical curve
fitting (including the application of extreme value theory), causal (Bayesian) modelling and the
extension of dynamic financial analysis to include operational risk. It suggests there is no one
correct approach and that the choice of parameters and modelling assumptions is critical. It lists
a number of other techniques for future consideration.
There is a section about how ‘soft issues’ including dominance risk, the impact of belief
systems and culture, the focus of performance management systems and the psychology of
organisations affect operational risk. An approach to rating the people aspects of risk in parallel
with quantification may help give a better overall assessment of risk and improve the
understanding for capital implications.
The paper concludes with a brief review of implications for reporting and considers what
future work will help develop the actuarial contribution. It is hoped the paper will sow seeds for
the development of best practice in dealing with operational risk and increase the interest of
actuaries in this emerging new topic.
keywords
Operational Risk; Enterprise Risk Management; Risk Management; General Insurance;

Quantification of Risk; Financial Services; Capital Assessment; Capital Management; People
Process and Systems; Curve Fitting; Risk Management; Regulations and Risk; Capital
Regulations; Risk Categorisation; Risk Indicators; Bank of International Settlements;
Extreme Value Theory; Stress Tests; Scenario Analysis; Causal Models; Bayesian Techniques;
Soft Issues; Myers-Briggs; Belbin Team Roles; Risk Reporting; Professional Guidance
contact address
Michael Howard Tripp, Watson Wyatt LLP, Watson House, London Road, Reigate, Surrey
RH2 9PQ, U.K. Tel: +44 (0)1737-241144; Fax: +44 (0)1737-241496;
E-mail: [email protected]
919
920 Quantifying Operational Risk in General Insurance Companies
In physical science the first essential step in the direction of learning any subject is to find
principles of numerical reckoning and practicable methods for measuring some quality connected
with it. Lord Kelvin (1824-1907)
The lament of many an Institute working party member: ‘‘It is late and I want to go home.’’
Shaw (3 December 2003)
". Introduction
1.1 The Starting Point

1.1.1 In the wake of a number of recent business failures and results
which the stock markets see as overly volatile (or unpredictable), an even
more professional and thorough scientific approach to business and risk
management is increasingly sought.
1.1.2 The financial services industry (be it banking, insurance or
securities) has its own approaches to risk that recognise the specific nature of
the business environment. Led by the thinking of working groups at the
Bank of International Settlements (Basel), typical risk categories include
(a) credit risk; (b) market risk; (c) liquidity risk; (d) insurance risk;
(e) operational risk; (f) other risk (strategic and reputational risk); and
(g) group risk.
1.1.3 The immediate trigger for this paper is a call from the Financial
Services Authority (FSA) for insurance companies to improve their
approaches to operational risk. Perhaps it is natural, given that much work
has already been undertaken on the modelling and measurement of credit,
market, liquidity and insurance risk, for attention to turn to operational
risk.
1.1.4 In these early days it is not possible to claim that best practice
exists, so this paper attempts to set out the emerging actuarial approach
which we hope will be a useful contribution to the development of best
practice. Some of the techniques are not new, but their application in the
field of operational risk is. The working group hope that this is a timely
contribution to the development of actuarial thinking in a subject that might
otherwise get overlooked.
1.1.5 Another objective of this paper is to guide the reader through a
number of different areas related to the classification and quantification of
operational risk. A further goal is to raise sights and go beyond what some
may see as the typical actuarial world.
1.1.6 Operational risk can be described as “the risk of direct or indirect
loss resulting from inadequate or failed internal processes, people and
systems or from external events’’ (based on references from both Basel and
the FSA). Despite the fact that definitions of operational risk do vary, they
are broadly similar in scope, in that operational risk refers to loss from
internal failure and/or external events not captured in the other risk
categories. As discussed in the paper, aspects of ‘other risks’, such as
Quantifying Operational Risk in General Insurance Companies 921
strategic decision taking, reputational or legal, may or may not be regarded
as categories of operational risk, but clearly need including in any overall risk
framework. Indeed, it has been further pointed out that not having the right
processes in place to manage operational risk is indeed itself an operational
risk!
1.1.7 Considerable discussion has taken place about where operational
risk starts and where proper corporate governance, including strategic
decision taking, ends. In some ways this is a hollow debate. Ultimately to
mitigate and manage operational and strategic risk we need:
ö design: the right controls, people and processes;
ö implementation: reviews to make sure controls are implemented with
trained and motivated people working in an appropriate cultural
environment; and
ö review: processes to ensure a continual rethink and refresh of the whole.
1.1.8 The current strong push of emerging regulatory requirements

alluded to above is accepted, but we believe that the real driver is, and
properly should be, the pull of business benefits. Some of the best run and
most professional companies have been developing their own approaches to
operational risk for their own business reasons. Indeed, measurement of risk
is becoming an essential tool of effective business management. Some of the
best-run companies have now embedded risk management, in all its guises, in
their way of doing business. Clearly those companies believe that it
enhances their profitability, whilst at the same time reducing earnings
volatility (and ultimately the risk of going out of business). Some of their
thinking draws heavily on risk process and quality management in other
industries ö manufacturing, construction, retailing and engineering, to cite a
few examples. Some of it is directly related to the characteristics of the
finance industry. Much is possible now with the ever-increasing power of
computers that even a few years ago would have been infeasible. However,
discussion within Basel, the FSA and the European Union has clearly
accelerated interest, understanding and action and even these forward
thinking companies have benefited from the new ideas.
1.1.9 Whilst this paper originates from a General Insurance Research
Organisation (GIRO) working group on operational risk, its application is
much wider, covering life assurance, fund management, pension funds, other
forms of security business and banking ö indeed any organisation using
analytic approaches to risk identification, management and measurement,
including stochastic risk analysis modelling techniques.
1.1.10 In 2001 an operational risk working group was set up that
reported at the 2002 GIRO conference in Paris. A good start had been made,
but there was more to do, especially in the desire to be able to quantify
operational risks and understand both their magnitude and correlation (or
otherwise) with other risks. A second working party, with the colourful title
‘Measurement or Bust’, was established, and the authors of this paper were
the key contributors to that work.
1.1.11 Adding value to business management often requires
measurement and quantification. Management decisions are better informed
by a well considered understanding of the scale of investments and returns.
While it remains a challenge to keep the right balance between numerically-
based logical decisions and more intuitive qualitative thought, the better the
thinking from all angles the better the chance of success. In this context it is
still easy to shy away from discussing, let alone quantifying, risk appetite or
risk tolerance, and still unusual to find a well articulated well defined clear
statement at Board level. This state of affairs is unlikely to last long. This
paper contributes to resolving aspects of this quantification challenge.
1.1.12 Quantification requires data. The initial reaction is often that
operational risk is difficult, if not impossible, to quantify, and even if it were
not, the range of incidents (losses) would be hard to categorise, and hard to
predict. The conclusion is that collecting data is a time consuming and
possibly meaningless task. This thinking is being overturned, and, as an
example, the United Kingdom banks have contributed to a confidential
central database facilitated by the British Bankers Association (BBA) known
as Gold. Also, across the globe there are emerging proprietorial databases
such as ‘OpVar’, and Basel has just completed its second loss quantification
exercise.
1.1.13 Sadly, insurers appear to be lagging behind, and to be relatively
slow to get going, although the signs are that this is changing. One source of
data might be the claims experience emerging from insurance policies
underwritten to cover insurable risks arising from operational loss events.
From our point of view this is not only a difficult source to access because of
commercial confidentiality, but also the data are probably biased, due to
limitations of insurance policy wordings (coverage) and the way insurance
claims processes work, not to say incomplete.
1.2 The Actuarial Contribution

1.2.1 Operational risk is a broad subject. Many skills and types of
expertise are required, and whilst the authors feel that the actuarial
profession can make a significant contribution, we cannot do so alone.
Indeed, others have been working with operational risk problems for some
time, and we can listen and learn.
1.2.2 During the ground work for this paper we talked with a wide
variety of people and are particularly appreciative of their openness and
willingness to share ideas. Amongst others, members of the Institute of Risk
Management, insurance underwriters, auditors, compliance specialists,
engineers and occupational psychologists all have skills and all show a
considerable interest.
1.2.3 Typically, one of the actuary’s key tasks is to assist with the
quantification of capital and risk, preparing analyses and reporting to the
Board (or even the FSA). The profession has been actively involved during
development of recent FSA consultation papers and, over the years, with
appropriate guidance, in fulfilling a number of relevant and important
roles.
1.2.4 Our work involves liaison with other professions, with company
management or functional experts, and with external advisors. We can act as
integrators in the quantification task ö pulling the threads together on
behalf of a diverse set of contributors. Our profession supports us, and, if
necessary, enables us to say the unsayable.
1.2.5 This paper considers wide aspects of risk management and
mitigation, well beyond the identification and reporting of risks. We hope
that it will serve to provide pointers for the expansion of our thinking and
further involvement in wider aspects of projects to support capital
assessments for the FSA and for business management requirements. We
hope that it will encourage readers to explore what they can learn from
observing others and to develop new behaviours and techniques.
1.3 Quantification Techniques

1.3.1 There are many quantitative methods that might be applicable to
the problems of understanding and quantifying operational risk. It is still
early days in investigating which are likely to be most useful. We have set out
below a list of those which we feel represent emerging best practice, and
which we believe may have increased applicability in the future.
1.3.2 The methods are grouped into families under common broad
headings. Some of these methods are discussed in subsequent sections.
1.3.3 Statistical/curve fitting: the statistical/curve fitting methods cover:
(a) empirical studies; (b) maximum loss approach; (c) theoretical probability
distribution functions (PDFs); and (d) regression analysis.
1.3.4 Frequency/severity analysis: this covers: (a) frequency/severity
analysis; (b) extreme value theory (EVT) ö which is an advanced version of
frequency/severity analysis; and (c) stochastic differential equations.
1.3.5 Statistical (Bayesian): the statistical/Bayesian approach includes:
(a) systems (dynamic) models; (b) influence diagrams; (c) Bayesian belief
networks and Bayesian causal models; (e) process maps and assessments; and
(f) neural networks.
1.3.6 Expert: here we include: (a) fuzzy logic; (b) direct assessment
of likelihood/preference among bets; (c) the Delphi method (see {2.7.8);
(d) capital asset pricing models (CAPM) ö market view less insurance/asset
risk values; and (e) RAMP.
1.3.7 Practical: then there are the practical approaches of: (a) stress
testing and scenario analysis; (b) business/industry scenarios; (c) dynamic
financial analysis; and (d) market beta comparison for individual companies
within market sectors.
1.4 Paper Overview
1.4.1 The paper is both theoretical and illustrative. We start by
describing a hypothetical case study of an insurance company, the aptly
named Middle England Life & General plc, that brings out many of the
salient points and ideas.
1.4.2 The next two sections provide the background to the
quantification of operational risk. Building on the case study, we discuss the
framework for the assessment and quantification of operational risk and then
present details of an initial risk assessment.
1.4.3 The middle sections of the paper address the quantification of
operational risk. Stress testing and scenario analysis are discussed first,
followed by a section on frequency/severity modelling, then an introduction
to the not so well known topic of causal/Bayesian approaches to risk, and
finally reference to the more widely used dynamic financial analysis methods.
1.4.4 The penultimate two sections look at the bigger picture. Various
pitfalls and soft issues are addressed, followed by a section considering how
to report the results and bring the threads together.
1.4.5 Finally, we make some concluding remarks and set out ideas for
future work.
Æ. Case Study
2.1 Introduction
2.1.1 A key objective of this paper is to examine the applicability of
various methods for quantifying operational risk, and quantification requires
data. In the absence of any reliable data sources, we decided to create an
illustrative case study: Middle England Life & General plc (MELG). Whilst
we have not illustrated every aspect of operational risk, we have attempted to
ensure the case study is:
ö based in reality ö by pooling data from public and private sources the
underlying figures are intended to be reasonably illustrative of the type of
losses, both in terms of order of magnitude (severity/impact) and
likelihood (frequency);
ö practical ö by building on the personal experiences of working party
members as well as published case studies, we hope it is sufficiently ‘real
life’ to be a helpful tool ö not just for this paper, but for other uses;
and
ö easy for readers to relate it to their own circumstances.
2.1.2 This case study is illustrative only. While the company overview,
historic accounts and other data are based on realistic elements of various
U.K. companies using FSA returns, any resemblance to any specific
company is purely coincidental.
2.1.3 For the sake of clarity, the case study only discusses the general
insurance aspects of the business. Furthermore, only a small number of risks
are described in detail, out of a possible 135 or so possible categories (e.g.
as given by the BBA level three categories discussed later and referred to in
Appendix A). These include fraud, systems development, implementation of
strategic decisions and reputational risk.
2.1.4 Following a FSA Arrow visit, the newly appointed director of
group risk has been charged with producing a report that:
ö reviews the enterprise wide risk management practices of MELG plc,
with particular reference to operational risks;
ö ensures that MELG plc takes steps to establish and maintain appropriate
risk management practices, adhering to any FSA regulatory guidelines
about operational risk management and other best practices; and
ö informs the group risk committee about past and current enterprise
wide risk management issues, with a focus on exposure to operational
risks.
2.1.5 The report is to investigate the past, current and projected future
of the company, quantifying issues wherever possible and setting out
findings, without fear of retribution, under the ‘whistle blowing rules’ from
the group procedures.
2.2 Historical Beginnings of MELG

2.2.1 The company’s origins in the U.K. may be traced back to the
early 1900s, when it started as a small life office, based in the Midlands. In
these early days it diversified into non-life business and started to offer
private motor insurance and then other ‘personal’ lines. A direct writing
operation was launched in 1993. Although MELG developed a commercial
motor account, it was only when it acquired a commercial insurance
company in 1995 that it became a serious commercial insurer. In 1997
MELG restructured into three separate business units ö commercial,
personal intermediary and personal direct.
2.2.2 In the summer of 1998 MELG became the target of a hostile
takeover bid, which was eventually successful. The company became, with
effect from January 1999, the U.K. subsidiary of a large multinational
company, with its parent Megacentral Insurance Corporation Inc (MICI)
based in New York, United States of America.
2.3 Current Operations of MELG

2.3.1 MELG plc currently operates through three major sites (one in the
north, one in the Midlands, where its head-office is co-located with an
operational site, and one in the south) with ten local offices. Its profitability
has generally been in line with market averages.
2.3.2 There are 2,600 general insurance staff, of whom 900 are in its
Midlands office, 600 in each of its north and south offices and 500 spread
over the ten local offices, including a team of 50 inspectors (broker sales
force), and 50 external claims staff.
2.3.3 It operates a matrix management philosophy. Each executive team
member ‘owns’ one of its key business processes and has a responsibility for
its improved quality across the whole organisation. The organisation is now
considered as three main strategic businesses:
ö commercial insurance;
ö personal intermediary insurance; and
ö personal direct insurance.
2.3.4 The parent company (MICI) is a global insurance company based

in New York, U.S.A. It is a centrally managed global firm with operations in
50 countries. The group management is powerful, and has tended to
determine the group strategic and investment policy on behalf of its local
operations. MICI now finds itself to be in financial difficulties with its U.K.
operations, which it believes to be the result of the way in which group
strategic investment policy was implemented and a large external weather
catastrophe in 2000, combined with inadequate reinsurance purchasing.
2.3.5 MELG plc has a limited degree of autonomy from its parent
company. Its Board meets eight times a year and as well as three executive
directors, the chief executive officer, the finance director and the group
operations director, it has three representatives from its parent and two
external non-executive directors. It is required to submit plans on an annual
basis with results and updated forecasts on a monthly basis. As well as
accounts, it submits a series of key performance indicator management
information.
2.3.6 MELG plc still faces multiple legacy systems and ongoing systems
integration problems, so production of consolidated management
information is unreliable. This further hampers the effective running of the
group from the centre. The three separate business units underwrite diverse
types of cover with different structures and cultures, which again make
centralised control difficult and exacerbate already poor financial and
underwriting disciplines. There appears to be a very high expense ratio and
large losses as the firm tried to compensate for previous under-pricing and
under-reserving. Merger costs were higher than expected, and MELG has
made little headway in achieving the forecast cost-side synergies.
2.4 MICI imposes Investment and Business Strategy

2.4.1 In July 2000 the parent company (MICI) set an aspect of policy
for MELG that was based on group investment objectives. This proved to
have a detrimental effect on MELG. MELG plc did not completely check the
suitability of the investments made on its behalf by the parent company. It
appears that the MELG plc balance sheet was used to make strategic
Table 2.4.2. 70/30 personal/commercial split; projection
» million 2001 2002 2003 2004 2005 2006 2007 2008
Personal lines 1,609 1,808 1,900 1,995 2,095 2,199 2,353 2,518
Commercial lines 298 497 697 1,045 1,145 1,219 1,294 1,369
Total gross premium 1,907 2,305 2,597 3,040 3,240 3,418 3,647 3,887
U/W result ÿ102 ÿ76 ÿ50 0 32 3 ÿ50 ÿ100
Net assets 1,700 857 1,043 1,332 1,538 1,747 1,917 2,128
Solvency ratio 75% 51% 55% 60% 65% 70% 72% 75%
Table 2.4.3. 90/10 personal/commercial split; projection

» million 2001 2002 2003 2004 2005 2006 2007 2008
Personal lines 1,609 1,808 1,900 1,995 2,095 2,199 2,353 2,518
Commercial lines 298 497 500 450 425 400 380 360
Total gross premium 1,907 2,305 2,400 2,445 2,520 2,599 2,733 2,878
U/W result 4 ÿ76 ÿ50 10 52 75 100 50
Net assets 1,700 857 1,050 1,350 1,600 1,850 2,150 2,600
Solvency ratio 75% 51% 63% 81% 96% 109% 120% 135%
investments for the U.S.A. parent. The subsequent market downturn in the
U.S.A. has resulted in significant losses, as at April 2003.
2.4.2 A group management decision to aim for 70% personal lines and
30% commercial lines business mix was also taken in July 2000. This split
was to be achieved for 2001 onwards. This business mix decision was
imposed on the U.K. management team, who, at the time, would have
preferred to maintain a 90/10 split between personal lines and commercial
lines. Anecdotal reports suggest that the U.K. management may have been
pressurised to improve the projected commercial lines loss ratios beyond that
which they felt was realistically achievable during the business planning
process. Their final projection for group on a 70/30 split was as shown in
Table 2.4.2.
2.4.3 The U.K. management team’s minority report at the time showed
the final projections on a 90/10 basis (adjusted in all other respects to be
comparable with the 70/30 split), and this is shown in Table 2.4.3.
2.5 Management Changes

2.5.1 The MELG management decision-making process changed during
1999, following its acquisition by MICI. Prior to that time it operated a more
consensus, delegated decision-making style. The cultural change has been a
difficult one, leading to the eventual resignation of its long-standing U.K.
chief executive officer, followed by that of the finance director a few months
later. There was no suggestion of poor management at that time ö the
culture clash had become a real point of friction. However, since then many
problems have been blamed on previous management.
2.5.2 The current CEO was appointed by the parent company (MICI) in
January 2002. After three months he appointed his own senior team,
including one outsider, the FD who came from a ‘FMCG’ background, and
who had previously worked with the CEO in a retail environment.
2.5.3 The new CEO’s own background included spells at a bank, and
before that at a firm of accountants; he accepts that this was some time ago.
His most recent experience was in retail (in New York, U.S.A.), where he
gained a reputation for acquiring smaller businesses and implementing
centralised back office functions that enabled significant staffing level
reductions and associated cost savings. He is also a personal friend of the
parent company chairman, having known him since their university days.
2.5.4 In mid 2002 MELG had its first FSA Arrow monitoring visit. The
net result was that the FSA required MICI to transfer »100m to its U.K.
subsidiary.
2.6 Current Position

The date is now April 2003. The company has just finalised its 2002
accounts. The gross earned premiums and some financial outcomes for 1995-
2002 (actual) are set out in Table 2.6.1, together with a breakdown between
personal and commercial lines.
2.7 Some Major Historical Actions and Incidents

2.7.1 Launch of direct writing. In mid 1993 a new direct channel was
launched. The projected cost at that time was »30m to P & L, based on a new
marketing budget of »10m per annum, extra staff costs and a »5m
investment in systems, all offset by growth of business and eventual profit. A
retrospective analysis undertaken in 1998 suggested that the actual cost was
in the region of »70m, partly due to expense overruns and partly to lower
than expected business growth. For the ease of analysis, it was decided to
assume that the loss had an effective date of 1998, as the problems were more
to do with the later stages of implementation than the original decision.
Table 2.6.1. Gross earned premiums and some financial outcomes for
1995 to 2002
» million 1995 1996 1997 1998 1999 2000 2001 2002
Personal lines 1,001 1,204 1,297 1,305 1,409 1,597 1,609 1,808
Commercial lines 77 112 139 167 199 240 298 497
Gross premium 1,079 1,316 1,436 1,473 1,609 1,838 1,907 2,305
U/W result 189 270 82 74 ÿ19 ÿ206 ÿ102 ÿ76
Net assets 539 865 1,170 1,427 1,735 1,500 1,700 857
Solvency ratio 50% 81% 101% 118% 131% 99% 75% 51%
2.7.2 Outsourcing of claims handling. The structure of the three current,
separate strategic business units was established in January 1997. The
commercial insurance business was self-contained and largely staffed by
people from the acquired commercial company. The personal direct business,
which had previously been considered a sales channel of the personal lines
business, was now given autonomy for all aspects of its business. In the
event, it decided to outsource its claims handling (post initial notification) to
the personal intermediary business. The projected cost of this change was
»10m. A retrospective analysis suggested that the real cost had been nearer
»50m, comprising loss of revenue »30m, extra expenses »5m (not saved) and
poorer loss ratio »15m as a result of attention being distracted from
underwriting and inadequate monitoring of claims handling. There were also
a number of cultural tensions.
2.7.3 External supplier fraud. A major case of external fraud led to a
further loss of »5m, spread throughout the 1999 calendar year. The fraud
involved a third-party supplier selected by the U.K. company to provide
services to insurance clients. The fraud was reported in 2001 by an employee
at the supplier. Subsequent investigation revealed that a junior manager at
the company was aware of potential irregularities, but had not disclosed this
information due to lack of confidence in whistle blowing procedures.
2.7.4 Reinsurance failure to respond. Group management also overrode
local management with respect to reinsurance policy. On the occurrence of a
large external catastrophe in March 2000, with a gross loss of »100 million,
only »10 million was recovered from the catastrophe XL reinsurance treaties,
instead of the »50 million that had been expected. The local U.K.
management blamed the group risks department in New York, which had
reviewed the reinsurance programme and agreed the terms with the lead
reinsurer. The group risks department blamed the U.K. management for
failing to spot the problems with the final draft reinsurance treaties that had
been sent to the U.K. team for final approval. The group internal audit
department blamed both parties for their evident lack of communication. The
overall result was an unexpected loss of »40m.
2.7.5 Block account loss. A key corporate relationship for MELG plc
collapsed in January 2001, primarily as a result of the group initiated
management changes at MELG plc in September 2000. The key corporate
partner was unimpressed by the new business development manager from
Chicago (U.S.A.), and decided to invite competitive tenders for the contract
renewal on 1 January 2001. As a result, this »100m ‘block account’ (personal
lines) was lost, with an assumed profit value of »20m.
2.7.6 Loan default investment loss. As previously mentioned, the parent
company had, in effect, set an aspect of investment policy that had a
detrimental effect on MELG plc because it put group objectives before the
prudent management of the U.K. insurance firm. Group management in the
U.S.A. overrode local decisions in the U.K. Local management either lost
Table 2.7.8. Delphi assessment
Year Number of incidents Total cost (»m)
1995 15,000 3.000
1996 12,000 2.220
1997 22,000 4.510
1998 15,000 3.255
1999 23,000 5.060
2000 20,000 5.040
2001 15,000 4.005
autonomy or they did not properly check the suitability of the investments
being made. Group management in the U.S.A. effectively used MELG’s
balance sheet to make strategic investments on a group wide basis. One such
strategic investment, a loan to a key producer in the U.S.A., defaulted (in
October 2002), costing »75m.
2.7.7 Stop loss reinsurance loss. The MELG stop loss reinsurance
treaties for its gross loss of »50m should have recovered »25m, but there was
a nil recovery. Once again the MICI group risks department and the U.K.
management blamed each other for the problem. The result was an
unexpected loss of »25m.
2.7.8 Delphi assessment. A recent ‘Delphi’ method assessment of
fraudulent and ‘misrepresented’ claims leakage led to an assessment of over
payment of claims, which is shown in Table 2.7.8. (This had involved seeking
the views of 12 internal claims personnel and three external experts based
on some preliminary data analysis, summarising these views, replaying them
back to the 15 people, letting them refine their views in light of the comments
of others and then collating these refined views.)
2.7.9 Systems overspend loss. Systems developments often led to
overspends. In the last seven years there have been twelve major overspends
averaging »2.2 million. A new project was planned recently, again influenced
by group management. Its outline budget cost for 2003/04 is »20 million. It
is already three months behind schedule and there is an overspend of »2m
compared to the phased budget. The system specification had been developed
to incorporate group and company requirements, but without effective
co-ordination. The result also appears to be probable weaknesses in reporting
of third-party supplier transactions.
2.7.10 There were one or two other relevant losses, and two ‘near
misses’. The risk director was told during various interviews about a number
of things that could have gone wrong, but did not, and hence the data
collected refer to events which could have resulted in larger operational risk
losses.
2.8 Consequences
2.8.1 The FSA Arrow monitoring visit in mid 2002 highlighted a series
of issues and concerns which led to a full investigation. The net result was
that the FSA required MICI to transfer »100m to the U.K. to maintain an
adequate solvency margin for MELG plc over the foreseeable future.
2.8.2 Although generally holding a good reputation with both
intermediaries and customers, following the loss of its major household-
based account and the stock market collapse there has been recent press
speculation that its financial position is less than satisfactory. The claim was
that its American parent might not stand behind it if the worst came to the
worst, and this is causing a nervous reaction from smaller brokers and direct
purchasers. This reputational risk could easily blow up into a full scale
crisis.
2.9 Initial Response to Information

2.9.1 The risk director’s first step was to ensure that the role was clear
and that the reporting lines were independent of any senior management
influence.
2.9.2 The next consideration was to understand the key drivers behind
the task ö the rationale for the operational risk review in terms of meeting
the FSA requirements and improving management disciplines. The initial
discussions highlighted a lack of any real concerted discipline to risk
management or any clear risk management policies. There was, for example,
no agreed Board statement about risk appetite or tolerance under the
current circumstances.
2.9.3 The subsequent more detailed preliminary investigations
highlighted the fact that the available information was incomplete. In
addition, there were too few losses on which to base any model. The known
losses did not reflect all the risks that the company faced, as all those risks
that had not so far resulted in a loss had been ignored.
2.9.4 The risk director therefore decided to pursue additional information
including, for the known losses:
ö clarification of when losses were recognised financially (i.e. the
accounting quarter that took the hit) and when they were incurred (i.e.
when the action or decision that led to them occurred);
ö the expected profit that would have accrued to the bottom line if the
block account had not been lost, as the lost income was not a good
measure of the actual financial loss;
ö where a loss amount is made up of several effects, such as expense
overrun, inappropriate assumptions and resourcing issues, then whether
it is possible to isolate the losses due to each effect; and
ö interviewing staff involved in the project/area where each loss arose,
and those who investigated it, on an informal basis, in order to get a
more detailed picture of events. The aim is to understand what happened,
look for the causes of the loss, and look for points at which the loss
could have been mitigated, but was not, and the reasons.
Table 2.9.9. Operational risk loss events
Year Year Loss Loss category Loss
incurred recognised amount
1993 1998 Launch of direct writing Strategic implementation »40m
and expense overrun
1997 1997 Outsourcing of claims handling Outsourcing/supplier »50m
control
2000 2002 Stop loss reinsurance fails to Legal (contract wording) »25m
respond
2000 2002 Investment strategy Loan default »75m
2000 2001 Loss of block account Loss of a key customer »20m
1999 2001 External supplier fraud External fraud »5m
2000 2000 Reinsurance fails to respond Legal (contract wording) »40m
1995 1995 Claims leakage ö model External claims fraud various
to date to date separately
1996 1996 System overspend ö model Expense overrun various
to date to date separately
1996 1998 Various facilities contracts placed Internal fraud »7m
with friends of the facilities
manager
1999 1999 Flaws in cascaded training Transaction capture, »15m
programme for new system (near execution and
miss) maintenance/training
1999 1999 Telephone system failure (near Damage to systems »9m
miss)
2001 2001 Error in delegated authority Legal (contract wording) »12m
contract wording (near miss ö
not spotted by cover holder)
Table 2.9.10. Summary operational risk loss events

Financial year Number of losses Amount of losses (»m)
1995 0 0.0
1996 0 0.0
1997 1 50.0
1998 2 47.0
1999 2 24.0
2000 1 40.0
2001 3 37.0
2002 2 100.0
Total 11 298.0
2.9.5 There is a high probability that the list might not include all the
losses that have actually been incurred. The risk director, therefore, decided
to interview key staff who might be aware of other losses that were either
turned around in time or which were absorbed in the financial results (say by
improvements elsewhere covering them up), to see if any additional losses
or near misses should be added to the list for analysis.
2.9.6 In addition, he decided that a full risk identification exercise would
have to be performed, in order to find those risks that have not yet resulted
in losses. We do not discuss this exercise further, as it is outside the scope of
this paper.
2.9.7 There is also the question of whether all the losses in the list are,
in fact, operational losses. For instance, there is some debate as to whether
the loss due to the impact of the imposed business mix is an operational loss
or a strategic loss. Some would argue that this is a separate category of risk
and not part of operational risk, because a strategic decision is a deliberate
choice made about the direction of the business. The alternative view is that,
while strategic choice clearly influences the chance of success and risk
profile, many aspects of the strategic decision process and the execution of
that strategy are operational. In this instance the group risk director decided
that this was a strategic loss, and omitted it from his list of operational
losses.
2.9.8 There is the further question as to whether some of the reinsurance
losses were inadvertent insurance losses. In the event the risk director decided
that the lack of proper procedures meant they should be classed as
operational losses.
2.9.9 As a result of these additional investigations the complete list of
operational risk loss events is prepared as in Table 2.9.9.
2.9.10 In order to facilitate modelling, including where judged relevant
near misses, this can be re-expressed as in Table 2.9.10.
2.9.11 The group risk director now has some data to use in the modelling
that he intends to perform. The approach to this modelling is described in
later sections.
â. A Framework for Assessing and Quantifying Operational Risk
3.1 MELG’s new risk director next looked at the overall framework for
risk management, and how operational risk could be defined. The consideration
of what data might be required in order to model operational risk was part
of this exercise.
3.2 A Risk Management Framework

3.2.1 The risk director found that, although there was some awareness
of risk management issues in MELG, there was no effective framework. This
had been noticed by the FSA in their Arrow visit, and was a major issue in
the resulting mitigation plan (FSA, 2003a). A top priority was therefore to
set up a risk management framework that worked.
3.2.2 The risk director decided to adopt the commonly used control
cycle approach (IRM, 2002), as shown diagrammatically in Figure 3.2.2.
3.2.3 MELG’s risk director found that, because of the matrix
Review
Report
Monitor Establish context

Risk indicators Scope
Purpose
Objectives
Decide
Control
Mitigate
Exploit
Fund Identify
Ignore Describe
Postpone Report
Understand
Assess
Measure
Estimate
Figure 3.2.2. A basic risk management control cycle
management structure, there had been confusion as to how risk information

was disseminated within the organisation. The priority given to risk
management had varied widely, and there had been no consistent assessment
of risk appetite.
3.3 Risk Management Maturity Model

3.3.1 There have been several attempts to describe the evolution of risk
management practices in general, and those for operational risk management
in particular (BBA, 1999; Fox, 2005; Hall, 2002; Risksig, 2002; Hoffman,
2002). They differ in detail, but agree on the overall outline, shown in
Figure 3.3.1.
3.3.2 MELG’s risk director concluded that MELG was at or near the
awareness stage. It had been relying on traditional measures (internal
controls, internal audit, quality of its staff) to control operational risk, but
had just realised that this was not enough ö hence the new risk director
role.
3.3.3 The risk director felt that a reasonable short-term goal was to
reach the monitoring stage. To do this would mean having a comprehensive
Evolution of Operational Risk Practices
Integration
Quantify
Monitor
Awareness
Traditional
Figure 3.3.1. Risk management maturity model
set of risk indicators along with corresponding escalation triggers and an

effective risk reporting process.
3.3.4 In the medium term the risk director wanted to achieve
quantification, with a loss database, quantitative targets for improvement
and good techniques in place for predictive analysis.
3.3.5 The risk director decided to propose to the Board that reaching
the integrated stage should be a strategic goal for MELG. This would require
MELG also to identify correlations between risk indicators and to link
compensation to returns adjusted for operational risk as well as other risk
types. The Board would need to consider the cost, in management time and
overall resources, before deciding whether this was an appropriate strategic
goal, or whether reaching the quantification stage was sufficient.
3.3.6 In the context of these goals, the push from the FSA Arrow visit
and the internal drive for value added, the risk director’s first formal paper to
the group risk committee was one which set out the background to the
project and initiated a discussion about what risk MELG could face, given its
capital and business position. The paper referred to the FSA’s comment
about a 1 in 200 risk of insolvency in a one-year time frame, it then explored
the concept of both one-year and longer-term (five-year) projections, and
discussed the difficulties in using either on their own ö the longer-term
projections needed to allow for the impact of management action, while the
shorter-term projections did not bring all the issues to light. It was noted
that, as a rule of thumb, a 97.5% chance of not becoming insolvent in a
five-year projection was seen as similar to a 99.5% of not becoming insolvent
in a one-year projection ö both based on the 1 in 200 thinking.
3.3.7 The opportunity was also taken to question whether the work was
starting too far along the management thought processes, and that some
issues related to the Board’s own values, objectives and incentives. The paper
articulated the need to balance short and longer-term views, encouraging
openness and transparency as well as voicing concerns about dominance risk.
Some thought that this was a brave move, but knowing that support, if
needed, would come from his profession, the risk director was confident that
voicing the more important or sensitive things as soon as possible was the
right approach.
3.4 Risk Categories

3.4.1 Throughout the identification, assessment (understanding) and
decision stages of the risk management control cycle, it is useful to have a
system of risk classification. This provides a framework for the reporting of
risk, and helps to place risks within coherent groups.
3.4.2 The term enterprise risk is often used to refer to the sum total of
all the risks faced by an organisation. Enterprise risk is commonly split into
core business risk and operational risk, each of which is then further
subdivided. Simplistically, core business risks are those that arise as a result
of the business decisions taken by the management, while operational risks
are those that arise as a result of the implementation of those decisions, or
from outside factors.
3.4.3 In the Prudential Sourcebook (FSA, 2003c), the FSA identifies six
major risk categories:
ö credit risk;
ö market risk;
ö liquidity risk;
ö insurance underwriting risk;
ö operational risk; and
ö group risk.
3.4.4 This classification is consistent with a division into core business

risk and operational risk, though it should be noted that the FSA is not
concerned with enterprise risk per se, but only with risks to its own
objectives. Also, since then an ‘other’ category has been introduced to include
strategic, reputational and even legal risk.
3.4.5 Other splits are possible. The Casualty Actuarial Society (CAS)
uses four categories in its generic, all industry, classification (CAS, 2001;
CAS, 2003): hazard risk, financial risk, operational risk and strategic
risk.
3.4.6 This is difficult to reconcile with the division into core business
risk and operational risk. Many of the risks classified by the CAS as hazard
risks (e.g. theft/crime, fire/property damage, windstorm/other natural
perils) would be considered as operational risks by non-insurance companies.
It is not clear where insurance risk fits into the CAS classification ö
possibly as a financial risk for insurance companies. Of the four CAS
categories, hazard risk and operational risk together seem to correspond to
our notion of operational risk, and financial risk and strategic risk to our
notion of core business risk.
3.4.7 Needless to say, once particular examples are examined, this neat
two-way division appears increasingly arbitrary.
3.4.8 Other categorisations are possible. Basel has moved towards six
(internal fraud; external fraud; employment practice/workplace safety;
clients, products and business practices; damage to physical assets; delivery
and process management), the FSA having started from four (people;
process; systems; external events), seems to be moving towards Basel; based
on the FSA’s original thinking, the BBA, in establishing their database
(Gold) for banking operational risk, mapped level 2 and level 3 categories on
to these four headings: people, process, systems and external events (see
Appendix A), but are in the process of refining their thinking ö possibly to
move more in line with Basel. If this happens, it may be that the 135 to 140
level 3 categories they use will reduce to something smaller, e.g. between 60
and 80.
3.4.9 Consider reinsurance risk. The question of what reinsurance cover
to take out is a business decision, so the risk that the reinsurance programme
is poorly chosen and proves unsuitable is a core business risk. So is the risk
that the reinsurers do not pay claims. However, the risk that the chosen
programme is not, in fact, implemented is not a core business risk: it would
happen for reasons such as poor communication or incompetence, and is
therefore an operational risk. This means that a failure to recover the
expected amount could be a result of either a core business risk or an
operational risk; we have to look to the cause rather than the consequence to
determine which.
3.4.10 Reputational risk, i.e. the risk that loss will be suffered because
of damage to the firm’s reputation, is similar. If a systems failure leads to a
breakdown in claims handling, a poor reputation and lower sales, then the
loss has clearly arisen from an operational risk. However, if the firm loses its
reputation because of a strategic decision that fails publicly, the resultant
loss is a core business risk. Reputational risk is particularly interesting, as
nearly anything that can go wrong, whether core business or operational, can
result in a loss of reputation and a financial loss. Many risk classifications
include reputational risk as an operational risk, regardless of the originating
cause of loss.
3.5 Cause and Consequence

3.5.1 These and similar examples have led us to propose a basic
framework for analysing risk based on the notions of cause and consequence.
A single consequence can have more than one cause; a single cause can have
more than one consequence; in order to analyse risk effectively it is necessary
to work from the causes rather than from the consequences.
3.5.2 Figure 3.5.2 illustrates the way in which a number of causes can
contribute to a collection of events that in turn have a number of
consequences.
Causes Consequences
Figure 3.5.2. Causes, events and consequences
3.5.3 For example, consider some of the adverse financial effects in the
Chicago Board Options Exchange (CBOE) that resulted from the market
crash of 1987. Some examples of systems problems that arose during the
crash are given below (MacKenzie & Millo, 2001):
ö CSLOUCH, the risk management system at O’Connor & Associates,
could only accommodate a move of 12% in one day, based on the worst
daily loss that had been seen in the past (12.8% in 1929).
ö The systems of many clearing firms could cope only with double-digit
dollar prices. When option prices rose to, say, $106, they appeared as $6,
and the trading firm’s accounts with the clearing firms were off by
millions of dollars, but at any one time it was impossible to tell in which
direction.
ö The markets at CBOE and the Mercantile Exchange were intimately
connected, but their clearing systems were not linked, so what were
actually well-hedged positions could be subject to huge margin calls.
3.5.4 In each of these examples a single consequence, a financial loss,

arose from more than one cause; in these cases, from adverse market events
exacerbated by systems problems. The loss is not solely attributable to any
single cause, and so cannot be attributed to either a core business risk (the
market event) or an operational risk (the systems failures); it is a
combination of the two.
3.5.5 In general, we believe that, by analysing risk of financial losses in
terms of causes, rather than consequences, it is possible to avoid many of the
problems of double counting and omission. Reputational risk is often
proposed as a risk category; however, loss of reputation is usually a
consequence of other problems, such as systems failure or failure to meet
regulatory requirements. If reputational risk is treated as a separate category,
alongside systems risk and regulatory risk, there is a risk of double
counting.
3.6 Defining Operational Risk

3.6.1 There is no single risk classification that suits all purposes.
However it seems that many U.K. insurance companies are adopting the
definition used by the Basel committee (BCBS, 2001) as a starting point. This
defines operational risk as the risk of loss resulting from inadequate or
failed internal processes, people and systems or from external events.
3.6.2 Other definitions include:
ö a measure of the link between a firm’s business activities and the
variation in its business results (King, 2001);
ö operational risk is the risk of adverse impact to business as a
consequence of conducting it in an improper or inadequate manner and
may result from external factors (Doerig, 2000);
ö operational risk results from costs incurred through mistakes made in
carrying out transactions such as settlement failures, failures to meet
regulatory requirements, and untimely collections (Pyle, 1997);
ö operational risk is the potential for adverse financial developments due
to effects that are attributable to customers, inadequately defined
controls, system or control failures, and unmanageable events (Laycock,
1998); and
ö risks deriving from a company’s reliance on systems, processes and
people. These include succession planning, human resources and
employment, information technology, accounting, audit and control
systems and compliance with regulations (McDonnell, 2002).
3.6.3 The FSA gives a number of examples of operational risks (FSA,

2003c), but avoid a hard and fast definition. They are more concerned that
all risks are subject to effective management, rather than with the precise
classification used.
3.7 Practical Considerations

3.7.1 In practice, what matters is that the organisation has good
definitions for all its risk categories and uses them consistently. It may well
be necessary to use different definitions for different purposes. For example,
during the main part of the risk management control cycle it is very
important to use a definition based on causes. The risk director saw that
most of MELG’s claims related problems were the result of operational
failures, and hence were classified as operational risks.
3.7.2 However, when it came to assessing the capital that was required
to fund the risks, the situation was different. The poor claims handling
processes had hitherto been reflected implicitly as part of the loss reserves,
as their effects were not stripped out of the claims data. Pending more
sophisticated data collection procedures, the risk director was forced to leave
them included in the insurance risk category.
3.7.3 MELG’s risk director is aware of the categorisation problem for
the claims handling risks, but it is easy to see that these risks could have been
double counted.
3.7.4 A risk matrix is often used as a framework for risk identification
and assessment. The probability assessment matrix used in the FSA’s Arrow
process is a handy example (FSA, 2003c). This gives a breakdown of risks by
category and subcategory; each is then assigned a rating of high, medium or
low.
3.7.5 Appendix B shows the risk director’s first draft of a fairly high
level risk assessment matrix that he uses as the basis of the process for
MELG.
3.7.6 In many organisations the risk matrix is designed centrally, and
then completed separately by each business unit. The matrix that we show
has two levels of categorisation, not unusual for the firm-wide matrix. Within
each business unit, however, there may be further levels of categorisation
which may not be shown on reports sent outside the unit.
3.7.7 In practice, the risk matrix serves as a working definition for the
different types of risk, including operational risk. If a risk appears in an
operational category, then it counts as an operational risk. This is an
effective operational definition.
3.8 Data Collection

3.8.1 While considering the overall risk management framework of
MELG, the risk director started to address the problems of data collection.
3.8.2 The accuracy of risk measurement methods crucially depends on
both the appropriateness of the model and the availability of data. An
appropriate model requires a thorough understanding of what underlies the
risk under consideration. This understanding is inherently linked to data
availability, and thus the occurrence of events. Incidents also provide the
basis for statistical testing of risk models. Furthermore, the accuracy of risk
models depends upon the measurability of outcomes, and thus upon sound
definition and understanding of effects.
3.8.3 Some form of data collection is vital for any effective risk
management process; the problem is to decide what data should be collected.
There are two opposing views: data collection should be driven by the needs
of the models; or the models that are used should be driven by the
availability of data.
3.8.4 MELG’s risk director decided to steer a middle course, realising that
the process of building effective models is iterative, and that the models and
data requirements will be refined in the light of experience and lessons learned.
3.8.5 A good starting point is to collect data relating to losses and
exposure.
3.8.6 The loss database is conceptually fairly simple, as it is very like an
insurance claims database. For each event it should record information such
as date incurred, date reported, the development of the loss amount
(including initial estimate), and so on. It should also include the cause of the
loss, in a form consistent with the categorisation used in the firm-wide risk
matrix, as well as the consequence (how the loss manifested). However, there
are a number of decisions that must be made:
ö How are losses that arise from more than one cause tested? Should the
loss amount be split between causes, or should the whole amount be
allowed to appear under each cause?
ö Should data on near misses ö incidents that did not in the end result in
any monetary loss ö be collected?
ö How should a blame-free procedure for reporting both actual losses and
near misses, and avoid under-reporting be set up?
3.8.7 It is much more difficult to collect exposure data, as often there

are no commonly agreed measures of exposure. To start with, some of the
data used for risk indicators (see below) may serve as a proxy, such as
number of renewals, number of claims, and so on. Another avenue of
investigation would be the data used for activity-based costing; indeed, many
of the difficulties associated with activity-based costing will arise in
collecting exposure data for operational risks.
3.9 Risk Indicators

3.9.1 The risk director also decided that he needed to set up a
monitoring system to provide early warning of high or increased risk. He
chose to design a system of risk indicators.
3.9.2 Ideally, risk indicators should be both easy to calculate and predictive.
This is difficult to achieve. However, they can be an extremely valuable
component of the overall risk management process, for a number of reasons.
3.9.3 First, they can help with a qualitative assessment of risk. Even if
an organisation cannot yet measure operational risk quantitatively, some sort
of assessment is needed. The behaviour of risk indicators can indicate that
qualitative, subjective assessments need to be changed or updated (FSA,
2003b). Second, risk indicators can be used for all risks, not only those with
past losses. Third, risk indicators can be used to gauge the effectiveness of
systems and controls. When a risk indicator falls outside its normal range, it
indicates a possible operational issue (Hallock et al., 2001). Fourth, risk
indicators can be the basis of penalties and positive incentives that encourage
managers to operate in a way that contributes to the reduction of enterprise
wide operational risk exposures. They thus help to create a culture of risk
awareness throughout the company.
3.9.4 There are several challenges to be overcome if the use of risk
indicators is to be effective. It is often difficult to find good indicators;
insurance companies have a plethora of risks and identifying the most
appropriate is not simple. In practice, the difficulty of finding good
indicators may limit the choice.
3.9.5 Although risk indicators can help to encourage a culture of risk
awareness, they are difficult to use unless the company already has at least a
rudimentary risk management framework. The risk director understood
this, and saw the introduction of risk indicators as a good short to medium-
term goal that would help to focus risk management efforts. He realised that
he needed to set up some form of database in order to track the risk
indicators, as well as to introduce a reporting system, so that the right people
receive information that they can act upon.
3.10 Finding Risk Indicators

3.10.1 The FSA lists a number of risk indicators, including the number
of customer complaints, processing volumes, employee turnover, large
numbers of reconciling items, process or system failures, fragmented systems,
systems subject to a high degree of manual intervention, and transactions
processed outside a firm’s mainstream operations (FSA, 2003b). Some of
these may be difficult to quantify (such as fragmented systems), and others
are not very predictive (system failures).
3.10.2 The desirability of leading over lagging indicators means that
causal risk maps are useful guides when building a set of risk indicators. Risk
indicators should be based on underlying causes, or intended to expose poor
processes, rather than relying on outcomes.
3.10.3 Risk indicators can be classified into three categories:
ö Exposure-related indicators, such as the number of claims handled.
These are volume-based indicators that typically measure the throughput
of processes with the potential for operational failure. Whilst helpful
while the rate of loss remains constant, they cannot pick up changes in
the loss rate. Also, operational risk is multi-dimensional, so that an array
of such measures is required;
ö Loss-related indicators, such as the number of customer complaints.
These measure events associated with operational losses. However, they
measure outcomes so are lagging indicators, and therefore insufficient on
their own;
ö Cause-related indicators, such as staff turnover. These generally measure
factors identified as drivers for operational losses, and are therefore
leading indicators. They are often the most difficult to identify, relying
on a causal relationship between the indicator and an associated
operational loss. They are often more complex than other types of
indicators, but are the most valuable in use.
Table 3.11.1. Illustrative risk indicators
Risk indicator Category Comments
Number of Cause Requires a tracking system for internal audit. Also
unresolved internal consider using the number of internal audit issues
audit issues rated unresolved after two years.
‘severe’
Staff turnover Cause May be a lagging indicator, as it may be symptomatic
of other problems as well as leading to problems itself.
Training hours (or Cause Low numbers are bad here.
pounds spent) per
staff member
Number of staff Cause Measure separately for each staff category and type of
members who training. Consider using ratio of untrained to trained
require training staff.
Number of different Cause Inconsistencies can lead to problems, especially for
desktop computer inadequately trained staff.
configurations in
use
Hours of paid Cause May indicate that resources are stretched.
overtime per staff
member
Number of claims Exposure May be a leading indicator, as it may indicate
processed increased pressure on claims handlers. Consider using
claims processed per claims handler.
Number of Loss A lagging indicator, but nonetheless useful. Consider
complaints using ratio of complaints to claims processed.
Growth in sales Exposure Can be used to detect anomalies. May be a leading
indicator for some risks.
Budget overruns Loss Consistent overruns may indicate failures in the
budgeting process.
Number of large Exposure Indicator for possible reinsurance problems.
claims
Sizes of outsourcing Exposure If significant, may need more indicators from the
contract outsourcing supplier.
Numbers of IT Exposure Potential integration problems and over-stretch of
projects under way resource.
Percentage of Exposure Calculated separately for each category of supplier.
business given to Can be used detect anomalies and measure exposure
each supplier by to supplier failure.
volume and pound
amount
3.10.4 It is possible to base risk indicators on qualitative assessments.

For example, managers can be asked to assess the levels of various types of
risk in the areas for which they are responsible in terms of a simple numerical
scale. Over time, the emerging relationships between actual loss experience
and the judgement-based risk indicators can help managers to refine their
estimates (BCBS, 2003). A similar technique can be used to set the ranges
that are initially considered normal for risk indicators.
3.11 Risk Indicators for MELG
3.11.1 The risk director has considered a number of risk indicators for
use in MELG. Some of these are summarised in Table 3.11.1.
3.11.2 Table 3.11.1 does not include specific indicators for computer
systems risks such as viruses, security breaches and so on.
3.11.3 Many of these indicators are fairly generally applicable, and are
useful at the level of individual business units or processes as well as
company-wide. Others are specific to particular business processes or risks.
3.11.4 The risk director considered appropriate indicators for dominance
risk. He found it difficult to see how the personality of the chief executive
could be quantified in an acceptable way, and doubted whether self-
assessment would be accurate. However, he hoped to introduce self-
assessment indicators in a number of areas, such as the availability of
management information and the complexity of reinsurance arrangements.
He also planned to develop a questionnaire, for example seeking personal
views about the general approach to governance, remuneration and
performance management, culture, management and staff shareholdings,
existence of succession plans, general atmosphere and approach to upwards
and/or downwards feedback.
ª. Capital Requirements ö Stress and Scenario Testing
4.1 General Approach

4.1.1 Stress testing and scenario analysis are part of best practice in the
overall management of a non-life insurance company. Stress testing and
scenario analyses, being based on an analysis of the impact of unlikely, but
not impossible events, enable a company to gain a better understanding of
the risks that it faces under extreme conditions. Together with dynamic
financial analysis (DFA ö discussed in Section 7), they provide a basis for
estimating capital requirements.
4.1.2 Further impetus to perform this type of analysis has recently come
from the FSA (FSA, 2003b). Companies will be required at all times to
maintain overall financial resources, including capital and liquidity resources,
which are adequate to ensure that there is no significant risk that liabilities
cannot be met as they fall due. Stress testing and scenario analysis are
valuable ways of testing and demonstrating financial adequacy, particularly
when capital levels are relatively high compared to that envisaged as required
in most reasonable adverse scenarios. The FSA guidance states that major
sources of risk must be identified, and stress and scenario tests performed for
each.
4.2 Theory
4.2.1 Stress testing is the process of evaluating a number of statistically
defined possibilities to determine the most damaging combination of events,
and the loss that they would produce. Stress testing also answers the question
of how far the risk factor must go to give negative surplus (or surplus below
a certain amount) over a specified time horizon. The likelihood of the
assumptions that lead to this outcome is then assessed and compared to a
threshold level in order to determine whether they are significant or not.
4.2.2 Scenario analysis is the process of evaluating the impact of
specified scenarios on the financial position of a company. The emphasis here
is on specifying the scenarios and following through their implications.
Scenario analysis typically refers to a wide range of parameters being varied
at the same time. The scenarios could be chosen as events that are intended
to have a defined probability of occurrence, e.g. a one in 100 year event. It is
still not entirely clear if the present FSA guidelines about assessing overall
risk tolerance apply to aggregated risks or for each risk independently in
turn.
4.2.3 There are two types of events that may be considered. Historical
events are often more easily understood, and are sometimes considered to be
less arbitrary, while hypothetical events may provide a more thorough and
systematic analysis, but anticipate risk with no historical parallel.
4.2.4 The tests should be carried out at least annually, or more often,
depending on the possible impact of the risks. For example, a sudden change
in the economic outlook may prompt a company to revise the parameters of
some of its stress tests and scenario analyses. Stress tests may be added if a
company has recently been exposed to a particular sectoral concentration.
4.4 Case Study Application

4.4.1 In order to carry out stress and scenario tests, MELG’s risk
director considered the specific circumstances of MELG. It was decided that
reviewing the operational risk exposures under the categories ö
administration, compliance, event, fraud, governance, strategic and
technology risks ö would capture the dynamics of the business and give a
reasonable set of realistic possible events. For each of these sources of
operational risk, the risk director carried out appropriate, separate tests.
4.4.2 Administration risk. In order to set up stress tests and scenario
analyses for administration risk, the risk director looked at the past
administrative deficiencies, taking account of both the actual losses recorded
in the exception reports and the results of the Delphi analysis (see {2.7.8).
Other relevant factors include the nature and extent of centralised and
decentralised functions and the segregation of duties between staff. The risk
director was satisfied that there was an adequate segregation of duties
between underwriting, claims and payments divisions in terms of acceptance,
authorisation and payments. He was also satisfied that there was sufficient
interaction between the front, middle and back offices in terms of financial
control and risk management.
4.4.3 The risk director considered that the administration risks were
reasonably well understood and predictable, and was able to suggest a single
point estimate of the capital required to cover these risks, given the group
risk committee’s previously agreed risk tolerance or risk appetite.
4.4.4 Compliance risk. MELG’s risk director considered the principal
compliance risk to arise from the risk of non-adherence to legislative and
internal company requirements. An investigation into compliance over the
last five years found no history of non-compliance with policy and control
systems, nor had there been any reported areas of non-compliance with
legislation or other requirements.
4.4.5 However, there are regulatory reforms due in the next five years,
and therefore the risk director felt it prudent to assume an increase in the
expense ratio over this time period. Given the relatively low level of
uncertainty, the risk director was again able to come up with a single point
estimate of the capital required to cover these risks.
4.4.6 Event risk. Event risk is the risk associated with the potential
impact of significant events on the company’s operations. The risks are those
that are directly related to the products and services offered, and not to
events impacting other business risk areas, e.g. non-life insurance business,
credit exposure or market risk. MELG’s risk director concluded that no
additional capital was required for this type of risk.
4.4.7 Fraud risk. In assessing fraud risk, MELG’s risk director was able
to use the major incident that involved fraudulent activity in relation to an
external supplier (see {2.7.3), which resulted in a loss of »5m. After allowing
for the improvements in controls that resulted from this incident, the
scenario analysis produced a range of estimates for the amount of capital
required to cover future fraud.
4.4.8 Governance risk. Governance risk is the risk that the Board and/or
senior management will not perform their respective roles effectively.
MELG’s risk director investigated the existence and level of directors and
officers insurance in place, and compared it to the known incidence of claims
of this type.
4.4.9 The current level of corporate governance was considered, and an
assessment made of the likelihood that its shortcomings might result in the
Board and/or senior management not adequately undertaking their roles. In
addition, costs of altering or strengthening the current Board structure were
analysed. Given the uncertainties involved, the risk director was unable to
come up with a single point estimate of the capital required, and instead used
a range of estimates.
4.4.10 Strategic risk. Strategic risk arises from an inability to implement
appropriate business plans and strategies, make decisions, allocate resources
or adapt to changes in the business environment.
4.4.11 MELG’s risk director assessed the prudence and appropriateness
of the future business strategy in the context of the competitive and economic
environment. In particular, the assumptions, forecasting and projections
were assessed, considering the possibility of a fundamental market change
due to higher numbers of competitors, changes in sales channels, new forms
of insurance or changes in legislation. This review included whether the
reinsurance programme is appropriate for the risks selected and whether it
adequately takes account of the underwriting and business plans of the
company in general. It was considered that the likelihood of a fundamental
strategic shift was too remote to include within the scenario, given the
maturity of the market in which MELG operates.
4.4.12 On the other hand, the risk director was concerned about the
effect of the losses caused by the decision to move to a 70/30 personal/
commercial business mix. Although he had, in the end, decided not to treat
this as an operational loss (see {2.9.7), he thought it worthwhile to
investigate its effect.
4.4.13 To determine the loss size, he first selected a time horizon over
which to estimate the cumulative losses. It seemed that the business plans and
strategy ought at least to be reviewable on an annual basis, but that, in this
case, it might take more than a year to achieve the required correction to
business mix. Hence, he estimated the effect on bottom line of having written
a 70/30 mix rather than the preferred 90/10 for the third and fourth years
from implementation ö that is an estimate for 2003 and 2004. To do this he
compared the results from the DFA model with implicit allowance for
operational risk on a 70/30 mix with those on a 90/10 split.
4.4.14 Technology risk. MELG’s risk director considered the risk of
error or failure associated with the technological aspects (IT systems) of
MELG’s operations, including both hardware and software risk.
4.4.15 The risk director considered the past reliability and future
functionality of the information systems to be adequate. However, past
software projects have resulted in significant cost overruns (see {2.7.9), and,
although there were no definite plans to replace systems or make major
modifications future problems could not be ruled out.
4.4.16 Plans for business continuity management and disaster recovery
are reviewed regularly and tested quarterly. There is a back-up site with full
recovery capabilities. When performing the scenario analysis, the risk
director allowed for the costs associated with utilising the site and the
associated business interruption insurance. The risk director was able to
estimate a range for the capital required to cover these risks.
4.5 Overall Assessment

4.5.1 After assessing each risk area individually, MELG’s risk director
considered the capital that was estimated to be absorbed under each scenario.
He considered how many of these scenarios might reasonably occur within
a period and the extent to which capital could be replaced within that period.
The analysis took into account scenarios which might reasonably be linked,
the difficulty with which capital might be replaced if the scenarios occurred,
and the changes in strategy which might need to be adopted if the scenarios
occurred.
4.5.2 Finally, the risk director estimated the range of capital that would
be absorbed by the worst realistic combination of circumstances that might
arise. The group risk committee noted that this was useful, but did not give a
quantified probabilistic assessment, which made it difficult to compare with
their agreed risk appetite.
4.5.3 In effect, he was using his best business judgement to determine
how many losses might occur in a poor, but not impossibly bad, year.
4.5.4 Readers should also recall that this illustration uses only a few risk
categories compared to the full range that would need examining in real life.
4.6 Comments and Observations

4.6.1 Stress testing and scenario analysis are very practical and easy to
understand methods. They require significant judgement and experience, and
may possibly need to incorporate external advice.
4.6.2 In the case of a relatively well capitalised business, these relatively
simple approaches can help provide significant evidence about capital
adequacy.
4.6.3 The method does not give a mathematically-based view on
probabilities of different scenarios ö it relies on the modeller’s assessment of
likelihood, based on discussion with, and the input of, experts, as
appropriate.
ä. Frequency and Severity Analysis (including EVT)

5.1.1 The frequency and severity analysis approach is already well
documented in actuarial literature (e.g. IoA Claims Reserving Manual). The
basic principle is to use separate statistical models to generate the number of
losses and the size of each loss. The parameters for the models are derived
by fitting distributions to the actual historical loss data. Standard Monte
Carlo techniques can then be used to simulate the number of events and a
loss size for each event.
5.1.2 Distributions that are often used for the frequency include Poisson,
negative binomial and binomial. Lognormal, weibull, or gamma are common
choices for the severity distribution.
5.1.3 Typically, the first step is to modify the actual historical data to
allow for future likely changes in claim numbers, to take account of known
changes in controls and procedures. Then a number of curves from different
families are fitted to the data. The criteria for choosing the curve with the
best fit depend on the circumstances: for example, when considering the
value of an excess of loss reinsurance contract, the modeller might be more
concerned over the fit of the model for large than for small claims. Tools that
can be used at this stage include standard maximum likelihood and other
goodness of fit tests, such as plotting the actual and modelled distributions
and considering the fit at decile or percentile points in the most relevant part
of the curve.
5.1.4 Operational losses tend to have even more skew distributions than
actuaries are used to, with extremely low frequency and high impact losses
being very significant.
5.1.5 Extreme value theory (EVT) is a tool that has its roots in the
physical sciences, and is increasingly being applied in insurance (Embrechts
et al., 1977). As its name suggests, it focuses on extreme values rather than
on measures of central tendency, such as the mean. Traditional statistical
techniques are inaccurate when estimating the values of larger losses, because
of their emphasis on the area round the mean of the distribution. EVT, on
the other hand, ignores the majority of the underlying loss data in order to
provide better estimates in the tails of the distribution.
5.1.6 EVT can be seen as simply another family of distributions that can
be fitted to either severity data or to total annual losses (combining the
frequency and severity distributions into a single distribution). The basic idea
is to pick a threshold for large losses, and then to use a generalised pareto
distribution (GPD) to determine the severity of a loss given that it exceeds
the threshold.
5.2 Theory
5.2.1 As a reminder, EVT uses the cumulative distribution function
1 ÿ lð1 þ xðx ÿ uÞ=sÞÿ1=x , where u is large, and is the threshold size above
which the distribution holds, and l ¼ PrðX 5 uÞ, x and s are shape and scale
parameters.
5.2.2 A simple approach to parameterising the extreme value
distribution starts with the determination of u, the loss size above which the
extreme value distribution is assumed to be appropriate. This can be done by
plotting the mean excess above the threshold against the threshold value.
The point at which this graph becomes linear can be taken to be u. l is the
number of losses in excess of this threshold divided by the total number of
losses.
5.2.3 The shape and scale parameters x and s are then determined by
maximising the logPlikelihood function for the extreme value distribution
ðÿ log s ÿ ð1=x þ 1Þ logð1 þ xðxi ÿ uÞ=sÞÞ, for i ¼ 1 to r, the number of
observations larger than u).
5.2.4 This then defines the distribution for losses above u in size (or
alternatively for losses where PrðX 4 uÞ > l). For losses below u in size,
normal curve fitting approaches can be used to determine a distribution, and
this can be scaled so that at size u the two distributions meet smoothly.
5.3 Case Study Applications
5.3.1 As MELG’s risk director started to model the operational losses,
he realised that they fell neatly into three groups:
ö claims leakage/fraud losses, with high frequency and low severity;
ö system development overspend losses, with lower frequency and medium
severity; and
ö miscellaneous losses with low frequency and high severity.
5.3.2 The risk director decided to reflect these differing characteristics in

the model. By modelling the three sets of losses separately the homogeneity
of the losses was improved.
5.3.3 Neither the claims leakage nor systems overspend losses showed
any unusual characteristics, so standard statistical curve fitting was deemed
to be appropriate.
5.3.4 The risk director decided to model the absolute number of losses
rather than the frequency, as there were no good exposure data available
(that is no suitable information to form a denominator for calculating
frequencies) for more than the last year or two. The data used were as in
Tables 2.9.9 and 2.9.10. Details of the fitted distributions and parameters are
shown in Appendix C.
5.3.5 The set of low frequency/high severity losses required more
consideration on two grounds: first, whether all of them should be included;
and second, how to model them. As discussed in {2.9.7, the losses caused by
the imposed decision to move to a 70/30 personal/commercial business mix
were, in the end, omitted.
5.3.6 The risk director decided to model the loss numbers using the
same Poisson as for the other two groups of losses. However, he noted that
this might not be entirely appropriate, as the mean and variance of the actual
annual numbers of losses were rather different. On the other hand, he
reasoned that a start had to be made somewhere.
5.3.7 In order to model the severity, the risk director decided to
consider three standard distributions and the extreme value distribution to
begin with. MELG had not yet acquired proprietary curve fitting software,
so the risk director chose distributions that are simple to apply using Excel.
Three distributions that are commonly used to model insurance losses and
that meet this criterion are the lognormal, weibull and gamma distributions.
The parameters were estimated using the data, and then the fit examined by
graphing the cumulative distributions for the actual and model data and
looking at the Q-Q plots of actual versus model for each distribution. A chi-
squared test was also used to look at the appropriateness of the three
alternatives.
5.3.8 An extreme value distribution was then fitted to the set of low
frequency/high severity losses. This was done in a fairly pragmatic fashion.
A plot of mean excess losses was used to determine an appropriate threshold
Table 5.4.2. Summary expected losses at various percentile points
Poisson/ Poisson/ Poisson/ Poisson/
EVT lognormal weibull gamma
Expected annual loss (»m) 100.6 52.8 44.1 44.8
99.0th percentile 612.1 253.2 181.6 183.5
99.6th percentile 722.4 311.0 209.0 208.4
99.8th percentile 855.4 351.6 221.1 224.7
99.9th percentile 911.1 429.3 237.4 238.5
value for the distribution ö that is the loss size (or its corresponding
cumulative probability) above which the extreme value distribution applies.
Then maximum likelihood was used to find the values of the two remaining
parameters. The graph is shown in Appendix C.
5.3.9 The risk director decided to use EVT for the loss size distribution
for the low frequency high severity losses. He then used the frequency and
severity distributions for all three groups to model annual losses
stochastically for a large number of simulations using @Risk.
5.4 Results
5.4.1 The risk director found that visual inspection of the graphs
suggested that the weibull or lognormal better represented the loss experience
to date than the gamma, but no more than that. A simple chi-squared test
suggested that the lognormal was the better fit.
5.4.2 Table 5.4.2 shows the expected loss and a selection of the higher
percentile points for EVT and the standard distributions.

5.5.1 The main observation is that the values in the table vary
considerably. The variation is largely a result of the differing shapes of the
tails of the distributions, together with the small number of large losses.
There is less variability in the expected values, which is reasonable, given the
larger volume of data influencing their levels. The higher percentiles, which
have larger variability, are very important for the purposes of determining
the levels of capital required.
5.5.2 Although, in this case, the risk director felt that the choice of u,
the threshold loss size, was reasonably obvious from the mean excess plot, it
was clear that this might not always be true. The smaller the dataset used, the
more subjective the choice of u could become, as any linear relationship
becomes less obvious and more a matter of interpretation.
5.5.3 The risk director used Excel to perform all the calculations
required. This was a little time consuming to set up initially, although
thereafter could be easily replicated. He decided to investigate statistical
software packages so that he could do this more efficiently and more
robustly.
5.5.4 The risk director was interested to see that the extreme value
approach actually gave less extreme outcomes at the less extreme percentiles.
5.5.5 Overall, the risk director recognised that the analysis was
somewhat simplistic and failed to take into account a number of important
factors.
5.5.6 The decision of what losses to include failed to allow for the fact
that the causes of past losses may have been partly or wholly controlled
through changes in internal processes and procedures. Hence, it might be
argued that these losses would not be repeated in the future, and so should
not be included in the set of losses to which a model is to be fitted.
5.5.7 On the other hand, it is also likely that in future losses might
occur as a result of hitherto unencountered or unimagined underlying causes,
let alone the almost inevitable consequences of future strategic decisions.
For example, there might be future losses due to a brand new process using
technology that is not currently available. It is at least possible that these
future losses could be of broadly similar size to the losses that would be
omitted due to better controls. Given the uncertainties involved, the risk
director felt that the approach of simply including all the known losses was a
reasonable one.
5.5.8 There may also be types of losses that are already known about or
have been experienced by others in the industry, but that a particular
company has not yet experienced. For instance, MELG has suffered no cases
of physical damage losses (e.g. fire or flood damage to the company’s
offices). One approach to this problem is to use external data, or seek the
opinion of external experts on the frequency and severity distributions of
such losses, and ensure that the fitted model takes this into account. In the
case of external data, an important consideration is whether the organisation
suffering the loss is of a comparable size and type to the one for which the
modelling is being performed, and whether any adjustments are required to
reflect size or operational differentials. In practice, such adjustments are
extremely difficult: how would external loss data based on the experience of
banks be adjusted to be appropriate to a medium-sized general insurer, for
instance?
5.5.9 In addition, it may well be that some of the past losses would, if
they happened today, be of significantly different size. This could be the
result of, for example, claims inflation, expense inflation or changes in
volumes of business. In such circumstances, it would seem appropriate to
adjust the historic losses to allow for these effects so that they reflect the
position in the period of projection.
5.5.10 It therefore becomes important to develop appropriate measure(s)
of exposure for each type of loss. This could vary significantly ö for losses
caused by a problem with a particular transactional process, it might ideally
be the number of such transactions. For losses caused by an error in the
pricing of a particular product, it might be the number of policies or the
premium income for the product. For losses due to theft of moveable
equipment, it might be the number of units and their value. Note that the
measure of exposure for frequency and for severity may well be different. For
example, in the case of employment practice losses, frequency may be
related to the number of employees, but severity to average annual salary or
average court/tribunal awards. The next stage is to consider how the value of
the exposure measure has changed since the loss was incurred, and adjust
the historic loss to the same extent.
5.5.11 When modelling operational risk losses, as with any situation
where data are scarce, the scope for parameter error is relatively high. It will
be instructive to look at the standard error for the parameter and consider
the impact of changing the parameters by, say, one standard error, on the
outcome. Where capital requirements are set to reflect a relatively high level
of risk aversion, this could make a very significant difference to the amount
of capital that a model might suggest should be set aside in respect of
operational risk.
5.5.12 For the same reasons the choice of distribution is unlikely to be
clear cut, and so looking at the effects of changing the distribution would
also be instructive.
5.5.13 In MELG, the low frequency/high severity losses cover asset
failures, loss of key personnel, supplier fraud, failure to check contract
wordings and communication failure, expense overruns and poor project
evaluation and management. There are no identified examples of physical
damage to property, employment practice problems, health and safety
failures, mis-pricing or under-reserving. After discussion, it appeared to the
risk director that there were few obvious exposure measures that might shed
light on whether the historic loss amounts or numbers should be adjusted to
reflect the organisation’s current position. Hence, the risk director decided to
model the losses unadjusted, although he met some opposition to his
inclusion of the loan default loss.
5.5.14 However, there are very few very large events and some
clustering of loss sizes in the »20m to »50m range. The risk director therefore
sought views from a management consultancy on their perceptions of
MELG’s potential exposure to very large losses and to losses of types not yet
reflected in the internal data ö in particular to mispricing, under-reserving
and physical damage on a scale likely to disrupt business ö given their
knowledge of the financial services market. He incorporated their views into
the tail of the severity and frequency distributions by ensuring that the
cumulative distributions passed through certain points.
5.5.15 The risk director was at least comforted that some traditional
actuarial methods were found to be readily applicable to modelling operational
risk losses.
å. Causal Modelling and Bayesian Methods

6.1.1 In this section we discuss causal risk mapping as a technique for
analysing and understanding risks. The technique can be extended to give a
quantitative assessment of risk by making use of Bayesian probability theory.
The resulting Bayesian networks can be used to gauge the effect of possible
risk control strategies, as well as to estimate risk probabilities. They are a
useful tool during the understanding and decision stages of the risk
management control cycle.
6.1.2 In this section we have illustrated a simple example of a Bayesian
causal network. We then illustrate how this technique may be applied in a
simple way to the case study. See our previous paper (GIRO, 2003) and the
texts that have been written on the subject (such as King, 2001) for more
information on the background and theory of these models.
6.2 Theory
6.2.1 Bayesian networks are useful when modelling situations in which
causality plays a role, but where our understanding is incomplete (Charniak,
1991). Probabilistic reasoning is a powerful tool under these circumstances.
6.2.2 A simple example, based on one taken from Pearl (1988), illustrates
the approach. Normally, the sound of a burglar alarm going off means that
there has been a burglary. However, it might also be triggered by an
earthquake. If there is an earthquake, one may hear a radio announcement to
the effect that an earthquake has occurred. We can use conditional
probabilities to model the situation as follows (assuming one lives in a high
crime neighbourhood in California):
ö There is a 95% chance that an attempted burglary will trigger the alarm:
Pðalarmjburglary) ¼ 0:95.
ö If there is an earthquake, there is a 20% chance that the alarm will be
triggered: Pðalarmjearthquake) ¼ 0:20.
ö If there is an earthquake, there is a 40% chance that one will hear an
announcement on the news: P(announcementjearthquake) ¼ 0:40.
ö There is a 1% chance that a given house will be burgled on a given
night: P(burglary) ¼ 0:01.
ö There is a 1% chance that there will be an earthquake on a given night:
P(earthquake) ¼ 0:01.
6.2.3 Figure 6.2.3 shows that the probability of hearing the burglar
alarm on any given night is 1.15%.
6.2.4 We can use the same network to analyse the situation in the case
that one actually hears the burglar alarm. Figure 6.2.4 shows that if the
burglar alarm is heard, but no radio announcement, then the probability that
there was a burglary is 88%.
Figure 6.2.3. Probability of hearing the burglar alarm
Figure 6.2.4. Probability of a burglary if the alarm is heard, but no radio

announcement
6.2.5 On the other hand, if both the alarm and a radio announcement of
an earthquake are heard, the probability of a burglary falls to under 5%, as
shown in Figure 6.2.5.
6.2.6 The network provides a clear visual representation of the
connections, and there are reasonably efficient algorithms for performing the
probabilistic inference (Pearl, 1988; Charniak, 1991; Netica, 1997).
6.2.7 The prior and conditional probabilities that are used in the
network need not be very precise in order for useful information to be
gained. Moreover, they can be adjusted as more experience of the risks and
the relationships of their causes and effects is acquired. In this way the model
can ‘learn’ from the data that are collected and fed into it in a Bayesian
fashion. The model starts with some initial beliefs about a process with little
data and ends up reflecting the reality of the information that is collected
over time.
Figure 6.2.5. Probability of a burglary if the alarm is heard and there is a

radio announcement
6.3 Case Study Application

6.3.1 The first step in applying this methodology is to set out a risk map
of the process in question. Having decided to use Bayesian networks to help
understand MELG’s overall risk vulnerability, MELG’s risk director started
with an enterprise wide risk map set out by McDonnell (McDonnell, 2002),
shown in Figure 6.3.1.
6.3.2 In this high level model policyholder harm results from financial
outcomes including underwriting or other revenue losses, balance sheet losses
Figure 6.3.1. Enterprise wide risk map

or a loss of reputation or goodwill. Such losses are brought about as the
results of risk decisions, which may relate to underwriting, expenses,
reinsurance, investments or more general business risk. For operational risk,
incorrect decisions are attributable to inadequate or failed internal processes,
people or systems. Behind the failed processes, risk decisions and financial
outcomes are underlying or trigger causes, which may be internal
(management, governance or ownership related) or external (wider changes,
as well as event or market specific changes).
6.3.3 It is worth noting that this cause-effect risk map has an adaptive
feedback control loop, from risk decisions to financial outcomes to incorrect
evaluation of financial outcomes, and back to risk decisions. This requires
some iteration in the modelling.
6.3.4 An example of using a Bayesian causal approach at this enterprise
level is given in our GIRO paper (GIRO, 2003). For the purposes of this
paper, the risk director has decided to drill down further and use risk
mapping by cause and effect to investigate what can be learnt about the risks
faced by the company from the known losses.
6.3.5 The purpose of this form of risk mapping is to document and
make explicit the causal chain that has brought about a particular risk
outcome, and the impact of the outcome on the business.
6.3.6 The risk director has constructed a causal risk map for each of the
losses experienced by MELG. The first is the loss incurred as a result of the
launch of direct writing and is displayed in Figure 6.3.8. Similar risk maps
for the remaining losses in the case study are shown in Appendix D.
6.3.7 The cost incurred in launching the direct writing operation was in
the region of »70m, partly due to expense overruns and partly to lower than
expected business growth. The risk decisions involved were faulty because of
Reputational damage
Figure 6.3.8. Risk map

inadequate information and over-optimistic assumptions. Process failures
caused poor availability of information on customers and expenses. External
drivers included the turmoil in general insurance distribution generally, and
particularly the gain in overall market share by direct distribution against
other channels. Internal drivers included management pursuing a ‘me-too’
strategy and also the clash of cultures between the direct and the traditional
areas of the business.
6.3.8 The resulting risk map is shown in Figure 6.3.8.
6.4 Case Study Results

6.4.1 MELG’s risk director has converted this risk map into a Bayesian
network appropriate for a causal model. The loss is applied and interpreted
into the network as follows:
ö failed processes and internal drivers: interpreted as a ‘weak’ score for
‘failed internal processes’ and a bias towards ‘weak’ for internal
‘governance/control’;
ö faulty risk decisions: interpreted as a stronger relationship between
‘failed internal processes’ and the risk decision nodes of ‘investment/
ALM risk’, ‘reinsurance risk’, ‘insurance underwriting’, ‘expense risk’
and ‘business risk’, which biases these nodes towards ‘weak’; and
ö external drivers: scenarios were run with an unbiased view of the
external causes as well as with ‘weak’ external drivers (‘weak’ interpreted
here as external drivers that generate a disadvantageous situation for
the company).
6.4.2 The risk director then investigated how different levels of ‘outcome
evaluation’ impacted the outcome of the model. The outcome is given as a
measure of ‘policyholder harm’, whereby ‘strong’ is interpreted as little or no
negative impact on policyholders, and ‘disaster’ is interpreted as significant
negative impact on policyholders.
6.4.3 An example of the simple Bayesian model used is shown in
Figure 6.4.3, and the results of the various scenarios are summarised in
Table 6.4.3.
6.4.4 For the interested reader, the other Bayesian network model images
representing the scenarios in Table 6.4.3 can be found in Appendix D.
6.4.5 We might question, from the large number of operational incidents
and the varied misfortunes that have beset MELG in recent years, whether
the senior management team was incompetent rather than unlucky. There
were clearly also corporate governance and group controller issues and
concerns. It does not seem unreasonable to conclude that the initial belief
network should show a ‘weak’ score for failed internal processes.
6.4.6 Nevertheless, it can be seen that ‘weak’ failed internal processes
need not lead to a terminal prognosis for MELG plc. Much depends on the
‘downstream’ risk enterprise wide management issues, such as the ‘outcome
Figure 6.4.3. Simple Bayesian model
Table 6.4.3. Results from simple Bayesian model

Failed internal External Outcome Individual Prob (policyholder
processes trigger causes evaluation risks harm ¼ disaster)
Weak ^ ^ ^ 68.9%
Weak Weak ^ ^ 92.0%
Weak Strong ^ ^ 43.8%
Weak ^ Weak ^ 83.6%
Weak ^ Strong ^ 35.6%
Weak Weak Weak ^ 95.5%
Weak ^ ^ Weak 99.8%
Note: ‘^’ indicates default value
evaluation’ score, which represents the risk that financial outcomes are
incorrectly evaluated.
6.4.7 The financial prognosis for MELG plc is less severe if it can be
shown to have ‘strong’ management competencies in respect of ‘outcome
evaluation’. At least the senior management team would then be able to read
the early warning signals of the impending ‘policyholder harm’, rather than
be confused by the noise from a potentially inadequate ‘outcome evaluation’
process.
6.4.8 Indeed, it can be seen from the results that improving the ‘outcome
evaluation’ process from a ‘weak’ to a ‘strong’ score reduces the probability
of disastrous policyholder harm from 83.6% to 35.6%. It may be that this is
the first area where management should direct its attention in order to reduce
the operational risk exposure of the company.
6.4.9 The overall conclusion of MELG’s risk director was that the
upstream issues of corporate governance, a resilient senior management team
and sound internal business practices, processes and controls were necessary
to mitigate the potential for downstream operational loss incidents. This
ideal situation may take some time to achieve if it is not already working.
6.4.10 In the short term, however, the risk director could see that
improvements to MELG’s processes for handling and evaluating incidents
that do occur would mitigate significantly the potential loss from these
incidents. In this way, the risk director could start taking steps to ensure that
the absence of sound upstream practices does not lead to an accident prone
senior management team and the consequent risk of policyholder harm.

6.5.1 The causal risk maps provide a good structure for analysing the
known losses. First, they make the distinction between risk events and risk
outcomes clear. Second, they help to clarify the management decisions that
resulted in an adverse outcome. Management decisions may be driven by the
failure of processes, systems or people, by external events or by internal
factors or, more probably, by a combination of all three. The risk maps thus
make it evident that operational losses involve complex chains of cause and
effect.
6.5.2 Causal risk maps can also be used to analyse potential losses. The
analysis of both potential and actual losses helps to make the underlying
causes of operational risk clear, and assists in deciding what control or
mitigation steps to take.
6.5.3 Bayesian networks, in conjunction with causal risk mapping, can
be a valuable tool in the risk director’s armoury. Their calibration, however,
can be tricky, and requires a heavy investment of resources.
6.5.4 The most productive way of using Bayesian networks is in
conjunction with risk indicators. The risk indicators can be used to help
calibrate the model, which is used to aid the deeper understanding of the
risks. In turn, the model can help to suggest effective risk indicators for
ongoing risk monitoring.
6.5.5 Regarding the work performed, the next stage is to gather some
more data to put into the model and revise our initial beliefs about the
system in the light of this information. Intuitively, this model, with explicit
relationships and the ability to take initial beliefs and revise them in the light
of experience, is attractive for modelling operational risk. We need to do
some more work in this area to see the true potential benefits.
æ. DFA and Overall Risk Modelling
7.1 There are two methods of allowing for operational risk in a dynamic
financial analysis (DFA) model. The first method is to construct a model
using the actual financials, incorporating all the operational loss events.
Operational risk is included implicitly in this method. The second method is
to remove all the operational risk losses from the financial history and
construct a DFA model that models everything except operational risk. The
operational losses can then be modelled separately (maybe using similar
methods to those described in Section 5), and added back to the model,
which then allows for operational risk explicitly.
7.2 We suggest that, the second approach, explicitly modelling
operational risk, is preferred, and this is illustrated below.

7.3.1 The first stage of the modelling exercise is to construct a DFA
model of MELG plc using its actual results, which implicitly incorporate
all past operational risk loss events. As the focus of this paper is about
operational risk, we do not describe the DFA technique in any detail.
Readers may refer to various Casualty Actuarial Society papers
(www.cascat.org/research/dfa) or a recent paper from Converium Re (2003).
The model and software tool used for the illustrations below was similar to
that used in producing a paper entitled ‘Calibration of the general insurance
risk based capital model’ for the FSA, dated 25 July 2003.
7.3.2 The next stage is to consider which operational risk events will be
modelled explicitly. This will require judgement about the likely modelled
frequency/impact of different loss categories on the projected accounts. The
historic data are then recast, with these operational risk events removed.
7.3.3 A second DFA model is then produced which effectively considers
how the company would look if it could manage itself so that the chance of
such events (i.e. those explicitly modelled) was zero (assuming, for the sake of
clarity, that there is no need to add in any of the probable costs of
management/mitigation). Even using this na|« ve assumption gives an
interesting insight into the quantum and variance resulting from the selected
operational risk events.
7.3.4 The selected operational risk losses are then modelled separately
using probability distributions and parameters that reflect past data,
including wider market insights if relevant. These are then combined into the
second model, to produce one where the explicit effect of the selected loss
events can be seen.
7.3.5 In effect, we now have one set of selected distributions and
parameters feeding two different financial models, so that the differences can
be studied in a meaningful way. A further task is to consider whether the
assumptions based on past data are appropriate for modelling the future. It
may be that the risk indicators and recent changes to processes or systems of
control suggest either a degradation or improvement in likely future
experience. The quantified impact of revised assumptions can be explored
under different scenarios, and, in effect, the implicit and explicit model
outcomes can then be compared.
7.3.6 The modelling can be built to explicitly allow for forms of
management/mitigation other than solvency capital.
7.3.7 This modelling approach integrates all risks, and is a useful tool
for assessing the total impact of credit, market, liquidity, insurance and
group risk, as well as operational risk.
7.4 General Theory (More Detail about the DFA Models)

7.4.1 Computer modelling techniques have resulted in a wide range of
simulation methods, based on Monte-Carlo (time series analysis or use of
random numbers to generate a number of outcomes that simulate assumed
underlying probability distributions). When modelling operational losses, the
selection of assumed distributions for event frequency/severity will depend on
the nature of the loss category. It may be appropriate to consider whether the
loss category is, in effect, an attritional loss (high frequency/relatively small
variation), or moderate frequency/medium sized (either skewed or symmetrical
about the mean) or low frequency/high impact with extreme skewness.
7.4.2 The illustrated loss events modelled were the fraud issues
(affecting claims), the systems project issues and aspects of major decisions
(affecting claims and expenses), and the forced strategic asset mix matter
(affecting investment income and asset values). As already discussed (see
{2.9.7), it was decided not to include the impact of the strategic risk resulting
from the determination to grow commercial lines against the will of the
U.K. management team.
7.4.3 A further consideration is whether operational loss events should
be considered to be independent or correlated with other losses. For example,
should external economic assumptions affect fraud levels as well as asset
values, claims frequency and/or severity and expenses? As another example,
should management change be modelled as a driver of insurance result
volatility as well as of large operational losses following strategic decisions?
DFA techniques enable the modeller to establish driver assumptions and
correlations.
7.4.4 The underlying business assumptions in the illustration are based
on the combined FSA return data of the major household and employers
liability companies, as discussed in Section 2 (the case study). A proprietary
market consistent time series investment model is used.
7.4.5 We hope that readers will focus on the underlying approach and
type of analyses rather than on the details and results of the model.
7.4.6 The DFA approach makes combining different risk distributions
relatively easy. Although our illustrations assume no correlations, the
approach also allows the modeller to experiment with correlation coefficients
until the modelled outcomes look fully realistic. It also facilitates
investigating linkages ö when one thing goes wrong, does everything else go
wrong at the same time?
7.4.7 Once parameterisation and model construction was complete,
1,000 simulations were carried out.
7.5 Case Study ö Applications and Results

7.5.1 The explicitly modelled operational risk losses consisted of a set of
high frequency/low severity claims leakage/fraud losses, a set of lower
frequency/medium severity system development overspend losses and a set of
more disparate low frequency/high severity losses. We reflected the different
characteristics of these events in our model using Poisson distributions for
frequency, and normal, gamma and pareto for amounts ö in order to
improve the model fit for roughly homogenous groupings of losses. Appendix
E shows summary results.
7.5.2 As indicated, we appreciate that the exclusion of some of the latter
group of losses (as they are judged strategic rather than operational) may be
open to discussion. The risk director cannot ignore such events, and has to
make a judgement about how to treat them, as they are both real and have a
material impact.
7.5.3 For ease, we also assumed that the modelled events were
independent and uncorrelated with other risks.
7.5.4 In practice there would be additional considerations around the set
of losses being modelled and the appropriateness of their inclusion. It may be
that the causes of these past losses have been mitigated partly or wholly
through changes in internal processes and procedures. Hence, it might be
argued that these losses would not be repeated in the future, and so should
not be included in the set of losses to which a model is to be fitted.
7.5.5 On the other hand, it is also likely that in future losses with
similar statistical characteristics might occur as a result of rather different
underlying causes ö perhaps causes not even contemplated or possible at the
moment ö but which may well be of broadly similar sizes to those losses
from the past now felt unlikely to be repeated; for example, losses as the
result of a brand new process using technology that is not currently available.
This may be a reason for not discounting losses where the causes have been
mitigated. In effect, we have taken this approach for the case study.
7.5.6 Many other considerations are similar to those already discussed
in {{5.5.8 to 5.5.14, and not repeated here.
7.5.7 The risk director’s first observation was that the base case
mean projection was more cautious than the management’s deterministic
corporate plan. This would require more discussion to determine whether
the model assumptions, based on historic performance, could justifiably
be changed to bring them in line with the corporate plan. On the other
hand, it was possible that the management assumptions might prove
unrealistic ö over optimistic planning being another form of operational
risk, and for the time being the risk director decided to use his
projections.
7.5.8 To compare the models and scenarios various indicators are used
ö the chance of ruin in five years, various other percentile solvency levels in
five years and the difference in capital required between scenarios based on
the year one (2003) ruin probability at the 99.5 percentile point, as shown in
Table 7.5.8.
7.5.9 The risk director observed that using the implicit approach led to
a model where the starting capital (»857m or 50.6% solvency ratio) rose to a
mean level of »1.86bn or 72.5%, but also gave approximately a 5% chance
of insolvency by the end of year five (2007). He found that the company
required »950m to reduce the chance of insolvency within five years to 2.5%.
He found it difficult to interpret this. He also calculated that a starting
capital of »450m would be sufficient to meet the 99.5% requirement of
remaining solvent over year one (2003).
7.5.10 He recognised that these models incorporated all modelled forms
of risk, including insurance and market risks. He decided to concentrate on
the differences between scenarios in order to better understand operational
risk.
7.5.11 The scenario where operational risk was eliminated led to a mean
level of net assets at the end of year five of »2.1bn or 82.6% solvency ratio,
with roughly a 1.5% chance of insolvency by the end of year five. This
encouraged him to see that operational risk control, whilst not necessarily as
important as insurance or market risk, was significant, and that its proper
management could make a difference. The starting capital to meet the 99.5%
chance of remaining solvent in year one (2003) fell to »370m.
Table 7.5.8. Illustrative figures to accompany {7.5.8

2007 mean 2007 five 2.5 Probability Capital to
solvency median percentile percentile that net achieve year
ratio solvency solvency solvency assets >0 one ruin
% ratio ratio end ratio end over probability of
% 2007 % 2007 % five years 0.5%
Implicit OR 72.5 77.7 0.5 ÿ20 95% »450m
No OR 82.6 84.2 20.4 1 98.5% »370m
Explicit OR 79.2 82.7 19.1 5 98% »440m
OR with improved 81.4 85.5 21.7 10 99% »380m
procedures
7.5.12 He then looked at the model including operational risk on an
explicit basis. This showed a mean level of net assets at the end of year five of
»2.03bn or 79.2% solvency, with marginally less than a 2% chance of
insolvency. The required capital to meet the year one 99.5% criteria was
»440m, roughly in line with the implicit model. This showed that, while the
magnitude of outcome was similar to that in the implicit model, the effect of
assuming that operational losses were not correlated with underwriting or
market movements reduced the chance of ruin.
7.5.13 His final scenario assumed that systems of control and other
changes reduced the chance of large loss events by one third ö this produced
calculated outcomes nearer the ‘no operational loss’ ones (e.g. the projected
mean net assets of »2.08bn, 81.4% solvency and only 1% chance of ruin, the
capital required to meet the year one 99.5% level was reduced to »380m, and
the chance of insolvency in five years was less, and he considered that this
was strong enough evidence to drive home the need for improved systems of
control around executive decision taking. (Note the 1% chance of ruin over
five years is less than that modelled with no operational risk ö this apparent
inconsistency would need further investigation, bearing in mind that it is
dependent on the very small number of adverse scenarios produced with only
1,000 runs.)

7.6.1 The approach enables the modeller to obtain an overview of the
consequences of combining all loss categories ö so called enterprise risk, and
an understanding of the probability distribution of outcomes. It is clearly
highly dependant on the quality of available data and modelling
assumptions.
7.6.2 Many points apply to all DFA models and all modelled risk
categories.
7.6.3 Specifically, the DFA model approach can be used to explore the
quantifiable effects of improving the control or mitigation of certain
operational risks on items such as the probability of ruin in a given (e.g. one
or five-year) timescale and the resulting change in capital requirement. This,
along with estimates of the associated costs, can be used to determine which
of a number of proposed mitigation or control improvements is the most cost
effective, and hence to prioritise the order in which they are tackled.
7.6.4 The model can also be used to understand the extent to which the
possible extreme outcomes are more severe, using explicit modelling of large
loss events (as once in a while a much larger separate loss occurs than
implicit modelling would identify) and which event types are contributing to
this most.
7.6.5 It can be used to explore features such as:
ö quantifying the probability of ruin in a given (e.g. one or five-year)
timescale;
ö quantifying the reduced capital requirement due to diversification
benefit;
ö understanding whether the possible extreme outcomes are poorer with
explicit modelling of large loss events (as, once in a while, a much larger
separate loss occurs than implicit modelling would identify); and
ö the effect of management action, or assumed improvements in
operational controls or strategic decision taking processes in so far as
they affect operational losses.
7.6.6 One issue is that the relative effect of explicitly modelling

operational risk compared to other risks appears small ö is it worth the
effort? The response to this is that any modelling repays the thought given to
it, and if the underlying cause of loss is operational, this modelling can get
more focus into what is going on, and improve management or stakeholder
value creation.
7.6.7 The apparent differences when comparing this modelled approach
with the calculations in Section 5 need further consideration. In this
illustration different assumptions and parameters have been used, and the
results are not directly comparable. In particular, the DFA model uses a
pareto distribution for the amount of large losses as opposed to the EVT
generalised pareto.
7.6.8 The choice and fit of model (distribution/parameter) can be tested
fairly easily, and this of itself can help improve understanding. Of course, the
whole approach depends on the availability of suitable data.
7.6.9 We feel that the approach can be tailored to reflect beliefs about
underlying business decisions. Admittedly this means a degree of subjectivity
in the work, but then, perhaps, risk management is as much an art as a
science.
ð. Pitfalls and Consideration of Soft Issues
Does the Organisation have the Ability to Admit Mistakes?

8.1 The focus of this paper up until this point has been on those aspects
of operational risk that lend themselves to being readily measured. However,
as our discussion of risk mapping in Section 6 illustrated, there are other
aspects that are not so easy to quantify and hence to model. These are
sometimes referred to as ‘soft’ issues. In this section we consider the nature of
soft issues and how they might be measured. We highlight the importance
of cultural factors, and look at an approach to measuring cultural risk. We
examine people risk and how it interacts with other risks. Finally, we
emphasise the complexity of soft risks, and use our case study to relate them
back to our earlier discussion of risk indicators.
8.2 The FSA quotes the Basel Committee on Banking Supervision, who
define operational risk as: “the risk of loss resulting from inadequate or
failed internal processes, people and systems, or from external events.’’ One
possible definition of soft issues is the people element of this definition.
Consultation Paper 142 says:
“The way in which a firm manages its employees can be a major source of operational
risk. Poorly trained or overworked employees may inadvertently expose a firm to
operational risk (for example by processing errors). In addition, a firm may find the
availability of its employees, or its ability to replace them, can influence its ability to
recover from interruptions to the continuity of its operations.’’
8.3 Operational risk has a different application for each company,

depending upon its own particular circumstances. The FSA’s draft guidance
suggests:
“A firm should try to understand the types of operational risk that are relevant to its
particular circumstances and the impact that these risks may have on the incidence of
financial crime, the fair treatment of its customers and its own solvency. This might
include, but is not limited to, the following issues:
ö The inappropriate management of a firm’s people is an important source of
operational risk (people refers to the employees of a firm and all the other human
resources that are involved in its operations);
ö Both IT and manual systems and their related processes are a source of operational
risk;
ö Operational losses may occur as a direct or indirect result of operational risk events;
ö Operational risk events may have immediate tangible effects that can be easily
quantified (e.g. monetary) and intangible and possibly delayed effects that cannot be
easily quantified (e.g. reputational damage);
ö The extent to which outsourced processes, people and systems remain a source of
operational risk;
ö Which external events represent sources of operational risk.’’
8.4 The accuracy of risk measurement methods depends on the risk

model and data availability. Risk models require a thorough understanding
of recurrent risk patterns, and their appropriateness is inherently linked to
data availability and thus the occurrence of events. Incidents help provide
better understanding of underlying risk structures and also provide the basis
for statistical testing of risk models. Furthermore, the accuracy of risk
models depends upon the measurability of outcomes and thus goes hand in
hand with sound definition and understanding of effects.
8.5 Operational risk encompasses events with very different frequencies
and possible patterns of occurrence and severities. It has been suggested that,
as a first step in determining the applicability of statistical analysis, the
potential incidents should be categorised into a frequency/severity matrix
based on experience and expert opinion (Muermann & Oktem, 2002). See
Figure 8.5.
8.6 The highest attention will obviously be paid to the high frequency/
high severity risks, which threaten the very existence of the operation.
Figure 8.5. Frequency/severity matrix
Relatively little attention will be paid to low frequency/low severity risk

risks. A lot of effort is often spent on the low frequency/high severity events,
but, by definition, these have very few data points and the estimation of
probabilities and loss distributions thus produce highly unreliable results.
Other approaches are therefore needed.
8.7 By contrast, high frequency/low severity events hold out the
possibility of creating large databases to which statistical analysis can be
accurately applied. Historical data, if based on a thorough definition of
outcomes related to operational losses, can be used to estimate the loss
distribution, i.e. the probabilities of such events and subsequent losses
within certain time periods. However, because we are still in the early
stages of the evolution of operational risk management in the industry,
these databases are not yet available, and will take a period of years to build
up. In the meantime, we are compelled to use more crude and subjective
methods.
8.8 This approach is advocated by the FSA, who says in Consultation
Paper 142 (FSA, 2002):
ö “A key issue is operational risk measurement. Due to both data limitations and lack
of high-powered analysis tools, a number of operational risks cannot be measured
accurately in a quantitative manner at the present time. So we use the term risk
assessment in place of measurement, to encompass more qualitative processes,
including for example the scoring of risks as ‘high’, ‘medium’ and ‘low’. However, we
would still encourage firms to collect data on their operational risks and to use
measurement tools where this is possible and appropriate. We believe that using a
combination of both quantitative and qualitative tools is the best approach to
understanding the significance of a firm’s operational risks.’’
8.9 The nature of soft issues is such that they are difficult to make
explicit. They include factors such as morale and organisational culture, and
other factors which impact on culture such as top leadership values and
behaviour, communication and performance orientation. A recent British
Bankers Association survey (BBA, 2002) suggests that there are a number of
factors that reflect or influence the company’s culture, including the style of
decision making, the level of formal processes and the attributes of the core
processes. All of the components are important, and they complement one
another. The BBA suggests an enterprise wide operational risk framework
that pulls the pieces into an integrated whole:
ö Strategy: risk management starts with the overall strategies and
objectives of the organisation and the goals for the individual business
units, products or managers, followed by identification of the associated
inherent risks in the strategy and objectives.
ö Risk policies: strategy is complemented by operational risk management
policies, which are a formal communication to the organisation as a whole
on the approach to, and importance of, operational risk management.
ö Risk management processes: these will encompass controls, assessment,
measurement and reporting.
ö Risk mitigation: specific controls or programmes designed to reduce the
exposure, frequency or severity of an event or the impact of an event or
eliminate or transfer an element of operational risk.
ö Operations management: the day-to-day processes, both front office and
back office, are involved in doing business.
Figure 8.9. Enterprise wide operational risk management framework

ö Organisational culture: includes communication, the ‘tone at the top’
clear ownership of each objective, training, performance measurement
and knowledge-sharing.
8.10 At this point the impact of performance incentive schemes and any
explicit or implicit organisational values should also be considered.
8.11 Robert Simons looked at cultural factors in an article in the
Harvard Business Review (Simons, 1999). He highlights how an aggressive
can-do culture often arises when a company’s sales and profits soar, and
leads to bold initiatives and satisfied clients, but also can end up silencing
any messenger carrying bad news. Simons has developed a tool that he calls
the risk exposure calculator, which shows the pressure points present in every
organisation that lead to increased risk and are a function of the company’s
circumstances and management style. There are three dimensions to this
tool:
ö Growth. This looks at the pressures for performance within the
organisation, the rate of expansion of the business and the level of
inexperience in key employees.
ö Culture. This covers the rewards the organisations gives for
entrepreneurial risk-taking, the level of executive resistance to bad news
and the amount of internal competition.
ö Information management. This focuses on the complexity and velocity of
transactions in the business, the amount of gaps in diagnostic
performance measures and the degree to which decision-making is
decentralised.
8.12 Though the scores from the tool are purely subjective, they are
intended to raise awareness of the issues and indicate whether the
organisation is fundamentally safe. It needs to be careful, or it is at risk and
needs to take action to address the level of risk. Simons concludes his article
by suggesting five questions which each organisation needs to ask itself:
ö Belief systems. Have senior managers communicated the core values of
the business in a way that people understand and embrace?
ö Boundary systems. Have managers in the organisation clearly identified
the specific actions and behaviours that are off-limits?
ö Diagnostic control systems. Are the diagnostic control systems adequate
at monitoring critical performance variables?
ö Interactive control systems. Are the control systems interactive and
designed to stimulate learning?
ö Internal controls. Is sufficient attention paid to traditional internal
controls?
8.13 Simon’s philosophy is clearly similar to that of the FSA, which, in

its draft guidance, says:
“A firm should ensure that all employees are aware of their responsibility and role in
operational risk management, and are suitable and capable of performing these
responsibilities, through the establishment and maintenance of:
(1) appropriate segregation of duties and supervision of employees in the performance of
their responsibilities;
(2) appropriate recruitment and, as necessary, subsequent review processes to consider
the fitness and propriety of employees, including their honesty, integrity and
reputation, competence and capability and financial soundness;
(3) appropriate systems and procedures manuals that employees may refer to as required;
(4) training processes that enable employees to attain and maintain appropriate
competence;
(5) appropriate disciplinary and termination of employment policies and procedures that
are enforced.
When controlling the impact that employees may have on its susceptibility to operational
losses, a firm should pay particular attention to approved persons and other positions of
high personal trust (for example, security administration, payment and settlement
functions). There are specific rules and guidelines for approved persons and for the
apportionment of senior management responsibilities.’’
8.14 It is clear that the people side of the organisation is of fundamental

importance in looking at operational risk: “Does the organisation have the
ability to admit mistakes? Does it suffer from ‘key person syndrome’? To
what extent is, and should, maverick behaviour be tolerated or encouraged?’’
The recruitment and selection process will influence how those in the
organisation behave, as will the approach to training and development.
8.15 Modern organisational development theory has a lot to say about
the composition of teams, especially top teams, and their impact on
organisational effectiveness. Perhaps the best known indicator in this field is
the Myers-Briggs type indicator (MBTI) (Quenk, 1999). This is an instrument
that has been exhaustively researched over 50 years, and seeks to make Carl
Jung’s theory of psychological types understandable and useful in a business
environment. It examines differences in preferences in individuals that result
from:
ö whether they prefer to focus externally or internally (extraversion/
introversion);
ö the way in which they prefer to take in information;
ö how they prefer to make decisions; and
ö how they orient themselves to the external world.
8.16 From this, it categorises respondents into one of 16 types. Jung’s

theory suggests that differences in behaviour result from people’s inborn
tendencies to use their minds in different ways. As people act on these
tendencies, they develop patterns of behaviour. The 16 types reflect the
different patterns of behaviour that are observed.
8.17 A team that works well together is not a chance event. When teams
understand their own styles and those of others, they are more likely to be
effective. Research has shown that the more similar the psychological types
in a team, the sooner the team members will understand each other.
However, while groups with high similarity may reach decisions more
quickly, they are more likely to make errors, owing to inadequate
representation of all viewpoints. Groups with many different types will reach
decisions more slowly and painfully, but may reach better decisions because
more viewpoints are considered.
8.18 Another well-known approach to enhancing team performance is
Belbin’s team role theory (Belbin, 1995). This has been developed through
rigorous analysis of a wide range of teams over an extended period, which
has led to the identification of different clusters of behaviour that underlie
the success of teams. This has been formulated in nine team roles:
ö action oriented roles: shaper, implementer and completer finisher;
ö people oriented roles: co-ordinator, team worker and resource investigator;
ö and
ö cerebral roles: plant, monitor evaluator and specialist.
8.19 Belbin’s research showed that there are a finite number of

behaviours or team roles, which comprise certain patterns of behaviour,
which can be adopted naturally by the various personality types found
among people at work. It is argued that the accurate delineation of these
team roles is critical to understanding the dynamics of any management or
work team.
8.20 Clearly, these two approaches share some similarities. Indeed,
research has been done to demonstrate the correlations between the MBTI
types and the Belbin team roles (Higgs, 1996). The key point from an
operational risk perspective is that the balance of a team, however measured,
can significantly influence its risk profile.
8.21 People risks interact with other risks. The most obvious example of
this is that people are always involved with computer systems, both in their
design and development, and in their operation. As the FSA says, in
Consultation Paper 142 (FSA, 2002):
“The automation of processes and systems may reduce a firm’s susceptibility to some
‘people risks’ (for example, by reducing human errors or controlling access rights to enable
the segregation of duties and information security) but will increase a firm’s dependency
on the reliability of its IT systems.’’
8.22 This is a key point, which raises other issues such as:
ö system design, the active involvement of users to make it work more
effectively for them, and so avoid errors or misunderstandings;
ö use of drop down lists/avoidance of manual typing; it is human nature
to be lazy ö do the first items on the drop down lists or the default
values appear more often than seems reasonable, are there useful data
fields that are optional;
ö building in data entry checks to minimise poor data entry;
ö ensuring that data entry staff understand the importance of entering the
correct items and the possible results of poor data entry; and
ö if staff raise issues with the system or if particular errors are cropping
up regularly, is anything actually done about it? If not, staff will stop
bothering to report problems or to monitor errors ö as well as getting
the impression that what they think does not matter.
8.23 Systems are implicated in many of the operational losses suffered

by the company in our case study. Perhaps the most obvious example is the
losses suffered in the direct writing arm.
8.24 The level of change will have a major impact on the level of people
risk. Major re-engineering and downsizing projects can lead to a significant
loss of experience by the organisation. A recent A M Best survey showed that
one of the major causes of insurance company failure was excessive growth.
Major expansion often leads to problems, particularly when it is unplanned.
8.25 The FSA highlights the fact that major change will result in an
alteration of a firm’s risk profile. Their draft guidance is as follows:
“Before, during and after a significant change to its organisation, infrastructure or business
operating environment, a firm should assess and monitor how this change will affect its risk
profile. In particular, there may be an increase in operational risk from:
(1) untrained or de-motivated employees or an expected significant loss of employees
during a period of change or subsequently;
(2) inadequate human resources or inexperienced employees carrying out routine business
activities owing to the prioritisation of resources to the programme or project;
(3) process or system instability and poor management information due to failures in
integration or increased demand;
(4) inadequate or inappropriate processes following business re-engineering.’’
8.26 Relationships are at the root of the soft issues. Personal

relationships with those inside and outside the organisation can have a
disproportionate influence on decisions made by senior executives. Given the
central role of personal relationships, the development of appropriate
measurements may have to come from a source such as occupational
psychology models. The future of any business depends upon its customer
relationships, so processes like complaint management and service level
monitoring are critical. Those businesses that have the Government as a
customer are exposed to significant political risk, as the recent experience of
BAe Systems has demonstrated.
8.27 The case study company is clearly one with a high level of
cultural risk. The pressure for growth from the U.S. parent has led to a
series of risk decisions that have led to operational losses, for example the
expansion into unfamiliar lines of business. The culture clash was
responsible for the losses suffered, e.g. the failure of the stop loss
reinsurance and the default on the loan. Deficiencies in management
information have been implicated in the excessive losses in the direct writing
arm, the change in the composition of the business mix and the IT
overspends. In Simon’s terms, there does not appear to be a common belief
system, there is a lack of clarity in boundaries and the different types of
controls are significantly deficient.
8.28 The case study is silent on the company’s approach to segregation
of duties and supervision of employees’ recruitment and review processes, the
availability of systems and procedure manuals, training and development
processes and the company’s approach to disciplinary action and termination
of employment. However, from the other evidence available about the
culture of the company, it seems likely that there were significant
deficiencies. We have no direct evidence from our case study about the
composition of the top team, but its performance would lead us to believe
that it is dysfunctional. The use of the type of tools such as the MBTI and
Belbin team role analysis, previously described, may have highlighted the
imbalance in the team and suggested ways in which the level of risk could
have been reduced.
8.29 The company in our case study has undergone a series of major
changes, such as taking over a competitor and then being acquired itself by
an overseas company, which has clearly increased the level of risk. The
evidence suggests that the case study company paid inadequate attention to
the quality of its relationships with its key stakeholders. These issues are
highlighted by the circumstances leading to the loss of the block account in
the case study.
8.30 In summary, the case illustrates a wide range of eventualities
where failure to attend to soft issues has been the cause of real and
significant operational losses. The traditional approach to the management
of insurance businesses has emphasised the measurement of hard
factors, but our case study suggests that something more is required. It
is here that the risk indicator approach, described in Section 3, comes into its
own.
8.31 The cause/effect risk map and the causal modelling work outlined
above (see Section 6) illustrated that, although the manifestation of a loss
may be simple enough to identify, the cause is likely to be considerably more
complex, being a causal chain that involves both soft and ‘non-soft’ factors.
Modelling it, even assuming that the data are available, is therefore a non-
trivial task.
8.32 The consideration of soft issues leads to the question as to whether
some form of qualitative risk rating would be desirable. For example, based
on agreed criteria, such as the amount of change, the management
experience, the personality profiles of the top team and a process quality
rating, is it possible that a ‘soft risk rating index’ (similar to those used by
ratings agencies) could be developed?
æ. Reporting and Pulling the Threads Together
9.1 This section is intended to help readers formulate reports in a

cohesive and useful manner. It considers what aspects should be included,
where caveats may be appropriate to avoid misunderstandings, and provides
one or two suggested communication tools.
9.2 The key aim is to communicate the burden of findings to, say, a risk
committee, which is likely to be a group with diverse experiences and
abilities, limited time and a mixture of personal interests. The first challenge
is to tailor the communication to the audience ö to communicate what is
important, to get the right level of supporting technical information and to
make it as clear as possible. The risk committee is unlikely to want to know
about how the EVT threshold was determined or the number of iterations in
the causal model, but it will want to know how serious is the risk that it
might not be in business in the foreseeable future.
9.3 The group risk committee (on behalf of the board) and senior
management want to know what operational risk means for their business.
The report is likely to include:
ö a preamble covering scope and purpose;
ö an executive summary;
ö some background about the company and its current situation;
ö a summary of the key risks being faced and the organisation’s risk
appetite or ability to withstand risk, including any soft issues or
qualitative aspects;
ö an assessment of the key operational risks (size, volatility, how vital);
ö an understanding and possibly quantification, possibly including
implications for an internal capital assessment (ICA) and FSA
discussions;
ö the approach to managing, mitigating and generally coping with the
risks ö an assurance or otherwise that the business can cope;
ö a comment on how the risks will be kept under review (monitored), how
the risk process will be refreshed and reviewed;
ö a description of the approach, work completed, methods and data ö
including any gaps and key assumptions;
ö suggested way forward, for example future process improvements or
data collection; and
ö the overall results and conclusions.
9.4 The headings chosen naturally reflect the risk management cycle,
although it is accepted that other structures may be equally valid. Each is
discussed in more detail below.
9.5 A preamble covering scope and purpose may include the intended
readership (the group risk committee), the purpose (to consider the capital
requirements of MELG plc in respect of operational losses), the scope
(definitions such as the risk of loss, resulting from inadequate or failed
internal processes, people and systems or from external events, categories,
cause/event/consequences and exclusions), and the context (e.g. first time
such an exercise has been completed and the relation with other risk
categories). Reliances and limitations (e.g. people’s views taken at face value,
limited data checking, constraints on data and quantification), other caveats
(e.g. the assumed experience of the reader, the need to read the report as a
whole) and the qualifications of the author would also be included.
9.6 Professional matters may need discussing. At this stage this would
mean referring to Guidance Note 12, but arguably further more specific
guidance may need to be considered by the profession; maybe some comment
whether these risks have been considered in isolation and what attempts
have been made to see how they integrate with other risks faced by the
organisation in association with DFA modelling. (The question of
professional liability is returned to in the conclusions.)
9.7 For the purpose of this paper we take the structure of the executive
summary as read.
9.8 Background. This will include comment about the company and its
current situation ö what sort of changes it is undertaking, its business plans,
any recent FSA visits or audit reports that are relevant, a general comment
on organisation structure, systems of control and any recent or planned
senior management change. It might also discuss the risk management cycle
and how it is embedded in the organisation, or what the operational risk
aspects are. It would be worth commenting on whether the review is being
driven by fear of missing FSA approval, or due to perceived business (value)
gains.
9.9 Key risks. This would summarise the key risk being faced by the
organisation and discuss its risk appetite or ability to withstand risk. This is
also a useful place to include comment on any soft issues or qualitative
aspects. One communication tool is a chart like Figure 9.9.
9.10 Such a tool may be called a risk map, or a profile of risks. It
displays the risks according to their frequency and severity of the loss when
an event occurs:
ö The bottom left quadrant represents low frequency, low severity losses
that are not of significant cost to the organisation. These elements make
up a ‘background noise’ level of operating loss that should be expected;
they are not the ones for which capital needs to be set aside.
ö The top left quadrant represents operational losses that are still
relatively small in amount (severity), but are more frequent. These losses
will represent a greater level of loss to the organisation. This is an area
where use of risk mitigation controls could reduce the frequency of
losses. The cost of additional controls in this case could well be cost
effective. Fluctuations in frequency could lead to variation in the level of
loss and, as such, capital should be set aside.
Low Severity High Severity
High Frequency
Frequency
Low Frequency
Severity
Figure 9.9. Frequency vs severity of operational losses
ö The bottom right quadrant also represents risks of which the company
should take note. These risks result in large losses on an infrequent basis.
These losses can be difficult to control against, because they happen
infrequently, but the amounts involved mean that it is worth considering
risk mitigation measures. They are also likely to introduce the most
volatility to losses experienced and, as such, it will be important to
consider the capital that should be set aside to protect against any level
of unexpected loss.
ö The top right quadrant needs to be empty for a healthy business. In the
case study there is one such risk edging into this area. The size of these
losses and the frequency with which they occur mean that they are the
single biggest cause of operational loss within the organisation, costing
on average »50 million per year. We would recommend that controls are
put in place to mitigate the loss from this risk area as a matter of
priority.
9.11 Another technique is to display risks by risk type in descending

order to focus attention on those risks where mitigation measures may have
the most significant benefit.
9.12 A third communication method is shown in Figure 9.12, illustrating
loss variability by customer process and strategic business line. Each bar
represents the spread of operational losses that may arise from a particular
process and class. The bar is not intended to cover every possible loss
Figure 9.12. Operational loss by customer process and strategic business

line
exhaustively, but represents the majority of possible outcomes (approximately

90% of losses).
9.13 The marker roughly in the middle of each bar represents the mean
level of loss. This is not an indication of the capital required to withstand an
unexpected level of loss. The unexpected level of loss is represented better
by a point towards the top of each bar. The ideal is for a short bar (little
variability) that lies close to the axis (limited average loss).
9.14 It can be seen that this section starts to include an assessment of
key risks. A separate comment may help give the right focus, and may
include further discussion about where the key risks lie ö which functions
are key, which processes, how they are monitored, their size, the likely
volatility and in general how vital to the functioning of the organisation. To
the extent not already covered, it would be the place to describe soft issues,
the current change climate ö any key recent or impending decisions and the
general approach to strategic, reputational and legal risk.
9.15 The next section would set out an understanding or possibly
quantification of risk, probably including implications for ICA and FSA
discussions. In this way, the report would try to convey a sense of the
character of each risk and how it was best dealt with. This is the section that
needs to convey the importance of tackling any given risk in a new way,
and whether there are possible combinations of events that could cause
unexpected consequences. The understanding part of the report is critical, as
quantification may be misleading or undesirable for certain categories of
loss.
9.16 The next section would set out the approach to managing,
mitigating and generally coping with the risks. In this section an indication
would be given about how each risk is being (or will be) handled, and where
overlaps/sharing of approach would be appropriate. There would be
comment on the use of systems and controls, the use of capital, the impact
of insurance lay-offs or reinsurance and other mitigation approaches. In
some cases, it would be appropriate to accept the risk. It would also be the
place, subject to professional caveats, to set out an assurance or otherwise
that the business can cope. It is, of course, possible to envisage that the
sections could be set out by considering each of the last four headings a risk
at a time.
9.17 There should then be some comment on how the review and
refreshment of the risk analysis should be achieved. This is to assure the
readers, as the organisation keeps developing and changing, and as external
circumstances move on, so all categories of operational risk will be kept
under review, that they will be monitored, and in some ways, more
importantly, that the risk management process itself will be refreshed,
reviewed and subject to continual improvement.
9.18 The next section should include a description of the approach,
work completed, methods and data ö including any gaps and key
assumptions. Each method used and relevant results might be discussed.
9.19 The suggested way forward would, for example, discuss future
process improvements or data collection.
9.20 The final section would be a conclusion and setting out of overall
results. This might be the place where correlations and independence are
discussed, and where comment is made as to how the capital required by
considering operational risk explicitly differs from the capital requirement of
the enterprise using a DFA model based on implicit assumptions.
9.21 It would be important to comment on areas where the operational
risk is such that risk mitigation measures could be considered as a means of
reducing the overall capital requirement; further, where an allocation of
operational risk across different business units might lead to a different
allocation of risk based capital, and the implications for unit (and personal)
profit goals.
9.22 If possible, there should be a simple summary of required
operational risk capital and its split by line (e.g. in the case study between
commercial, personal intermediary and personal direct).
9.23 Finally, there might be some comment on the key learning points
identified during the exercise, e.g. the areas where risk management tools are
inadequate and the cost benefit for improving them is positive, where
systems or processes are clearly more or less reliable and where fraud losses
are clearly too high.
"ò. Conclusions
10.1 We hope that this paper has served its purpose, which is to set out
an overview of the landscape relating to operational risk, indications of
possible approaches/current seeds of best practice, and to excite further
attention to what could be a new area for developing actuarial involvement.
We hope that it is timely in setting a framework for the profession.
10.2 It is still early days, and we must be wary of running too fast ö
before we can walk. It has to be a matter of ‘first things first’. Whilst not
strictly actuarial in some past senses of the word, this means beginning by
identifying, assessing and understanding operational risk, and being able to
view various forms of control as important, as well as understanding their
impact ö all before using statistical measurement techniques. This requires
insight into, and understanding of, process management, organisational
design including defining roles and responsibilities, occupational psychology
and general management. The actuarial analytic training is good grounding
for such work, but by no means a passport to success.
10.3 Starting at the beginning also implies asking questions about
whether the board understands risk ö the organisation’s ability to bear risk
or its risk appetite, as well as considering how the board itself can be a source
of risk. Strategic error is often critical, particularly if combined with
dominance risks, a culture which encourages risk taking and achievement,
and incentive plans that encourage short-term delivery at the expense of
medium-term value and capital management. Strategic error and risk are
inevitably closely connected with operational risk.
10.4 We believe that ultimately understanding operational risk should
be driven by the desire for business success and value creation ö more so
than the fear of failing FSA tests or even the risk of complete ruin ö vital
though these later two motivators should be.
10.5 Moving to a vision for the future management of operational risk
will mean the need to start to collect data as soon as sensible. Appropriate
liaison with other interested parties may help, and design of relevant
reporting forms (or on-line mechanisms) might involve capturing ‘near
misses’ and organisational culture issues, as well as ensuring sensible capture
of useful statistical data.
10.6 The relative importance of operational risk compared with insurance
or market risk is unclear. Our illustrations show a relatively small
operational risk ö only 2% of net premiums on average. Further work is
needed to quantify the real impact: it could easily be three, four or more
times the illustrated level, and comment would be welcome.
10.7 Nevertheless, we believe that, as thinking develops, operational risk
will assume greater importance in terms of capital requirements and
management thinking than at present. The rationale for this is that much of
what is now considered insurance risk (be it based on premium or reserve),
and even market risk, has its root cause in poor operational process. The
concept of cause/event/consequence will inevitably drive attention to
operational causes of loss.
10.8 Naturally, as this happens, there will be an increasing need for, and
interest in, quantification. This will lead to discussion about methods and
then about underlying assumptions and concepts to do with diversification of
risk, correlation, new mitigation techniques and so on. These are fields
ready and waiting for actuarial involvement.
10.9 We strongly believe that the actuarial profession should be
considering how to better position itself. This could mean development of
new courses, training and exams (or wider risk qualifications); it could
mean development of new actuarial guidelines; it could mean involvement
through the risk and regulatory co-ordinating group of an impact study
across industry boundaries; it could mean sponsoring academic and
practical research; it could mean starting something as basic as a life,
general and pensions industry operational risk database. Whatever it
means, we can only see good in it for adventurous and outward looking
actuaries.
10.10 Independence and the ability to speak the unspeakable are
valuable contributions that a well disciplined profession can make.
10.11 Equally, while an actuarial role exists and can be developed, we
are not alone and need to work with other professions. Our ability to
contribute may require development, but it also requires interaction or
liaison and a new mind set.
10.12 As well as being an opportunity, there are huge concerns for the
profession. As always, we need to be clear about claiming expertise that we
do not possess. Our skills involve synthesising information from others,
working with others to make sense of information, and possibly designing
frameworks for quantification.
10.13 In terms of future work, again there is no shortage of things to be
done. Here is a preliminary list; we would welcome comment from the
profession to help strengthen and prioritise these topics:
ö developing a deeper understanding of causal modelling techniques and
their implication for risk modelling and analysis;
ö a quantitative impact study, to help obtain industry based estimates on
the quantum of operational risk;
ö more detailed development of risk indicators and exposure to risk
measures;
ö development of a more consistent categorisation framework; while we
understand the importance of defining risk tailored to a given
organisation’s needs, we think, ultimately, that this will slow down
progress, as too much time will be taken in comparing categories which
fundamentally have minimal difference;
ö commencing a shared, confidential data collection service for the industry;
ö developing new methods based on value at risk approaches, market
measures (betas) and other techniques;
ö deepening our understanding of systems, processes, controls and
organisational design (roles and responsibilities); changing our own
attitudes to ‘soft issues’ and building insights into the vital areas of
culture and behaviours ö we may not wish to become experts in all these
fields, but our thinking should be good enough to ensure we can act
sensibly as facilitators and integrators;
ö considering new forms of risk management or mitigation, including use
of insurance, cross sector aggregation, securitisation and other
alternative forms of risk transfer ö this might go as far as insurance
product design to handle operational risk and subsequent rating; and
ö ensuring that professional guidance and education are adapted to meet
emerging needs.
10.14 “I do not want my house to be walled in on all sides and my

windows to be stuffed. I want the skills and experiences of all peoples and
professions to be blown through my house as freely as possible. But I refuse
to be blown off my feet by any.’’ With due respect and regard for Mahatma
Gandhi, whose wonderful words we have taken the liberty of adapting, we
need to learn from others, be open to new ways, but be strong enough to
work out our own role. There is much to do and much to gain by open-
minded exploration of this new area.
Acknowledgements
This has been a true team effort. Each member of the working party has
made a significant contribution. Like all such efforts, with more time the
output could be further improved, and we collectively shoulder the
responsibility for errors and omissions.
We would particularly like to thank the scrutineers for their helpful
contributions, attendees at GIRO for their observations, Andrew Hitchcox
and the General Insurance Board for their support, Charles Ng for his
assistance with modelling, Marie-Jose¤ Gaze'res de Baradieu for her
unflappable positive help with typing and meeting organisation, the Institute
staff, all those who we spoke with in developing ideas, and any one else
who has helped in whatever way.
References
BBA (1999). Operational risk management ö the new frontier. The British Bankers Association.
BBA (2002). Operational risk management ö the new frontier. British Bankers Association.
BBA, CA. Operational risk database loss categorisation. British Bankers Association.
http://www.bba.org.uk/xl/45716.xls
BCBS (2001). Working paper on the regulatory treatment of operational risk. Bank for
International Settlements, Basel Committee on Banking Supervision.
BCBS (2003). Sound practices for the management and supervision of operational risk. Bank for
International Settlements. Basel Committee Publications No 96.
Belbin, M.R. (1995). Team roles at work. Butterworth-Heinemann.
CAS (2001). Final report of the advisory committee on enterprise risk management. Casualty
Actuarial Society.
CAS (2003). Overview of enterprise risk management. Enterprise Risk Management Committee,
Casualty Actuarial Society.
CAS various. www.casact.org/research/dfa/index.html
Charniak, E. (1991). Bayesian networks without tears. AI Magazine, 12(4), 50-63.
http://www.aaai.org/Library/Magazine/Vol12/12-04/Papers/AIMag12-04-007.pdf
Converium Re. (2003). Dynamic financial analysis understanding risk and value creation in
insurance. econwpa.wustl.edu/eps/ri/papers/0306/0306002.pdf
Doerig, H.-U. (2000). Operational risks in financial services: an old challenge in a new
environment. Paper presented to Institut Internationale d’Etudes Bancaires, London.
Available from http://www.risklab.ch/kaufmann/RM.html
Embrechts, P., Klueppelberg, C. & Mikosch, T. (1997). Modelling of extremal events for
insurance and finance. Springer.
Fox, N.J. (2005). Capability maturity model (RM-CMM) for risk management.
http://www.siliconrose.com.au/Articles/RiskCMM.htm
FSA (2002). Operational risk systems and controls. Financial Services Authority.
FSA (2003a). The firm risk assessment framework. Financial Services Authority.
FSA (2003b). Enhanced capital requirements and individual capital assessments for non-life
insurers. Financial Services Authority.
FSA (2003c). Integrated prudential sourcebook ö near-final text on prudential risks systems
and controls. Financial Services Authority.
GIRO (2002). Report of the Operational Risks Working Party to GIRO 2002.
GIRO (2003). Operational risk: measurement or bust. Report of the working party to GIRO
2003.
Hall, D.C. (2002). Using a risk management maturity-level model. Software Risk Magazine,
Vol. 2, No. 4.
Hallock, Micah, Heintz & Kourtney (2001). Measuring operational risk. Bank Accounting
and Finance, Vol. 14, Issue 4.
Higgs (1996). Comparison of Myers Briggs type indicator profiles and Belbin team roles. Henley
Management College.
Hoffman, D. (2002). Managing operational risk: 20 firmwide best practice strategies. John
Wiley & Sons.
Institute of Actuaries. Claims reserving manual.
IRM (2002). A risk management standard. The Institute of Risk Management, ALARM (The
National Forum for Risk Management in the Public Sector) and AIRMIC (The
Association of Insurance and Risk Managers).
King, J.L. (2001). Operational risk: measurement and modelling. Wiley Finance.
Laycock et al. (1998). Operational risks and financial institutions. Risk Publications/Arthur
Andersen.
MacKenzie, D. & Millo, Y. (2001). Negotiating a market, performing theory: the historical
sociology of a financial derivatives exchange. Paper presented at European Association
for Evolutionary Political Economy conference, Siena, November 8-11, 2001.
McDonnell, W. (2002a). Managing risk: practical lessons from recent ‘failures’ of E.U. insurers.
McDonnell, W. (2002b). Financial Services Authority occasional paper, 20 December 2002.
Muermann, A. & Oktem, U. (2002). The near-miss management of operational risk. The
Journal of Risk Finance.
Netica (1997). Netica TM users guide, Norsys Software Corporation, http://www.norsys.com
Pearl, J. (1988). Probabilistic reasoning in intelligent systems. Morgan Kaufmann.
Pyle, D.H. (1997). Bank risk management: theory. Paper presented at the Conference on Risk
Management and Regulation in Banking, Jerusalem, May 17-19, 1997.
Quenk, N.L. (1999). Essentials of Myers-Brigss type indicator assessment. John Wiley & Sons
Inc.
Risksig (2002). Risk management maturity level development. Risk Management Specific
Interest Group. http://www.risksig.com/projects/report.html
Simons, R. (1999). How risky is your company? Harvard Business Review.
APPENDIX A
BBA GOLD DATABASE CATEGORIES
The British Bankers Association co-ordinate a database of operational

risk losses on behalf of many U.K. banks. These losses are categorised at
three levels, and the headings (publicly available from the BBA website) are
set out below for interest, and although they relate to banks, we believe that
they can act as a useful indicator of the type of operational risks that might
be experienced. As mentioned in the paper, these headings are under current
review.
Tier 1 ö People
Tier 2 Tier 3
Employee fraud/malice (criminal) ö Collusion
ö Embezzlement
ö (Deliberate) sabotage of bank
reputation
ö (Deliberate) money laundering
ö Theft ö physical
ö Theft ö intellectual property
ö Programming fraud
ö Other
Unauthorised activity/rogue ö Misuse of privileged information
trading/employee misdeed ö Churning
ö Market manipulation
ö Activity leading to deliberate
mis-pricing
ö Activity with unauthorised
counterparty
ö Activity in unauthorised product
ö Limit breach
ö Incorrect models (intentional)
ö Activity outside exchange rules
ö Illegal/aggressive selling tactics
ö Ignoring/short-circuiting procedures
(deliberate)
ö Other
Employment law ö Wrongful termination
ö Discrimination/equal opportunity
ö Harassment
ö Non-adherence to other employment
law
ö Non-adherence to health and safety
regulations
ö Other
Workforce disruption ö Industrial action
ö Other
Loss or lack of key personnel ö Lack of suitable employees
ö Loss of key personnel
ö Other
Tier 1 ö Process
Tier 2 Tier 3
Payment/settlement ö Failure of/inadequate internal
payment/settlement processes
Delivery risk ö Losses through reconciliation failure
ö Securities delivery errors
ö Limit breach
ö Insufficient capacity of people or
systems to cope with volumes
ö Other
Documentation or ö Document not completed properly
contract risk ö Inadequate clauses/contract terms
ö Inappropriate contract terms
ö Inadequate sales records
ö Failure of due diligence
ö Other
Valuation/pricing ö Model risk
ö Input error
ö Other
Internal/external reporting ö Inadequate exception reporting
ö Accounting/book-keeping failure/
inadequate data
ö Inadequate risk management
reporting
ö Inadequate regulatory reporting
ö Inadequate financial reporting
ö Inadequate tax reporting
ö Inadequate stock exchange/securities
reporting
ö Non-adherence to Data Protection
Act/Privacy Act/similar
ö Other
Compliance ö Failure to adhere to internal
compliance procedures
ö Failure of external compliance
procedures
ö Breach of Chinese walls
Project risk/change management ö Inadequate project proposal/plan
ö New product process inadequacies
ö Project overruns
ö Other
Selling risks ö Inappropriate product selection
ö Product complexity
ö Poor advice (including securities)
ö Other
Tier 3 ö Systems
Tier 2 Tier 3
Technology ö Inappropriate architecture
Investment risk ö Strategic risk (platform/suppliers)
ö Inappropriate definition of business
requirements
ö Incompatibility with existing systems
ö Obsolescence of hardware
ö Obsolescence of software
ö Other
Systems development ö Inadequate project management
and implementation ö Cost/time overruns
ö Programming errors (internal/
external)
ö Failure to integrate and/or migrate
with/from existing systems
ö Failure of system to meet business
requirements
ö Other
Systems capacity ö Lack of adequate capacity planning
ö Software inadequate
ö Other
Systems failures ö Network failure
ö Interdependency risk
ö Interface failures
ö Hardware failure
ö Software failure
ö Internal telecommunication failures
ö Other
Systems security breach ö External security breaches
ö Internal security breaches
ö Programming fraud
ö Computer viruses
ö Other
Tier 1 ö External Events
Tier 2 Tier 3
Legal/public liability ö Breach of environmental
management
ö Breach of fiduciary/agency duty
ö Interpretation of law
ö Misrepresentation
ö Other
Criminal activities ö External frauds/cheque fraud/forgery
ö Fraudulent account opening by client
ö Masquerade
ö Blackmail
ö Robberies (þ theft)
ö Money laundering
ö Terrorism/bomb
ö Disruption to business
ö Physical damage to property
ö Arson
ö Other
Outsourcing/supplier risk ö Bankruptcy of supplier
ö Breach of responsibility (misuse of
confidential data)
ö Inadequate contract
ö Breach of service level agreement
ö Supplier/delivery failure
ö Inadequate management of suppliers/
service providers
ö Other
Insourcing risk ö Insourcing failure
Disasters and infrastructural ö Fire
utilities failures ö Flood
ö Other natural (geological/
meteorological)
ö Civil disasters
ö Transport failure
ö Energy failure
ö External telecommunications failure
ö Disruption to water supply
ö Unavailability of building
ö Other
Regulatory risk ö Regulator changes rules in industry/
country
Political/government risk ö War
ö Expropriation of assets
ö Business blocked
ö Change of tax regime
ö Other changes in law
ö Other
APPENDIX B
MELG: ILLUSTRATIVE RISK MATRIX
The risk classification shown in this matrix is based on that in Annex 2 of

the working paper on the treatment of operational risk (BCBS, 2001). We
show the perceived risk to the bottom line of MELG for three business units.
Event type Event sub type MELG broker MELG direct MELG
(category 1) (category 2) personal lines personal lines commercial lines
Internal fraud Unauthorised Low, due to Low, due to Moderate, due to

activity generally low generally low higher authorities
underwriting underwriting and bigger
authorities and authorities and individual risks
fixed rates fixed rates
Theft and fraud Low Low Moderate (more
flexibility
available on
admin systems)
External fraud Theft and fraud Moderate (but Moderate (but Moderate to high
should be low) should be low)
Systems security Moderate to high Moderate Moderate to high
(weak legacy (weak legacy
systems) systems)
Employment Employee Moderate (broker Moderate (broker Low (culturally
practices/ relations vs direct culture vs direct culture largely intact
workplace safety issues) issues) post merger/
acquisition)
Safe environment Low Low Low
Diversity and Low Low Low
discrimination
Clients, products Suitability, Moderate Moderate Low
and business disclosure and
practices fiduciary
Improper Low (no recent Low (no recent Low (no recent
business/market sign of cartels, sign of cartels, sign of cartels,
practice dodgy deals, etc.) dodgy deals, etc.) dodgy deals, etc.)
Product flaws Moderate (low Moderate (low High (high
individual, high individual, high individual sizes,
volume) volume) moderate volume,
possible lack of
experience in
senior team?)
Selection, Moderate to high Moderate to high Moderate to high
sponsorship and ö should be low ö should be low ö should be low
exposure (not much (not much (not much
sponsorship or sponsorship or sponsorship or
advertising to advertising to advertising to
general public), general public), general public),
but rumour over but rumour over but rumour over
parental support parental support parental support
Advisory None (broker’s Low to moderate None (broker’s

activities role) (care over role)
whether staff in
practice offer
advice or make
suggestions to
customers)
Damage to Disasters and Moderate Moderate to high High ö should
physical assets other events (brokers can ö need to be moderate as
carry on with maintain direct per personal
some activity in contact and broker but
the meantime) response to disaster recovery
customers plan is several
years out of date
and misses out a
key new system
Systems High (high High (even more Moderate (lower
volume) dependent than volumes and
personal broker) many products
could be rated
using paper
tables if required)
Execution, Transaction High (garbage in, High (GIGO) High(er) as data
delivery and capture, garbage out) are more complex
process execution and than for personal
management maintenance lines
Monitoring and High (legacy High (as per High ( as per
reporting systems make personal broker, personal broker
accurate reporting but no legacy plus more
difficult, penalties system issues) complex
including reserving issues)
prevention form
writing and adverse
publicity for
missing regulatory
deadlines, etc.
directors’ and
officers’ claims
potential if
misreport to
markets
Customer intake High (high High (as personal Moderate to high
and volume and broker plus more (lower volumes)
documentation standard docs so automation of
any problem document
would be production so
multiplied) more process
risk)
Customer/client Do not handle Do not handle Do not handle
account mgt customer customer customer
accounts/ accounts/ accounts/
deposits (broker) deposits deposits (broker)
Trade Moderate (broker Low (reinsurer Moderate (broker
counterparties and reinsurer exposure) and reinsurer
exposure) exposure)
Vendors and High ö gaps in High ö gaps in High ö gaps in

suppliers supplier supplier supplier
monitoring monitoring monitoring
capability in new capability in new capability in new
system plus system plus system plus
historic problems historic problems historic problems
APPENDIX C
DETAILS OF CURVE FITTING
C.1 Frequency
C.1.1 Poisson distribution l ¼ 1:83.
C.2 Severity
C.2.1 Lognormal distribution a ¼ 2:979; b ¼ 0:872.
C.2.2 Weibull distribution a ¼ 1:144; b ¼ 25:444.
C.2.3 Gamma distribution a ¼ 1:468; b ¼ 16:561.
C.2.4 Extreme value distribution u ¼ 25, l ¼ 0:364, x ¼ 0:000000295,
s ¼ 105:002.
C.3 Q-Q Plots of Standard Curve Fitting Distributions
Lognormal
70
60
model values £m
50
40
30 model
20 ideal
10
0
0 10 20 30 40 50 60 70
observed values £m
Weibull
100%
model percentiles
80%
60% model
40% ideal
20%
0%
0% 20% 40% 60% 80% 100%
observed percentiles
Gamma
60
50
model values £m
40
model
30
ideal
20
10
0
0 10 20 30 40 50 60
observed values £m
C.4 Extreme Value Distribution Mean Excess Plot
Mean Excess Plot
30
25
Mean Excess £m
20
15
10
5
0
0 20 40 60 80
Threshold value £m
C.5 Graph of Distribution of Outcomes for Modelled Annual Losses
CDF of Annual Losses
120%
100%
80%
percentile
EVT
lognormal
60% weibull
40% gamma
20%
0%
0 200 400 600 800 1,000
annual loss £m
APPENDIX D
CAUSAL RISK MAPS AND BAYESIAN NETWORKS
Sections D.1 to D.8 show additional examples of causal risk maps for
some of the other operational risk events that have affected MELG.
The first diagram in Section D.9 shows pictorially a Bayesian network for
MELG with central assumptions, using one of the proprietary software
packages available. The subsequent diagrams show the effect of changing the
assumptions for some of the nodes. In each case the nodes for which
assumptions are being changed are shown as shaded.
D.1 Launch of Direct Writing
D.2 Outsourcing of Claims Handling

D.3 Failure of Stop Loss Reinsurance
D.4 Loan Default

D.5 Loss of Block Account
D.6 External Supplier Fraud

D.7 Imposed Business Mix
D.8 IT Over Spends

D.9 Bayesian Causal Network Images
APPENDIX E
DFA MODEL DETAILS
The following pages summarise the four versions of a DFA model, as

described in Section 7 of the paper. The underlying assumptions are based on
the historic results (as shown in the FSA returns) of the leading six insurers’
household and employer’s liability accounts. These have been overlaid with
the operational losses shown in Section 2.
Each version is summarised in terms of the mean projection of key financial
items, followed by a funnel plot showing the probability distribution of the
solvency ratio about its mean. The model is developed using a proprietary
DFA package developed in excel and visual basics, as used in producing a
paper ‘Calibration of the general insurance risk based capital model’ for the
FSA, dated 25 July 2003.
Table E.1. Middle England Life and General plc, implicit operational risk, mean projections
Period ending 31/12/02 31/12/03 31/12/04 31/12/05 31/12/06 31/12/07 31/12/08
Net written premiums 1,694.625 1,798.400 2,027.619 2,241.204 2,396.794 2,560.651 2,788.329
Net earned premiums 1,643.664 1,560.205 1,771.082 2,130.014 2,315.385 2,474.922 2,669.691
Underwriting surplus for the year (149.532) (124.558) (133.318) 31.367 3.294 (94.212) (217.356)
Profit after tax (452.882) 101.997 117.239 304.487 282.507 192.944 92.433
Total shareholders’ funds 857.118 959.115 1,076.355 1,380.841 1,663.348 1,856.293 1,948.725
Net loss ratio 75.8% 70.6% 66.5% 60.7% 62.1% 65.9% 69.9%
Expense ratio 32.4% 36.2% 36.8% 36.6% 36.7% 36.9% 37.1%
Combined ratio 108.2% 106.8% 103.3% 97.3% 98.9% 102.8% 107.0%
Return on capital employed ÿ15.8% 4.0% 4.5% 10.1% 8.7% 5.8% 3.1%
Solvency ratio 50.6% 53.3% 53.1% 61.6% 69.4% 72.5% 69.9%
Quantifying Operational Risk in General Insurance Companies
1005
1006
Funnel plot
Funnel plot
200.0%
150.0%
100.0%
50.0%
Solvency ratio
0.0%
31/12/2002 31/12/2003 31/12/2004 31/12/2005 31/12/2006 31/12/2007 31/12/2008
-50.0%
-100.0%
Year
1%-5% 5%-10% 10%-20% 20%-30% 30%-40% 40%-50% 50%-60% 60%-70%

70%-80% 80%-90% 90%-95% 95%-99% Median
Table E.2. Middle England Life and General plc, no operational risk
Period ending 31/12/02 31/12/03 31/12/04 31/12/05 31/12/06 31/12/07 31/12/08
Profit after tax (452.882) 153.551 165.730 350.762 331.767 252.857 172.902
Total shareholders’ funds 857.118 1,010.669 1,176.399 1,527.162 1,858.928 2,111.785 2,284.687
Net loss ratio 75.8% 67.4% 64.0% 58.8% 60.1% 63.6% 67.3%
Expense ratio 32.4% 36.1% 36.6% 36.5% 36.6% 36.7% 36.9%
Combined ratio 108.2% 103.4% 100.6% 95.3% 96.7% 100.4% 104.2%
Solvency ratio 50.6% 56.2% 57.8% 68.1% 77.4% 82.6% 82.2%
1007
1008
Funnel plot
200.0%
150.0%
100.0%
50.0%
Solvency ratio
0.0%
31/12/2002 31/12/2003 31/12/2004 31/12/2005 31/12/2006 31/12/2007 31/12/2008
-50.0%
Year
1%-5% 5%-10% 10%-20% 20%-30% 30%-40% 40%-50% 50%-60% 60%-70%

70%-80% 80%-90% 90%-95% 95%-99% Median

Table E.3. Middle England Life and General plc, explicit operational risk
Period ending 31/12/02 31/12/03 31/12/04 31/12/05 31/12/06 31/12/07 31/12/08
Profit after tax (452.882) 138.094 146.072 333.504 318.205 235.410 169.952
Net loss ratio 75.8% 68.4% 65.1% 59.7% 60.9% 64.5% 67.4%
Expense ratio 32.4% 36.1% 36.6% 36.5% 36.6% 36.8% 36.9%
Combined ratio 108.2% 104.5% 101.8% 96.2% 97.5% 101.2% 104.3%
Underwriting profit ÿ8.8% ÿ4.8% ÿ5.2% 2.5% 1.5% ÿ2.2% ÿ5.2%
Solvency ratio 50.6% 55.3% 56.2% 65.8% 74.8% 79.2% 78.8%
1009
1010
Funnel plot
Funnel plot
200.0%
150.0%
100.0%
50.0%
Solvency ratio
0.0%
31/12/2002 31/12/2003 31/12/2004 31/12/2005 31/12/2006 31/12/2007 31/12/2008
-50.0%
Year
1%-5% 5%-10% 10%-20% 20%-30% 30%-40% 40%-50% 50%-60% 60%-70%
70%-80% 80%-90% 90%-95% 95%-99% Median

Table E.4. Middle England Life and General plc, explicit operation risk (improved systems of control)
Period ending 31/12/02 31/12/03 31/12/04 31/12/05 31/12/06 31/12/07 31/12/08
Profit after tax (452.882) 140.381 152.968 351.172 332.356 245.322 162.181
Net loss ratio 75.8% 68.2% 64.7% 58.9% 60.3% 64.1% 67.9%
Expense ratio 32.4% 36.1% 36.6% 36.5% 36.6% 36.8% 36.9%
Combined ratio 108.2% 104.4% 101.4% 95.3% 96.8% 100.9% 104.9%
Solvency ratio 50.6% 55.4% 56.6% 67.0% 76.5% 81.4% 80.6%
1011
1012
Funnel plot
Funnel plot
200.0%
150.0%
100.0%
50.0%
Solvency ratio
0.0%
31/12/2002 31/12/2003 31/12/2004 31/12/2005 31/12/2006 31/12/2007 31/12/2008
-50.0%
Year
1%-5% 5%-10% 10%-20% 20%-30% 30%-40% 40%-50% 50%-60% 60%-70%

70%-80% 80%-90% 90%-95% 95%-99% Median

Quantifying Operational Risk PDF

Uploaded by

Copyright:

Available Formats

Quantifying Operational Risk PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Quantifying Operational Risk PDF

Uploaded by

Copyright:

Available Formats

B.A.J.

10, V, 919-1026 (2004)

QUANTIFYING OPERATIONAL RISK IN

Developed by a Giro Working Party

[Presented to the Institute of Actuaries, 22 March 2004]

Operational Risk; Enterprise Risk Management; Risk Management; General Insurance;

1.1 The Starting Point

1.1.8 The current strong push of emerging regulatory requirements

1.2 The Actuarial Contribution

1.3 Quantification Techniques

2.2 Historical Beginnings of MELG

2.3 Current Operations of MELG

2.3.4 The parent company (MICI) is a global insurance company based

2.4 MICI imposes Investment and Business Strategy

Table 2.4.3. 90/10 personal/commercial split; projection

2.5 Management Changes

2.6 Current Position

2.7 Some Major Historical Actions and Incidents

2.9 Initial Response to Information

Table 2.9.10. Summary operational risk loss events

â. A Framework for Assessing and Quantifying Operational Risk

3.2 A Risk Management Framework

Monitor Establish context

Figure 3.2.2. A basic risk management control cycle

management structure, there had been confusion as to how risk information

3.3 Risk Management Maturity Model

Figure 3.3.1. Risk management maturity model

set of risk indicators along with corresponding escalation triggers and an

3.4 Risk Categories

3.4.4 This classification is consistent with a division into core business

3.5 Cause and Consequence

Figure 3.5.2. Causes, events and consequences

3.5.4 In each of these examples a single consequence, a financial loss,

3.6 Defining Operational Risk

3.6.3 The FSA gives a number of examples of operational risks (FSA,

3.7 Practical Considerations

3.8 Data Collection

3.8.7 It is much more difficult to collect exposure data, as often there

3.9 Risk Indicators

3.10 Finding Risk Indicators

3.10.4 It is possible to base risk indicators on qualitative assessments.

ª. Capital Requirements ö Stress and Scenario Testing

4.1 General Approach

4.4 Case Study Application

4.5 Overall Assessment

4.6 Comments and Observations

ä. Frequency and Severity Analysis (including EVT)

5.1 General Approach

5.3.2 The risk director decided to reflect these differing characteristics in

5.5 Comments and Observations

6.1 General Approach

Figure 6.2.3. Probability of hearing the burglar alarm

Figure 6.2.4. Probability of a burglary if the alarm is heard, but no radio

Figure 6.2.5. Probability of a burglary if the alarm is heard and there is a

6.3 Case Study Application

Figure 6.3.1. Enterprise wide risk map

Figure 6.3.8. Risk map

6.4 Case Study Results

Figure 6.4.3. Simple Bayesian model