See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/319753715

An Empirical Study of Root-Cause Analysis in Information Security

Management

Conference Paper · September 2017

CITATIONS READS

13 10,601

4 authors, including:

Gaute Bjørklund Wangen Niclas Hellesen

Norwegian University of Science and Technology, Gjøvik, Norway Norwegian University of Science and Technology
28 PUBLICATIONS 401 CITATIONS 3 PUBLICATIONS 18 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Security awareness View project

Implementation of Information Security Management System and Risk Management View project

All content following this page was uploaded by Gaute Bjørklund Wangen on 03 February 2018.

The user has requested enhancement of the downloaded file.

 An Empirical Study of Root-Cause Analysis in Information Security Management

Gaute Wangen, Niclas Hellesen, Henrik Torres, and Erlend Brækken

NTNU Gjøvik
Teknologiveien 22,
2802 Gjøvik, Norway
Email: [email protected], [email protected],
[email protected], [email protected]

Abstract—This paper studies the application of Root-cause problem-solving paradigms, business process improvement,
analysis (RCA) methodology to a complex socio-technical in- benchmarking, and continuous improvement [4]. The ISRA
formation security (InfoSec) management problem. InfoSec risk and RCA approaches are different in that RCA investigates
assessment (ISRA) is the common approach for dealing with incidents that have occurred with some frequency aiming to
problems is InfoSec, where the main purpose is to manage risk understand and eliminate the problem from a socio-technical
and maintain an acceptable risk level. In comparison, the RCA
tools are designed to identify and eliminate the root-cause of a
perspective. While ISRA attempts to estimate the risk and
reoccurring problem. Our case study is a complex issue regarding propose and implement risk treatments based on the results
multiple breaches of the security policy primarily through access to achieve acceptable risk.
control violations. By running a full-scale RCA, this study finds The case study presented in this paper extends the ISRA
that the benefits of the RCA tools are a better understanding of a complex socio-technical problem with RCA and discusses
of the social aspects of the risk; RCA highlighted previously the cost/benefit of the results. The objective of ISRM is to
unknown social and administrative causes for the problem reduce risk to an acceptable level. A typical ISRA would be to
which in turn provided an improved decision-basis. The problem estimate annual incident cost, compare it to risk appetite, and
treatments recommended by the ISRA and the RCA differed
in that the ISRA results recommended technical controls, while if found unacceptable: implement a treatment to address either
the RCA suggested more administrative treatments. Furthermore, probability, consequence, or both, to maintain the risk within
we found that the ISRA and RCA can complement each other in acceptable levels, while RCA aims to remove the problem in
administrative and technical issues. The main drawback was that its entirety. However, both approaches seek to treat the problem
our cost-benefit analysis regarding hours spent on RCA was on at hand, which makes the output comparable. The application
the borderline of being justifiable. As future work, we propose of formal RCA tools is an area that has remained largely
to develop a leaner version of the RCA scoped for information unexplored in InfoSec literature. Therefore, the problem we are
security problems. addressing in this study is to determine the utility of RCA for
Keywords—Information Security; Root cause analysis; Risk InfoSec and if it provides useful input to the decision-making
Management; Case study. process beyond the ISRA. The problem is investigated using a
case study, qualitative assessment of results, and cost-benefit
I. I NTRODUCTION analysis.
Judging by the available literature on standards and The case is of breaches to the access control (AC)
methods, the common approach to dealing with problems security policy (SecPol), such as access card and Personal
in information security (InfoSec) is risk assessments. Risk Identification Number (PIN) exchange between employees.
assessment aims to estimate the probability and consequence This complex problem is located at the intersection of the
of an identified scenario or for reoccurring incidents, and social and technological aspects that many organizations may
propose risk treatments based on the results. By estimating the face. The Scandinavian organization in our case study had
expected risk of repeating incidents or an identified scenario, logged multiple occurrences of policy violations together with
risk assessment aims at proposing risk treatments based on costly incidents as a consequence. This study investigates if
the estimated results. The InfoSec risk assessment (ISRA) has RCA can be applied as a useful extension to the ISRM process
been developed to analyze risks that occur when applying for the AC SecPol problem. To investigate this issue, we
technology to information, and revolve around securing the qualitatively assess the results of a RCA conducted as an
confidentiality, integrity, and availability of information or extension to a high-level ISRA of the problem. Further, we
other assets [1]. By focusing on assets and vulnerabilities, discuss if RCA can be justified for complex InfoSec problems
these assessments tend to have a technical scope [2] [3] through cost-benefit analysis. This paper applies the seven-
with estimates of consequences and respective probabilities of step process RCA methodology [4] for comparison of results.
events as key outputs. Although the InfoSec risk management The data collected for this study was primarily from historical
(ISRM) approach is useful for maintaining acceptable risk observations and data in the target institution together with
levels, they are not developed to solve complex socio-technical qualitative interviews of thirty-six representatives from six
problems. In comparison, the Root Cause Analysis (RCA) is relevant stakeholder groups.
”a structured investigation that aims to identify the real cause The remainder of the paper is structured as follows:
of a problem and the actions necessary to eliminate it.” [4] The following section addresses previous work on RCA in
RCA incorporates a broad range of approaches, tools, and tech- InfoSec. Section III provides a description of the applied
niques to uncover causes of problems, ranging from standard
ISRA method and the RCA tools methods including statistical re-occurring InfoSec problems. The studies we found provided
analysis. Further, we present the results from the ISRA and the positive results and motivation for further experiments with
RCA. Lastly, we discuss the qualitative differences and discuss RCA for InfoSec problems.
cost-benefit. Finally, we discuss the limitations, propose future
work, and conclude the results. III. M ETHOD
The primary research approach was a case study which
II. R ELATED W ORK was conducted in a Scandinavian R&D institution to inves-
RCA was developed to solve practical problems in tradi- tigate the complex problem of internal AC policy violations.
tional safety, quality assurance, and production environments The ISRA was conducted as a high-level risk assessment for
[4]. However, RCA has also been adopted in selected areas the institution which revealed the need for deeper analysis
of InfoSec: Julisch [5] studied the effect of the RCA, by of the problem. Three independent researchers conducted the
considering RCA for improvement of decision-making for RCA and gathered data from 36 scientific interviews and
handling alarms from intrusion detection systems. The study applied historical data on incidents caused by unauthorized
provides evidence towards the positive contribution of RCA, access.
but it does not apply the RCA tools as they are proposed in the Further, we qualitatively compare the results where we
recent literature [4], [6], [7]. Julisch builds on the notion that analyze the differences in approaches, findings, and treatment
there are root causes accounting for a percentage of the alarms, recommendation. Additionally, we applied a cost-benefit anal-
but proposes his tools for detecting and eliminating root causes ysis to measure resources regarding time spent on conducting
outside of the problem-solving process, Fig. 1. A more recent RCA and benefits concerning additional knowledge about the
study conducted by Collmann and Cooper [8] applied RCA for problem.
an InfoSec breach of confidentiality and integrity in the health- The following section briefly describes the ISRA ap-
care industry. Based on a qualitative approach, the authors proach applied in this study, while the second section de-
find the root cause of an incident and propose remediation. scribes the RCA approach. The latter contains a description
Their results also show a clear benefit from applying RCA, of the seven-step RCA process, the tools used, data collection
although their RCA approach seems non-standardized, being method, and a brief overview of the statistical methods used
primarily based on previously published complex problem- for data analysis.
solving research articles. Wangen [9] utilizes RCA to analyze
a peer review ring incident, where an author managed to game A. ISRA Method
the peer review process and review his papers. This incident The ISRA method applied for the case study is based on
is analyzed by combining RCA tools and the Conflicting the standard ISO/IEC 27000-series [1]. Further substantiated
Incentives Risk Analysis (CIRA) to understand the underlying with the Wangen et.al. [13] [14] approaches which centers on
incentives and to choose countermeasures. Further, Abubakar estimations of asset value, vulnerability, threat, and control
et.al. [10] applied RCA as a preliminary tool to investigate the efficiency, these are combined with available historical data
high-level causes identity theft. The study applies a structured to obtain both quantitative and qualitative risk estimations.
RCA approach [7] and identifies multiple causes and effects The applied method identifies events together with adverse
for setbacks to the investigation of identity theft. The Abubakar outcomes and uses conditional probability to estimate the risk
et.al. study shows the utility of RCA for InfoSec by providing of each identified outcome. The results section provides a
an insight into a complex problem such as identity theft. summary of the initial ISRA results.
Hyunen and Lenzini [11] discuss RCA application in InfoSec
by contrasting the traditional approaches to Safety and Secu- B. Approach to Root cause analysis
rity to highlight shortcomings of the latter. Furthermore, the In choosing a RCA framework, we looked at compre-
authors propose an RCA-based tool for InfoSec management hensiveness, academic citations, and availability. Based on the
to address said shortcomings and demonstrate the tool on a use criteria, our study chose to follow the seven-step RCA process
case. The tool is designed to reveal vulnerable socio-technical proposed by Andersen and Fagerhaug [4], as shown in Fig.
factors. 1. Each step consists of a set of tools to produce the results
Some of the tools applied in an RCA are also recog- needed to complete the subsequent steps, whereas step 7 is
nizable in the risk assessment literature, for example, instru- out of scope. Each step consists of different tools to solve
ments such as Flowcharts and Tree diagrams model processes problems where one or more are required to complete the
and events visually. Typical comparable examples from risk RCA and conclude the root cause(s). As recommended in the
assessment are Event-tree and Fault-tree analysis, where the methodology, we chose tools per step based on our judgment
risk is modeled as a set of conditional events, however, these of suitability. The RCA in this study was conducted by a three-
approaches are not specifically developed for InfoSec risk person team supported by a mentor. We have anonymized in-
analysis. Schneier adapted the Fault-tree analysis mindset and formation according to the employer’s requests. The following
created Attack Trees [12]. These tools resemble those of RCA. subsections describe each step in the RCA process and our
However, the frame for applying them is different in the sense selected tools (see [4] for further description).
that attack trees focus on the technical threat and vulnerability Step 1 - Problem understanding, Performance Ma-
modeling, while RCA tools focus on problem-solving. trices. The goal of this step is to understand the problem and
Although there are a couple of published studies on rank the issues. Performance Matrices are used to illustrate
the application and utility of formal RCA methodologies, the the target system’s current performance and importance. The
previous work on RCA in InfoSec is scarce, and there is a performance matrix contributes towards establishing priority of
research gap in experimenting with the RCA tools for solving the different problems, factors, or problems in the system [4]
 Further, we counted the occurrence of each theme and sum-
marized the responses. We also applied the Affinity diagram
for analyzing our qualitative data, which is a RCA tool for
grouping data and discovering underlying relationships.
Step 5 - Root Cause Identification - Cause-and-
Effect Charts. The goal of this step is to identify the
root cause(s) of the problem. For this task, we applied the
Fig. 1: Seven step process for RCA [4]. Cause-and-Effect chart (Fishbone diagram) which is a tool for
identifying the major causes of a problem, together with the
secondary causes/factors influencing the problem. The results
from this process should map to the undesired effect, the
problem.
(P.36-41): (i) which part of the problem is the most important
to address, and (ii) which problem will reduce the highest Step 6 - Problem elimination - Systematic Inventive
amount of symptoms. The problems are qualitatively identified Thinking (SIT). The goal of this step is to propose solutions
and ranked on a scale from 1 to 9, on performance (x-axis) to deal with the root causes of the problem, Andersen and
and importance (y-axis). Fagerhaug [4] describe primarily two types of tools for drafting
treatments; one is designed to stimulate creativity for new
Step 2 - Problem cause brainstorming. The main idea solutions, while the other is designed for developing solu-
of this step is to cover other possible issues that may be tions.
causing the problem, not thought of in Step 1. For this purpose,
we applied unstructured Brainstorming, which is a technique IV. C ASE S TUDY: ACCESS C ONTROL P OLICY
where the participants verbally suggested all possible causes V IOLATIONS
they could think of, which was immediately noted on a
whiteboard and summarized together at the end. In this section, we first present a summary of the results
from the ISRA, in terms of risk estimation and proposed
Step 3 - Problem Cause Data Collection - Interviews.
treatment. Further, we present the results from our RCA for
RCA recommends several data collection techniques [4], this
comparison.
study chose scientific interviews as the main data collection
approach as the study required an in-depth understanding of the The case data was collected from an institution whose
motivations for AC SecPol violation problem. The interviews IT-operations delivers services to about 3000 users. The organi-
were conducted in a face-to-face setting, and was designed zation is a high-availability academic organization providing a
using category, ordinal, and continuous type questions together range of services to the users, mainly in research, development,
with open-ended interview questions for sharing knowledge and education. The IT Operations are the internal owners of
about the problem. The interview subjects were primarily the AC regimes and most of the lab equipment; they represent
categorized as representatives of key stakeholder groups within the principal in this study. The objectives of the IT-operations
the organization and one group of external contractors. Each is to deliver reliable services with minimal downtime, together
interview had twenty-six questions with follow-up questions with information security solutions.
if deemed necessary to clarify the opinion or to extract During the last years, the Institution has experienced
valuable knowledge from particularly knowledgeable individ- multiple incidents of unauthorized access to its facilities. The
uals. recurring events primarily lead to theft and vandalism of
Step 4 - Problem Cause Data Analysis - Statistics equipment in a range of cost that is deemed unacceptable.
& Affinity diagram. We applied a variety of statistical Thus, the hypothesis is that this has partially been caused
data analysis methods specified in the results, and the IBM by employees and students being negligent of the SecPol
SPSS software for the statistical analysis. A summary of the regarding AC, providing unauthorized access to the facilities.
statistical tests used in this research is as follows. While the SecPol explicitly states that both the token and
the PIN are personal and shall not be shared, there has been
For Descriptive analysis on continuous type questions, registered multiple incidents of this occurring.
we applied the median as the primary measure of central
tendency. We also conducted Univariate analysis of individual
A. The Risk of Access control policy violations
issues and Bivariate analysis for pairs of questions, such as a
group belonging and a continuous question, to see how they The goal of the ISRA was to derive the annual risk of
compare and interact. As the Likert-scale seldom will satisfy the incidents. This section summarizes the asset identification
the requirements of normality and not have a defined scale of and evaluation, vulnerabilities assessment, threat assessment,
measurement between the alternatives, we restricted the use of control efficiency, and outcomes.
mean and standard deviation. We analyzed the median together The Institution had two key asset groups: (i) hard-
with an analysis of range, minimum and maximum values, ware and (ii) physical sensitive information, both stored in
and variance. This study also analyses the distributions of the access controlled facilities. The hardware’s primary protec-
answers, for example, if they are normal, uniform, bimodal, or tion attribute was availability, and the value was estimated
similar. We used Pearson two-tailed Correlation test to reveal in the range of moderate according to the budget, with a
relationships between pairs of variables as this test does not low to medium importance in the day-to-day business pro-
assume normality in the sample. cesses.
The questionnaire had several open-ended questions The two controls in place are primarily (i) AC mech-
which we treated by listing and categorizing the responses. anisms - physical control in place to prevent unauthorized
accesses and mitigate the risk of theft. (ii) The SecPol -
administrative control, which is a written statement concerning
the proper use of AC mechanisms.
For the vulnerability assessment, experience showed that
illegitimate users were accessing the facilities on a daily
basis. We identified two primary vulnerabilities; (i) lack of
security training and awareness, whereas the stakeholders
do not understand the risk exposure of the organization.
(ii) Insufficient organizational security policies, whereas the
SecPol itself lacks clear consequences for breaches, leaving
the personnel complacent. The main attack for exploiting these
two vulnerabilities was social engineering, where the attacker
either manages to get a hold of a security token and PIN.
Alternatively, the attacker manages to gain unauthorized access
to the facilities by entering with others who have legitimate
access (tailgating). With the number of stakeholders having Fig. 2: Performance matrix.
access, both attacks are easy for a motivated threat actor. The
exposure is summarized in Table I.
risk treatment also subjects the organization to requirements
TABLE I. SUMMARY OF VULNERABILITY from data privacy protection laws. Neither did it address the
ASSESSMENT. socio-technical problem with the SecPol, card swapping, and
card lending.
Vulnerability Attack Attack Vulnerability Exposure
Scenario
Description description Difficulty Severity Assessment
Lack of Security Social Engineering - V. ROOT C AUSE A NALYSIS R ESULTS FOR A
Training and Employee or Student
A1 Awareness, Gives away Token Medium Very High High S OCIO -T ECHNICAL PROBLEM
Insufficient and PIN (Likely)
InfoSec Policies In this section, we present the results from conducting
Lack of security
training and
Social Engineering-
Employee or Student
the RCA according to the method described in Section III-B.
A2 awareness, leaves doors opened Easy Medium Medium The results are derived from conducting RCA on the previously
Insufficient for convenience
InfoSec Policies outlined problem and risk; we outline the hypothesized root
causes and proposed treatments.
For the threat assessment, the experts identified one
threat group motivated by a financial incentive with the intent A. RCA Process, Step 1 & 2 - Problem Understanding and
of stealing either physical equipment or sensitive information, Cause Brainstorming
with two actors; (i) Actors who frequently steals small items, The goal of these steps is to scope the RCA and center on
representing high frequency - low impact risk. (ii) Actors who the preliminarily identified problem causes. The performance
conduct a few significant thefts, representing the low frequency matrix, Fig. 2, is used to rank the identified causes on their
- high impact risk. Importance and Performance. With the help of resource per-
sons, the team derived six topics from the preliminary RCA
B. Risk Analysis Results. steps 1 & 2, Fig. 1): (i) Theoretical knowledge of the SecPol
The ISRA results showed that the most severe risk for AC, (ii) Practical implementation of the SecPol for AC,
facing the organization is theft of sensitive information, while (iii) Consequences for policy breaches, (iv) Security Culture,
physical theft of equipment is also a grave risk. According (v) Backup solutions for forgotten and misplaced cards, and
to past observations, the risk is greatest during holidays (vi) Card hand out for new employees. The RCA team and
with few people on campus. The two primary risks were the expert ranked the issues and prioritized the data collection
major equipment thefts during the holiday season and several step accordingly, illustrated in Fig. 2.
minor equipment thefts that aggregated into an unacceptable
amount. B. RCA Process Step 3 - Data Collection

C. Implemented Treatment - Camera Surveillance

As a result of the ISRA, the treatment implemented to
reduce the two risks was camera surveillance of the main entry
points of buildings. Firstly, this treatment has a preventive
effect in the sense that it will heighten the attack threshold
for threat actors. Besides, it will provide audit trails that will
be useful in future investigations. Camera surveillance had also
been proven to reduce the number of incidents as well as
increasing the amount of solved crimes in similar institutions.
This data indicates a high control efficiency; however, the mea-
sure also comes with some drawbacks, such as equipment cost
together with the required resources to operate the system. Due Fig. 3: Stakeholder groups included in the study
to the data collection on employees surveillance brings, this
 TABLE II. DEMOGRAPHICS INCLUDING AGE AND borrowing among employees more severely than men. The
SEX DISTRIBUTIONS women in our sample also believe that it is more likely
that employees admit to borrowing cards. Another visible
Age Sex difference between the stakeholder groups was who had read
Group Freq. Percent Group Freq. Percent the policy, where all the representatives of the Management
20-29 8 22,2 Women 10 27,8 and IT and Security groups had read it. The Ph.D. Fellows
30-39 7 19,4 Valid Men 26 72,2
40-49 10 27,8 Total 36 100,0 and the student groups scored the lowest on having read the
Valid 50-59 8 22,2 policy. Another observable finding was that the waiting time
60-69 3 8,3
Total 36 100,0 varied between the groups, whereas the permanent employees
perceived the shortest waiting times, Table III.
2) Qualitative analysis of differences between groups:
For the categorical analysis, the team used age, gender, IT and Security. The IT operations owned much of the hard-
and stakeholder group as the primary categories, with the ware in the facilities and was in charge of both designing, im-
emphasis on the latter as our hypothesis was that parts of the plementing, and operating the AC policy. Both representatives
root cause are found in conflicting interests between internal had read the policy and considered it important that staff and
groups. The team interviewed thirty-six people located at the students also know the policy. The IT operations believed that
site, Fig. 3 displays age and gender distributions, with the card lending is an increasing problem within the institution,
six primary stakeholder groups. The interview subjects for the especially in the modern facilities where AC mechanisms are
academic staff, Ph.D. Fellows, B.Sc. and M.Sc. students were more frequent. One also answered that since he had been
chosen at random. The representatives of management and IT involved in developing the policy, he felt more ownership of it
and security were key stakeholders in the organization, such and, therefore, experienced a greater responsibility to follow it
as decision-makers and policy writers. than other departments. They also felt the legal responsibility
not to break the policy due to owning the AC system.
C. RCA Step 4 - Problem Cause data analysis Management. This group consists of middle and upper
The Descriptive analysis showed that about half of the management, which had all read the SecPol. Half believed
respondents had read the SecPol. All but two reported that it it was important to have those who will be subject to the
is was not allowed to lend away cards, whereas the remaining policy involved in the policy development process. When we
two did not know, indicating a high level of security awareness asked this group about what they saw as the worst scenario,
for the issue. Also, the study uncovered uncertainty among this group had similar opinions: their main concerns was loss
the respondents when we asked them about what the potential and compromise of information together with relevant legal
consequences for breaching the SecPol would bring for the aspects. Two members of this group reported that they did
employees. Whereas most of them assumed no consequence, not get the service they expected from IT regarding forgotten
and none perceived any severe consequences. We also uncov- cards. Three out of four said that they believed the security
ered that most people would be reluctant to admit to sharing culture to be good, while the last one reported the security
cards. Further, we asked them ”How often do you think access controls to be cumbersome.
cards are shared at the Institution?” on a scale from 1 - 5 Senior academic staff. Consists of different types of
(1- Never, Yearly, Monthly, Weekly, 5 -Daily), to which the professors, researchers, and lecturers, and represents the ma-
respondents thought that this is an issue that occurs on at jority of employees in the case. This group was the largest with
least a weekly basis (Median 4). Using the same scale, the the most widespread opinions. Regarding the SecPol, several
team asked how often the respondents had the need to borrow expressed discontent and said that it was neither security
cards from others. Over half reported to not ever had the need, department or IT service that should be responsible for it.
while twelve reported having had to lend cards on an annual The organization should provide the content of the policy to
basis, only two reported having the problem more than that. ensure that it was not an obstacle in the day to day work.
However, half of the respondents said to have been asked by Further, delivering on the aims and goals of the organizational
others to borrow cards, which documented the frequency of assignment should be compared to the potential harm from
the problem. card swapping incidents, meaning that the policy should be
designed with a better understanding of risk. An example of
TABLE III. NOTABLE DIFFERENCES BETWEEN this was that employees must have access to rooms to do their
GROUPS ON ”HOW LONG DID IT TAKE FOR YOU TO job where a too-strict policy would stand in the way. Regarding
GET ACCESS TO THE FACILITIES YOU NEEDED?” this, several mentioned that if the cards were not lent to other
(BETWEEN 1 VERY LONG - 6 IMMEDIATE ACCESS) employees, it would be very problematic due to the lack of
backup solutions. They missed good fallback solution if one
Category N Range Median Minimum Maximum Variance had forgotten access card.
Management 3 0 6,00 6 6 0,000
Senior Academic Staff 17 4 6,00 2 6 1,654 Ph.D. Fellows. Out of this group, only one had read
Ph.D. Students 7 5 5,00 1 6 3,238 the SecPol. Most assumed it was not allowed to lend out
BSc. and MSc. Students 3 4 3,00 1 5 4,000
External Contractors 3 3 4,00 1 4 3,000 their access cards, but two said they did not know. One
Total 33 5 5,00 1 6 2,729 expressed discontent from not receiving his access card quick
enough, which he hypothesized as one of the reasons for
1) Summary of categorical analysis: The statistical anal- borrowing other people’s cards. Longer times to hand out
ysis showed differences between the responses of men and access cards may force them to lend cards internally in an
women; where the latter viewed incidents involving card office. Another issue was that Ph.D. Fellows occasionally
worked with students and that they often needed access to modern buildings, there is a very strict AC regime in place,
restricted facilities to be able to work. This issue required where low-level security rooms and facilities are regulated.
the Ph.D. fellow either to open the door physically for the Several of the respondents highlighted this as the main reason
students or to loan them their card. When we asked about for card lending. These low-security rooms only required the
the security culture, the responses were split: Two did not card and not the PIN code, so the respondents did not consider
know, one thought that security was good, another one said this a serious breach of policy. Several of our respondents said
that people trust each other, one said it was wrong, while one that this was too much security and could not understand the
said that people knew that they should not lend it to others. reasoning underlying this decision.
The last one said that others could borrow it for practical 5. Lack of risk awareness and consequences. 33 out of
reasons. 36 defined possible negative consequences for the institution,
Students. Represents the main bulk of people with so, the awareness around possible risks for the institution was
access to the main facilities, but with limited access to offices high. However, we found that less than half of the respondents
and employee areas. Only one of the students had read the had read the overarching SecPol and that the respondents
policy, and none of the students who participated knew of any were unaware and uncertain about the organization’s and their
instances of card lending, although two out of three had been personal risk if their cards went astray. Everybody agreed that
asked by someone if they could lend them their cards. it was a bad thing, but nobody could say with certainty what
External contractors. Represents the contractors in the consequences would be, if any at all.
charge of running the physical facilities, such as cleaning
personnel and physical maintenance. In the External group, E. RCA Step 6 - Proposed root cause treatments
only one had read the policy. All believed that it was not Based on our findings we conducted Systematic In-
allowed to borrow cards and that the school saw this as a ventive Thinking and came up with following root cause
serious offense. Only one of them reported having had the treatments:
need to borrow a card. Improve fallback solutions. Regarding root cause 1 and
2, the RCA team proposed to develop a solution for reserve
D. RCA Step 5 - Identified Root causes access cards with adequate and tailored room access. The solu-
The interviews with the groups provided an insight into tion should provide basic access to low-security level facilities,
the many views on this problem and the complexity it entails, with tailored room access according to stakeholder needs. This
visualized with the Fishbone diagram in Fig. 4. Based on our suggestion should be a public and low threshold offer for those
RCA we found five possible root causes: who have forgotten or misplaced their cards.
1. Uncertainty regarding fallback solutions. We found Align SecPol with objectives. Regarding root causes
that there was uncertainty surrounding available backup so- 3 and 4, the RCA team proposed to risk assess the need
lutions among all the stakeholder groups. Where 14 of the for physical security and AC for the facilities based on the
31 respondents were undecided if there existed any fallback organizational goals, employee needs, and the assets stored in
solution, and suggested to create better backup solutions. 17 the room. Include key stakeholders in the process and focus
said there existed backup solutions, but we uncovered different on balancing productivity and security to revise the security
opinions regarding what these were and who was responsible baseline.
for them. For example, six respondents thought they could Improve the overarching SecPol. Regarding cause 5,
summon the IT department, three thought the student help the RCA team proposed to improve the overarching SecPol,
desk, while the remainder thought either management could the suggestions were: (i) clarify consequences for breaches of
help or ask a colleague to lend them access cards. Even from policy, (ii) assigning a responsible for sanctions per depart-
the two key stakeholders in IT the replies were contradic- ment, (iii) including the employees in the shaping of policy,
tory. and (iv) increase the accessibility of the policy.
2. Discomfort when using fallback solutions. Two of Improving risk awareness. Regarding root cause 5, we
our respondents reported to have forgotten their cards and had also propose to improve risk awareness among the stakehold-
contacted the on-campus card distributor to use the fallback ers, by running awareness campaigns including both the risks
solution. The respondents meant they had not been well- the organization and employees are facing. As a part of this,
received and had not gotten the help they needed. Overall, we proposed to create an information bank regarding risks,
they reported the situation to be discomforting, which was fallback solutions, and how to make use of them.
unfortunate, as this may lead to the employees using different
methods for solving the problem. VI. D ISCUSSION
3. Misaligned SecPol regarding authorization. Our This section discusses the additional insight gained from
interviews highlighted that being able to do their work is the the RCA; first qualitatively, and then through cost/benefit
most important goal for every employee. Thus, the SecPol analysis.
should aim to facilitate this aim. Too strict AC will in some
cases lead to obstruction in day-to-day tasks and lead to em- A. Additional insight gained from RCA
ployees finding workarounds which may compromise security, Upon completing the RCA, we see that the results from
such as asking trusted co-workers to borrow cards. Some of the ISRA and RCA provide different models of the same
the respondents reported not having been included in the de- problem. The information gathered from the ISRA process
velopment of the SecPol and felt that it was misaligned. was scoped towards technical risks with solutions for reducing
4. Too much security. In especially one of the most probability and consequence. Furthermore, we found the RCA
 Fig. 4: Fishbone diagram illustrating contributing causes to the main problem.

to work better to visualize complexity and providing insight TABLE IV. TOTAL HOURS SPENT CONDUCTING RCA
into the human aspects of the problem. However, the RCA FOR AN UNTRAINED THREE MAN TEAM
process was resource intensive and required extra training to (APPROXIMATELY 220 HOURS PER TEAM MEMBER)
complete. The RCA process also required the inclusion of
more stakeholders than the ISRA. Step Phase Tasks Time spent
Preliminary Preparations Collecting available data 100 hours
The results show that the benefits of the RCA are a better Preliminary Preparations Testing and choosing tools 72 hours
understanding of the social dimensions of the problem, such 1 Problem Understanding Performance Matrix 3 hours
2 Problem cause brainstorming Brainstorming 1 hours
as conflicts between users and the security organization. This 3 Problem cause Data Collection Planning interviews 150 hours
insight provides an improved decision basis and an opportunity 3 Problem cause Data collection Conducting interviews 100 hours
4 Data analysis Qualitative & Statistical 220 hours
for reaching a compromise with the risk treatment. The risk 5 Root cause identification Fishbone 7 hours
assessment team were aware of two (cause 3. and 5.) out of 6 Root cause elimination SIT 7 hours
Total 660 h.
the five identified root causes of the problem. Thus, in our case Only RCA Process Total 488 h.
study, the RCA did provide a valuable extension to the risk
assessment for solving the problem. The RCA results showed
all root causes to be on the administrative and human side the main task of the root cause identification phase was to
of the problem. Thus, the treatments produced from the two formalize the causes and effects, and the elimination was used
approaches were different; ISRA produced a technical treat- to propose treatments.
ment in camera surveillance, while RCA produced multiple
administrative treatments, each for addressing separate root As the team gain experience with using RCA on cases,
causes. the time estimate should be significantly be reduced. For
example, our study spent 172 hours in the preparation phases
Although the ISRA did highlight the vulnerabilities gathering data on the problem and testing tools. With more ex-
related to the human factor and risk perception as one of perience, the preliminary steps will be significantly shortened.
the risk factors, in this case, the decision-makers did not Our team also estimated that the whole process itself would
opt for revision of the AC policy. To summarize, the ISRA become leaner with practice.
findings viewed card lending as a technical security problem,
while RCA extended the knowledge into the administrative To summarize, we derived the primary benefit from
problem. the problem cause data collection and analysis phases, which
enabled the root cause identification. Furthermore, the group
benefited from working on the performance matrix, which set
B. Cost-benefit analysis the direction for the remainder of the project. Regarding the
For cost-benefit analysis, we consider time spent on tasks remaining tools, the benefits the problem cause brainstorming
and usefulness of the task. Table IV shows that the process of was that it helped to provide an overview of the problem space
achieving desired results was time and resource consuming for and invited creative thinking. The advantage of the Fishbone
our team. The reported hours are the total amount from start tool was to group and visualize the identified problems in the
to end without having a budget constraint. The reported hours context. Further, the process step contributed to determine and
does contain resources spent beyond the three-man team, e.g. analyze causes. The SIT tool has a series of five principles
from interview attendance and supervision. that attempts to discover how to solve the components of the
The most time consuming and crucial tasks were the steps 3 root cause. This tool offers a well-structured way to traverse
and 4, data collection and analysis. Further, the table shows a problem situation but could be resource intensive when
that the resource demand for the Root cause identification and handling many problems with all their components.
elimination phases as low, this is because the team primarily Issues of minor importance should not be subject to such
identified the root causes during the data analysis. While an extensive effort as RCA requires. During the preparations
for this study, we ran RCA for minor issues and found it of the time and resources invested in the project is on the
not worthwhile as it was unproductive to use a complicated borderline of being justifiable, and the cost of the problem
problem-solving process to less costly problems. However, should be considered before launching a RCA. Thus, the
future projects should consider RCA when they perceive the RCA provides a viable option when dealing with complex and
issue as important and do not know its nature or cause. The costly InfoSec problems and should be a part of the InfoSec
problem should be expensive, complicated, and cannot be management toolbox.
addressed sufficiently with less comprehensive methods. These
properties make conducting an RCA on the project justifiable ACKNOWLEDGEMENTS
and a valuable addition to the decision-making process. The authors acknowledge the help and support from Professor
Einar Snekkenes, Christoffer Hallstensen and Stian Husemoen.
C. Limitations & Future Work We also extend our gratitude to all the participants in our study
The case study presented in this article is specific to and to the anonymous reviewers.
the organization and culture; thus our results have limited R EFERENCES
generalizability, but the RCA method and results provide an [1] Information technology, Security techniques, Information Security
insight into what to expect from the process. Another aspect Risk Management, International Organization for Standardization Std.,
is that our RCA team was inexperienced and other more ISO/IEC 27005:2011.
experienced teams will run the process more efficiently with a [2] G. Wangen and E. Snekkenes, “A taxonomy of challenges in in-
better cost-benefit. Another issue is if a similar insight could formation security risk management,” in Proceeding of Norwegian
have been gained if we delegated a similar amount of resources Information Security Conference - NISK 2013 - Stavanger, vol. 2013.
Akademika forlag, 2013, pp. 76–87.
into the ISRA to investigate the problem. It is possible that the
results of the ISRA would have overlapped more with the RCA [3] P. Shedden, W. Smith, and A. Ahmad, “Information security risk
with more time and resources spent on the former. However, assessment: towards a business practice perspective,” in Australian In-
formation Security Management Conference. School of Computer and
the ISRA process does not argue for such a deep dive into Information Science, Edith Cowan University, Perth, Western Australia,
the problem as the RCA process and does not provide tools 2010, pp. 119–130.
for doing so. It is therefore unlikely that a more thorough [4] B. Andersen and T. Fagerhaug, Root cause analysis: simplified tools
ISRA process would have produced a similar result. However, and techniques. ASQ Quality Press, 2006.
the incentive for such an investigation was not there, and we
[5] K. Julisch, “Clustering intrusion detection alarms to support root
perceive the ISRA methodologies as immature in this area [14]. cause analysis,” ACM transactions on information and system security
Instead of considering the RCA as an extension of the ISRA, a (TISSEC), vol. 6, no. 4, pp. 443–471, 2003.
possible path for future work is to conduct case studies where [6] P. F. Wilson, Root cause analysis: A tool for total quality management.
the researchers invest a similar amount of resources into both ASQ Quality Press, 1993.
the RCA and ISRA and then compare results.
[7] A. M. Doggett, “Root cause analysis: a framework for tool selection,”
An additional direction for future work is to apply RCA The Quality Management Journal, vol. 12, no. 4, p. 34, 2005.
to more and diverse case studies to get a better understand- [8] J. Collmann and T. Cooper, “Breaching the security of the kaiser
ing of the contributions and limitations of the approach for permanente internet patient portal: the organizational foundations of
InfoSec. Recent work has also proposed a novel approach for information security,” Journal of the American Medical Informatics
conducting socio-technical security analysis [11], and a path Association, vol. 14, no. 2, pp. 239–243, 2007.
for future work is to adapt, develop, and improve RCA tools for [9] G. Wangen, “Conflicting incentives risk analysis: A case study of the
InfoSec. Furthermore, the future efforts could research RCA normative peer review process,” Administrative Sciences, vol. 5, no. 3,
efficiency through automation of tasks and build knowledge p. 125, 2015. [Online]. Available: http://www.mdpi.com/2076-3387/5/
3/125
repositories. Regarding the latter, a repository of tools for
data collection would help streamline step 3 in the RCA [10] A. Abubakar, P. B. Zadeh, H. Janicke, and R. Howley, “Root cause
analysis (rca) as a preliminary tool into the investigation of identity
process. theft,” in Cyber Security And Protection Of Digital Services (Cyber
Security), 2016 International Conference On. IEEE, 2016, pp. 1–5.
VII. C ONCLUSION [11] J.-L. Huynen and G. Lenzini, “From situation awareness to action: An
This study has applied RCA tools to propose a solution information security management toolkit for socio-technical security
to a complex socio-technical InfoSec problem and found the retrospective and prospective analysis,” in Proceedings of the 3rd
International Conference on Information Systems Security and Privacy,
RCA method a valid but costly extension to the ISRA. Running 2017, pp. 213 – 224.
a full-scale RCA requires a lot of time and resources and
the problem should be expensive enough to justify the RCA. [12] B. Schneier, “Attack trees,” Dr. Dobbs journal, vol. 24, no. 12, pp.
21–29, 1999.
The results from the RCA overlapped slightly with the initial
ISRA. The main differences were that the RCA team proposed [13] G. Wangen, A. Shalaginov, and C. Hallstensen, “Cyber security risk as-
sessment of a ddos attack,” in International Conference on Information
administrative treatments aimed at solving problems in the Security. Springer, 2016, pp. 183–202.
social domain, while the ISRA produced a more technical
analysis and treatment for the problem. We conclude that prac- [14] G. Wangen, C. Hallstensen, and E. Snekkenes, “A framework for
estimating information security risk assessment method completeness,”
titioners should look at these two approaches as complimentary International Journal of Information Security, Jun 2017. [Online].
for dealing with complex socio-technical risks and problems. Available: http://dx.doi.org/10.1007/s10207-017-0382-0
The combination of the ISRA and RCA will also have utility
when planning for defense-in-depth, where administrative and
technical risk controls can work in coherence to mitigate
threats. The main drawback was that our cost-benefit analysis

View publication stats